Skip to content

Commit c04240b

Browse files
vstinnergpsheadethanfurmanEclips4
committed
gh-161705: Prevent shadowing stdlib in gzip/tarfile/zipfile CLI (#161706)
Remove sys.path[0] in gzip, tarfile and zipfile command line interface to prevent shadowing standard library modules from extracted files. * Add sys._path0 variable. * pymain_sys_path_add_path0() sets sys._path0. * Add runpy._remove_sys_path0(). * gzip, tarfile and zipfile main functions now call _remove_sys_path0() to prevent shadowing stdlib modules from extracted files. Co-authored-by: Gregory P. Smith <68491+gpshead@users.noreply.github.com> Co-authored-by: Ethan Furman <ethan@stoneleaf.us> Co-authored-by: Kirill Podoprigora <kirill.bast9@mail.ru> (cherry picked from commit e891a91)
1 parent 679b376 commit c04240b

14 files changed

Lines changed: 195 additions & 21 deletions

File tree

‎Doc/c-api/init_config.rst‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1216,8 +1216,8 @@ PyConfig
12161216
12171217
.. c:member:: int safe_path
12181218
1219-
If equals to zero, ``Py_RunMain()`` prepends a potentially unsafe path to
1220-
:data:`sys.path` at startup:
1219+
If equals to zero, ``Py_RunMain()`` prepends a potentially unsafe
1220+
:data:`sys._path0` to :data:`sys.path` at startup:
12211221
12221222
* If :c:member:`argv[0] <PyConfig.argv>` is equal to ``L"-m"``
12231223
(``python -m module``), prepend the current working directory.

‎Doc/library/sys.rst‎

Lines changed: 18 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1516,6 +1516,24 @@ always available. Unless explicitly noted otherwise, all variables are read-only
15161516
* Module :mod:`site` This describes how to use .pth files to
15171517
extend :data:`sys.path`.
15181518

1519+
.. data:: _path0
1520+
1521+
Path prepended to :data:`sys.path` at Python startup:
1522+
1523+
* Current directory if running a module
1524+
with ``python -m module`` command line.
1525+
* Script's directory if running a script
1526+
with ``python script.py``.
1527+
* Otherwise an empty string, which means the current working directory,
1528+
for ``python -c code`` or ``python`` (REPL) command line.
1529+
1530+
If :option:`-P` or :option:`-I` command line options are used or if
1531+
:envvar:`PYTHONSAFEPATH` environment variable is used, :data:`sys._path0` is
1532+
``None`` and :data:`sys.path` is left unchanged.
1533+
1534+
.. versionadded:: next
1535+
1536+
15191537
.. data:: path_hooks
15201538

15211539
A list of callables that take a path argument to try to create a

‎Doc/using/cmdline.rst‎

Lines changed: 11 additions & 13 deletions
Original file line numberDiff line numberDiff line change
@@ -313,10 +313,12 @@ Miscellaneous options
313313
Run Python in isolated mode. This also implies :option:`-E`, :option:`-P`
314314
and :option:`-s` options.
315315

316-
In isolated mode :data:`sys.path` contains neither the script's directory nor
317-
the user's site-packages directory. All ``PYTHON*`` environment
318-
variables are ignored, too. Further restrictions may be imposed to prevent
319-
the user from injecting malicious code.
316+
In isolated mode, Python does not prepend the potentially unsafe path
317+
:data:`sys._path0` to :data:`sys.path`, and ignores all ``PYTHON*``
318+
environment variables, e.g. :envvar:`PYTHONPATH` and :envvar:`PYTHONHOME`,
319+
that might be set.
320+
Further restrictions may be imposed to prevent the user from injecting
321+
malicious code.
320322

321323
.. versionadded:: 3.4
322324

@@ -344,14 +346,8 @@ Miscellaneous options
344346

345347
.. option:: -P
346348

347-
Don't prepend a potentially unsafe path to :data:`sys.path`:
348-
349-
* ``python -m module`` command line: Don't prepend the current working
350-
directory.
351-
* ``python script.py`` command line: Don't prepend the script's directory.
352-
If it's a symbolic link, resolve symbolic links.
353-
* ``python -c code`` and ``python`` (REPL) command lines: Don't prepend an
354-
empty string, which means the current working directory.
349+
Don't prepend the potentially unsafe path :data:`sys._path0` to
350+
:data:`sys.path`.
355351

356352
See also the :envvar:`PYTHONSAFEPATH` environment variable, and :option:`-E`
357353
and :option:`-I` (isolated) options.
@@ -784,7 +780,9 @@ conflict.
784780
.. envvar:: PYTHONSAFEPATH
785781

786782
If this is set to a non-empty string, don't prepend a potentially unsafe
787-
path to :data:`sys.path`: see the :option:`-P` option for details.
783+
path :data:`sys._path0` to :data:`sys.path`.
784+
785+
See also the :option:`-P` option.
788786

789787
.. versionadded:: 3.11
790788

‎Lib/gzip.py‎

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -695,6 +695,11 @@ def decompress(data):
695695

696696
def main():
697697
from argparse import ArgumentParser
698+
import runpy
699+
700+
# Prevent shadowing a stdlib module from the extracted file
701+
runpy._remove_sys_path0()
702+
698703
parser = ArgumentParser(description=
699704
"A simple command line interface for the gzip module: act like gzip, "
700705
"but do not delete the input file.",

‎Lib/runpy.py‎

Lines changed: 20 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -317,6 +317,26 @@ def run_path(path_name, init_globals=None, run_name=None):
317317
pass
318318

319319

320+
def _remove_sys_path0():
321+
"""
322+
Remove sys._path0 from sys.path. Do nothing if sys._path0 is
323+
sys._stdlib_dir or if it has already been removed. Similar to using -P
324+
(safe path) command line option.
325+
"""
326+
path0 = sys._path0
327+
if path0 is None:
328+
return
329+
330+
if path0 == sys._stdlib_dir:
331+
return
332+
333+
# Remove all path0 copies, not only the first one.
334+
# We are intentionally not trying to normalize & match alternate
335+
# representations of path0, this is only intended to remove copies
336+
# of the exact value the runtime itself may have added.
337+
sys.path[:] = [path for path in sys.path if path != path0]
338+
339+
320340
if __name__ == "__main__":
321341
# Run the module specified as the next command line argument
322342
if len(sys.argv) < 2:

‎Lib/tarfile.py‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -3136,6 +3136,10 @@ def is_tarfile(name):
31363136

31373137
def main():
31383138
import argparse
3139+
import runpy
3140+
3141+
# Prevent shadowing stdlib modules from extracted files
3142+
runpy._remove_sys_path0()
31393143

31403144
description = 'A simple command-line interface for tarfile module.'
31413145
parser = argparse.ArgumentParser(description=description, color=True)

‎Lib/test/test_runpy.py‎

Lines changed: 92 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -20,6 +20,8 @@
2020
requires_subprocess,
2121
verbose,
2222
)
23+
from test import support
24+
from test.support import os_helper
2325
from test.support.import_helper import forget, make_legacy_pyc, unload
2426
from test.support.os_helper import create_empty_file, temp_dir, FakePath
2527
from test.support.script_helper import make_script, make_zip_script
@@ -875,5 +877,95 @@ def test_pymain_run_module(self):
875877
self.assertSigInt(["-m", ham.stem], cwd=ham.parent)
876878

877879

880+
class MiscTests(unittest.TestCase):
881+
def test_remove_sys_path0(self):
882+
# Check that all path0 copies are removed
883+
PATH0 = '/path/zero'
884+
STDLIB_PATH = '/to/stdlib'
885+
OTHER_PATH = '/other/path'
886+
sys_path = [PATH0, STDLIB_PATH, PATH0, '', OTHER_PATH, PATH0]
887+
expected = [STDLIB_PATH, '', OTHER_PATH]
888+
with (support.swap_attr(sys, '_stdlib_dir', STDLIB_PATH),
889+
support.swap_attr(sys, '_path0', PATH0),
890+
support.swap_attr(sys, 'path', sys_path)):
891+
runpy._remove_sys_path0()
892+
self.assertEqual(sys.path, expected)
893+
894+
# Calling it twice does nothing
895+
runpy._remove_sys_path0()
896+
self.assertEqual(sys.path, expected)
897+
898+
# Do nothing if sys._path0 is sys._stdlib_dir
899+
sys_path = ['', STDLIB_PATH, OTHER_PATH]
900+
expected = sys_path.copy()
901+
with (support.swap_attr(sys, '_stdlib_dir', STDLIB_PATH),
902+
support.swap_attr(sys, '_path0', STDLIB_PATH),
903+
support.swap_attr(sys, 'path', sys_path)):
904+
runpy._remove_sys_path0()
905+
self.assertEqual(sys.path, expected)
906+
907+
# Do nothing if sys._path0 is None
908+
sys_path = ['', STDLIB_PATH, OTHER_PATH]
909+
expected = sys_path.copy()
910+
with (support.swap_attr(sys, '_stdlib_dir', STDLIB_PATH),
911+
support.swap_attr(sys, '_path0', None),
912+
support.swap_attr(sys, 'path', sys_path)):
913+
runpy._remove_sys_path0()
914+
self.assertEqual(sys.path, expected)
915+
916+
def run_python(self, *args, PYTHONPATH=None):
917+
cmd = [sys.executable, *args]
918+
env = support.make_clean_env()
919+
if PYTHONPATH:
920+
env['PYTHONPATH'] = PYTHONPATH
921+
proc = subprocess.run(cmd, text=True, stderr=subprocess.PIPE, env=env)
922+
self.assertEqual(proc.returncode, 0, proc.stderr)
923+
924+
@support.requires_subprocess()
925+
def test_remove_sys_path0_run_python(self):
926+
CODE = textwrap.dedent("""
927+
import runpy, sys
928+
929+
# Remove sys.path[0]
930+
orig_path = sys.path.copy()
931+
path0 = sys._path0
932+
runpy._remove_sys_path0()
933+
934+
if path0 is not None and path0 in sys.path:
935+
raise Exception(f"path0 {path0!r} found in sys.path {sys.path!r}")
936+
if not sys.flags.safe_path:
937+
if sys.path == orig_path:
938+
raise Exception("sys.path was not changed")
939+
else:
940+
if sys.path != orig_path:
941+
raise Exception("sys.path was changed")
942+
943+
# Calling it twice must do nothing
944+
clean_path = sys.path.copy()
945+
runpy._remove_sys_path0()
946+
if sys.path != clean_path:
947+
raise Exception("sys.path was changed")
948+
""")
949+
950+
# Test "python -c code"
951+
self.run_python('-c', CODE)
952+
self.run_python('-P', '-c', CODE)
953+
954+
with tempfile.TemporaryDirectory() as tmpdir:
955+
name = 'test_remove_sys_path0'
956+
filename = os.path.join(tmpdir, f'{name}.py')
957+
with open(filename, 'w') as fp:
958+
fp.write(CODE)
959+
960+
# Test "python script.py"
961+
self.run_python(filename)
962+
self.run_python('-P', filename)
963+
964+
with os_helper.change_cwd(tmpdir):
965+
# Test "python -m module"
966+
self.run_python('-m', name)
967+
self.run_python('-P', '-m', name, PYTHONPATH=tmpdir)
968+
969+
878970
if __name__ == "__main__":
879971
unittest.main()

‎Lib/test/test_sys.py‎

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1453,6 +1453,15 @@ class PythonType:
14531453
self.assertFalse(is_immortal(b'abc'.decode()))
14541454
self.assertFalse(is_immortal('abc'.encode()))
14551455

1456+
def test_path(self):
1457+
self.assertIsInstance(sys.path, list)
1458+
for path in sys.path:
1459+
self.assertIsInstance(path, str)
1460+
1461+
def test_path0(self):
1462+
self.assertTrue(sys._path0 is None or isinstance(sys._path0, str),
1463+
sys._path0)
1464+
14561465

14571466
@test.support.cpython_only
14581467
class UnraisableHookTest(unittest.TestCase):

‎Lib/test/test_zipfile/test_core.py‎

Lines changed: 17 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -53,6 +53,15 @@ def get_files(test):
5353
test.assertFalse(f.closed)
5454

5555

56+
@contextlib.contextmanager
57+
def save_restore_sys_path():
58+
old_path = sys.path.copy()
59+
try:
60+
yield
61+
finally:
62+
sys.path[:] = old_path
63+
64+
5665
class LazyImportTest(unittest.TestCase):
5766
@cpython_only
5867
def test_lazy_import(self):
@@ -3817,17 +3826,18 @@ def test_add_comment(self):
38173826
def test_cli_with_metadata_encoding(self):
38183827
errmsg = "Non-conforming encodings not supported with -c."
38193828
args = ["--metadata-encoding=shift_jis", "-c", "nonesuch", "nonesuch"]
3820-
with captured_stdout() as stdout:
3821-
with captured_stderr() as stderr:
3829+
with (captured_stdout() as stdout,
3830+
captured_stderr() as stderr,
3831+
save_restore_sys_path()):
38223832
self.assertRaises(SystemExit, zipfile.main, args)
38233833
self.assertEqual(stdout.getvalue(), "")
38243834
self.assertIn(errmsg, stderr.getvalue())
38253835

3826-
with captured_stdout() as stdout:
3836+
with captured_stdout() as stdout, save_restore_sys_path():
38273837
zipfile.main(["--metadata-encoding=shift_jis", "-t", TESTFN])
38283838
listing = stdout.getvalue()
38293839

3830-
with captured_stdout() as stdout:
3840+
with captured_stdout() as stdout, save_restore_sys_path():
38313841
zipfile.main(["--metadata-encoding=shift_jis", "-l", TESTFN])
38323842
listing = stdout.getvalue()
38333843
for name in self.file_names:
@@ -3846,7 +3856,9 @@ def test_cli_with_metadata_encoding_extract(self):
38463856
except UnicodeEncodeError:
38473857
self.skipTest(f'cannot encode file name {fn!a}')
38483858

3849-
zipfile.main(["--metadata-encoding=shift_jis", "-e", TESTFN, TESTFN2])
3859+
with save_restore_sys_path():
3860+
zipfile.main(["--metadata-encoding=shift_jis", "-e",
3861+
TESTFN, TESTFN2])
38503862
listing = os.listdir(TESTFN2)
38513863
for name in self.file_names:
38523864
self.assertIn(name, listing)

‎Lib/zipfile/__init__.py‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -2413,6 +2413,10 @@ def _compile(file, optimize=-1):
24132413

24142414
def main(args=None):
24152415
import argparse
2416+
import runpy
2417+
2418+
# Prevent shadowing stdlib modules from extracted files
2419+
runpy._remove_sys_path0()
24162420

24172421
description = 'A simple command-line interface for zipfile module.'
24182422
parser = argparse.ArgumentParser(description=description, color=True)

0 commit comments

Comments
 (0)