Skip to content

profile and cProfile cannot profile a directory or PEP-441 zip file  #100961

Description

@huonw

Bug report

Python can execute zip files (PEP-441) and directories directly, e.g. python code.zip or python directory, if they're constructed appropriately (such as via zipapp or https://github.com/pantsbuild/pex). These don't work with profile or cProfile modules, e.g. python -m profile code.zip fails with ValueError: source code string cannot contain null bytes and python -m cProfile directory fails with IsADirectoryError: [Errno 21] Is a directory: 'directory'.

For example: https://gist.github.com/huonw/52574abce4057afd99abb5e4a7f82b61 has a simple __main__.py and a zip file that contains it:

import sys
print(f"Hello from {sys.argv[0]}")

There's a script that runs all 9 combinations of zip vs. file vs. directory, and no profiling vs. profile vs cProfile. It can be executed via:

git clone https://gist.github.com/huonw/52574abce4057afd99abb5e4a7f82b61
cd 52574abce4057afd99abb5e4a7f82b61
./script.sh

Output of script.sh:

updating: __main__.py (stored 0%)
Archive:  code.zip
 extracting: directory/__main__.py   
Python 3.11.0


*** Running without profiling
Hello from __main__.py
Hello from code.zip
Hello from directory


*** Running with 'profile'
Hello from __main__.py
** BUG:
Traceback (most recent call last):
  File "<frozen runpy>", line 198, in _run_module_as_main
  File "<frozen runpy>", line 88, in _run_code
  File "/Users/huon/.pyenv/versions/3.11.0/lib/python3.11/profile.py", line 610, in <module>
    main()
  File "/Users/huon/.pyenv/versions/3.11.0/lib/python3.11/profile.py", line 591, in main
    code = compile(fp.read(), progname, 'exec')
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
ValueError: source code string cannot contain null bytes
Traceback (most recent call last):
  File "<frozen runpy>", line 198, in _run_module_as_main
  File "<frozen runpy>", line 88, in _run_code
  File "/Users/huon/.pyenv/versions/3.11.0/lib/python3.11/profile.py", line 610, in <module>
    main()
  File "/Users/huon/.pyenv/versions/3.11.0/lib/python3.11/profile.py", line 590, in main
    with open(progname, 'rb') as fp:
         ^^^^^^^^^^^^^^^^^^^^
IsADirectoryError: [Errno 21] Is a directory: 'directory'


*** Running with 'cProfile'
Hello from __main__.py
** BUG:
Traceback (most recent call last):
  File "<frozen runpy>", line 198, in _run_module_as_main
  File "<frozen runpy>", line 88, in _run_code
  File "/Users/huon/.pyenv/versions/3.11.0/lib/python3.11/cProfile.py", line 190, in <module>
    main()
  File "/Users/huon/.pyenv/versions/3.11.0/lib/python3.11/cProfile.py", line 171, in main
    code = compile(fp.read(), progname, 'exec')
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
ValueError: source code string cannot contain null bytes
Traceback (most recent call last):
  File "<frozen runpy>", line 198, in _run_module_as_main
  File "<frozen runpy>", line 88, in _run_code
  File "/Users/huon/.pyenv/versions/3.11.0/lib/python3.11/cProfile.py", line 190, in <module>
    main()
  File "/Users/huon/.pyenv/versions/3.11.0/lib/python3.11/cProfile.py", line 170, in main
    with open(progname, 'rb') as fp:
         ^^^^^^^^^^^^^^^^^^^^
IsADirectoryError: [Errno 21] Is a directory: 'directory'

Workarounds:

  • for a zip, manually unzip, and then run the __main__.py, e.g. unzip code.zip -d directory && python -m cProfile directory/__main__.py.
  • for a directory, run the __main__.py, e.g. python -m cPorfile directory/__main__.py

Your environment

  • CPython versions tested on: 3.9.10, 3.10.4, 3.11.0
  • Operating system and architecture: python -m platform == macOS-12.5.1-arm64-arm-64bit

Linked PRs

Activity

  1. added
    type-bugAn unexpected behavior, bug, or error
    on Jan 11, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    type-bugAn unexpected behavior, bug, or error

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions