[shoumikhin]

Problem

The Linux CUDA wheel cannot be used with a CPU-only PyTorch. Four of its libraries record a load-time dependency (DT_NEEDED) on libtorch_cuda.so, one also on libc10_cuda.so:

• extension/training/pybindings/_training_lib
• extension/llm/runner/_llm_runner
• extension/llm/custom_ops/libcustom_ops_aot_lib.so
• kernels/quantized/libquantized_ops_aot_lib.so

None of them uses those libraries, yet with a CPU-only torch they fail to load:

ImportError: libtorch_cuda.so: cannot open shared object file: No such file or directory

It comes from linking the torch CMake target (or ${TORCH_LIBRARIES}). In a CUDA build of PyTorch that target adds libtorch_cuda.so wrapped in -Wl,--no-as-needed, so it is recorded even when unused, and --as-needed on our side cannot drop it.

Fix

Link torch_cpu_library instead of torch where these libraries are built. torch is torch_cpu_library plus torch_cuda_library, so the CPU half is linked exactly as before, including the --as-needed handling that keeps unused libraries such as MKL out of DT_NEEDED. With a CPU-only PyTorch the link is unchanged.

The CUDA wheel smoke test gets a new check that fails if any shipped library depends on libtorch_cuda.so or libc10_cuda.so.

Testing

Built the CUDA 13.0 wheel on Linux x86_64 before and after the change.

readelf -d on _llm_runner, before:

libexecutorch.so libtorch_python.so libtorch.so libc10.so libc10_cuda.so libtorch_cpu.so libtorch_cuda.so libcudart.so.13

after:

libexecutorch.so libtorch_python.so libcudart.so.13 libtorch_cpu.so libc10.so

The other three lose libtorch_cuda.so the same way.

• The new check fails on the old wheel and passes on the new one.
• With CUDA torch 2.14.0 on an A100, the CUDA wheel smoke test gives the same results before and after, apart from the new check. That includes running models through the CUDA delegate and building a C++ app against the wheel. Checks needing the full release setup (every GPU architecture, the Qualcomm backend) could not run locally.
• The Python tests for the training bindings, the LLM runner bindings, the custom ops and the quantized ops give the same results before and after.
• With a PyTorch that links MKL as shared libraries, linking plain torch_cpu would pull MKL into DT_NEEDED and the operator libraries then fail to load (undefined symbol: __kmpc_flush). torch_cpu_library keeps MKL out and they load.
• In a fresh environment with torch 2.14.0+cpu: before, the training bindings, the LLM runner and both operator libraries fail to load. After, import executorch works, all four modules import, and both operator libraries load.

[pytorch-bot]

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/24060

• 📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

⏳ No Failures, 2 Pending

As of commit e5d9535 with merge base d1f7f1f ():
💚 Looks good so far! There are no failures yet. 💚

This comment was automatically generated by Dr. CI and updates every 15 minutes.

[Copilot]

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

[Copilot]

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.