Skip to content

fix(cpu): count only the CPUs the process can use (affinity and cgroup quota) - #1476

Open
manunicholasjacob wants to merge 1 commit into
mlco2:masterfrom
manunicholasjacob:feat/cpu-count-affinity-cgroup-quota
Open

manunicholasjacob wants to merge 1 commit into
mlco2:masterfrom
manunicholasjacob:feat/cpu-count-affinity-cgroup-quota

Conversation

@manunicholasjacob

@manunicholasjacob manunicholasjacob commented Nov 5, 2026 •

Copy link
Copy Markdown

Description

In codecarbon/core/util.py, count_cpus() currently returns psutil.cpu_count() in the non-SLURM case. This reports the total number of logical CPUs on the host, even when the process has access to only a subset of them.

There are two constraints that need to be considered:

  1. CPU affinity: restrictions imposed by taskset, Docker cpusets, or cpuset cgroups can limit which CPUs a process can use. On Linux, os.sched_getaffinity(0) provides the CPUs available to the current process.
  2. CPU quota: container limits such as Docker's --cpus or Kubernetes CPU limits can restrict CPU usage independently of affinity. These limits can also be fractional.

The CPU count is used when estimating CPU power consumption. If the CPU model is not available in CodeCarbon's registry, the estimate is calculated using the number of threads multiplied by DEFAULT_POWER_PER_CORE (4 W). For example, a container limited to two CPUs running on a 64-core host could be reported as using 256 W instead of approximately 8 W. This significantly overestimates power consumption and affects the resulting energy and CO2 calculations.

Related Issue

No existing issue or PR covers this (checked before opening).

Motivation and Context

Update count_cpus() to report the effective number of CPUs available to the process rather than the total host count:

  • Use os.sched_getaffinity(0) when available to determine the number of CPUs allowed by the affinity mask.
  • Read the cgroup CPU quota and use the lower of the quota and affinity counts.
  • Reuse the existing _read_cgroup_cpu_limit / _cgroup_cpu_limit helpers, which already support both cgroup v1 and v2 and return None when no CPU limit is configured.
  • Preserve the existing psutil.cpu_count() behaviour when neither affinity nor a CPU quota imposes a restriction.
  • Handle unsupported platforms and missing cgroup files gracefully so that the function does not fail when these mechanisms are unavailable.

CodeCarbon already does allocation-aware counting on SLURM (parsing NumCPUs=); this extends the same intent to the common container and cpuset case so the power model and metadata are not computed against cores the process cannot use.

How Has This Been Tested?

tests/test_core_util.py, run locally:

  • a 2-CPU affinity mask yields 2;
  • a 2.0-CPU cgroup quota caps an 8-CPU affinity to 2;
  • affinity unavailable falls back to psutil.cpu_count();
  • cgroup v2 cpu.max and v1 cfs_quota/cfs_period parsing (no root, no real cgroup);
  • existing SLURM tests unchanged and passing.

black, isort (--profile black) and flake8 (--config=.flake8) are clean on the two changed files.

Types of changes

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to change)

AI Usage Disclosure

  • 🟥 AI-vibecoded: You cannot explain the logic. Car analogy : the car drive by itself, you are outside it and just tell it where to go.
  • 🟠 AI-generated: Car analogy : the car drive by itself, you are inside and give instructions.
  • ⭐ AI-assisted. Car analogy : you drive the car, AI help you find your way.
  • ♻️ No AI used. Car analogy : you drive the car.

I wrote this implementation, reusing my own cgroup-limit technique from numexpr PR #584 (_read_cgroup_cpu_limit / _cgroup_cpu_limit and the sched_getaffinity preference). An AI assistant produced an initial draft, which I rewrote in my own style, and the test scaffolding, which I reviewed.

Checklist:

  • My code follows the code style of this project.
  • My change requires a change to the documentation.
  • I have updated the documentation accordingly.
  • I have read the docs/how-to/contributing.md document.
  • I have added tests to cover my changes.
  • All new and existing tests passed.

@manunicholasjacob
manunicholasjacob requested a review from a team as a code owner November 5, 2026 20:18
@github-actions github-actions Bot added the size/M label Nov 5, 2026
@SaboniAmine

Copy link
Copy Markdown
Member

Thanks for this contribution, which will unlock some use cases on Kubernetes which we couldn't adress, due to our lack of knowledge of the way cgroups work.
Would it be possible for you to run some benchmarks, to show us how it behaves live in this kind of deployments ? This would be a great learning opportunity for us.

Outside SLURM, count_cpus() returned psutil.cpu_count(), the machine's logical core count, ignoring a restricted CPU affinity mask (taskset, docker run --cpuset-cpus, a cpuset) and a cgroup CPU quota (docker run --cpus, a Kubernetes CPU limit). In a container pinned or limited to a few cores on a large host, and when the CPU model is absent from the power registry, the TDP fallback estimates threads * DEFAULT_POWER_PER_CORE against the host count, so the CPU power and the reported cpu_count come out far too high.

Honour the affinity mask where the OS exposes one (os.sched_getaffinity) and a cgroup v2 cpu.max / v1 cpu.cfs_quota_us quota, rounded up, falling back to psutil.cpu_count() off Linux. The SLURM path is unchanged.
@manunicholasjacob
manunicholasjacob force-pushed the feat/cpu-count-affinity-cgroup-quota branch from 2379a4f to 977df8b Compare November 5, 2026 20:42

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants