Skip to content

Send SIGXCPU and SIGKILL at the guest CPU limits - #475

Merged
jserv merged 2 commits into
sysprog21:mainfrom
xalestar:rlimit-cpu
Nov 5, 2026
Merged

jserv merged 2 commits into
sysprog21:mainfrom
xalestar:rlimit-cpu

Conversation

@xalestar

@xalestar xalestar commented Oct 26, 2026 •

Copy link
Copy Markdown
Collaborator

Intent: a guest RLIMIT_CPU limits the guest's CPU time as it does on Linux. This is follow-up 2 of #448, which stores the limit in the guest table and leaves it unenforced.

Failure fixed: on main, a guest that sets RLIMIT_CPU to 1 s and spins gets no signal and runs until the vCPU watchdog ends elfuse after 10 s with status 124. On Linux the guest gets SIGXCPU after 1 s of CPU time.

This PR follows check_process_timers (kernel/time/posix-cpu-timers.c). At the soft limit the process gets SIGXCPU and the soft limit rises by one second, so a process that catches or ignores SIGXCPU gets another one each CPU second. At the hard limit it gets SIGKILL; when soft equals hard, SIGKILL alone. A guest spinning at EL0 takes no exit where elfuse could check the limit, so rlimit_cpu_main samples the host process CPU time, the clock guest CLOCK_PROCESS_CPUTIME_ID reads, from a thread of its own. The thread starts on the first finite soft limit (boot seed, fork-child restore, or prlimit64) and sleeps for the remaining CPU time divided by the online CPU count, at least 10 ms. A limit change wakes it.

Review focus:

  • rlimit_lock is a leaf, so the sampler drops it before signal_queue_info, which takes sig_lock.
  • The soft limit is raised before SIGXCPU is queued, so a handler that calls getrlimit reads the new value, as on Linux.
  • Both signals are queued as SI_KERNEL with si_pid 0, matching SEND_SIG_PRIV in check_rlimit.
  • A fork child starts the sampler through sys_rlimit_cpu_start, called right after signal_set_state, since that call replaces the shared pending set and would drop a signal queued earlier.
  • elfuse ignores host SIGXCPU and lifts its own RLIMIT_CPU soft limit to the hard one, as eeada5b does for RLIMIT_FSIZE. XNU sends nothing at the hard limit, so the sampler sends SIGKILL at the lower of the guest and host hard limits, which keeps a guest with euid 0 from raising its way past the host limit.

Known gaps: a fork child is a new host process, so its CPU time starts with what its elfuse process spent before the guest resumes (a fresh launch measured 6 to 17 ms), where a Linux child starts at 0. The signal can land up to 10 ms after the limit, the sampler's shortest sleep. A fatal default signal taken by a worker thread ends only that thread (#476), so the sampler sends SIGKILL again every 10 ms, and a default SIGXCPU taken by a worker is not sent again. Under a finite host RLIMIT_CPU hard limit, elfuse stalls before the guest runs, on main as well (#474), so the host hard limit is checked only with a build that reads it from the environment. A CLONE_VM child without CLONE_THREAD runs in its parent's host process, so the two share one CPU clock and the parent can be signalled for the child's CPU time (#478).

Verified on 8134b88, rebased onto 3347188: make check-format and make check pass; make test-matrix-elfuse-aarch64 gives 317 passed, 0 failed, 10 skipped.

Comment thread src/syscall/sys.c Resolved
Comment thread tests/test-rlimit-cpu.c Resolved
Comment thread src/syscall/sys.c Resolved
Comment thread src/syscall/sys.c Resolved
Comment thread src/syscall/sys.c Resolved
Comment thread src/syscall/sys.c Resolved
Comment thread tests/test-rlimit-cpu.c Resolved
Comment thread src/syscall/sys.c Resolved
@xalestar
xalestar force-pushed the rlimit-cpu branch 3 times, most recently from 9822261 to b95b96a Compare October 28, 2026 16:05
Comment thread src/syscall/sys.c Resolved
Comment thread src/syscall/sys.c Resolved
Comment thread tests/test-rlimit-cpu.c Resolved
Comment thread tests/test-rlimit-cpu.c Resolved
Comment thread src/syscall/sys.c Resolved
Comment thread src/syscall/sys.c Resolved
Comment thread src/syscall/sys.c Resolved
Comment thread tests/test-rlimit-cpu.c Resolved
Comment thread src/syscall/sys.c Resolved
Comment thread src/syscall/sys.c Resolved
Comment thread tests/test-rlimit-cpu.c Resolved
Comment thread src/syscall/sys.c Resolved
Comment thread tests/test-rlimit-cpu.c Resolved
Comment thread tests/test-rlimit-cpu.c Resolved
Comment thread src/main.c Resolved
Comment thread src/syscall/sys.c Resolved
Comment thread tests/test-rlimit-cpu.c Resolved
Comment thread tests/test-rlimit-seed.sh Resolved
Comment thread src/syscall/sys.c Resolved
Comment thread src/syscall/sys.c Resolved

@cubic-dev-ai cubic-dev-ai Bot left a comment •

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All reported issues were addressed across 11 files

Tip: Get a Slack DM when cubic finishes reviewing your PRs. Connect Slack

View guided diff | Re-trigger cubic

Comment thread tests/test-rlimit-seed.sh Resolved
Comment thread src/main.c Resolved
Comment thread tests/test-rlimit-cpu.c Resolved
Comment thread tests/test-rlimit-cpu.c
sigaddset(&xcpu, SIGXCPU);
sigprocmask(SIG_BLOCK, &xcpu, NULL);
set_cpu_soft(0);
pid_t pid = fork();

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This case passes without exercising the fork-child start. signal_get_state copies the shared pending set into the snapshot and signal_set_state restores it, so the SIGXCPU the parent earned while blocked arrives already pending in the child. The child dies the moment it unblocks, and deleting the sys_rlimit_cpu_start call in fork_child_main leaves the test green.

Drain the pending signal in the parent before forking (sigtimedwait with a zero timeout), so the child inherits only the limit and has to be killed by its own sampler.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in 669ca51: the parent now spins until it takes SIGXCPU before it forks.

Comment thread src/runtime/forkipc.c
* are properly restored to the thread entry.
*/
signal_set_state(&sig);
if (!sys_rlimit_cpu_start()) {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This failure exit comes after the fork was committed: the parent has already returned the child pid, and proc_lifecycle_sync_self has run. If pthread_create fails here, the guest sees a successful fork whose child exits 1 with no proc_process_exit record, where Linux would have failed the fork with EAGAIN.

Consider creating the sampler thread disarmed before the commit (next to sys_rlimit_restore) and having sys_rlimit_cpu_start only arm it. Then a thread-creation failure takes one of the existing pre-commit exits, and this call can no longer fail.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is beyond this PR's subject, so I'll fix that in a follow-up: #479

Comment thread src/syscall/sys.c Resolved
A guest RLIMIT_CPU lives in the guest rlimit table and nothing acted on
it, so a guest that spins past its limit runs until the vCPU watchdog
fires after 10 s. Linux sends SIGXCPU at the soft limit and raises the
soft limit by one second, so a process that survives gets another
SIGXCPU each CPU second, and sends SIGKILL at the hard limit
(check_process_timers in kernel/time/posix-cpu-timers.c). Both are
sent as SEND_SIG_PRIV, so a handler sees SI_KERNEL and no sender.

A guest spinning at EL0 takes no exit where elfuse could check the
limit, so rlimit_cpu_main samples the host process CPU time from a
thread of its own. It starts on the first finite soft limit, from the
boot seed, prlimit64, or in a fork child after signal_set_state, since
that call replaces the shared pending set and would drop a signal the
sampler queued earlier. It sleeps for the remaining CPU time divided
by the online CPU count, from 10 ms up to an hour. A limit change wakes
it, so the cap only keeps a soft limit near saturation out of the host
timed wait. After a SIGXCPU it waits the 10 ms before checking again,
so a soft limit set far below the CPU time already used rises one step
per check, as it does per tick on Linux, not in one burst whose
signals merge into one.

Linux checks the limits on a tick that lands while the process runs,
so a process idle past a limit gets nothing until it runs again. The
sampler sends only when the process spent 1 ms of CPU outside the
sampler since its previous check, or since the limit last changed. An
idle elfuse spends far less than that per 10 ms check, about 0.002 ms
in nanosleep and 0.09 ms in sigtimedwait (sysprog21#480).

At the hard limit the sampler sends SIGKILL again every 10 ms rather
than once. A CLONE_THREAD worker that dequeues a fatal signal ends
alone instead of taking the thread group with it as do_group_exit
does, so a single send could leave the other threads running past the
limit with nothing left to enforce it.

When the sampler thread cannot be created the limit would go
unenforced, so prlimit64 restores the old slot and fails with EAGAIN,
and boot and the fork child fail rather than run the guest without its
limit.

A signal the sampler queues raises attention in the guest shim
globals, so launch_destroy_guest stops the sampler first and waits out
a send already in flight, before guest_destroy unmaps that memory.
The sampler queues with rlimit_lock dropped, so prlimit64 waits out
that send too before it changes RLIMIT_CPU, and a signal decided under
the old limit never arrives after the new one is in place.

tests/test-rlimit-cpu covers SIGXCPU at its default action, the repeated
SIGXCPU with the raised soft limit and its siginfo, a soft limit lowered
below the CPU time used catching up, SIGKILL at the hard limit and at
{0, 0}, a sleeping process that is not charged, a process sleeping past
its soft limit that earns no SIGXCPU, a fork child that dies of an
inherited soft limit, and a sleeping leader whose spinning worker
crosses the hard limit. That fork child's parent takes its own SIGXCPU
with sigtimedwait before the fork, so the child inherits the limit but
no pending signal, and only the child's sampler can end it. One step per
check is checked as a lower bound, soft limit 4 not visible within 1 ms
of the lowering, which a busy host can only make longer; a burst reaches
4 in about 0.05 ms, and Linux at HZ 1000 takes 2 ms. The parent kills a
child still alive after 10 s, so a limit that is never enforced fails
its case rather than the run; alarm in the child would not, because
elfuse checks a guest ITIMER_REAL only when the vCPU exits. The test
skips under an inherited hard limit below 4 s, since one child needs 3.5
s of CPU.
sys_rlimit_init copies a finite host RLIMIT_CPU into the guest rlimit
table, where rlimit_cpu_main enforces it, but the host soft limit stays
armed on elfuse itself. Past it the host kernel sends elfuse SIGXCPU,
whose default action terminates elfuse even when the guest caught
SIGXCPU or raised its own soft limit.

Ignore SIGXCPU next to SIGXFSZ, and lift the host soft limit to the
hard one, as sys_rlimit_init already does for RLIMIT_FSIZE, so the
guest table decides. XNU sends nothing at the hard limit: a host
process that ignores SIGXCPU spins to 4 CPU seconds under {1, 2} and
{2, 2} and exits 0. With the soft limit lifted, nothing on the host
stops elfuse, and a guest with euid 0 can raise its own hard limit
through prlimit64. rlimit_cpu_main therefore sends SIGKILL at the
lower of the guest hard limit and the host hard limit, and runs while
either the guest soft limit or the host hard limit is finite. The host
hard limit is read at boot and in a fork child, which inherits it.

Under ulimit -St 1 and under ulimit -St 1000 with SIGXCPU ignored in
the shell, test-hello stalled at boot until a 20 s timeout in 3 of 3
runs before this change and exits 0 in 3 of 3 after. A finite hard
limit (ulimit -Ht 100000) still stalls in 3 of 3 runs, see sysprog21#474, so
the host hard limit cannot be tested end to end until it is fixed. A
build that reads a host hard limit of 2 from the environment kills a
fakeroot guest that raised its limits to unlimited after 2 to 3 s
with status 137.

tests/test-rlimit-seed.sh spins a guest until times reports 1 s of
user time, after the guest raises its soft limit above a 1 s host soft
limit.
@jserv
jserv merged commit 987201d into sysprog21:main Nov 5, 2026
16 checks passed
@jserv

jserv commented Nov 5, 2026

Copy link
Copy Markdown
Contributor

Thank @xalestar for contributing!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants