Repository navigation
Send SIGXCPU and SIGKILL at the guest CPU limits - #475
Conversation
9822261 to
b95b96a
Compare
There was a problem hiding this comment.
All reported issues were addressed across 11 files
Tip: Get a Slack DM when cubic finishes reviewing your PRs. Connect Slack
View guided diff | Re-trigger cubic
| sigaddset(&xcpu, SIGXCPU); | ||
| sigprocmask(SIG_BLOCK, &xcpu, NULL); | ||
| set_cpu_soft(0); | ||
| pid_t pid = fork(); |
There was a problem hiding this comment.
This case passes without exercising the fork-child start. signal_get_state copies the shared pending set into the snapshot and signal_set_state restores it, so the SIGXCPU the parent earned while blocked arrives already pending in the child. The child dies the moment it unblocks, and deleting the sys_rlimit_cpu_start call in fork_child_main leaves the test green.
Drain the pending signal in the parent before forking (sigtimedwait with a zero timeout), so the child inherits only the limit and has to be killed by its own sampler.
There was a problem hiding this comment.
Fixed in 669ca51: the parent now spins until it takes SIGXCPU before it forks.
| * are properly restored to the thread entry. | ||
| */ | ||
| signal_set_state(&sig); | ||
| if (!sys_rlimit_cpu_start()) { |
There was a problem hiding this comment.
This failure exit comes after the fork was committed: the parent has already returned the child pid, and proc_lifecycle_sync_self has run. If pthread_create fails here, the guest sees a successful fork whose child exits 1 with no proc_process_exit record, where Linux would have failed the fork with EAGAIN.
Consider creating the sampler thread disarmed before the commit (next to sys_rlimit_restore) and having sys_rlimit_cpu_start only arm it. Then a thread-creation failure takes one of the existing pre-commit exits, and this call can no longer fail.
There was a problem hiding this comment.
This is beyond this PR's subject, so I'll fix that in a follow-up: #479
A guest RLIMIT_CPU lives in the guest rlimit table and nothing acted on it, so a guest that spins past its limit runs until the vCPU watchdog fires after 10 s. Linux sends SIGXCPU at the soft limit and raises the soft limit by one second, so a process that survives gets another SIGXCPU each CPU second, and sends SIGKILL at the hard limit (check_process_timers in kernel/time/posix-cpu-timers.c). Both are sent as SEND_SIG_PRIV, so a handler sees SI_KERNEL and no sender. A guest spinning at EL0 takes no exit where elfuse could check the limit, so rlimit_cpu_main samples the host process CPU time from a thread of its own. It starts on the first finite soft limit, from the boot seed, prlimit64, or in a fork child after signal_set_state, since that call replaces the shared pending set and would drop a signal the sampler queued earlier. It sleeps for the remaining CPU time divided by the online CPU count, from 10 ms up to an hour. A limit change wakes it, so the cap only keeps a soft limit near saturation out of the host timed wait. After a SIGXCPU it waits the 10 ms before checking again, so a soft limit set far below the CPU time already used rises one step per check, as it does per tick on Linux, not in one burst whose signals merge into one. Linux checks the limits on a tick that lands while the process runs, so a process idle past a limit gets nothing until it runs again. The sampler sends only when the process spent 1 ms of CPU outside the sampler since its previous check, or since the limit last changed. An idle elfuse spends far less than that per 10 ms check, about 0.002 ms in nanosleep and 0.09 ms in sigtimedwait (sysprog21#480). At the hard limit the sampler sends SIGKILL again every 10 ms rather than once. A CLONE_THREAD worker that dequeues a fatal signal ends alone instead of taking the thread group with it as do_group_exit does, so a single send could leave the other threads running past the limit with nothing left to enforce it. When the sampler thread cannot be created the limit would go unenforced, so prlimit64 restores the old slot and fails with EAGAIN, and boot and the fork child fail rather than run the guest without its limit. A signal the sampler queues raises attention in the guest shim globals, so launch_destroy_guest stops the sampler first and waits out a send already in flight, before guest_destroy unmaps that memory. The sampler queues with rlimit_lock dropped, so prlimit64 waits out that send too before it changes RLIMIT_CPU, and a signal decided under the old limit never arrives after the new one is in place. tests/test-rlimit-cpu covers SIGXCPU at its default action, the repeated SIGXCPU with the raised soft limit and its siginfo, a soft limit lowered below the CPU time used catching up, SIGKILL at the hard limit and at {0, 0}, a sleeping process that is not charged, a process sleeping past its soft limit that earns no SIGXCPU, a fork child that dies of an inherited soft limit, and a sleeping leader whose spinning worker crosses the hard limit. That fork child's parent takes its own SIGXCPU with sigtimedwait before the fork, so the child inherits the limit but no pending signal, and only the child's sampler can end it. One step per check is checked as a lower bound, soft limit 4 not visible within 1 ms of the lowering, which a busy host can only make longer; a burst reaches 4 in about 0.05 ms, and Linux at HZ 1000 takes 2 ms. The parent kills a child still alive after 10 s, so a limit that is never enforced fails its case rather than the run; alarm in the child would not, because elfuse checks a guest ITIMER_REAL only when the vCPU exits. The test skips under an inherited hard limit below 4 s, since one child needs 3.5 s of CPU.
sys_rlimit_init copies a finite host RLIMIT_CPU into the guest rlimit
table, where rlimit_cpu_main enforces it, but the host soft limit stays
armed on elfuse itself. Past it the host kernel sends elfuse SIGXCPU,
whose default action terminates elfuse even when the guest caught
SIGXCPU or raised its own soft limit.
Ignore SIGXCPU next to SIGXFSZ, and lift the host soft limit to the
hard one, as sys_rlimit_init already does for RLIMIT_FSIZE, so the
guest table decides. XNU sends nothing at the hard limit: a host
process that ignores SIGXCPU spins to 4 CPU seconds under {1, 2} and
{2, 2} and exits 0. With the soft limit lifted, nothing on the host
stops elfuse, and a guest with euid 0 can raise its own hard limit
through prlimit64. rlimit_cpu_main therefore sends SIGKILL at the
lower of the guest hard limit and the host hard limit, and runs while
either the guest soft limit or the host hard limit is finite. The host
hard limit is read at boot and in a fork child, which inherits it.
Under ulimit -St 1 and under ulimit -St 1000 with SIGXCPU ignored in
the shell, test-hello stalled at boot until a 20 s timeout in 3 of 3
runs before this change and exits 0 in 3 of 3 after. A finite hard
limit (ulimit -Ht 100000) still stalls in 3 of 3 runs, see sysprog21#474, so
the host hard limit cannot be tested end to end until it is fixed. A
build that reads a host hard limit of 2 from the environment kills a
fakeroot guest that raised its limits to unlimited after 2 to 3 s
with status 137.
tests/test-rlimit-seed.sh spins a guest until times reports 1 s of
user time, after the guest raises its soft limit above a 1 s host soft
limit.
|
Thank @xalestar for contributing! |
Intent: a guest
RLIMIT_CPUlimits the guest's CPU time as it does on Linux. This is follow-up 2 of #448, which stores the limit in the guest table and leaves it unenforced.Failure fixed: on main, a guest that sets
RLIMIT_CPUto 1 s and spins gets no signal and runs until the vCPU watchdog ends elfuse after 10 s with status 124. On Linux the guest getsSIGXCPUafter 1 s of CPU time.This PR follows
check_process_timers(kernel/time/posix-cpu-timers.c). At the soft limit the process getsSIGXCPUand the soft limit rises by one second, so a process that catches or ignoresSIGXCPUgets another one each CPU second. At the hard limit it getsSIGKILL; when soft equals hard,SIGKILLalone. A guest spinning at EL0 takes no exit where elfuse could check the limit, sorlimit_cpu_mainsamples the host process CPU time, the clock guestCLOCK_PROCESS_CPUTIME_IDreads, from a thread of its own. The thread starts on the first finite soft limit (boot seed, fork-child restore, orprlimit64) and sleeps for the remaining CPU time divided by the online CPU count, at least 10 ms. A limit change wakes it.Review focus:
rlimit_lockis a leaf, so the sampler drops it beforesignal_queue_info, which takessig_lock.SIGXCPUis queued, so a handler that callsgetrlimitreads the new value, as on Linux.SI_KERNELwithsi_pid0, matchingSEND_SIG_PRIVincheck_rlimit.sys_rlimit_cpu_start, called right aftersignal_set_state, since that call replaces the shared pending set and would drop a signal queued earlier.SIGXCPUand lifts its ownRLIMIT_CPUsoft limit to the hard one, aseeada5bdoes forRLIMIT_FSIZE. XNU sends nothing at the hard limit, so the sampler sendsSIGKILLat the lower of the guest and host hard limits, which keeps a guest with euid 0 from raising its way past the host limit.Known gaps: a fork child is a new host process, so its CPU time starts with what its elfuse process spent before the guest resumes (a fresh launch measured 6 to 17 ms), where a Linux child starts at 0. The signal can land up to 10 ms after the limit, the sampler's shortest sleep. A fatal default signal taken by a worker thread ends only that thread (#476), so the sampler sends
SIGKILLagain every 10 ms, and a defaultSIGXCPUtaken by a worker is not sent again. Under a finite hostRLIMIT_CPUhard limit, elfuse stalls before the guest runs, on main as well (#474), so the host hard limit is checked only with a build that reads it from the environment. ACLONE_VMchild withoutCLONE_THREADruns in its parent's host process, so the two share one CPU clock and the parent can be signalled for the child's CPU time (#478).Verified on
8134b88, rebased onto3347188:make check-formatandmake checkpass;make test-matrix-elfuse-aarch64gives 317 passed, 0 failed, 10 skipped.