Skip to content

feat(natsock): cover validated Netty epoll images and composed lifecycle (PROF-14931) - #730

Draft
kaahos wants to merge 36 commits into
paul.fournillon/taskblock-native-iofrom
paul.fournillon/native_socket_netty
Draft

kaahos wants to merge 36 commits into
paul.fournillon/taskblock-native-iofrom
paul.fournillon/native_socket_netty

Conversation

@kaahos

@kaahos kaahos commented Aug 10, 2026 •

Copy link
Copy Markdown
Contributor

What does this PR do?:

Extends the shared native I/O patches from #666 to Netty's native epoll transport library, including shaded copies outside the JDK directory.

  • Netty images are selected by the exact epoll JNI load/unload markers cached at symbol ingestion, not by file name. Discovery validates the mapped base and fingerprint and pins each image until restoration.
  • When the cached path cannot pin the image (deleted file, removed symlink), discovery pins by the loader-owned name, outside the patch lock, and validates the fingerprint again.
  • Adds optional epoll_pwait2 handling with the same errno-preserving scope as the other epoll waits.
  • ThreadFilter::refreshOwnerSlotForRecording() validates a retained native owner (TID, lifecycle generation, recording epoch) under the registry lock before it provisions or resets recording payload.

Motivation:

Netty epoll calls libc directly and does not go through the JDK networking imports, so #666 does not see its waits. Shading and deleted extraction files defeat path-based discovery. Patching by file name, or scanning arbitrary mapped symbols, could patch an unrelated library.

A Netty worker often blocks across a recording restart. Its retained owner must get the new recording's projection without clearing a different thread's native lifecycle.

Additional Notes:

  • Stack: builds on feat(taskblock): compose Linux native I/O producers with shared socket hooks (PROF-14931) #666.
  • Reviewer acknowledgment, epoll_pwait2 on older glibc: when libc has no epoll_pwait2 (glibc before 2.35), every epoll_pwait2 import stays untouched, including a non-NULL lazy-binding stub. A NULL weak import stays NULL, so Netty's availability probe keeps its meaning. Neither case disables the hooks or skips the image.
  • Reviewer acknowledgment, per-image skip: a failure local to one Netty image (failed pin, incomplete imports, GOT not writable, missing import slot) skips only that image and increments native_io_netty_images_skipped. Other libraries keep their patches and both owners stay active. Why: one unusual Netty build must not disable all socket hooks. The skip is keyed by base and fingerprint, so other cached descriptions of the same image are also skipped. The next installation (normally the next recording) retries it once.
  • Reviewer acknowledgment, bounded skip table: the table holds 64 images. When it is full, the next skip fails closed: both owners are disabled and every image is restored. The same applies to a missing libc original, an allocation failure, any JDK image failure, and a missing slot found after writes started.
  • Initial ingestion is not guaranteed when neither path can retain the image. Unrelated JNI libraries stay out of scope.
  • Newly covered Netty waits pay the inherited native-scope and timeout costs from feat(taskblock): compose Linux native I/O producers with shared socket hooks (PROF-14931) #666. Cold retained-owner refresh can allocate context sidecars and take the registry mutex; a failed allocation is retried later.
  • Test-only dependencies: io.netty:netty-transport-classes-epoll and netty-transport-native-epoll 4.1.137.Final (x86_64 and aarch_64) in testCommon. No production dependency. The real-Netty fixture qualifies Linux glibc only, because the published Netty native artifacts need glibc; musl keeps the profiler's own epoll, weak-import and lifecycle tests.
  • Test teardown retries only named transient stop outcomes within a fixed budget (AbstractProfilerStopRetryTest); terminal errors still fail the test.
  • CI evidence (PROF-14931): .github/scripts/prepare_reports.sh copies ddprof-test/build/netty-epoll (child JVM argv, logs, hs_err files and JUnit XML) into test-reports/, with a case in test_prepare_reports.sh.
  • Contracts: NativeIoHooks.
  • Human review: mandatory (label human_review:mandatory). Reason: patches GOT entries of third-party (Netty) DSOs outside the JDK, and changes failure handling from fail-closed to per-image skip.

How to test the change?:

Added coverage:

  • gtests: nettyEpollDiscovery_ut (NettyEpollDiscoveryTest, NettyEpollAliasDiscoveryTest for removed symlinks and deleted backing files, NettyEpollImageSkipTest for each skip reason, skip-table capacity, pin release, writable GOT, stale generation and missing libc epoll_pwait2), LibraryPatcherSocketStateTest, NativeSocketInterposerHookTest, NativeBlockScopeTest, nativeIoTimeout_ut, threadFilter_ut (ThreadRegistryTest retained-owner refresh).
  • Java: NettyEpollTaskBlockTest (positive-timeout waits on a verified EpollEventLoop worker, default select strategy, JFR fields and stacks), NettyEpollIsolationTest, NettyEpollSyscallAbiTest, ProfilerEpollFixtureTest, NettyEpollTaskBlockAssertionsTest, TimedNativeWorkerTest, NativeWaitTestSupportTest, AbstractProfilerStopRetryTest, TaskBlockRecorderCountAssertionsTest, JavaProfilerTaskBlockLightweightTest.
  • Script: .github/scripts/tests/test_prepare_reports.sh.

Run:

  • ./.claude/commands/build-and-summarize :ddprof-lib:gtestDebug_nettyEpollDiscovery_ut
  • ./.claude/commands/build-and-summarize :ddprof-test:testDebug -Ptests='*NettyEpoll*' on Linux glibc
  • bash .github/scripts/tests/test_prepare_reports.sh

For Datadog employees:

  • If this PR touches code that signs or publishes builds or packages, or handles credentials of any kind, I've requested a security review (run the dd:platform-security-review skill, or file a request via the PSEC review form). bewaire also runs automatically on every PR.
  • This PR doesn't touch any of that.
  • JIRA: PROF-14931

Unsure? Have a question? Request a review!

@dd-octo-sts

dd-octo-sts Bot commented Aug 10, 2026 •

Copy link
Copy Markdown
Contributor

CI Test Results

Run: #44817218192 | Commit: efed463 | Duration: 34m 3s (longest job)

❌ 7 of 32 test jobs failed

Status Overview

JDK glibc-aarch64/debug glibc-amd64/debug musl-aarch64/debug musl-amd64/debug
8 - ✅ - -
8-ibm - ❌ - -
8-j9 ✅ ✅ - -
8-librca - - ✅ ❌
8-orcl - ✅ - -
11 - ✅ - -
11-j9 ✅ ✅ - -
11-librca - - ✅ ❌
17 ✅ ✅ - -
17-graal ✅ ✅ - -
17-j9 ✅ ✅ - -
17-librca - - ✅ ❌
21 ✅ ✅ - -
21-graal ✅ ✅ - -
21-librca - - ✅ ❌
25 ✅ ✅ - -
25-graal ✅ ❌ - -
25-librca - - ✅ ❌

Legend: ✅ passed | ❌ failed | ⚪ skipped | 🚫 cancelled

⚠️ Flaky tests — failed, then passed on retry

Test Cells Message
NettyEpollIsolationTest.survivingOwnerAndNewWorkerAreDisjointAcrossRestart__ glibc-25-graal-debug-amd64 org.opentest4j.AssertionFailedError: isolated Netty test failed; retained log=/home/runner/work/java-profiler/java-profi
NettyEpollIsolationTest.defaultEventLoopWaitProducesTaskBlock__ glibc-8-ibm-debug-amd64 java.lang.AssertionError: isolated Netty JVM exceeded outer deadline; log=build/netty-epoll/defaultEventLoopWaitProduces

These fail the build. Passing on a second run makes a test flaky, not passing. Fix it, or quarantine it against a ticket so the debt is tracked rather than forgotten.

Consider quarantining these — click for ready-made entries

A quarantined test still runs and still reports; its failures just stop
turning CI red. To quarantine one:

  1. Open a PROF ticket for the test, linking the failing job.
  2. Append the line below to ddprof-test/quarantine.txt, replacing
    PROF-XXXXX with the ticket number.
  3. Check the cells and reason columns — the proposal only knows what
    failed in this run, and a narrower cells glob keeps the same test
    gating everywhere it has not misbehaved.

CI fails once review_by passes, so an entry expires instead of piling up.

# test | ticket | added | review_by | cells | reason
com.datadoghq.profiler.wallclock.NettyEpollIsolationTest.survivingOwnerAndNewWorkerAreDisjointAcrossRestart | PROF-XXXXX | 2026-11-20 | 2027-02-18 | glibc-25-graal-debug-amd64 | org.opentest4j.AssertionFailedError: isolated Netty test failed; retained log=/home/runner/work/java-profiler/java-profiler/ddprof-test/build/netty-epoll/survivingOwnerAndNewWorkerAreDisjointAcrossRes (seen in: glibc-25-graal-debug-amd64)
com.datadoghq.profiler.wallclock.NettyEpollIsolationTest.defaultEventLoopWaitProducesTaskBlock | PROF-XXXXX | 2026-11-20 | 2027-02-18 | glibc-8-ibm-debug-amd64 | java.lang.AssertionError: isolated Netty JVM exceeded outer deadline; log=build/netty-epoll/defaultEventLoopWaitProducesTaskBlock-4211798787592456365/child.log (seen in: glibc-8-ibm-debug-amd64)

Retried 2 of 32 cells.

Failed Jobs

Summary: Total: 32 | Passed: 25 | Failed: 7


Updated: 2026-11-20 21:42:50 UTC

@kaahos
kaahos force-pushed the paul.fournillon/native_socket_netty branch from 9de97d6 to ddd6385 Compare August 16, 2026 14:26
@datadog-official

This comment has been minimized.

@dd-octo-sts

dd-octo-sts Bot commented Aug 16, 2026 •

Copy link
Copy Markdown
Contributor

Scan-Build Report

User:runner@runnervm76f27
Working Directory:/home/runner/work/java-profiler/java-profiler/ddprof-lib/src/test/make
Command Line:make -j4 all
Clang Version:Ubuntu clang version 18.1.3 (1ubuntu1)
Date:Fri Aug 21 10:42:24 2026

Bug Summary

Bug TypeQuantityDisplay?
All Bugs1
Logic error
Dereference of null pointer1

Reports

Bug Group Bug Type ▾ File Function/Method Line Path Length
Logic errorDereference of null pointerprofiler.hfindLibraryByAddress63828

@kaahos
kaahos force-pushed the paul.fournillon/native_socket_netty branch from ddd6385 to cdf93b0 Compare August 16, 2026 15:00
@dd-octo-sts

dd-octo-sts Bot commented Aug 16, 2026 •

Copy link
Copy Markdown
Contributor

✅ All 40 integration tests passed

📊 Dashboard · 👷 Pipeline · 📦 c43caa27

@kaahos
kaahos force-pushed the paul.fournillon/native_socket_netty branch from cdf93b0 to 003c443 Compare August 16, 2026 18:24
@kaahos
kaahos force-pushed the paul.fournillon/native_socket_netty branch from 003c443 to 1347a00 Compare August 17, 2026 12:38
@kaahos
kaahos force-pushed the paul.fournillon/native_socket_netty branch from 1347a00 to 54c2392 Compare August 18, 2026 08:51
@dd-octo-sts

dd-octo-sts Bot commented Aug 18, 2026 •

Copy link
Copy Markdown
Contributor

Reliability & Chaos Results

✅ All reliability & chaos checks passed Pipeline: https://gitlab.ddbuild.io/DataDog/java-profiler/-/pipelines/154352147

@kaahos
kaahos force-pushed the paul.fournillon/native_socket_netty branch from 54c2392 to 0bd231e Compare August 21, 2026 10:40
@kaahos
kaahos marked this pull request as ready for review August 21, 2026 14:22
@kaahos
kaahos requested a review from a team as a code owner August 21, 2026 14:22

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 0bd231e185

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

@kaahos kaahos changed the title feat(natsock): cover validated Netty epoll images and composed lifecycle (PROF-16669) feat(natsock): cover validated Netty epoll images and composed lifecycle (PROF-14931) Nov 20, 2026
@kaahos kaahos added the human_review:mandatory Human review required — see PR description label Nov 20, 2026
@kaahos kaahos added the dog-sitter Actively watched by babysit-pr: fixes pushed through the repo gates label Nov 20, 2026
@kaahos

kaahos commented Nov 20, 2026 •

Copy link
Copy Markdown
Contributor Author

🐶 babysit-pr is watching this PR — CI breakages and mechanical findings are fixed and pushed automatically; design decisions are always brought to you.

This run
  • 13:29 · started · CI on head ab25d4c10 in progress
  • 13:33 · CI red · CI Run 44744251316 cancelled (superseded by run 44744736568 on the same head after the dog-sitter label event) — not a failure, waiting
  • 13:31 · answered · both chatgpt-codex-connector threads on libraryPatcher_linux.cpp (deleted Netty images, full symbol scan) — still fixed at head via hasNettyEpollMarkers()
  • 14:05 · CI red · test-linux-musl-amd64 (8/21/25-librca, debug): tests passed, job failed in Gradle cache post-step (hashFiles failed in container) — infra, will retry once when run 44744736568 finishes
  • 14:20 · CI red · glibc-17-graal-debug-amd64: NettyEpollIsolationTest.nativeSocketSamplerCoexistsWithTaskBlock (observed=0 timed intervals, record_failed=0; passed on in-job retry); glibc-17-debug-amd64: JavaProfilerTaskBlockApiTest.tooShortIntervalStillClearsLifecycle[20] (parent-PR test) — retrying once
  • 14:53 · retried · CI Run 44744736568 rerun --failed (one retry): glibc-amd64 (8-ibm, debug) hung 70+ min — gdb watchdog shows SIGVTALRM WallClockASGCT -> J9 AsyncGetCallTrace re-entering omrthread_monitor_exit on a mutex the interrupted frame already held (cancelled run to collect it); musl hashFiles post-step; Netty and parent-PR TaskBlock flakes
  • 14:56 · Bits · posted @DataDog review on draft head ab25d4c
  • 15:08 · Bits · review returned FAIL on ab25d4c with 2 P2 findings (datadog-official): ASan JVM args not forwarded to isolated child JVMs (NettyEpollIsolationTest, TimedNativeWorkerTest); shared recorder-failure credit across both workers in NettyEpollTaskBlockTest — both verified, not fixable inside this run's cap, handed to the session
  • 15:21 · stopped · base paul.fournillon/taskblock-native-io advanced 9383cbf → 38f29dd mid-run (session steer: stop, no rebase/merge). Commits pushed: none. Rerun of CI Run 44744736568 (attempt 2): glibc-amd64 (8-ibm, debug) fails deterministically in NativeIoErrnoTest and NativeIoCombinedErrnoTest .preservesErrnoDuringPreOriginalLoggingAndSampledRecording (both attempts; parent-PR tests); musl-amd64 debug cells again fail only in the Gradle cache hashFiles post-step; 3 cells still running. Open for the author: Bits P2 threads 4643925657 (ASan JVM args in isolated child launchers) and 4643925667 (shared recorder-failure credit across workers) — verified, unreplied, unfixed. Also flagged: J9 self-deadlock (SIGVTALRM ASGCT re-entering omrthread_monitor_exit) hung 8-ibm debug in attempt 1; intermittent NettyEpollIsolationTest.nativeSocketSamplerCoexistsWithTaskBlock observed=0. bits_triggered: ab25d4c. Replied: 3831037092, 3831037100.

✅ stopped 15:21 UTC · base advanced mid-run; CI red (IBM errno tests in parent layer, musl cache post-step); Bits FAIL with 2 P2 open

@kaahos

kaahos commented Nov 20, 2026

Copy link
Copy Markdown
Contributor Author

@DataDog review

@datadog-official datadog-official Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Bits Code Review: FAIL

Netty qualification can accept missing intervals because recorder-rejection credit is reused across workers. The isolated JVM launchers also omit the ASan heap-placement settings used by the parent tests.

Open Bits AI session

🤖 Bits Code Review · Commit ab25d4c

Path run = Files.createTempDirectory(directory, method + "-");
List<String> command = new ArrayList<>();
command.add(Paths.get(System.getProperty("java.home"), "bin", "java").toString());
command.add("-Xmx512m");

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Preserve ASan JVM settings in isolated child commands

On Linux ASan runs with JDK 12+, the child inherits LD_PRELOAD and ASAN_OPTIONS but omits the parent's HeapBaseMinAddress and CompressedClassSpaceSize settings. ConfigurationPresets.configureAsan() supplies these specifically to prevent JVM heap placement from colliding with ASan shadow memory on affected kernels. The isolated tests can therefore fail during JVM startup before exercising Netty. Preserve the required sanitizer JVM settings in both this launcher and TimedNativeWorkerTest.childCommand(), which has the same omission.

React: 👍 useful · 👎 wrong · 😕 unclear · 👀 right, but not acting on it

What each reaction means
  • 👍 Useful: correct, and you're fixing it or will.
  • 👎 Wrong: the finding is incorrect, for example the bug can't happen or the code was misread.
  • 😕 Unclear: you can't tell what it means or why it matters.
  • 👀 Right, not acting: correct, but intended, out of scope for this PR, a known risk, or too minor.

🤖 Bits Code Review · Open Bits AI session

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in 0965577: child JVMs now inherit the parent's -Xmx, -XX:HeapBaseMinAddress= and -XX:CompressedClassSpaceSize= through NettyEpollJvm.heapLayoutArgs() (last occurrence wins; -Xmx512m only when the parent sets none). Used by NettyEpollIsolationTest and TimedNativeWorkerTest.childCommand(); covered by NettyEpollJvmArgsTest and an ASan run of NettyEpollIsolationTest.

}
stopProfiler();
JfrEvents events = verifyEvents("datadog.TaskBlock", false);
assertEpollTaskBlock(events, name, "io.netty.", recordFailures() - failuresBefore);

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Avoid crediting one recorder rejection to both workers

Both workers' assertions receive the same recording-wide recorder-failure delta. Four ordinary events, four timed events, and one recorder rejection satisfy both five-interval assertions, although two missing intervals require explanation. This lets lost Netty intervals pass qualification. The shaded and restart cases repeat the pattern. Use separate measurement windows or consume a shared rejection budget across all workload assertions so each rejection explains at most one missing interval.

React: 👍 useful · 👎 wrong · 😕 unclear · 👀 right, but not acting on it

What each reaction means
  • 👍 Useful: correct, and you're fixing it or will.
  • 👎 Wrong: the finding is incorrect, for example the bug can't happen or the code was misread.
  • 😕 Unclear: you can't tell what it means or why it matters.
  • 👀 Right, not acting: correct, but intended, out of scope for this PR, a known risk, or too minor.

🤖 Bits Code Review · Open Bits AI session

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in 302370d: both workers now draw on one rejection budget. TaskBlockRecorderTestSupport.assertCompletedIntervals(recordFailures, worker(...)...) requires the summed shortfall to fit the single delta, and a surplus on one worker cannot offset another's shortfall. Applied to the default loop, shaded image, restart and survivor cases; covered by TaskBlockRecorderCountAssertionsTest.

@kaahos kaahos removed the dog-sitter Actively watched by babysit-pr: fixes pushed through the repo gates label Nov 20, 2026

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

human_review:mandatory Human review required — see PR description test:reliability

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant