Skip to content

Forbid C++ exceptions in native code: AGENTS.md rule + qualityChecks enforcement (PROF-16668) - #1039

Draft
jbachorik wants to merge 7 commits into
mainfrom
drydock/jb-clever-mossy-beetle
Draft

jbachorik wants to merge 7 commits into
mainfrom
drydock/jb-clever-mossy-beetle

Conversation

@jbachorik

Copy link
Copy Markdown
Collaborator

What does this PR do?:

Forbids C++ exceptions (throw/try/catch) in ddprof-lib/src/main/cpp and
enforces the rule mechanically. Three changes:

  • CallTraceStorage::allocateTables() no longer uses exceptions: the three
    tables are allocated with new (std::nothrow) and the eager
    unordered_set pre-sizing (rehash) is dropped — std containers cannot
    pre-size without the throwing allocator contract. Bucket arrays now
    allocate on first rotation; per-rotation node allocation is unchanged.
  • utils/check-cpp-exceptions.py flags throw/try/catch and the
    throwing std helpers (rethrow_exception, throw_with_nested,
    __cxa_throw, ...) in native sources after blanking comments and string
    literals. Wired into :ddprof-lib:qualityChecks (CI source-invariant-checks
    job). The exception list starts empty; stale entries fail the check.
  • The rule lives in AGENTS.md ("C++ exceptions"): bugs are assert +
    abort(), expected failures are error codes, OOM that must be survivable
    uses nothrow/mmap allocation. The existing "contract violations → throw"
    bullet is scoped to the Java API.

Motivation:

PROF-16668. PR #943 introduced the first throw sites in native code
(std::logic_error on an out-of-contract tag), reachable from the JVMTI
FollowReferences callback on the VM thread; the review replaced them with
assert + abort. An uncaught exception escaping through a JVMTI callback
terminates the JVM — like abort, but opaque — and the unwinder conflicts
with the static-libgcc/-static-libstdc++ link (libraryPatcher_linux.cpp) and
the pthread-cancellation cleanup rules (j9WallClock.cpp). The rule makes the
policy explicit and mechanically checked so it does not regress.

Additional Notes:

  • Accepted behavior change: OOM while pre-sizing the processTraces
    collections (~33 KB of bucket arrays) used to fail acquire() cleanly via
    the caught std::bad_alloc; it now surfaces at the first processTraces()
    rotation, where bad_alloc was never caught (pre-existing hazard). In
    practice the pre-size only held until 2048 traces, after which the sets
    rehashed anyway, and clear() keeps the buckets at their high-water mark.
  • The removed pre-sizing also removes its fault-injection hook
    (failAllocationForTest); its test intent (OOM at start fails cleanly,
    nothing published) is covered by the mmap-failure path, and a new
    nullTableForTest hook covers the nothrow-new failure path plus the
    partial-failure cleanup (SIGABRT-pinned death tests per the PR refchain: reduce ancestor-fanout and batch allocation churn (PROF-16467) #943
    precedent).
  • Agentic review (Sphinx, standard effort, two passes). Full range: 0
    CRITICAL / 0 HIGH; 5 LOW + 1 INFO. Fixed: the checker docstring no longer
    claims "never silent misses" next to a bullet describing a silent miss
    (commit 69834100c). Dismissed with evidence: the raw-string regex ends at
    the first )delim", which is exactly C++ raw-string termination semantics.
    Accepted, no fix: _liveness_checkers.reserve(4) can throw on OOM — that is
    the documented convention (std containers terminate on OOM; never caught).
    Listed, out of scope (pre-existing design/tests): liveness checkers run
    under the _liveness_lock shared lock, so a slow checker stalls a
    concurrent recording start/stop; the unregisterLivenessChecker path has
    no dedicated test. One generalist reviewer worker settled without
    submitting; coverage shows 0 uncovered files. Fix range re-reviewed: 0
    findings.
  • Review: gate addition — reviewed per the pre-pr-review skill gate row
    (Sphinx over origin/main..HEAD, plus a re-review of the fix commit).
  • No -fno-exceptions build flag: the ticket's build plan is the rule + the
    check; a flag change would force further refactoring of std-container use
    and is out of scope.
  • Human review: recommended — the change adds an enforcement gate and changes
    the OOM-at-start behavior described above; both deserve an engineer's
    confirmation beyond the agentic review.

How to test the change?:

  • utils/tests/test_quality_check_scripts.sh — hermetic cases T10a–T10h cover
    keyword detection, comment/string/digit-separator blanking, tracked-mode
    source filtering, the empty exception list, exemption and stale-entry
    behavior (284 groups pass on Linux).
  • python3 utils/check-cpp-exceptions.py against the pre-refactor sources
    reports exactly the four removed sites; against this tree it is clean.
  • :ddprof-lib:gtestDebug (incl. the reworked CallTraceStorageAcquireTest
    death tests), :ddprof-lib:gtestAsan_test_callTraceStorage and
    :ddprof-lib:gtestTsan_test_callTraceStorage in the glibc aarch64
    container — all pass on the final candidate, LeakSanitizer clean on the
    partial-failure paths.
  • :ddprof-lib:qualityChecks passes on the final tree.
  • Agentic review marker: Sphinx over origin/main..HEAD + fix range, both
    clean of CRITICAL/HIGH findings.

For Datadog employees:

  • If this PR touches code that signs or publishes builds or packages, or handles
    credentials of any kind, I've requested a security review (run the dd:platform-security-review
    skill, or file a request via the PSEC review form).
    bewaire also runs automatically on every PR.
  • This PR doesn't touch any of that.
  • JIRA: [JIRA-XXXX]

Unsure? Have a question? Request a review!

The only throw/try/catch sites in ddprof-lib/src/main/cpp were here: a
try/catch(std::bad_alloc) guarding the unordered_set pre-size calls and a
UNIT_TEST throw in unusableTableForTest. C++ exceptions are now forbidden in
native code (follow-up commit adds the AGENTS.md rule and the qualityChecks
enforcement).

allocateTables() allocates the three tables with nothrow new and drops the
eager rehash() pre-sizing: std containers cannot pre-size without the
throwing allocator contract. Bucket arrays allocate on first rotation;
per-rotation node allocation is unchanged. Out-of-contract test-hook input
aborts instead of throwing, pinned with SIGABRT death tests; a new
nullTableForTest hook covers the nothrow-new failure path and the partial-
failure cleanup.
utils/check-cpp-exceptions.py flags throw/try/catch and the throwing std
helpers in ddprof-lib/src/main/cpp after blanking comments and string
literals; the exception list starts empty and stale entries fail the check.
Wired into :ddprof-lib:qualityChecks, documented in utils/README.md, hermetic
cases added to test_quality_check_scripts.sh.
New AGENTS.md rule: no throw/try/catch in ddprof-lib/src/main/cpp — bugs are
assert/abort, expected failures are error codes, survivable OOM uses nothrow/
mmap allocation; plain new and std containers still terminate on OOM. The
contract-violation bullet is scoped to the Java API, and the native AGENTS.md
carries the pointer invariant.
Exception-list entries become (path, keyword, occurrence) triples that exempt
exactly one occurrence; raw string bodies are parsed and blanked, character
literals match only the single-character form so digit separators cannot hide
a keyword, and the docstring states the blanking limits honestly. T10 grows
the mutation-sensitive cases (digit separators, raw strings, helper keywords,
occurrence semantics, environment error, file-mode stale skip) and the rule
wording in AGENTS.md and utils/README.md is scoped to the listed helpers with
the libraryPatcher_linux.cpp forced-unwind mechanism stated precisely. Stale
exception-safety comments in flightRecorder and the collection comments are
updated to the no-exception reality, and the test sentinel is renamed to
NO_TABLE_FOR_TEST.
@jbachorik jbachorik added the human_review:recommended Human review recommended — see PR description label Nov 22, 2026
@jbachorik

Copy link
Copy Markdown
Collaborator Author

@DataDog review

@datadog-datadog-prod-us1-2 datadog-datadog-prod-us1-2 Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Bits Code Review: FAIL

The new exemption tests can pass when exempted exception keywords are incorrectly reported because their negative assertions quote the wildcard. A focused Bash reproduction confirmed the missed matches.

Open Bits AI session

🤖 Bits Code Review · Commit 6983410

Comment on lines +995 to +1006
if [[ "$out" == *"bad.cpp:2:* (try)"* ]]; then
fail "T10g the listed occurrence is not exempt:"$'\n'"$out"
else
pass
fi
rc=0; out=$(run_exc_module exempt_second) || rc=$?
expect "T10g2 second try exempt, first still flagged" 1 "$rc" "$out" "$SRC/bad.cpp:2: C++ exceptions are forbidden in native code (try)"
if [[ "$out" == *"bad.cpp:3:* (try)"* ]]; then
fail "T10g2 the listed occurrence is not exempt:"$'\n'"$out"
else
pass
fi

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Unquote the wildcard in both exemption assertions

If the checker incorrectly reports an exempted try, both exemption tests still pass: Bash treats the wildcard inside the quoted diagnostic pattern literally. A focused Bash reproduction confirmed that neither negative assertion matches a checker-format diagnostic. The positive assertions only require the nonexempt occurrence, so broken exemption handling goes undetected. Unquote the internal wildcard in both assertions.

Suggested change
if [[ "$out" == *"bad.cpp:2:* (try)"* ]]; then
fail "T10g the listed occurrence is not exempt:"$'\n'"$out"
else
pass
fi
rc=0; out=$(run_exc_module exempt_second) || rc=$?
expect "T10g2 second try exempt, first still flagged" 1 "$rc" "$out" "$SRC/bad.cpp:2: C++ exceptions are forbidden in native code (try)"
if [[ "$out" == *"bad.cpp:3:* (try)"* ]]; then
fail "T10g2 the listed occurrence is not exempt:"$'\n'"$out"
else
pass
fi
if [[ "$out" == *"bad.cpp:2:"*" (try)"* ]]; then
fail "T10g the listed occurrence is not exempt:"$'\n'"$out"
else
pass
fi
rc=0; out=$(run_exc_module exempt_second) || rc=$?
expect "T10g2 second try exempt, first still flagged" 1 "$rc" "$out" "$SRC/bad.cpp:2: C++ exceptions are forbidden in native code (try)"
if [[ "$out" == *"bad.cpp:3:"*" (try)"* ]]; then
fail "T10g2 the listed occurrence is not exempt:"$'\n'"$out"
else
pass
fi

React: 👍 useful · 👎 wrong · 😕 unclear · 👀 right, but not acting on it

What each reaction means
  • 👍 Useful: correct, and you're fixing it or will.
  • 👎 Wrong: the finding is incorrect, for example the bug can't happen or the code was misread.
  • 😕 Unclear: you can't tell what it means or why it matters.
  • 👀 Right, not acting: correct, but intended, out of scope for this PR, a known risk, or too minor.

🤖 Bits Code Review · Open Bits AI session

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Verified: the quoted inner wildcard made both negative assertions no-ops — a checker that reported an exempted try still passed T10g/T10g2. Fixed in 5907474 with grep-based, line-bounded guards (the suggested unquoted glob would have matched across lines).

@dd-octo-sts

dd-octo-sts Bot commented Nov 22, 2026 •

Copy link
Copy Markdown
Contributor

CI Test Results

Run: #45069175053 | Commit: 1dec998 | Duration: 21m 19s (longest job)

✅ All 32 test jobs passed

Status Overview

JDK glibc-aarch64/debug glibc-amd64/debug musl-aarch64/debug musl-amd64/debug
8 - ✅ - -
8-ibm - ✅ - -
8-j9 ✅ ✅ - -
8-librca - - ✅ ✅
8-orcl - ✅ - -
11 - ✅ - -
11-j9 ✅ ✅ - -
11-librca - - ✅ ✅
17 ✅ ✅ - -
17-graal ✅ ✅ - -
17-j9 ✅ ✅ - -
17-librca - - ✅ ✅
21 ✅ ✅ - -
21-graal ✅ ✅ - -
21-librca - - ✅ ✅
25 ✅ ✅ - -
25-graal ✅ ✅ - -
25-librca - - ✅ ✅

Legend: ✅ passed | ❌ failed | ⚪ skipped | 🚫 cancelled

Summary: Total: 32 | Passed: 32 | Failed: 0


Updated: 2026-11-22 21:19:02 UTC

@jbachorik jbachorik added the dog-sitter Actively watched by babysit-pr: fixes pushed through the repo gates label Nov 22, 2026
@jbachorik

jbachorik commented Nov 22, 2026 •

Copy link
Copy Markdown
Collaborator Author

🐶 babysit-pr watched this PR — CI breakages and mechanical findings fixed automatically; design decisions brought to the author.

This run
  • 20:47 · started · CI on head 69834100c in progress
  • 20:5x · answered · Bits thread on test_quality_check_scripts.sh T10g/T10g2 guards — verified, escalated out of edit scope
  • 21:04 · pushed · 5907474c4 — operator session fixed the T10g/T10g2 no-op guards (grep-based, line-bounded)
  • 21:04 · Bits · @datadog review posted for 5907474c4 (draft PR) — bits-review/head PASS
  • 21:1x · CI red · test-matrix / linux-glibc-amd64 (8-orcl, debug) MallocArenaWarningTest — one rerun of run 45066127064, green
  • 21:4x · CI red · dd-gitlab integration-test [hotspot,8] / [openj9,8] — Fabric Egress Gateway truncated JDK download, 3/3 attempts, retried once each, still red — flagged for the author (infra, not fixable in this run)
  • 21:48 · stopped · blocked on the dd-gitlab infra failure and the codeowner gate

✅ stopped 21:48 UTC · bits-review/head PASS, one flake retried green; dd-gitlab [hotspot,8]/[openj9,8] red on a persistent Fabric egress download failure (infra — left for the author); codeowner approval on utils/* pending · 1 finding answered · 1 commit pushed by the operator session, none by the babysitter

The negative assertions quoted the inner wildcard, making both guards
no-ops: a checker that flagged an exempted occurrence would still pass
T10g/T10g2 (Bits finding on PR #1039). Replace them with line-bounded
grep guards; bash [[ == ]] globs span lines and over-match. Verified in
both directions: healthy run passes, an exemption-disabled checker
mutant fails the suite.
@dd-octo-sts

dd-octo-sts Bot commented Nov 22, 2026 •

Copy link
Copy Markdown
Contributor

❌ 38 passed, 0 failed, 2 without a result out of 40 expected configurations

Test Matrix

Platform JDK 8 JDK 11 JDK 17 JDK 21 JDK 25
glibc-x64-hotspot ❔ ✅ ✅ ✅ ✅
glibc-x64-openj9 ❔ ✅ ✅ ✅ ✅
glibc-arm64-hotspot ✅ ✅ ✅ ✅ ✅
glibc-arm64-openj9 ✅ ✅ ✅ ✅ ✅
musl-x64-hotspot ✅ ✅ ✅ ✅ ✅
musl-x64-openj9 ✅ ✅ ✅ ✅ ✅
musl-arm64-hotspot ✅ ✅ ✅ ✅ ✅
musl-arm64-openj9 ✅ ✅ ✅ ✅ ✅

No result produced

These configurations wrote no validation log, so their outcome is unknown and
they are counted as failures. The job usually timed out or failed during setup;
see its log in the pipeline.

  • glibc-x64-hotspot-jdk8
  • glibc-x64-openj9-jdk8

Links

@datadog-datadog-prod-us1-2

datadog-datadog-prod-us1-2 Bot commented Nov 22, 2026 •

Copy link
Copy Markdown

Pipelines

❌ Errors

Your PR has failed checks. Please review the issues below and take necessary action before merging.

🚦 4 Pipeline jobs failed

Bits review gate | publish (5907474)

View more details · View in GitHub Actions

DataDog/java-profiler | integration-test-x64-glibc: [hotspot, 8]

View more details · View in GitLab

DataDog/java-profiler | integration-test-x64-glibc: [openj9, 8]

View more details · View in GitLab

View all 4 failed jobs.

Useful? React with 👍 / 👎

This comment will be updated automatically if new data arrives.
🔗 Commit SHA: 5907474 | Docs | View more details | Give us feedback!

@jbachorik

Copy link
Copy Markdown
Collaborator Author

@DataDog review

@datadog-datadog-prod-us1-2 datadog-datadog-prod-us1-2 Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Bits Code Review: PASS

More details

The checker accepts the reviewed sources and detects the four removed exception sites. Moving bucket-allocation OOM failures to the first trace rotation is an explicitly accepted behavior change.

Was this helpful? React 👍 or 👎

Open Bits AI session

🤖 Bits Code Review · Commit 5907474

@jbachorik jbachorik removed the dog-sitter Actively watched by babysit-pr: fixes pushed through the repo gates label Nov 22, 2026

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

human_review:recommended Human review recommended — see PR description

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant