Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
69 commits
Select commit Hold shift + click to select a range
da98672
feat: squash commit for PR 664 rebase
kaahos Aug 19, 2026
fd415df
fix
kaahos Aug 20, 2026
b7784bd
Arm OBJECT_WAIT in the generic blockEnter TaskBlock hook
jbachorik Aug 26, 2026
a683dd2
Address review feedback for taskblock JVM producers
jbachorik Sep 1, 2026
8e894a8
fix: preserve wall-clock lifecycle state in JVM TaskBlock producers
kaahos Oct 21, 2026
0bacae6
test: make monitor contention threshold coverage deterministic
kaahos Oct 21, 2026
d6afb4b
fix: fix api drift and classload failure in chaos tests
kaahos Oct 22, 2026
7ae01fa
fix: address openj9 vt recognition issue
kaahos Oct 22, 2026
f90f104
fix: address weak tests
kaahos Oct 22, 2026
bc1c724
build: retain deterministic qualification reports and exact container…
kaahos Nov 10, 2026
2c89187
fix: qualify JVM stack admission and producer restart boundaries
kaahos Nov 10, 2026
2f247ab
test: preserve complete exit-time TaskBlocks across live dumps
kaahos Nov 10, 2026
c818000
test: retain engine-specific unowned wall sample weight contracts
kaahos Nov 10, 2026
bca23c0
docs: link blocked-interval token to the current exit hook
kaahos Nov 10, 2026
fb4c3c1
chore: clarify Gradle wrapper status and cleanup handling
kaahos Nov 11, 2026
ee26917
Merge taskblock-core into taskblock-jvm-producers
kaahos Nov 12, 2026
f911b5b
test: retain explicit uninitialized agent rejection in chaos coverage
kaahos Nov 12, 2026
d56da1b
Merge core signal-isolation fix into taskblock-jvm-producers
kaahos Nov 12, 2026
a30884a
fix: recording delivery custody, hook allocation and wrapper cleanup
kaahos Nov 12, 2026
32c0436
test: make compressed decoder loop termination explicit
kaahos Nov 12, 2026
3792bf5
fix: gate native delivery fixture on the shared skip-native parser
kaahos Nov 12, 2026
5a08e25
Merge taskblock review fixes into JVM producers
kaahos Nov 13, 2026
27c4e8f
Fix inherited context in unowned wall-tail test setup
kaahos Nov 14, 2026
9600d63
Stop in terminal error state when recording output is quarantined
kaahos Nov 15, 2026
f6fc17b
Clear allocator shadow before unmapping trace chunks
kaahos Nov 15, 2026
6e197a9
Seed retained wall-tail context transitions deterministically
kaahos Nov 15, 2026
e4b514c
Qualify available real JVM producers and protect container Git metadata
kaahos Nov 15, 2026
c34cb86
test: calibrate blocking-hook allocation accounting
kaahos Nov 16, 2026
e9fca19
Merge cumulative TaskBlock recovery into JVM producers
kaahos Nov 16, 2026
02106e8
Retain epoch fixture descriptors across consuming chunk delivery
kaahos Nov 16, 2026
0dc582c
Seed controlled wall tails only in a sampled epoch window
kaahos Nov 16, 2026
0996536
Complete terminal output teardown after a failed wall-tail flush
kaahos Nov 16, 2026
fc7b73d
Merge crash bootstrap and recorder telemetry into JVM TaskBlock produ…
kaahos Nov 17, 2026
016c9a5
Merge upstream #663 test follow-ups into #665
kaahos Nov 17, 2026
573e802
Merge upstream exit-order repair into #665
kaahos Nov 18, 2026
6602c85
Merge wall dump epoch guard into #665
kaahos Nov 18, 2026
eb1ea5c
Mark unprovisionable musl/OpenJ9 combinations not applicable
kaahos Nov 18, 2026
fb24e92
test(ci): make the pipeline exclusion literal explicit
kaahos Nov 18, 2026
37cc00d
Merge #915 repairs into #665 with quarantine-aware capacity recovery
kaahos Nov 19, 2026
df1afff
fix(taskblock): fail closed on non-HotSpot virtual-thread detection
kaahos Nov 19, 2026
c47fac2
fix(taskblock): restrict real-VM classifier probes to debug builds
kaahos Nov 19, 2026
dc71bbd
fix(taskblock): attach ProfiledThread before JNI in monitor callbacks
kaahos Nov 19, 2026
2c332dd
chore(stack): merge #915 fixes into #665
kaahos Nov 19, 2026
aa51890
test(taskblock): skip JNI-classified producer tests on non-HotSpot JD…
kaahos Nov 19, 2026
7a7c5c7
test(taskblock): skip JNI-classified producer variants on non-HotSpot…
kaahos Nov 19, 2026
20c5a3b
chore(stack): merge main and #915 fixes into #665
kaahos Nov 20, 2026
ce3dfbf
test(taskblock): use renumbered TaskBlock type id in receiving-scope …
kaahos Nov 20, 2026
fe0b920
chore(stack): merge #915 fixes into #665
kaahos Nov 20, 2026
1a4c719
fix(jfr): close the intern gate and latch ERROR when quarantined outp…
kaahos Nov 20, 2026
4c9c109
docs(java): state that legacy getInstance overloads can throw on an o…
kaahos Nov 20, 2026
136caa6
test(wall): prove producer-flush stripe and no-recording retention br…
kaahos Nov 20, 2026
76312c6
test(taskblock): cover owned-run admission flush branches and nested …
kaahos Nov 20, 2026
42b41d5
test(thread): compare park and monitor block slots after reset
kaahos Nov 20, 2026
1022cb1
test(jfr): cover dump and start output failure branches
kaahos Nov 20, 2026
a1421b4
test(wall): resolve the precheck wall engine once per assertion
kaahos Nov 20, 2026
2fc2227
test(taskblock): scope the context-window park check to its blocker
kaahos Nov 20, 2026
f57b579
ci(test): run the profiler start dispatch test with the utils python …
kaahos Nov 20, 2026
e209228
test(wall): read the park token from the reset snapshot slot
kaahos Nov 20, 2026
25d80c6
chore(stack): merge #915 fixes into #665
kaahos Nov 20, 2026
ccdeb08
chore(stack): merge #915 fixes into #665
kaahos Nov 20, 2026
1b28228
test(taskblock): add lower-bound recorder credit and retry #665 paire…
kaahos Nov 20, 2026
5410c90
test(taskblock): scope lightweight TaskBlock count to the worker thread
kaahos Nov 20, 2026
aec1f67
test(hotspot): drop continuation metadata test duplicated by continua…
kaahos Nov 20, 2026
77afe25
test(taskblock): skip pre-existing monitor tests without platform-thr…
kaahos Nov 20, 2026
e1a6712
test(taskblock): gate monitor tests on engine support and scope deleg…
kaahos Nov 20, 2026
f5e5610
fix(jfr): certify completed dump delivery when continuing the recordi…
kaahos Nov 20, 2026
4b55c75
test(taskblock): gate park producer tests on wall-engine owner tracking
kaahos Nov 20, 2026
48f9757
chore(stack): merge #915 fixes into #665
kaahos Nov 20, 2026
fbc7953
chore(stack): merge #915 fixes into #665
kaahos Nov 20, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
47 changes: 4 additions & 43 deletions .claude/commands/build-and-summarize
Original file line number Diff line number Diff line change
@@ -1,45 +1,6 @@
#!/usr/bin/env bash
# Copyright 2026, Datadog, Inc.
# Runs Gradle with the full output captured to a timestamped log so an agent
# session never ingests a multi-thousand-line build log. Prints the status
# line; on failure, extracts the "What went wrong" blocks and failing-test
# lines. The log file remains the source of truth for anything rarer.
#
# Usage: build-and-summarize [gradle-task...] (default: build)
#
# No live tail and no summary artifacts: a harness tool result arrives
# atomically after the run anyway, so streaming filters only added a
# background process to manage, and the failures section below is everything
# an agent needs before deciding to grep the full log.
set -u

ARGS=("$@")
[ "${#ARGS[@]}" -eq 0 ] && ARGS=(build)

STAMP="$(date +%Y%m%d-%H%M%S)-$$"
LABEL="$(printf '%s' "${ARGS[0]}" | tr '/:' '__')"
mkdir -p build/logs || exit 1
LOG="build/logs/${STAMP}-${LABEL}.log"
(umask 077; set -C; : > "$LOG") || exit 1

# Build logs are qualification evidence, including failed runs. Retention is
# managed by the caller; never remove earlier evidence as a side effect of a build.

echo "▶ ./gradlew ${ARGS[*]} -i --console=plain (full log: $LOG)"
./gradlew "${ARGS[@]}" -i --console=plain >"$LOG" 2>&1
status=$?

if [ "$status" -eq 0 ]; then
echo "✔ Gradle completed. Full log at: $LOG"
else
echo "✖ Gradle failed with status $status. Full log at: $LOG"
echo
echo "=== What went wrong ==="
sed -n '/^\* What went wrong:/,/^\* Try:/p' "$LOG" | head -40
echo
echo "=== Failing tasks/tests ==="
grep -E '^> Task .+FAILED$' "$LOG" | head -10
grep -E '[0-9]+ tests completed(, [0-9]+ failed)?' "$LOG" | head -5
grep -E '[a-zA-Z0-9_.$]+ > [A-Za-z0-9_.$()]+ FAILED$' "$LOG" | head -20
fi
exit "$status"
# Capture Gradle and produce deterministic local reports; never invoke a coding agent.
set -euo pipefail
ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/../.." && pwd)"
exec python3 "$ROOT/utils/gradle_summary.py" "$@"
12 changes: 11 additions & 1 deletion .claude/commands/build-and-summarize.md
Original file line number Diff line number Diff line change
@@ -1,11 +1,21 @@
# build-and-summarize

<!-- Copyright 2026, Datadog, Inc -->

Execute the bash script `.claude/commands/build-and-summarize` with all provided arguments.

This script will:
- Run `./gradlew` with the specified arguments (defaults to 'build' if none provided)
- Capture full output to a timestamped log in `build/logs/`
- Show minimal live progress in the console
- Delegate to the `gradle-logs-analyst` agent for structured analysis
- Write deterministic local summaries to `build/reports/claude/gradle-summary.md` and
`build/reports/claude/gradle-summary.json`, including on Gradle failure
- Return the actual Gradle exit status and terminate its owned process group on interruption

No external coding runner or agent is invoked. Python 3 is required; `stdbuf` is not.
Absent test counts/timings stay unknown. Inspect retained XML/JFR separately for actual
nonzero case counts, failures and skips; a successful build alone does not establish coverage.

The wrapper's inert tests run with `python3 -m unittest utils/test_gradle_summary.py`.

Pass through all arguments exactly as provided by the user.
72 changes: 72 additions & 0 deletions .github/scripts/assert_vm_qualification.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,72 @@
#!/usr/bin/env python3
# Copyright 2026, Datadog, Inc.
"""Require real-VM methods to pass, distinguishing unavailable delegated JVM support."""
import argparse
from pathlib import Path
import sys
import xml.etree.ElementTree as ET

PREFIX = "com.datadoghq.profiler.wallclock."
DELEGATED_METHOD = (PREFIX + "HotSpotDelegatedProducerLifecycleTest",
"originalJvmStacksAndDirectTaskBlocksSurviveRestart")
EXTENSION_UNAVAILABLE = "HOTSPOT_REQUEST_STACK_TRACE_UNAVAILABLE"
EXPECTED_METHODS = {
"hotspot": {
"HotSpotProducerLifecycleTest": "hotspotRecordingAndJniLifecycleSurviveRestart",
"HotSpotDelegatedProducerLifecycleTest": "originalJvmStacksAndDirectTaskBlocksSurviveRestart",
"RealVmThreadClassifierTest": "liveJniNullPlatformAndVirtualClassification",
},
"openj9": {
"OpenJ9ProducerLifecycleTest": "genuineOpenJ9RecordingAndJniLifecycleSurviveRestart",
"OpenJ9CpuProducerLifecycleTest": "genuineOpenJ9CpuSamplesSurviveRestart",
"RealVmThreadClassifierTest": "liveJniNullPlatformAndVirtualClassification",
},
}


def verify(vm, results):
"""Check each intended method, not suite totals that can hide skipped qualifications."""
expected = {(PREFIX + cls, method) for cls, method in EXPECTED_METHODS[vm].items()}
observed = set()
deferred = False
for path in sorted(results.glob("TEST-*.xml")):
root = ET.parse(path).getroot()
for case in root.iter("testcase"):
key = (case.get("classname"), case.get("name", "").removesuffix("()"))
if key not in expected:
continue
failed = any(case.find(tag) is not None for tag in ("failure", "error"))
skipped = case.find("skipped") is not None
# Gradle writes empty <skipped/> elements; the test also retains its reason in stdout.
unavailable = any(EXTENSION_UNAVAILABLE in (out.text or "")
for out in root.iter("system-out"))
if not failed and skipped and key == DELEGATED_METHOD and unavailable:
deferred = True
elif failed or skipped:
raise ValueError("{}#{} did not pass without skips ({})".format(*key, path))
observed.add(key)
missing = expected - observed
if missing:
raise ValueError("Missing qualification methods: " + ", ".join(
"{}#{}".format(*key) for key in sorted(missing)))
return len(observed) - int(deferred), deferred


def main():
parser = argparse.ArgumentParser(description=__doc__)
parser.add_argument("--vm", required=True, choices=EXPECTED_METHODS)
parser.add_argument("--results", required=True, type=Path)
args = parser.parse_args()
try:
count, deferred = verify(args.vm, args.results)
except (OSError, ET.ParseError, ValueError) as error:
print("Real-VM qualification failed: " + str(error), file=sys.stderr)
return 1
print("Real-VM qualification: {} intended {} methods passed without skips".format(count, args.vm))
if deferred:
print("Delegated JVM-stack qualification deferred: " + EXTENSION_UNAVAILABLE)
return 0


if __name__ == "__main__":
sys.exit(main())
160 changes: 160 additions & 0 deletions .github/scripts/tests/test_assert_vm_qualification.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,160 @@
#!/usr/bin/env python3
# Copyright 2026, Datadog, Inc.
"""Inert XML fixtures for the real-VM CI execution gate."""
import importlib.util
from pathlib import Path
import subprocess
import sys
import tempfile
import unittest
import xml.etree.ElementTree as ET

SCRIPT = Path(__file__).resolve().parents[1] / "assert_vm_qualification.py"
ROOT = SCRIPT.parents[2]
spec = importlib.util.spec_from_file_location("assert_vm_qualification", SCRIPT)
qualification = importlib.util.module_from_spec(spec)
spec.loader.exec_module(qualification)


class AssertVmQualificationTest(unittest.TestCase):
def setUp(self):
self.temp = tempfile.TemporaryDirectory()
self.addCleanup(self.temp.cleanup)
self.results = Path(self.temp.name)

def write_results(self, vm):
suite = ET.Element("testsuite")
cases = []
for cls, method in qualification.EXPECTED_METHODS[vm].items():
cases.append(ET.SubElement(suite, "testcase", {
"classname": qualification.PREFIX + cls, "name": method + "()"}))
return suite, cases

def save(self, suite):
ET.ElementTree(suite).write(self.results / "TEST-qualification.xml")

def invoke(self, vm, success):
result = subprocess.run(
[sys.executable, str(SCRIPT), "--vm", vm, "--results", str(self.results)],
capture_output=True, text=True)
self.assertEqual(result.returncode, 0 if success else 1, result.stdout + result.stderr)
return result

def test_every_vm_requires_its_actual_source_methods(self):
for vm in qualification.EXPECTED_METHODS:
with self.subTest(vm=vm):
suite, cases = self.write_results(vm)
self.save(suite)
result = self.invoke(vm, True)
self.assertIn("3 intended " + vm + " methods passed without skips", result.stdout)
for cls, method in qualification.EXPECTED_METHODS[vm].items():
source = (ROOT / "ddprof-test/src/test/java/com/datadoghq/profiler/wallclock" /
(cls + ".java")).read_text()
self.assertIn("public void " + method + "()", source)

def test_each_missing_method_fails_despite_other_successes(self):
for vm in qualification.EXPECTED_METHODS:
for missing in range(len(qualification.EXPECTED_METHODS[vm])):
with self.subTest(vm=vm, missing=missing):
suite, cases = self.write_results(vm)
suite.remove(cases[missing])
self.save(suite)
result = self.invoke(vm, False)
self.assertIn("Missing qualification methods", result.stderr)
self.assertIn(cases[missing].get("name")[:-2], result.stderr)

def test_each_skipped_failed_or_errored_method_fails(self):
for vm in qualification.EXPECTED_METHODS:
for rejected in range(len(qualification.EXPECTED_METHODS[vm])):
for tag in ("skipped", "failure", "error"):
with self.subTest(vm=vm, rejected=rejected, tag=tag):
suite, cases = self.write_results(vm)
ET.SubElement(cases[rejected], tag)
self.save(suite)
self.assertIn("did not pass without skips", self.invoke(vm, False).stderr)

def test_zero_cases_missing_results_and_unrelated_cases_fail(self):
self.assertIn("Missing qualification methods", self.invoke("hotspot", False).stderr)
suite = ET.Element("testsuite", tests="100", skipped="0")
ET.SubElement(suite, "testcase", classname="UnrelatedTest", name="passes()")
self.save(suite)
self.assertIn("Missing qualification methods", self.invoke("openj9", False).stderr)

def test_malformed_or_unreadable_xml_fails(self):
path = self.results / "TEST-qualification.xml"
path.write_text("<broken")
self.assertIn("Real-VM qualification failed", self.invoke("hotspot", False).stderr)
path.unlink()
path.mkdir()
self.assertIn("Real-VM qualification failed", self.invoke("hotspot", False).stderr)

def test_failure_is_not_hidden_by_duplicate_success(self):
suite, cases = self.write_results("hotspot")
failed = ET.SubElement(suite, "testcase", cases[0].attrib)
ET.SubElement(failed, "failure")
self.save(suite)
self.invoke("hotspot", False)

def test_unavailable_delegated_extension_is_not_counted_as_a_pass(self):
suite, cases = self.write_results("hotspot")
delegated = next(case for case in cases if case.get("classname") == qualification.DELEGATED_METHOD[0])
ET.SubElement(delegated, "skipped")
ET.SubElement(suite, "system-out").text = qualification.EXTENSION_UNAVAILABLE
self.save(suite)
result = self.invoke("hotspot", True)
self.assertIn("2 intended hotspot methods passed without skips", result.stdout)
self.assertIn("Delegated JVM-stack qualification deferred", result.stdout)
self.assertNotIn("3 intended", result.stdout)
source = (ROOT / "ddprof-test/src/test/java/com/datadoghq/profiler/wallclock" /
"HotSpotDelegatedProducerLifecycleTest.java").read_text()
self.assertIn(qualification.EXTENSION_UNAVAILABLE, source)
self.assertIn("System.out.println(EXTENSION_UNAVAILABLE)", source)

def test_unavailable_marker_never_excuses_failures_or_errors(self):
for vm in qualification.EXPECTED_METHODS:
for rejected in range(len(qualification.EXPECTED_METHODS[vm])):
for tag in ("failure", "error"):
with self.subTest(vm=vm, rejected=rejected, tag=tag):
suite, cases = self.write_results(vm)
ET.SubElement(cases[rejected], "skipped")
ET.SubElement(cases[rejected], tag)
ET.SubElement(suite, "system-out").text = qualification.EXTENSION_UNAVAILABLE
self.save(suite)
self.invoke(vm, False)

def test_unavailable_marker_never_excuses_other_skips(self):
for vm in qualification.EXPECTED_METHODS:
for rejected in range(len(qualification.EXPECTED_METHODS[vm])):
suite, cases = self.write_results(vm)
if cases[rejected].get("classname") == qualification.DELEGATED_METHOD[0]:
continue
with self.subTest(vm=vm, rejected=rejected):
ET.SubElement(cases[rejected], "skipped")
ET.SubElement(suite, "system-out").text = qualification.EXTENSION_UNAVAILABLE
self.save(suite)
self.invoke(vm, False)

def test_unavailable_reason_must_be_in_the_delegated_result_file(self):
suite, cases = self.write_results("hotspot")
delegated = next(case for case in cases if case.get("classname") == qualification.DELEGATED_METHOD[0])
ET.SubElement(delegated, "skipped")
self.save(suite)
unrelated = ET.Element("testsuite")
ET.SubElement(unrelated, "system-out").text = qualification.EXTENSION_UNAVAILABLE
ET.ElementTree(unrelated).write(self.results / "TEST-unrelated.xml")
self.invoke("hotspot", False)

def test_workflow_enables_and_checks_explicit_applicable_cells(self):
workflow = (ROOT / ".github/workflows/test_workflow.yml").read_text()
job = workflow.split(" qualify-real-vm:\n", 1)[1]
for text in ("expected_vm: hotspot", 'java_version: "25"', "expected_vm: openj9",
'java_version: "17-j9"', "DDPROF_EXPECTED_VM: ${{ matrix.expected_vm }}",
"--tests '*ProducerLifecycleTest'", "--tests '*RealVmThreadClassifierTest'",
'assert_vm_qualification.py --vm "$DDPROF_EXPECTED_VM"',
"--results ddprof-test/build/test-results/testDebug", "--rerun-tasks",
"if: always()", "ddprof-test/build/test-results/"):
self.assertIn(text, job)


if __name__ == "__main__":
unittest.main()
53 changes: 53 additions & 0 deletions .github/workflows/test_workflow.yml
Original file line number Diff line number Diff line change
Expand Up @@ -790,3 +790,56 @@ jobs:
name: asan-logs-musl-${{ matrix.java_version }}-aarch64-${{ inputs.slow_tests && 'slow' || 'regular' }}
path: /tmp/asan.log.*
if-no-files-found: ignore

qualify-real-vm:
needs: cache-jdks
if: ${{ !inputs.slow_tests && contains(fromJson(inputs.configuration), 'debug') }}
name: qualify-real-vm (${{ matrix.expected_vm }}, ${{ matrix.java_version }})
runs-on: ubuntu-latest
timeout-minutes: 60
strategy:
fail-fast: false
matrix:
include:
- expected_vm: hotspot
java_version: "25"
- expected_vm: openj9
java_version: "17-j9"
env:
DDPROF_EXPECTED_VM: ${{ matrix.expected_vm }}
LIBC: glibc
SANITIZER: debug
KEEP_JFRS: "true"
steps:
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
with:
fetch-depth: 0
persist-credentials: false
- name: Setup cached qualification JDK
uses: ./.github/actions/setup_cached_java
with:
version: ${{ matrix.java_version }}
arch: amd64
- name: Setup OS
run: |
sudo apt-get update
sudo apt-get install -y libgtest-dev libgmock-dev binutils
- name: Qualify real VM lifecycle and classification
run: |
"$JAVA_TEST_HOME/bin/java" -version
./.claude/commands/build-and-summarize :ddprof-test:testDebug -Pskip-gtest -PkeepJFRs \
--tests '*ProducerLifecycleTest' --tests '*RealVmThreadClassifierTest' \
--rerun-tasks --no-daemon --max-workers=1
python3 .github/scripts/assert_vm_qualification.py --vm "$DDPROF_EXPECTED_VM" \
--results ddprof-test/build/test-results/testDebug
- name: Retain qualification evidence
if: always()
uses: actions/upload-artifact@cf430e030ddbb5b0abf93d22962f4752f3646cd9 # v7.0.2
with:
name: real-vm-qualification-${{ matrix.expected_vm }}-${{ matrix.java_version }}
path: |
build/logs/
build/reports/claude/
ddprof-test/build/test-results/
ddprof-test/build/reports/tests/
/tmp/recordings/
Loading
Loading