Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
44 commits
Select commit Hold shift + click to select a range
18a212a
threadFilter: non-blocking TLS-destructor retirement and index lifeti…
jbachorik Oct 23, 2026
d02e008
wallClock: per-tick recording epoch, join-failure handling, block-tra…
jbachorik Oct 23, 2026
5258186
profiler: clear _activated_mask at start so a failed start cannot lea…
jbachorik Oct 23, 2026
ae54777
javaApi: move test seams onto JavaProfiler, DEBUG-gated; relocate Jav…
jbachorik Oct 23, 2026
3f77b78
tests: disarm toggles in teardown, null-safe counter reads, bounded c…
jbachorik Oct 23, 2026
30c3ef3
chaos: drain the active worker batch on shutdown instead of abandonin…
jbachorik Nov 2, 2026
02a1576
build: carry upstream formatter fix for stacked wall-clock PR
kaahos Nov 11, 2026
c78cbf6
Fix reusable unowned wall-clock roots and timer cadence
kaahos Nov 11, 2026
c0b1fc1
Merge refreshed wall-clock foundation into review fixes
kaahos Nov 11, 2026
cb7f255
Fix build log discovery and review gate fallbacks
kaahos Nov 12, 2026
70b60f9
Share the chaos profiler API through bootstrap
kaahos Nov 12, 2026
199d3be
Merge #663 deep fix into #890 with retained epoch provenance
kaahos Nov 12, 2026
e938bab
Wait for actual reservoir coverage and diagnose missing threads
kaahos Nov 12, 2026
e5c07be
Merge #663 comment fixes into #890 retained wall sampling
kaahos Nov 13, 2026
3bab2a7
Preserve originating wall-tail epoch metadata across dumps
kaahos Nov 15, 2026
4112b2e
Preserve wall-clock epoch metadata across dump and timer shutdown
kaahos Nov 15, 2026
9f25e1d
Merge wall-clock foundation fixes into retained-epoch follow-ups
kaahos Nov 16, 2026
2335980
Merge crash-handler bootstrap fix into wall-clock review fixes
kaahos Nov 17, 2026
4cb19a3
Merge #663 test follow-ups into #890
kaahos Nov 17, 2026
8b8a4c8
Merge upstream #663 exit-order repair into #890
kaahos Nov 17, 2026
2fc80fd
Mark unprovisionable musl/OpenJ9 combinations not applicable
kaahos Nov 18, 2026
79bf589
test(ci): make the pipeline exclusion literal explicit
kaahos Nov 18, 2026
4fa7f9e
Merge #663 native-memory and JVM installer repairs into #890
kaahos Nov 19, 2026
7a5f0ae
chore(stack): merge #663 fixes into #890
kaahos Nov 19, 2026
386215b
style(wall): drop review identifiers from timer-thread comments
kaahos Nov 19, 2026
862b476
chore(stack): merge main and #663 fixes into #890
kaahos Nov 19, 2026
ae148c4
chore(stack): merge #663 review fixes into #890
kaahos Nov 19, 2026
30ad6f4
fix(jfr): prune closed wall epochs without retained segments between …
kaahos Nov 19, 2026
d08e22c
fix(wall): close one sampling epoch per statistics change
kaahos Nov 19, 2026
db7962e
refactor(wall): derive unowned context cell word counts from DD_TAGS_…
kaahos Nov 19, 2026
83db661
docs(jfr): move stack-pool header comment back above writeStackTraceH…
kaahos Nov 19, 2026
36b67ad
test(wall): require a retained tail write after rotation dumps
kaahos Nov 19, 2026
0910dd2
test(wall): cover stale-owner exit and rootless epoch-start guards
kaahos Nov 19, 2026
8da846c
fix(wall): allocate unowned context cells for retained slots at filte…
kaahos Nov 19, 2026
8bdcfc7
chore(stack): merge #663 accounting fixes into #890
kaahos Nov 19, 2026
07b4d53
fix(wall): keep opening-pass suppressions from closing the new epoch
kaahos Nov 20, 2026
da34ffe
perf(jfr): prune closed wall epochs rarely behind a per-stripe barrier
kaahos Nov 20, 2026
8636481
fix(wall): count context allocation failures and skip chunks without …
kaahos Nov 20, 2026
df52625
test(wall): trigger the retained tail write while the worker stays bl…
kaahos Nov 20, 2026
3476389
chore(stack): merge #663 CI test fixes into #890
kaahos Nov 20, 2026
37fe822
fix(build): define Profiler::getLockIndex in the header so gtests lin…
kaahos Nov 20, 2026
20ea66d
fix(wall): index unowned context cells with an unsigned chunk offset
kaahos Nov 20, 2026
33c481f
fix(wall): accumulate suppressions into the open sampling epoch
kaahos Nov 20, 2026
048d8e5
chore(stack): merge #663 fixes into #890
kaahos Nov 20, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 6 additions & 5 deletions .claude/commands/build-and-summarize
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
#!/usr/bin/env bash
# Copyright 2026, Datadog, Inc.
# Runs Gradle with the full output captured to a timestamped log so an agent
# session never ingests a multi-thousand-line build log. Prints the status
# line; on failure, extracts the "What went wrong" blocks and failing-test
Expand All @@ -15,14 +16,14 @@ set -u
ARGS=("$@")
[ "${#ARGS[@]}" -eq 0 ] && ARGS=(build)

STAMP="$(date +%Y%m%d-%H%M%S)"
STAMP="$(date +%Y%m%d-%H%M%S)-$$"
LABEL="$(printf '%s' "${ARGS[0]}" | tr '/:' '__')"
mkdir -p build/logs || exit 1
LOG="build/logs/${STAMP}-${LABEL}.log"
mkdir -p "$(dirname "$LOG")"
(umask 077; set -C; : > "$LOG") || exit 1

# One log per build, hundreds of KB each, and agent sessions generate many:
# prune to the newest 20 so the directory stays bounded.
ls -t build/logs/*.log 2>/dev/null | tail -n +21 | xargs rm -f 2>/dev/null || true
# Build logs are qualification evidence, including failed runs. Retention is
# managed by the caller; never remove earlier evidence as a side effect of a build.

echo "▶ ./gradlew ${ARGS[*]} -i --console=plain (full log: $LOG)"
./gradlew "${ARGS[@]}" -i --console=plain >"$LOG" 2>&1
Expand Down
6 changes: 3 additions & 3 deletions .claude/hooks/require-fresh-review.py
Original file line number Diff line number Diff line change
Expand Up @@ -87,14 +87,14 @@ def repo_flag(tokens):

def ready_target(args, repo, cwd):
"""(PR number, node id, head oid, check arguments) for the PR the PR-ready subcommand
targets, from GitHub so the commits the PR holds are checked; ("", "", "", []) means
the current branch, None that it cannot be resolved."""
targets, from GitHub so the commits the PR holds are checked; None means the PR
cannot be resolved."""
selector = pr_selector(args)
out = gh_query(["pr", "view", *selector[:1], *(["-R", repo] if repo else []), "--json",
"id,number,headRefName,headRefOid,baseRefName", "-q",
'"\\(.id) \\(.number) \\(.headRefName) \\(.headRefOid) \\(.baseRefName)"'], cwd)
if not out or len(out.split()) != 5:
return None if selector else ("", "", "", [])
return None
pr_id, number, head, oid, base = out.split()
return number, pr_id, oid, ["--branch", head, "--commit", oid, "--base", origin(base)]

Expand Down
7 changes: 5 additions & 2 deletions .pi/extensions/require-fresh-review.ts
Original file line number Diff line number Diff line change
Expand Up @@ -9,13 +9,16 @@ import { join } from "node:path";
export default function (pi: ExtensionAPI) {
pi.on("tool_call", async (event, ctx) => {
if (event.toolName !== "bash") return undefined;
const hook = join(ctx.cwd, ".claude", "hooks", "require-fresh-review.py");
// Pi may run in a subdirectory: the hook lives at the checkout's top level.
const top = spawnSync("git", ["rev-parse", "--show-toplevel"], { cwd: ctx.cwd, encoding: "utf8" });
const root = top.status === 0 ? top.stdout.trim() : ctx.cwd;
const hook = join(root, ".claude", "hooks", "require-fresh-review.py");
if (!existsSync(hook)) return undefined; // a checkout from before the gate
const payload = JSON.stringify({ cwd: ctx.cwd, tool_input: { command: String(event.input.command ?? "") } });
const res = spawnSync("python3", [hook], {
input: payload,
encoding: "utf8",
env: { ...process.env, CLAUDE_PROJECT_DIR: ctx.cwd },
env: { ...process.env, CLAUDE_PROJECT_DIR: root },
timeout: 60_000,
});
return res.status === 2 ? { block: true, reason: res.stderr.trim() } : undefined;
Expand Down
7 changes: 5 additions & 2 deletions .pi/extensions/require-gradle-wrapper.ts
Original file line number Diff line number Diff line change
Expand Up @@ -11,13 +11,16 @@ export default function (pi: ExtensionAPI) {
if (event.toolName !== "bash") return undefined;
const command = String(event.input.command ?? "");
if (!command.includes("gradlew")) return undefined; // most commands: skip the python3 spawn
const hook = join(ctx.cwd, ".claude", "hooks", "require-gradle-wrapper.py");
// Pi may run in a subdirectory: the hook lives at the checkout's top level.
const top = spawnSync("git", ["rev-parse", "--show-toplevel"], { cwd: ctx.cwd, encoding: "utf8" });
const root = top.status === 0 ? top.stdout.trim() : ctx.cwd;
const hook = join(root, ".claude", "hooks", "require-gradle-wrapper.py");
if (!existsSync(hook)) return undefined; // a checkout from before the hook
const payload = JSON.stringify({ cwd: ctx.cwd, tool_input: { command } });
const res = spawnSync("python3", [hook], {
input: payload,
encoding: "utf8",
env: { ...process.env, CLAUDE_PROJECT_DIR: ctx.cwd },
env: { ...process.env, CLAUDE_PROJECT_DIR: root },
timeout: 60_000,
});
return res.status === 2 ? { block: true, reason: res.stderr.trim() } : undefined;
Expand Down
19 changes: 14 additions & 5 deletions ddprof-lib/src/main/cpp/AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,13 +9,13 @@ before working on a subsystem.

| Subsystem | Details |
|---|---|
| Call trace storage (triple-buffered ACTIVE/STANDBY/SCRATCH, 64-bit instance+slot trace IDs, lock-free rotation, freed on stop) | [doc/architecture/CallTraceStorage.md](/doc/architecture/CallTraceStorage.md) |
| Call trace storage (triple-buffered ACTIVE/STANDBY/SCRATCH, 64-bit instance+slot trace IDs, lock-free rotation, freed on stop, reusable unowned wall-clock roots) | [doc/architecture/CallTraceStorage.md](/doc/architecture/CallTraceStorage.md) |
| String dictionary (lazily allocated arena chunks, rotation/clear/release protocols) | [doc/architecture/StringDictionary.md](/doc/architecture/StringDictionary.md) |
| RefCountGuard (lock-free RAII reader-drain before reclamation) | [doc/architecture/RefCountGuard.md](/doc/architecture/RefCountGuard.md) |
| TLS context (thread-local tracing context, distributed tracing) | [doc/architecture/TLSContext.md](/doc/architecture/TLSContext.md) |
| TLS context (thread-local tracing, owner context-window membership, retained unowned wall context and sampling-epoch weight provenance) | [doc/architecture/TLSContext.md](/doc/architecture/TLSContext.md) |
| JVM support (HotSpot, J9, Zing) | [doc/architecture/JvmSupportLayer.md](/doc/architecture/JvmSupportLayer.md) |
| Profiling engines (CPU, wall-clock, allocation, native memory, liveness; filtered wall prechecks preserve reservoir selections in epoch accounting; timer TID metadata lookup is capped at 256 probes and fails open to ordinary signals; wall block backing is lazy and retained until tracker destruction; allocation/liveness stack capture via `allocstack`, with JVMTI retry for unsafe frames, depth-limit truncation and separate verification scratch) | [doc/architecture/ProfilingEngines.md](/doc/architecture/ProfilingEngines.md) |
| JFR integration (FlightRecorder, metadata, constant pools, buffers) | [doc/architecture/JfrIntegration.md](/doc/architecture/JfrIntegration.md) |
| Profiling engines (CPU, monotonic wall-clock cadence, allocation, native memory, liveness; prechecks preserve reservoir selections in epoch accounting; timer TID metadata lookup is capped at 256 probes and fails open to ordinary signals; lazy wall-block backing survives recording resets; allocation/liveness stack capture via `allocstack`, with JVMTI retry for unsafe frames, depth-limit truncation and separate verification scratch) | [doc/architecture/ProfilingEngines.md](/doc/architecture/ProfilingEngines.md) |
| JFR integration (FlightRecorder, metadata, constant pools, buffers; wall epochs settle before chunk seals; retained tails carry originating metadata across dumps) | [doc/architecture/JfrIntegration.md](/doc/architecture/JfrIntegration.md) |
| Live-heap reference chains (implementation, leak-signal gate, scheduling, design) | [doc/architecture/LiveHeapReferenceChainsImplementation.md](/doc/architecture/LiveHeapReferenceChainsImplementation.md), [doc/architecture/ReferenceChainsSignalsExplained.md](/doc/architecture/ReferenceChainsSignalsExplained.md), [doc/ReferenceChainsDesign.md](/doc/ReferenceChainsDesign.md), [doc/ReferenceChainsCollectionSummary.md](/doc/ReferenceChainsCollectionSummary.md) |
| Native memory profiling (malloc interception installed per recording and restored on stop, raw-identity exit admission, sampling) | [doc/architecture/NativeMemoryProfiling.md](/doc/architecture/NativeMemoryProfiling.md) |
| glibc malloc arena warning (lock-free arena walk, per-chunk rule, `MallocArenaWarning`/`MallocArenaStats` events, `arenawarn`) | [doc/architecture/MallocArenaWarning.md](/doc/architecture/MallocArenaWarning.md) |
Expand All @@ -24,7 +24,7 @@ before working on a subsystem.
| Event type system | [doc/reference/EventTypeSystem.md](/doc/reference/EventTypeSystem.md) |
| Memory requirements & budgets | [doc/reference/ProfilerMemoryRequirements.md](/doc/reference/ProfilerMemoryRequirements.md) |
| Remote symbolication (+frame types) | [doc/reference/RemoteSymbolication.md](/doc/reference/RemoteSymbolication.md), [doc/reference/RemoteSymbolicationFrameTypes.md](/doc/reference/RemoteSymbolicationFrameTypes.md) |
| Per-sampler perf counters (`-PenableSamplerPerf`) | [doc/reference/SamplerPerfCounters.md](/doc/reference/SamplerPerfCounters.md) |
| Per-sampler perf counters (`-PenableSamplerPerf`) and wall-timer diagnostics | [doc/reference/SamplerPerfCounters.md](/doc/reference/SamplerPerfCounters.md) |

## Critical Invariants

Expand Down Expand Up @@ -189,6 +189,15 @@ gates and skip a `NULL` engine slot. Details:
[ProfilingEngines.md](/doc/architecture/ProfilingEngines.md#thread-registration),
[SigsegvPatching.md](/doc/architecture/SigsegvPatching.md#initialization-flow).

**Reusable wall-clock roots.** Hold the recording stripe through capture, cached-ID publication and tail commit.
Do not consume a deferred segment after a failed write or apply a later sampling epoch's scale.
Retained units serialize at their original epoch-window start; `stop()` drops pending tails when it releases the storage.
See [CallTraceStorage](/doc/architecture/CallTraceStorage.md#reusable-unowned-wall-clock-roots).

**Context-window membership.** Owner TLS is the logical source; registry slots are recording projections.
Only owner-side entry/exit changes membership; signals and timers do not walk remote TLS.
See [TLSContext](/doc/architecture/TLSContext.md#context-window-membership).

**Atomic memory ordering (arm64).** arm64 is weakly ordered (x86 is TSO); relaxed-ordering
bugs are silent arm64-only lockups. Cross-thread reads: acquire ordering
(`__ATOMIC_ACQUIRE` / `std::memory_order_acquire`), including loads of pointers published
Expand Down
18 changes: 14 additions & 4 deletions ddprof-lib/src/main/cpp/callTraceHashTable.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -520,7 +520,11 @@ void CallTraceHashTable::collect(CallTraceSet &traces, std::function<void(CallTr
}


void CallTraceHashTable::putWithExistingId(CallTrace* source_trace, u64 weight) {
#ifdef UNIT_TEST
u64 CallTraceHashTable::failedPreservationIdForTest = 0;
#endif

bool CallTraceHashTable::putWithExistingId(CallTrace* source_trace, u64 weight) {
// Trace preservation for standby tables (no contention with new puts)
// This is safe because new put() operations go to the new active table

Expand All @@ -538,13 +542,13 @@ void CallTraceHashTable::putWithExistingId(CallTrace* source_trace, u64 weight)
search_table != nullptr; search_table = search_table->prev()) {
CallTrace *existing_trace = findCallTrace(search_table, hash);
if (existing_trace != nullptr) {
return;
return existing_trace->trace_id == source_trace->trace_id;
}
}

LongHashTable *table = __atomic_load_n(&_table, __ATOMIC_ACQUIRE);
if (table == nullptr) {
return; // Table allocation failed
return false; // Table allocation failed
}

u64 *keys = table->keys();
Expand All @@ -564,7 +568,11 @@ void CallTraceHashTable::putWithExistingId(CallTrace* source_trace, u64 weight)
// Create a copy of the source trace preserving its exact ID
const size_t header_size = sizeof(CallTrace) - sizeof(ASGCT_CallFrame);
const size_t total_size = header_size + source_trace->num_frames * sizeof(ASGCT_CallFrame);
void *memory = _allocator.alloc(total_size);
void *memory = nullptr;
#ifdef UNIT_TEST
if (source_trace->trace_id != failedPreservationIdForTest)
#endif
memory = _allocator.alloc(total_size);
if (memory != nullptr) {
// Use placement new to invoke constructor in-place
CallTrace* copied_trace = new (memory) CallTrace(source_trace->truncated, source_trace->num_frames, source_trace->trace_id);
Expand All @@ -577,6 +585,7 @@ void CallTraceHashTable::putWithExistingId(CallTrace* source_trace, u64 weight)
// Grow like put(): one entry per live trace ID can exceed the
// initial capacity (LivenessTracker tracks up to 262,144 objects).
expandTableIfNeeded(table, table->incSize());
return true;
} else {
// Allocation failure - clear the key we claimed
__atomic_store_n(&keys[slot], 0, __ATOMIC_RELEASE);
Expand All @@ -592,4 +601,5 @@ void CallTraceHashTable::putWithExistingId(CallTrace* source_trace, u64 weight)
break;
}
}
return false;
}
10 changes: 8 additions & 2 deletions ddprof-lib/src/main/cpp/callTraceHashTable.h
Original file line number Diff line number Diff line change
Expand Up @@ -175,9 +175,15 @@ class CallTraceHashTable {
/**
* Copies a trace into a scratch/standby table, deduplicating by exact trace ID.
* Must not mix with put(): clear the table before reusing it for active sampling.
* The caller must exclude concurrent writers.
* The caller must exclude concurrent writers. Returns true only for a
* confirmed existing exact ID or a successful copy; allocation/capacity
* failure returns false without confirming preservation.
*/
void putWithExistingId(CallTrace* trace, u64 weight);
bool putWithExistingId(CallTrace* trace, u64 weight);
#ifdef UNIT_TEST
// Fail only the exact-ID arena copy, without changing active sampling.
static u64 failedPreservationIdForTest;
#endif
void setInstanceId(u64 instance_id) {
// Use atomic store with RELEASE ordering to ensure visibility across threads
_instance_id.store(instance_id, std::memory_order_release);
Expand Down
33 changes: 27 additions & 6 deletions ddprof-lib/src/main/cpp/callTraceStorage.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -57,6 +57,7 @@ bool CallTraceStorage::allocateTables() {

_preserve_set_buffer.max_load_factor(0.75f);
_preserve_set_buffer.rehash(static_cast<size_t>(1024 / 0.75f));
_collector_set_buffer.max_load_factor(0.75f);

#ifdef UNIT_TEST
if (_fail_allocation_for_test) {
Expand All @@ -80,6 +81,7 @@ bool CallTraceStorage::allocateTables() {
// Give the pre-sized bucket arrays back, as release() does.
CallTraceSet().swap(_traces_buffer);
CallTraceIdSet().swap(_preserve_set_buffer);
CallTraceIdSet().swap(_collector_set_buffer);
return false;
}

Expand Down Expand Up @@ -109,6 +111,7 @@ void CallTraceStorage::release() {
// with empty sets frees their bucket arrays (clear() would keep them).
CallTraceSet().swap(_traces_buffer);
CallTraceIdSet().swap(_preserve_set_buffer);
CallTraceIdSet().swap(_collector_set_buffer);
Counters::set(CALLTRACE_STORAGE_BYTES, 0);
Counters::set(CALLTRACE_STORAGE_TRACES, 0);
}
Expand Down Expand Up @@ -161,18 +164,19 @@ CallTrace* CallTraceStorage::getDroppedTrace() {
return &dropped_trace;
}

u64 CallTraceStorage::registerLivenessChecker(LivenessChecker checker) {
u64 CallTraceStorage::registerLivenessChecker(LivenessChecker checker,
std::function<void(const CallTraceIdSet&)> completion) {
ExclusiveLockGuard lock(&_liveness_lock);
const u64 token = _next_liveness_token++;
_liveness_checkers.emplace_back(token, std::move(checker));
_liveness_checkers.push_back({token, std::move(checker), std::move(completion)});
return token;
}

void CallTraceStorage::unregisterLivenessChecker(u64 token) {
ExclusiveLockGuard lock(&_liveness_lock);
_liveness_checkers.erase(
std::remove_if(_liveness_checkers.begin(), _liveness_checkers.end(),
[token](const std::pair<u64, LivenessChecker>& c) { return c.first == token; }),
[token](const LivenessRegistration& c) { return c.token == token; }),
_liveness_checkers.end());
}

Expand Down Expand Up @@ -252,8 +256,13 @@ void CallTraceStorage::processTraces(std::function<void(const CallTraceSet&)> pr
_preserve_set_buffer.clear();

for (const auto& checker : _liveness_checkers) {
checker.second(_preserve_set_buffer);
_collector_set_buffer.clear();
checker.collect(_collector_set_buffer);
// The heap collector clears its output, including while disabled.
// Transfer nodes so it cannot erase roots from earlier collectors.
_preserve_set_buffer.merge(_collector_set_buffer);
}
_collector_set_buffer.clear(); // Release duplicate nodes not transferred by merge.
}

// PHASE 2: 10-Step Triple-Buffer Rotation
Expand All @@ -273,7 +282,9 @@ void CallTraceStorage::processTraces(std::function<void(const CallTraceSet&)> pr
// so we can iterate the first one and at the same time add new elements to the second one safely
original_standby->collect(_traces_buffer, [&](CallTrace* trace) {
if (_preserve_set_buffer.find(trace->trace_id) != _preserve_set_buffer.end()) {
original_scratch->putWithExistingId(trace, 1);
if (original_scratch->putWithExistingId(trace, 1)) {
_preserve_set_buffer.erase(trace->trace_id);
}
}
});

Expand Down Expand Up @@ -309,10 +320,20 @@ void CallTraceStorage::processTraces(std::function<void(const CallTraceSet&)> pr
// Step 6: Collect from old active directly to _traces_buffer with hook for immediate preservation
original_active->collect(_traces_buffer, [&](CallTrace* trace) {
if (_preserve_set_buffer.find(trace->trace_id) != _preserve_set_buffer.end()) {
original_scratch->putWithExistingId(trace, 1);
if (original_scratch->putWithExistingId(trace, 1)) {
_preserve_set_buffer.erase(trace->trace_id);
}
}
});

// Report only unconfirmed requests, before source frames are reclaimed.
{
SharedLockGuard lock(&_liveness_lock);
for (const auto& checker : _liveness_checkers) {
if (checker.complete) checker.complete(_preserve_set_buffer);
}
}

// Step 7: Update the instance id for the recently retired active buffer
original_active->setInstanceId(getNextInstanceId());

Expand Down
Loading
Loading