Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
41 commits
Select commit Hold shift + click to select a range
afdb48e
feat(wallclock): add paired TaskBlock API with generation-based owned…
kaahos Oct 15, 2026
940d2e0
fix: reset TaskBlock state when returning a thread object to the pool
kaahos Oct 20, 2026
6c2846e
test: assert weighted coverage for unowned wall-clock sleeps
kaahos Oct 20, 2026
74a55d5
fix: preserve unowned wall-tail traces across JFR dumps
kaahos Oct 20, 2026
1690e26
perf: add benchmark for unowned wall-tail recording
kaahos Oct 20, 2026
2a6d2d8
fix: remove JVMTI unowned taskblock suppression
kaahos Oct 21, 2026
b52711a
Merge wallclock-all-threads into taskblock-core
kaahos Nov 12, 2026
1360385
fix: align owned-block chaos hooks with TaskBlock admission
kaahos Nov 12, 2026
a358d49
fix: isolate wall-tail dump test driver from signal handlers
kaahos Nov 12, 2026
2c639b4
Merge uniform wall-clock sampling into TaskBlock core
kaahos Nov 12, 2026
5f5a6b2
Fix JVMTI truncation and wall-clock regression coverage
kaahos Nov 13, 2026
bce1bd5
Merge #890 review fixes into #664 TaskBlock core
kaahos Nov 13, 2026
9895dbe
Bound TaskBlock admission retries under contention
kaahos Nov 13, 2026
9d3d5fd
Flush deferred wall-clock tails at teardown and gate TaskBlock tests …
kaahos Nov 15, 2026
9427d5b
Fix TaskBlock lifecycle coverage and wall-tail thread admission
kaahos Nov 15, 2026
8df0271
Fix optimized gtest linkage for profiler stripe lookup
kaahos Nov 16, 2026
7228d04
Merge retained wall epochs and bounded prechecks into TaskBlock core
kaahos Nov 16, 2026
b7858d9
feat: add DEBUG TaskBlock recorder telemetry
kaahos Nov 17, 2026
b41f98e
Merge crash-handler bootstrap fix into TaskBlock core telemetry
kaahos Nov 17, 2026
78e4978
Merge upstream #663 test follow-ups into #664
kaahos Nov 17, 2026
d407cf4
Merge upstream exit-order repair into #664 and isolate recorder test
kaahos Nov 17, 2026
8025e85
Serialize synchronous wall dump sampling with epoch writes
kaahos Nov 18, 2026
2980cf3
Mark unprovisionable musl/OpenJ9 combinations not applicable
kaahos Nov 18, 2026
454af9d
fix(jfr): recover thread-table capacity during stop tail flush
kaahos Nov 18, 2026
e37124f
test(jfr): choose stripe occupants independently of the tail tid
kaahos Nov 18, 2026
9467155
test(ci): make the pipeline exclusion literal explicit
kaahos Nov 18, 2026
bc07fe5
Merge #890 native-memory and JVM installer repairs into #664
kaahos Nov 19, 2026
e42b96b
chore(stack): merge #890 fixes into #664
kaahos Nov 19, 2026
dda5077
chore(stack): merge main and #890 fixes into #664
kaahos Nov 19, 2026
0243faf
fix(wall): keep stop teardown when a deferred tail cannot be written
kaahos Nov 20, 2026
86249d1
fix(wall): name safepoint waits in the task-block rotation timeout log
kaahos Nov 20, 2026
d5cf1d0
chore(stack): merge #890 fixes into #664
kaahos Nov 20, 2026
c93d943
test(wall): drop unwritten tails between wall-epoch fixture tests
kaahos Nov 20, 2026
c9ef659
docs(wall): say the stop drop counter counts tail weight
kaahos Nov 20, 2026
7b63225
test(wall): use the renamed owned-block suppression field
kaahos Nov 20, 2026
4c19e3b
test(wall): stream wall-tail context samples instead of materializing…
kaahos Nov 20, 2026
7e8361f
test(taskblock): account for best-effort recorder rejection in API tests
kaahos Nov 20, 2026
39f4c42
chore(stack): merge #890 fixes into #664
kaahos Nov 20, 2026
e87f5a9
test(taskblock): retry counter-accounted recorder rejections across T…
kaahos Nov 20, 2026
2033c1b
chore(stack): merge #890 fixes into #664
kaahos Nov 20, 2026
da82f07
chore(stack): merge #890 fixes into #664
kaahos Nov 20, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions ddprof-lib/src/main/cpp/AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,8 +14,8 @@ before working on a subsystem.
| RefCountGuard (lock-free RAII reader-drain before reclamation) | [doc/architecture/RefCountGuard.md](/doc/architecture/RefCountGuard.md) |
| TLS context (thread-local tracing, owner context-window membership, retained unowned wall context and sampling-epoch weight provenance) | [doc/architecture/TLSContext.md](/doc/architecture/TLSContext.md) |
| JVM support (HotSpot, J9, Zing) | [doc/architecture/JvmSupportLayer.md](/doc/architecture/JvmSupportLayer.md) |
| Profiling engines (CPU, monotonic wall-clock cadence, allocation, native memory, liveness; prechecks preserve reservoir selections in epoch accounting; timer TID metadata lookup is capped at 256 probes and fails open to ordinary signals; lazy wall-block backing survives recording resets; allocation/liveness stack capture via `allocstack`, with JVMTI retry for unsafe frames, depth-limit truncation and separate verification scratch) | [doc/architecture/ProfilingEngines.md](/doc/architecture/ProfilingEngines.md) |
| JFR integration (FlightRecorder, metadata, constant pools, buffers; wall epochs settle before chunk seals; retained tails carry originating metadata across dumps) | [doc/architecture/JfrIntegration.md](/doc/architecture/JfrIntegration.md) |
| Profiling engines (CPU, monotonic wall-clock cadence, allocation, native memory, liveness; prechecks preserve reservoir selections in epoch accounting; timer TID metadata lookup is capped at 256 probes and fails open to ordinary signals; lazy wall-block backing survives recording resets; unowned teardown tails retain slot custody until written, and stop counts the weight of the ones it cannot write as dropped; allocation/liveness stack capture via `allocstack`, with JVMTI retry for unsafe frames, depth-limit truncation and separate verification scratch) | [doc/architecture/ProfilingEngines.md](/doc/architecture/ProfilingEngines.md) |
| JFR integration (FlightRecorder, metadata, constant pools, buffers; bounded DEBUG TaskBlock transition receipts survive recording teardown; wall epochs settle before chunk seals; retained tails carry originating metadata across dumps; stop archives full thread tables before bounded tail-admission retries) | [doc/architecture/JfrIntegration.md](/doc/architecture/JfrIntegration.md) |
| Live-heap reference chains (implementation, leak-signal gate, scheduling, design) | [doc/architecture/LiveHeapReferenceChainsImplementation.md](/doc/architecture/LiveHeapReferenceChainsImplementation.md), [doc/architecture/ReferenceChainsSignalsExplained.md](/doc/architecture/ReferenceChainsSignalsExplained.md), [doc/ReferenceChainsDesign.md](/doc/ReferenceChainsDesign.md), [doc/ReferenceChainsCollectionSummary.md](/doc/ReferenceChainsCollectionSummary.md) |
| Native memory profiling (malloc interception installed per recording and restored on stop, raw-identity exit admission, sampling) | [doc/architecture/NativeMemoryProfiling.md](/doc/architecture/NativeMemoryProfiling.md) |
| glibc malloc arena warning (lock-free arena walk, per-chunk rule, `MallocArenaWarning`/`MallocArenaStats` events, `arenawarn`) | [doc/architecture/MallocArenaWarning.md](/doc/architecture/MallocArenaWarning.md) |
Expand Down
15 changes: 14 additions & 1 deletion ddprof-lib/src/main/cpp/counters.h
Original file line number Diff line number Diff line change
Expand Up @@ -79,6 +79,8 @@
X(THREAD_REGISTRY_CAPACITY_EXHAUSTED, "thread_registry_capacity_exhausted") \
X(THREAD_REGISTRY_INDEX_FAILURES, "thread_registry_index_failures") \
X(THREAD_REGISTRY_HOOK_REREGISTRATION, "thread_registry_hook_reregistration") \
X(THREAD_REGISTRY_BLOCK_GENERATION_SATURATED, "thread_registry_block_generation_saturated") \
X(THREAD_REGISTRY_UNREGISTER_ACTIVE_BLOCK_RUN, "thread_registry_unregister_active_block_run") \
X(JMETHODID_SKIPPED, "jmethodid_skipped_count") \
X(CODECACHE_NATIVE_SIZE_BYTES, "codecache_native_size_bytes") \
X(CODECACHE_NATIVE_COUNT, "native_codecache_count") \
Expand All @@ -88,17 +90,28 @@
X(AGCT_NATIVE_NO_JAVA_CONTEXT, "agct_native_no_java_context") \
X(AGCT_BLOCKED_IN_VM, "agct_blocked_in_vm") \
X(SKIPPED_WALLCLOCK_UNWINDS, "skipped_wallclock_unwinds") \
X(WC_SIGNAL_SUPPRESSED_SAMPLED_RUN, "wc_signals_suppressed_sampled_run") \
X(WC_PRECHECK_REGISTRY_LOOKUPS, "wc_precheck_registry_lookups") \
X(WC_PRECHECK_REGISTRY_PROBE_EXHAUSTED, "wc_precheck_registry_probe_exhausted") \
X(WC_PRECHECK_CANDIDATES_REJECTED, "wc_precheck_candidates_rejected") \
X(WC_SIGNAL_SUPPRESSED_OWNED_BLOCK, "wc_signals_suppressed_owned_block") \
X(WC_UNOWNED_BLOCKED_SUPPRESSED, "wc_unowned_blocked_suppressed") \
X(WC_UNOWNED_BLOCKED_TRACE_PRESERVATION_FAILED, "wc_unowned_blocked_trace_preservation_failed") \
X(WC_UNOWNED_BLOCKED_TAIL_RECORDED, "wc_unowned_blocked_tail_recorded") \
X(WC_UNOWNED_TAIL_DROPPED_AT_STOP, "wc_unowned_tail_dropped_at_stop") \
X(WC_UNOWNED_BLOCKED_RECORDED, "wc_unowned_blocked_recorded") \
X(WC_UNOWNED_CONTEXT_SNAPSHOT_UNAVAILABLE, "wc_unowned_context_snapshot_unavailable") \
X(WC_UNOWNED_CONTEXT_ALLOCATION_FAILED, "wc_unowned_context_allocation_failed") \
X(WC_SIGNAL_QUEUE_FULL, "wc_signals_queue_full") \
X(TASK_BLOCK_EMITTED, "task_block_emitted") \
X(TASK_BLOCK_SKIPPED_TRACE_CONTEXT, "task_block_skipped_trace_context") \
X(TASK_BLOCK_SKIPPED_CONTEXT_WINDOW, "task_block_skipped_context_window") \
X(TASK_BLOCK_SKIPPED_TOO_SHORT, "task_block_skipped_too_short") \
X(TASK_BLOCK_STACK_CAPTURE_FAILED, "task_block_stack_capture_failed") \
X(TASK_BLOCK_RECORD_FAILED, "task_block_record_failed") \
X(TASK_BLOCK_DROPPED_ROTATION, "task_block_dropped_rotation") \
X(TASK_BLOCK_DROPPED_ADMISSION, "task_block_dropped_admission") \
X(TASK_BLOCK_SKIPPED_THREAD_MISMATCH, "task_block_skipped_thread_mismatch") \
X(TASK_BLOCK_ROTATION_TIMEOUT, "task_block_rotation_timeout") \
X(UNWINDING_TIME_ASYNC, "unwinding_ticks_async") \
X(UNWINDING_TIME_JVMTI, "unwinding_ticks_jvmti") \
X(ALLOC_STACK_VM, "alloc_stack_vm") \
Expand Down
21 changes: 15 additions & 6 deletions ddprof-lib/src/main/cpp/event.h
Original file line number Diff line number Diff line change
Expand Up @@ -62,11 +62,10 @@ class ExecutionEvent : public Event {
u64 _weight;
// Deferred wall observations keep their sampling-epoch timestamp; zero means now.
u64 _sample_ticks;
u32 _call_trace_id;

ExecutionEvent()
: Event(), _thread_state(OSThreadState::RUNNABLE), _execution_mode(ExecutionMode::UNKNOWN),
_weight(1), _sample_ticks(0), _call_trace_id(0) {}
_weight(1), _sample_ticks(0) {}
};

class AllocEvent : public Event {
Expand Down Expand Up @@ -212,13 +211,13 @@ class WallClockEpochEvent {
u32 _num_failed_samples;
u32 _num_exited_threads;
u32 _num_permission_denied;
u64 _num_suppressed_sampled_run;
u64 _num_suppressed_owned_block;

WallClockEpochEvent(u64 start_time)
: _dirty(false), _start_time(start_time), _duration_millis(0),
_num_samplable_threads(0), _num_successful_samples(0),
_num_failed_samples(0), _num_exited_threads(0),
_num_permission_denied(0), _num_suppressed_sampled_run(0) {}
_num_permission_denied(0), _num_suppressed_owned_block(0) {}

bool hasChanged() { return _dirty; }

Expand Down Expand Up @@ -258,7 +257,7 @@ class WallClockEpochEvent {
}

// Accumulates into the open epoch; a suppression alone never closes it.
void addNumSuppressedSampledRun(u64 n) { _num_suppressed_sampled_run += n; }
void addNumSuppressedOwnedBlock(u64 n) { _num_suppressed_owned_block += n; }

void endEpoch(u64 millis) { _duration_millis = millis; }

Expand All @@ -267,7 +266,7 @@ class WallClockEpochEvent {
void newEpoch(u64 start_time) {
_dirty = false;
_start_time = start_time;
_num_suppressed_sampled_run = 0;
_num_suppressed_owned_block = 0;
}
};

Expand All @@ -292,4 +291,14 @@ typedef struct QueueTimeEvent {
u32 _queueLength;
} QueueTimeEvent;

typedef struct TaskBlockEvent {
u64 _start;
u64 _end;
u64 _blocker;
u64 _unblockingSpanId;
Context _ctx;
u64 _callTraceId;
OSThreadState _observedBlockingState;
} TaskBlockEvent;

#endif // _EVENT_H
82 changes: 77 additions & 5 deletions ddprof-lib/src/main/cpp/flightRecorder.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -1002,6 +1002,9 @@ Recording::Recording(int fd, Arguments &args)
}

Recording::~Recording() {
#ifdef UNIT_TEST
if (_fd == -1) return; // Buffer/table-only serializer fixture.
#endif
finishChunk(true);
close(_fd);
}
Expand Down Expand Up @@ -1903,7 +1906,9 @@ void Recording::writeThreads(Buffer *buf) {
// After flip: new samples go into the new active set
// We flush from old_index (the previous active set)

std::unordered_set<int> threads;
std::unordered_set<int> threads(_archived_thread_ids.begin(), _archived_thread_ids.end());
_archived_thread_ids.clear();
_thread_table_overflow.store(0, std::memory_order_release);
threads.insert(_tid); // always present: the recording thread itself

for (int i = 0; i < CONCURRENCY_LEVEL; ++i) {
Expand Down Expand Up @@ -2567,6 +2572,21 @@ void Recording::recordMethodSample(Buffer *buf, int tid, u64 call_trace_id,
flushIfNeeded(buf);
}

void Recording::recordTaskBlock(Buffer *buf, int tid, TaskBlockEvent *event) {
int start = buf->skip(1);
buf->putVar64(T_TASK_BLOCK);
buf->putVar64(event->_start);
buf->putVar64(event->_end - event->_start);
buf->putVar64(tid);
buf->putVar64(event->_blocker);
buf->putVar64(event->_unblockingSpanId);
buf->putVar64(event->_callTraceId);
buf->put8(static_cast<int>(event->_observedBlockingState));
writeContextSnapshot(buf, event->_ctx);
writeEventSizePrefix(buf, start);
flushIfNeeded(buf);
}

void Recording::recordWallClockEpoch(Buffer *buf, WallClockEpochEvent *event) {
DEBUG_ASSERT_NOT_IN_SIGNAL();
_wall_epochs.insert_or_assign(event->_start_time, *event);
Expand All @@ -2579,7 +2599,7 @@ void Recording::recordWallClockEpoch(Buffer *buf, WallClockEpochEvent *event) {
buf->putVar64(event->_num_failed_samples);
buf->putVar64(event->_num_exited_threads);
buf->putVar64(event->_num_permission_denied);
buf->putVar64(event->_num_suppressed_sampled_run);
buf->putVar64(event->_num_suppressed_owned_block);
writeEventSizePrefix(buf, start);
flushIfNeeded(buf);
}
Expand Down Expand Up @@ -2884,9 +2904,35 @@ void Recording::recordCpuLoad(Buffer *buf, float proc_user, float proc_system,
}

// assumption is that we hold the lock (with lock_index)
void Recording::addThread(int lock_index, int tid) {
bool Recording::addThread(int lock_index, int tid) {
int active = _active_index.load(std::memory_order_acquire);
_thread_ids[lock_index][active].insert(tid); // ThreadIdTable::insert is signal-safe (atomics only)
if (_thread_ids[lock_index][active].insert(tid)) return true;
if (tid != 0) _thread_table_overflow.fetch_or(1U << lock_index, std::memory_order_release);
return false;
}

bool Recording::recoverThreadTableCapacity() {
DEBUG_ASSERT_NOT_IN_SIGNAL();
u32 overflow = _thread_table_overflow.load(std::memory_order_acquire);
if (overflow == 0) return false;
int active = _active_index.load(std::memory_order_acquire);
std::unordered_set<int> threads;
for (int i = 0; i < CONCURRENCY_LEVEL; ++i) {
if (overflow & (1U << i)) _thread_ids[i][active].collect(threads);
}
// Allocation completes before any table is cleared, preserving every existing event reference.
_archived_thread_ids.insert(_archived_thread_ids.end(), threads.begin(), threads.end());
for (int i = 0; i < CONCURRENCY_LEVEL; ++i) {
if (overflow & (1U << i)) _thread_ids[i][active].clear();
}
_thread_table_overflow.store(0, std::memory_order_release);
return true;
}

bool FlightRecorder::recoverThreadTableCapacity() {
DEBUG_ASSERT_NOT_IN_SIGNAL();
ExclusiveLockGuard locker(&_rec_lock);
return _rec != nullptr && _rec->recoverThreadTableCapacity();
}

Error FlightRecorder::start(Arguments &args, bool reset) {
Expand Down Expand Up @@ -2998,6 +3044,31 @@ void FlightRecorder::recordQueueTime(int lock_index, int tid,
}
}

bool FlightRecorder::recordTaskBlock(int lock_index, int tid,
TaskBlockEvent *event
TASK_BLOCK_DIAGNOSTIC_ONLY(, const TaskBlockDiagnosticRecord* diagnostic, TaskBlockStage* rejection)) {
OptionalSharedLockGuard locker(&_rec_lock);
if (!locker.ownsLock()) {
TASK_BLOCK_REJECTION(REJECT_RECORDER_SHARED_LOCK, diagnostic, lock_index, rejection);
return false;
}
Recording* rec = _rec;
if (rec == nullptr) {
TASK_BLOCK_REJECTION(REJECT_NO_RECORDING, diagnostic, 0, rejection);
return false;
}
Buffer *buf = rec->buffer(lock_index);
// Thread admission must precede bytes that reference this chunk's thread pool.
if (!rec->addThread(lock_index, tid)) {
TASK_BLOCK_REJECTION(REJECT_EVENT_THREAD_ADMISSION, diagnostic, tid, rejection);
return false;
}
TASK_BLOCK_DIAGNOSTIC(BUFFER_SERIALIZATION_BEGIN, diagnostic, lock_index);
rec->recordTaskBlock(buf, tid, event);
TASK_BLOCK_DIAGNOSTIC(BUFFER_ACCEPTED, diagnostic, lock_index);
return true;
}

void FlightRecorder::recordDatadogSetting(int lock_index, int length,
const char *name, const char *value,
const char *unit) {
Expand Down Expand Up @@ -3107,10 +3178,11 @@ bool FlightRecorder::recordWallSample(int lock_index, int tid, u64 call_trace_id
}
Recording* rec = _rec;
if (rec == nullptr) return false;
// A retained tail must keep its weight if eventThread cannot enter this chunk.
if (!rec->addThread(lock_index, tid)) return false;
Comment thread
kaahos marked this conversation as resolved.
Resolved
RecordingBuffer* buf = rec->buffer(lock_index);
rec->recordMethodSample(buf, tid, call_trace_id, 0, event, &context);
rec->flushIfNeeded(buf);
rec->addThread(lock_index, tid);
return true;
}

Expand Down
24 changes: 19 additions & 5 deletions ddprof-lib/src/main/cpp/flightRecorder.h
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,7 @@
#include <string.h>

#include "arch.h"
#include "taskBlockDiagnostics.h"
#include "arguments.h"
#include "buffers.h"
#include "common.h"
Expand Down Expand Up @@ -265,10 +266,8 @@ class Recording {
friend ObjectSampler;
friend Profiler;
friend Lookup;
// Grants gtest access to the private countSerializableChildren() helper below,
// since Recording itself can't be constructed in a plain gtest binary (its
// constructor needs a live JVMTI environment). Same pattern as
// VMTestAccessor/ProfilerTestAccessor in the test sources.
// Gtest access to private serializer invariants; the production constructor
// requires a live JVMTI environment.
friend class RecordingTestAccessor;

private:
Expand Down Expand Up @@ -303,6 +302,9 @@ class Recording {
// we have a second dimension to allow a switch in the active table
ThreadIdTable _thread_ids[CONCURRENCY_LEVEL][2];
std::atomic<int> _active_index{0}; // 0 or 1 globally
std::atomic<u32> _thread_table_overflow{0};
// Stop archives full tables before reusing their slots; writeThreads unions the saved IDs.
std::vector<int, CountingAllocator<int, NM_JFR_BUFFERS>> _archived_thread_ids;

int _fd;
off_t _chunk_start;
Expand Down Expand Up @@ -375,6 +377,10 @@ class Recording {

public:
Recording(int fd, Arguments &args);
#ifdef UNIT_TEST
// Buffer/table-only recording for serializer tests without a live JVM.
Recording() : _fd(-1), _bytes_written(0) {}
#endif
~Recording();

void copyTo(int target_fd);
Expand Down Expand Up @@ -509,6 +515,7 @@ class Recording {
}
void recordTraceRoot(Buffer *buf, int tid, TraceRootEvent *event);
void recordQueueTime(Buffer *buf, int tid, QueueTimeEvent *event);
void recordTaskBlock(Buffer *buf, int tid, TaskBlockEvent *event);
void recordAllocation(RecordingBuffer *buf, int tid, u64 call_trace_id,
AllocEvent *event);
void recordMallocSample(Buffer *buf, int tid, u64 call_trace_id,
Expand All @@ -527,7 +534,9 @@ class Recording {
void recordCpuLoad(Buffer *buf, float proc_user, float proc_system,
float machine_total);

void addThread(int lock_index, int tid);
bool addThread(int lock_index, int tid);
// Normal thread, exclusive recording lock: preserve old event references before clearing capacity.
bool recoverThreadTableCapacity();

private:
void cleanupUnreferencedMethods();
Expand Down Expand Up @@ -677,6 +686,7 @@ class Lookup {
class FlightRecorder {
friend Profiler;
#ifdef UNIT_TEST
friend class FlightRecorderTestAccessor;
friend class ProfilerTestAccessor;
#endif

Expand All @@ -688,6 +698,7 @@ class FlightRecorder {
Recording* _rec;

Error newRecording(bool reset);
bool recoverThreadTableCapacity();

public:
FlightRecorder() : _rec(NULL) {}
Expand All @@ -699,6 +710,9 @@ class FlightRecorder {
void pruneWallEpochs(const WallClockBlockTracker::EpochStarts& pending);
void recordTraceRoot(int lock_index, int tid, TraceRootEvent *event);
void recordQueueTime(int lock_index, int tid, QueueTimeEvent *event);
bool recordTaskBlock(int lock_index, int tid, TaskBlockEvent *event
TASK_BLOCK_DIAGNOSTIC_ONLY(, const TaskBlockDiagnosticRecord* diagnostic = nullptr,
TaskBlockStage* rejection = nullptr));

bool active() const { return _rec != NULL; }

Expand Down
Loading
Loading