[datanexus-vincent]

Adds opt-in OpenTelemetry tracing to graph-node. With GRAPH_OTEL_TRACING=true, graph-node exports spans over OTLP (http/protobuf) to any collector or vendor agent. Off by default; when off, nothing is installed and the only new cost is one atomic load at the start and end of each instrumented section.

Documentation: docs/tracing.md, plus a section in docs/environment-variables.md and a NEWS entry.

Why

Prometheus metrics say how long block processing, writes and queries take on average. They cannot say where the time went in one slow block or one slow request, or which blocks a slow batch write came from. Traces can, and many enterprise-level operators already run OpenTelemetry collectors and vendor agents that accept OTLP.

Design

• Spans are derived from StopwatchMetrics sections. The runner, the writer and the WASM host already wrap their work in stopwatch sections. Each section now optionally opens a span, so span names are the existing deployment_sync_secs section names and the tree nests exactly as the sections do. No tracing crate, no changes to logging; slog stays as it is.
• Roots are explicit, not thread-local. process_block starts a trace per block, write_batch one per batch written by the write queue, graphql_query one per request. Nothing relies on the OpenTelemetry thread-local context except the one place where a GraphQL query runs on a dedicated blocking thread, which is where the store's db_query spans find their parent.
• Write batches link to blocks instead of nesting in them. A batch holds several blocks and is written after they were processed, so it is its own trace with a span link to each block's process_block span (capped at 32, overflow counted in dropped_links).
• Raw opentelemetry SDK behind a thin graph::telemetry module. Span, Context, extract/inject and the enabled flags are the only surface the rest of the code sees. The SDK reads the standard OTEL_* variables itself; graph-node adds only GRAPH_OTEL_TRACING and GRAPH_OTEL_SPANS.
• Span groups. GRAPH_OTEL_SPANS selects block, handler, host_fn, store, graphql. host_fn is off by default because a handler can make thousands of store and chain calls.
• Shutdown flushes. On SIGINT or SIGTERM the buffered spans are exported before the process exits.

What is traced

process_block {subgraph_id, block_number, block_hash, network, shard}
├── process_triggers
│   ├── match_and_decode
│   └── run_handler {handler, trigger, tx_hash?, log_index?}
│       └── host_export_* {host_function, ...}            (host_fn, off by default)
├── as_modifications
├── transact_block {entity_count}
└── update_proof_of_indexing

write_batch {subgraph_id, first_block, last_block, entity_count}   links -> process_block
└── transact_blocks_get_conn, apply_entity_modifications, transact_blocks_prune, ...

graphql_query {subgraph, api_version, query_id, deployment?}   parent <- traceparent header
└── db_query {subgraph_id, entity_count, sql?}

A request with a W3C traceparent header joins the caller's trace. query_id on the request span is the id the query timing logs and the SQL route comment carry, so spans, log lines and pg_stat_activity can be matched up. sql is only attached when GRAPH_LOG_QUERY_TIMING includes sql, the same gate as the logs.

Testing

• An in-memory exporter in graph::telemetry::testing lets tests assert on exported span trees. Tests cover the module itself, the stopwatch span derivation, the per-block trace and handler attributes (block_spans runner test), write batch links and db_query parenting (test-store), and the HTTP server's header extraction and error status.
• Tests that need the process-global exporter in a specific state run in a subprocess; the helper is in the same module.
• Verified manually against the OpenTelemetry Collector's debug exporter: a traced request produced graphql_query under the header's span with db_query under it, a failing query carried an error status, and spans sent just before SIGTERM arrived during shutdown.
• With tracing off, the unit, test-store and runner suites are unchanged.

Notes for reviewers

• Cargo.lock only gains the new packages; unrelated entries were left as they were.
• Along the way, two unrelated small things were fixed because the new code ran into them: the SQL text in query timing logs was truncated at a byte offset, which panics inside a multibyte character, and a stopwatch section that was never bound to anything was removed.
• Not in this PR: injecting the trace context into outbound RPC and IPFS requests. It is a small follow-up if requested.