Skip to content

perf(web): send the Home log volume read on its own so it stops holding the other cards - #2525

Merged
Makisuo merged 2 commits into
mainfrom
perf/home-log-volume-own-request
Nov 12, 2026
Merged

Makisuo merged 2 commits into
mainfrom
perf/home-log-volume-own-request

Conversation

@Makisuo

@Makisuo Makisuo commented Nov 12, 2026 •

Copy link
Copy Markdown
Collaborator

Problem

On Home, the time series and facets cards often wait several seconds for the log volume card.
On prd, FetchOverviewFacets, FetchOverviewLogVolume and FetchOverviewTimeSeries start
together and also end together. From 11-10 to 11-12 their end time (from the start of pageload Home) was
p50 ~3.0 s and p90 7.2 to 8.7 s.

The server spans in the same traces show which read is slow (prd, last 24 h, traces that hold a log volume read):

query.context p50 p95
logsTimeseries (log volume) 60 ms 6,257 ms
servicesFacets 205 ms 1,078 ms
tracesAllMetricsTimeseries (time series) 70 ms 535 ms

Cause

  • apps/web/src/page/overview/command.ts:161: FetchLogSeries goes through the shared execute
    batcher, like the time series and facets.
  • apps/web/src/data/execute-batcher.ts:18: a 15 ms window puts reads from separate commands
    into one /execute-batch request, and packages/domain/src/query-engine.ts:727 says it plainly:
    the response is all or nothing, so a batch answers at the speed of its slowest member.
  • The log volume read is slow on large orgs. At 24 h it uses 900 s buckets, which the hourly
    logs rollup cannot answer (canUseLogsAggregatesHourly, packages/query-engine/src/ch/queries/logs.ts:306),
    so it scans raw logs and reads the ResourceAttributes map on every row for the environment filter.
    On Maple's own org (5.3 M logs a day) that map read alone takes the query from ~0.37 s to ~1.2 s.

Fix

  • ExecuteBatcher and WarehouseExecutor get executeAlone: one query, sent at once as a batch of one.
  • outsideSharedBatch(effect) (@maple/web-data/warehouse/warehouse-services) runs an effect with
    its warehouse reads on executeAlone.
  • FetchLogSeries uses it. The time series and facets keep batching with each other and stop
    waiting for logs. The log volume card itself is not faster (that needs a minutely logs rollup,
    see below).

Cost: Home makes one more request per load.

Before / After

Measure Before After Where
Time series + facets end, p90 (from pageload Home start) 7.2 to 8.7 s expected ~2.5 s (start ~1.55 s + their own p95 ≤ 1.1 s) before: prd. after: projected from prd spans, confirm after deploy
Log volume end, p90 7.2 to 8.7 s unchanged prd
Batches on Home with a 6 s log read 1 request holding logs + time series 2 requests; time series settles at 300 ms, logs at 6 s local, command.test.ts (real commands and batcher, simulated send)
/execute-batch requests per Home load (3 cold loads, signed in) 1 holding all three reads 2 every time: logs count alone; traces time series + services facets together preview pr-2525, small data: shows the split, not the speed

Follow-up (not in this PR)

The log volume read itself (logsTimeseries p95 4.4 to 4.7 s on 11-10 and 11-12) needs a minutely logs rollup
with DeploymentEnv as a column, spliced like service_overview_minutely. That is a new
materialized view, a migration and a local schema bump, so it gets its own PR.

How to verify after deploy

Home children end time by day (run_sql). Time series and facets should end well before log volume:

SELECT toDate(c.Timestamp) day, c.SpanName, count() n,
  round(quantile(0.5)(dateDiff('millisecond', p.Timestamp, c.Timestamp) + c.Duration / 1e6)) end_p50_ms,
  round(quantile(0.9)(dateDiff('millisecond', p.Timestamp, c.Timestamp) + c.Duration / 1e6)) end_p90_ms
FROM traces c
INNER JOIN (SELECT TraceId, SpanId, Timestamp FROM traces
  WHERE $__orgFilter AND $__timeFilter(Timestamp) AND SpanName = 'pageload Home') p
  ON c.TraceId = p.TraceId AND c.ParentSpanId = p.SpanId
WHERE $__orgFilter AND $__timeFilter(c.Timestamp)
  AND c.SpanName IN ('FetchOverviewFacets', 'FetchOverviewLogVolume', 'FetchOverviewTimeSeries')
GROUP BY day, c.SpanName ORDER BY day, c.SpanName

logsTimeseries p95 by day (should not change with this PR; it is the follow-up's target):

SELECT toDate(Timestamp) day, count() n,
  round(quantile(0.5)(Duration) / 1e6) p50_ms, round(quantile(0.95)(Duration) / 1e6) p95_ms
FROM traces
WHERE $__orgFilter AND $__timeFilter(Timestamp) AND ServiceName = 'maple-api'
  AND SpanName = 'WarehouseQueryService.executeSql'
  AND SpanAttributes['query.context'] = 'logsTimeseries'
GROUP BY day ORDER BY day

Checks run

  • bunx vitest run src/page/overview src/data/execute-batcher.test.ts in apps/web: 56 passed.
    The new test fails without the outsideSharedBatch line (one batch holding both reads).
  • bun run --filter=@maple/web typecheck, --filter=@maple/web-data typecheck, oxlint -c .oxlintrc.effect.json on the changed files.

🤖 Generated with Claude Code


Autofix with [code]smith
Open in [code]smith · Need help on this PR? Tag @codesmith-bot with what you need.

Summary by CodeRabbit

  • Bug Fixes
    • Overview log-volume queries now run independently from other card queries. Slower raw-log scans no longer delay faster overview results, such as time-series data, which can complete and display while the log-volume request is still processing. Log-volume results continue to load once that request finishes.

…ng the other cards

The log volume read scans raw logs at 15 minute buckets and takes seconds on a
large org. It shared one execute-batch request with the time series and facets,
and a batch answers at the speed of its slowest member, so those cards waited
on it too.
@Makisuo Makisuo added the preview:web Deploy a Workers-only PR preview (no ingest/Electric); see docs/pr-previews.md label Nov 12, 2026
@maple-review-bot

This comment has been minimized.

@coderabbitai

coderabbitai Bot commented Nov 12, 2026 •

Copy link
Copy Markdown

Explain Change →

No actionable comments were generated in the recent review.

Walkthrough

The batcher and warehouse service now support executing a query outside the shared batch window. The overview log-series request uses this path, and a test checks its batching and completion timing.

Changes: Standalone query execution

Layer / File(s) Summary
Immediate single-request execution
apps/web/src/data/execute-batcher.ts
The batcher exposes executeAlone, which dispatches a request immediately as a one-item batch and returns its result.
Warehouse service support
packages/web-data/src/warehouse/warehouse-services.ts, apps/web/src/data/warehouse.ts
The warehouse executor exposes standalone execution. The outsideSharedBatch helper routes wrapped effects through it.
Overview log-series integration and test
apps/web/src/page/overview/command.ts, apps/web/src/page/overview/command.test.ts
FetchLogSeries uses outsideSharedBatch. The test checks that log and trace requests use separate batches and verifies their completion timing.

Estimated code review effort: 3 (Moderate) | ~20 minutes

Change: Bug fix

Sequence Diagram(s)

sequenceDiagram
  participant FetchLogSeries
  participant outsideSharedBatch
  participant WarehouseExecutor
  participant ExecuteBatcher
  FetchLogSeries->>outsideSharedBatch: Wrap the log-series request
  outsideSharedBatch->>WarehouseExecutor: Route enqueue through executeAlone
  WarehouseExecutor->>ExecuteBatcher: Execute one request immediately
Loading
Merge Risk: Minimal · up to 7e9f1

No merge-blocking issue is identified. The change separates log-volume requests so slower log reads need not delay the other Home cards; deployment timing improvements remain to be confirmed.

Pre-merge checks: Passed 4

Check name Status Explanation
Description Check Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check Passed The title names the main change: the Home log volume read is sent on its own (via executeAlone and outsideSharedBatch) so it no longer holds the time-series and facets cards that share the /execute-ba…
Linked Issues check Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check Passed Check skipped because no linked issues were found for this pull request.
Full details: Title check

Explanation

The title names the main change: the Home log volume read is sent on its own (via executeAlone and outsideSharedBatch) so it no longer holds the time-series and facets cards that share the /execute-batch request. The changeset matches this: ExecuteBatcher and WarehouseExecutor gain executeAlone, outsideSharedBatch is added, and FetchLogSeries uses it. The title is specific and uses a conventional commit prefix.

Finishing Touches

Generate unit tests (beta)

  • Commit to this branch
  • Create a new PR
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

Share

Recent review info

Run configuration

  • Source mode: Auto
  • Repository: Unavailable
  • Branch: Unavailable
  • File: No configuration file from this repository applied
  • Commit: Unavailable
  • Layers: defaults
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: 969faa7e-541e-47d4-a5ea-9cbecd61a7ea
Commits

Reviewing files that changed from the base of the PR and between 356b041 and 7e9f155.

Files selected for processing: 1

  • apps/web/src/page/overview/command.test.ts
Files skipped from review as they are similar to previous changes: 1

  • apps/web/src/page/overview/command.test.ts

Included review availability: This review used your included allowance. Your plan provides up to 4 included reviews per hour; 2 remain after this review.

@github-actions

github-actions Bot commented Nov 12, 2026 •

Copy link
Copy Markdown
Contributor

🍁 Maple PR preview

Note

Preview resources were removed when the preview label came off.

Final commit 7e9f155 · View workflow run

@Makisuo Makisuo removed the preview:web Deploy a Workers-only PR preview (no ingest/Electric); see docs/pr-previews.md label Nov 12, 2026
@Makisuo
Makisuo deployed to preview-tinybird November 12, 2026 10:22 — with GitHub Actions Active
@maple-review-bot

maple-review-bot Bot commented Nov 12, 2026 •

Copy link
Copy Markdown

Maple review

🟢 Confidence 10/10 · safe to merge. Contained to one page's command; the new test pins logs-alone and fails if outsideSharedBatch or executeAlone is dropped.
no findings · tests covered · risk medium

Adds executeAlone to the execute batcher and outsideSharedBatch, so Home's log volume read goes out in its own /execute-batch request and the time series and facets stop waiting on it. The new test pins the split; safe to merge.

  • FetchLogSeries runs through outsideSharedBatch, so log volume skips the shared batch
  • ExecuteBatcher.executeAlone sends one query at once as a batch of one
  • WarehouseExecutor gains executeAlone, wired to the batcher in warehouse.ts
  • New command.test.ts pins the split: logs alone at 6 s, series at 300 ms

7e9f155 · What was checked, coverage and telemetry details are on the Maple / review check. Reply "won't fix" to dismiss a finding, or mention @maple-review-bot to ask about one.

@Makisuo
Makisuo merged commit b511820 into main Nov 12, 2026
33 checks passed
@Makisuo
Makisuo deleted the perf/home-log-volume-own-request branch November 12, 2026 11:04
@maple-review-bot

Copy link
Copy Markdown

Maple: after this shipped

✅ Shipped clean. Nothing this change touches got worse in the hour after it deployed.

Deployed to maple-web (production) at 2026-11-12 11:53 UTC as b511820. Compared the 60 minutes before and after.

Operation Calls/h Errors p95
FetchOverviewLogVolume 14 → 16 0.0% → 0.0% 3593 → 2169 ms

Maple looks once, an hour after the first deploy that carries the merge.

This branch was successfully deployed

1 active and 1 inactive deployments
pr-preview — 7e9f1553 Deployed Nov 12, 2026 by Makisuo via deploy-pr-preview #11606
preview-tinybird — 7e9f1553 Deployed Nov 12, 2026 by Makisuo via branch #1177
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant