Skip to content

feat(mcp-cloudflare): Cache Sentry API responses in memory behind MCP_RESPONSE_CACHE - #1833

Open
sentry-junior[bot] wants to merge 6 commits into
mainfrom
feat/hosted-response-cache-memory
Open

sentry-junior[bot] wants to merge 6 commits into
mainfrom
feat/hosted-response-cache-memory

Conversation

@sentry-junior

@sentry-junior sentry-junior Bot commented Nov 3, 2026

Copy link
Copy Markdown
Contributor

Task 7.1 of the #1669 plan, plus the 7.5 metrics. It wires the shared memory store (#1696) and cache policy (#1688) into the hosted MCP behind MCP_RESPONSE_CACHE.

Off by default. Only memory or tiered turns it on. An unset, off, or unknown value means no caching. This PR changes no wrangler config, so every deployed environment stays uncached. tiered behaves like memory until the R2 tier lands in the stacked follow-up.

Identity isolation. Each cache key's identity is a SHA-256 of ["mcp-grant", userId, grantId]. Both values come from the validated wrapper token (userId:grantId:secret), so a grant never reads another grant's entries, even for the same user and URL. The identity survives a wrapper-token refresh. Requests without a grant ID are not cached, and that includes Sentry-Bearer direct tokens. In mcp-core, SentryApiService takes an optional responseCache and sends through a caching transport bound to its token. apiServiceFromContext drops the cache whenever a credentialRouter is set, because routed calls run with other tokens.

Storage. There is one 24 MiB memory store per isolate. Writes run in ctx.waitUntil(). Non-JSON bodies, such as attachments, are skipped before they are read. A write invalidates only the writing grant's entries.

Isolation tests:

  • Unit tests in workerd run two SentryApiService instances over one shared store, with an upstream that answers per Authorization. The same URL never returns the other identity's body, and a write by A leaves B's entry in place.
  • Handler tests send real /mcp tool calls. The same grant hits the cache. Another grant for the same user, another user, a malformed wrapper token, and a Sentry-Bearer token all go to the API. With the flag unset, nothing is cached.

Metrics (7.5). app.response_cache.lookup (hit/miss/stale), app.response_cache.write (with the body encoding), app.response_cache.eviction, and two gauges: memory.bytes and memory.entries. The first budget eviction in an isolate also writes a one-time info log. They are documented in docs/operations/monitoring.md, and the cache is described in docs/cloudflare/architecture.md. toolkit-core's memory store gains an onEvict callback for this.

Pending, needs a deployed Worker (CI deploys only from main, so this PR gets no preview):

  • 7.2 zstd: local workerd (miniflare 5.20261019.1-beta) has node:zlib zstd. On a deployed Worker with MCP_RESPONSE_CACHE=memory, app.response_cache.write grouped by app.response_cache.encoding should show zstd. If it only shows br, brotli stays the hosted codec, and the doc needs updating to say so.
  • 7.5 preview check: on the same Worker, confirm that the metrics above appear after a few repeated tool calls.
  • 7.4 is a separate human sign-off. Don't set the flag in production before it.

Not in scope: the fresh input (6.3). When it lands, its fresh() callback goes on createCachingTransport in mcp-core/src/api-client/client.ts.

via @BYK.

--

View Session [Sentry]

…_RESPONSE_CACHE

Wire the shared byte-budgeted memory store into the hosted MCP. The cache is
off unless MCP_RESPONSE_CACHE is `memory` or `tiered` (memory only until the
R2 tier lands). Entries are keyed by a SHA-256 of the MCP grant's user and
grant IDs from the wrapper token, so one grant never reads another's
responses; requests without a grant, such as Sentry-Bearer direct tokens,
are not cached.

SentryApiService takes an optional responseCache and sends through a
caching transport for its token. apiServiceFromContext drops it when a
credential router can swap tokens. Writes run in waitUntil.

Adds hit/miss/stale lookup, write (with encoding), and eviction metrics,
memory gauges, and a one-time log when an isolate reaches its budget.

Co-Authored-By: BYK <126780+BYK@users.noreply.github.com>
@vercel

vercel Bot commented Nov 3, 2026 •

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated
sentry-local Ready Ready Preview Nov 13, 2026 2:47pm UTC

Request Review

@cursor cursor Bot left a comment •

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Stale Bugbot comment from a previous run.

Cancel the transport's body copy whenever the cache does not read it,
including JSON responses the policy refuses to store. Let
getResponseCache infer its return type: CI lints before building
mcp-core, so the ResponseCache | undefined union tripped
no-redundant-type-constituents.

Co-Authored-By: BYK <126780+BYK@users.noreply.github.com>
Resolve conflicts with the @sentry/api client move (#1942):
- mcp-core client.ts: keep main's @sentry/api/client imports and
  `fresh: isFreshRead`, plus this PR's shared `isIdempotent` and caching
  transport, which now also passes `fresh`.
- toolkit-core memory-store: keep main's deletion. The memory store now
  comes from @sentry/api/client, which has no eviction hook, so
  createMeteredMemoryStore derives evictions from entry counts around
  each write.

Also point the cache at @sentry/api/client, add it to mcp-cloudflare's
dependencies, and call whoami directly in the handler tests now that
every tool is exposed.

@cursor cursor Bot left a comment •

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Stale Bugbot comment from a previous run.

The memory store drops oversized entries, but the write metric still
counted them, so app.response_cache.write and its encoding breakdown
included bodies that were never cached. Count a write only when the
store kept the entry.
Resolve the SentryApiService constructor conflict by keeping the response
cache transport and using main's API_BASE_PATH. Take main's lockfile,
with @sentry/api 2.0.0, and add only the mcp-cloudflare @sentry/api entry.

@cursor cursor Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes and found 2 potential issues.

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit 9f294c7. Configure here.

A GET's cache write runs in waitUntil after the response returns. If a
mutation invalidated while that write was still running, the write landed
afterwards and served the pre-mutation body until its TTL. Track each
store's running writes per identity, and have invalidate wait for them
before it deletes.

Also test that a mutation sent to a region host clears that host's
entries. Invalidation prefixes come from the mutation URL's own origin, so
apiBaseUrl only adds the control-host legacy issue path.

This branch was successfully deployed

1 active deployment
Preview — 74aa5ddb Deployed Nov 13, 2026 by vercel[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

risk: medium PR risk score: medium

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant