Skip to content

Answer identify and batch sync locally on every adapter - #1614

Open
dhruv8sh wants to merge 11 commits into
spec/edgezero-pr275-migrationfrom
feat/identify-batch-sync-local-all-adapters
Open

dhruv8sh wants to merge 11 commits into
spec/edgezero-pr275-migrationfrom
feat/identify-batch-sync-local-all-adapters

Conversation

@dhruv8sh

Copy link
Copy Markdown
Collaborator

Summary

  • Partner API requests leaked to the publisher origin. On Axum, Cloudflare and Spin, GET /_ts/api/v1/identify and POST /_ts/api/v1/batch-sync had no route, so they fell through to the publisher fallback, carrying the partner's Authorization: Bearer token and mapping body with them. On every adapter, Fastly included, any other method on those paths did the same. This PR answers every request on both paths locally (ID-3 of the server-side identity spec).
  • With an EC record store, identify and batch sync run as before (Fastly only today). Without one, they return a local 501 {"error":"ec_store_not_configured"}, and Fastly with ec.ec_store unset now returns 501 instead of its old 503 store error. Every other method returns 405 with an Allow header (GET, OPTIONS for identify, POST for batch sync). The store-less identify 501 applies the same origin policy as the store-backed path: 403 for a denied browser origin, and reflected CORS headers for an allowed one so the browser can read the 501.
  • A new CapturingOrigin test fixture records every request that reaches the publisher fallback. Each adapter's route tests use it to prove that no identify or batch-sync request, on any method, reaches the origin. A positive control shows that a normal page request does reach it.

Changes

File Change
crates/trusted-server-core/src/ec/identify.rs identify_not_supported(settings, req): 501 behind the identify origin policy. identify_method_not_allowed(): 405 with Allow: GET, OPTIONS. answer_identify_locally(): picks the response by method. Tests
crates/trusted-server-core/src/ec/batch_sync.rs batch_sync_not_supported() (501), batch_sync_method_not_allowed() (405 with Allow: POST), answer_batch_sync_locally(). Tests
crates/trusted-server-core/src/test_support.rs capturing_origin::CapturingOrigin fixture, behind test/test-utils
crates/trusted-server-adapter-fastly/src/app.rs Both paths claim every method. Without ec.ec_store, answers 501 (was 503). Other methods get 405. Test-only #[cfg(test)] services_override on AppState, so route tests can dispatch through the capturing origin. Tests with and without a store
crates/trusted-server-adapter-axum/src/app.rs, tests/routes.rs Routes both paths on every method to the core local answers. Tests
crates/trusted-server-adapter-cloudflare/src/app.rs, tests/routes.rs Same as Axum
crates/trusted-server-adapter-spin/src/app.rs, tests/routes.rs Same as Axum. Removes the stale comment that called these paths intentionally unrouted
docs/guide/api-reference.md Route availability table (identify GET and OPTIONS rows split). Identify and batch-sync contracts updated for 501, 405 and the origin policy

Notes:

  • The store-present ("runs") tests exist only on Fastly. Axum, Cloudflare and Spin have no EC record store on main, so they always answer 501. Plan tasks T8, T9 and A2 bring stores to those adapters.
  • services_override compiles out of non-test builds. Only the Fastly route tests set it.
  • Fastly answers 501 only when ec.ec_store is unset. A configured but unlinked store behaves as before: identify returns a degraded 200 and batch sync rejects items as kv_unavailable.
  • Admin lookup (already 501 locally) and pull sync (outbound only) are unchanged.

Closes

Closes #1582

Spec

Spec: server-side identity on every adapter, ID-3 (plan task T1 of #1558)

Test plan

  • justrust cargo test-fastly && justrust cargo test-fastly-reuse && justrust cargo test-axum && justrust cargo test-cloudflare && justrust cargo test-spin
  • justrust cargo clippy-fastly && justrust cargo clippy-axum && justrust cargo clippy-cloudflare && justrust cargo clippy-cloudflare-wasm && justrust cargo clippy-spin-native && justrust cargo clippy-spin-wasm
  • justrust cargo fmt --all -- --check
  • JS tests: cd crates/trusted-server-js/lib && npx vitest run (Node 24.20.0 via mise)
  • JS format and lint: cd crates/trusted-server-js/lib && npm run format && npm run lint
  • Docs format: cd docs && npm run format
  • WASM build: cargo build --package trusted-server-adapter-fastly --release --target wasm32-wasip1
  • Manual testing via fastly compute serve
  • Other: capturing-origin route tests on all four adapters (store absent: 501; store present on Fastly: real handlers, 401; every other method: 405; origin receives nothing)

Checklist

  • Feature PRs: spec linked above, ending in a review log of two or more agent review rounds
  • Changes follow AGENTS.md conventions
  • No unwrap() in production code — use expect("should ...")
  • Uses log macros (not println!)
  • New code has tests
  • No secrets or credentials committed

Adapters without an EC record store answer these partner APIs locally instead of proxying them to the publisher origin. Also add a capturing origin fixture so adapter route tests can prove a request never reaches the origin.

Signed-off-by: dhruv8sh <dhruv8sh@proton.me>
The dev server has no EC record store, so identify and batch sync return 501 and the identify preflight is answered locally; none reach the publisher origin.

Signed-off-by: dhruv8sh <dhruv8sh@proton.me>
Cloudflare has no EC record store, so identify and batch sync return 501 and the identify preflight is answered locally; none reach the publisher origin.

Signed-off-by: dhruv8sh <dhruv8sh@proton.me>
Spin has no EC record store, so identify and batch sync return 501 and the identify preflight is answered locally; none reach the publisher origin.

Signed-off-by: dhruv8sh <dhruv8sh@proton.me>
Without ec.ec_store, identify and batch sync now return the local 501 instead of a 503 store error. Route tests dispatch through a capturing origin to prove neither request reaches the publisher, with and without a record store.

Signed-off-by: dhruv8sh <dhruv8sh@proton.me>
Signed-off-by: dhruv8sh <dhruv8sh@proton.me>
Identify and batch sync now claim every method on Fastly, Axum, Cloudflare and Spin. Methods other than GET/OPTIONS on identify and POST on batch sync answer 405 with an Allow header, so no request or partner Bearer token on these paths reaches the publisher origin.

The store-less identify 501 now applies the identify origin policy: a denied origin gets 403 and an allowed one gets reflected CORS headers. Also document the panic contract on batch_sync_not_supported.

Signed-off-by: dhruv8sh <dhruv8sh@proton.me>
Signed-off-by: dhruv8sh <dhruv8sh@proton.me>
@dhruv8sh dhruv8sh self-assigned this Nov 16, 2026
Signed-off-by: dhruv8sh <dhruv8sh@proton.me>

@ChristianPavilonis ChristianPavilonis left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Summary

The adapter routing consistently keeps identify and batch-sync requests out of publisher fallback. Store-less responses reuse the shared method and origin policies, while Fastly retains the store-backed handlers. Capturing-origin tests include positive controls across all four adapters.

Reviewed head 3228f4cccf74aae9c8a21dd67ff0796b7ce38d3f against merge base 1810c763add220dd34fd5faaffe2c69aab1d556f.

CI Status

  • integration tests (Fastly EC lifecycle): PASS
  • integration tests: PASS
  • browser integration tests: PASS
  • trace runtime and browser acceptance: PASS
  • CodeQL: PASS
  • cargo check/build/test (spin native + wasm32-wasip1): PASS
  • cargo check (cloudflare native + wasm32-unknown-unknown): PASS
  • Analyze (javascript-typescript): PASS
  • Analyze (actions): PASS
  • vitest: PASS
  • ts dev lint domains (changed lines): PASS
  • cargo test (ts CLI, native) (ubuntu-latest): PASS
  • HTML EOF runtime (Fastly): PASS
  • cargo test: PASS (required)
  • cargo test (cross-adapter parity): PASS
  • cargo test (axum native): PASS
  • docs-proposal scripts: PASS
  • Analyze (rust): PASS
  • worker-deadlines (spin): PASS
  • prepare integration artifacts: PASS
  • format-docs: PASS (required)
  • cargo fmt: PASS (required)
  • release scripts: PASS
  • CLAUDE.md symlink guard: PASS
  • format-typescript: PASS (required)
  • CI Node pin guard: PASS
  • cargo test (ts CLI, native) (macos-latest): PASS
  • worker-deadlines (cloudflare): PASS
  • Analyze (javascript-typescript): PASS
  • Analyze (python): PASS

T1 now builds on the EdgeZero cutover, so the partner API change ports onto
#1506's adapters:

- Core batch sync keeps T1's local 501 and 405 answers next to #1506's async
  handle_batch_sync_with_writer.
- Fastly takes #1506's per-request services. The identify and batch-sync
  handlers answer locally unless the method is the primary one and an EC
  record store is configured, and both paths claim every method.
- The route tests drop the CapturingOrigin fixture, which implemented the
  platform HTTP client #1506 removed. They route through #1506's scripted
  outbound client instead and assert that no partner API request reaches
  it; a control test shows a publisher page does reach it.
@aram356
aram356 changed the base branch from main to spec/edgezero-pr275-migration November 17, 2026 18:03
aram356 added a commit that referenced this pull request Nov 17, 2026
#1614 now carries #1506's EdgeZero cutover, so #1525's per-provider HTTP
opt-in meets the cutover here, resolved the way rc/identity resolved it in
30a74de:

- The endpoint rule is the union: https, plain http to loopback (#1506), or
  any http with allow_insecure_http (#1525).
- Cloudflare keeps #1506's startup_loads_multi_provider_plan; the dormant
  multi-provider test is dropped because the cutover loads every plan.
- #1506's test ProviderConfig literals in the Fastly app and the auction
  orchestrator gain allow_insecure_http: false.
@aram356
aram356 added this pull request to stack #1621 November 17, 2026 18:05
aram356 added a commit that referenced this pull request Nov 17, 2026
#1525 now carries #1506's EdgeZero cutover and #1614, so #1526's awaitable EC
persistence meets them here. The resolution is the one rc/identity made in
30a74de and 4a414a0:

- Fastly runs the async EC finalize from finalize_response, which EdgeZero
  calls after its dispatch executor returns, and drops edgezero_main.
- Pull sync and batch sync keep #1506's EdgeZero send paths with #1526's
  awaits, and their KV-touching tests become async, including #1506's
  pull-sync group failure test.
- The post-delivery test blocks on the now-async snapshot load.

#1614's identify handler awaits #1526's async handle_identify.
@aram356

aram356 commented Nov 17, 2026

Copy link
Copy Markdown
Collaborator

This PR now builds on #1506's EdgeZero cutover, so it can join the identity integration branch (#1608) as a plain merge. 9627ca12a merges #1506's head 47a539528 into this branch and ports T1 onto the cutover; git show --remerge-diff 9627ca12a shows the port.

Area Port
Core batch sync Keeps this PR's batch_sync_not_supported, batch_sync_method_not_allowed and answer_batch_sync_locally next to #1506's async handle_batch_sync_with_writer
Fastly Takes #1506's per-request services. Identify and batch sync answer locally unless the method is the primary one and an EC record store is configured, and both paths claim every method, as before
Spin The two handlers take the request with ctx.into_request()?, the cutover's request API
Route tests, all four adapters CapturingOrigin implemented the platform HTTP client #1506 removed, so it is dropped. The tests route through #1506's scripted outbound client and assert that no partner API request reaches it. The control test that shows a publisher page does reach the origin now sends a body-less GET, because the cutover refuses a GET that carries body bytes

The base is now spec/edgezero-pr275-migration, so the diff still shows only this PR's changes, and the PR lands after #1506. Checked locally at 9627ca12a: cargo fmt, the six adapter clippy aliases, test-axum (63), test-cloudflare (65), test-spin (88) and test-fastly (3,729), all passing.

The chain is GitHub stack 1621: #1506, #1614, #1525, #1526, with the provider stack (#1487 to #1489) to join its top once it is ported.

The partner API requests target publisher.example.com, and the fallback control test scripts the origin URL from the test settings, as the cutover's other Fastly tests do.
aram356 added a commit that referenced this pull request Nov 17, 2026
Takes #1614's 9552a89, which moves its ported Fastly partner API tests onto allowlisted example hosts.
aram356 added a commit that referenced this pull request Nov 17, 2026
Takes #1525's f5d8b7f, which brings #1614's move of its ported Fastly partner API tests onto allowlisted example hosts.
aram356 added a commit that referenced this pull request Nov 17, 2026
… rc/identity

#1526's head b53c6e2 now stacks on #1525 (f5d8b7f), which stacks on #1614 (9552a89), which stacks on #1506. Each PR merged the one below it and resolved its own conflicts, so this merge needs no resolution. #1525's and #1526's resolutions against #1506 are the ones this branch made in 30a74de and 4a414a0, and #1614 (T1) is new here.
aram356 added a commit that referenced this pull request Nov 17, 2026
Brings PR #1487 onto PR #1526's base (PR #1506's EdgeZero cutover, PR #1614,
PR #1525 and PR #1526's awaitable EC persistence) and keeps PR #1487's
behavior in the new base's structure.

How each area was resolved:

- Provider lookups: the lookup transport now sends through EdgeZero's
  outbound client in the auction fan-out's shape. `LookupTransport::start`
  takes every built request and starts them as one batch with
  `HttpClient::start_batch_until` and a cutoff of the identity budget,
  taken on the client's own clock;
  `gather` reads the batch until it finishes, maps slots back to providers
  and reports unresolved slots as timeouts. `PlatformLookupTransport` and the
  coordinator's web-time `Deadline` are replaced by `OutboundLookupTransport`;
  requests keep the 64 KiB response cap, cache bypass and strict TLS.
- Deleted modules: `platform/http.rs` and the Fastly `platform/deadline.rs`
  stay deleted. PR #1487's final-snapshot change to the Fastly deadline
  select (a 1 ms surfacing wait per pending body and counting arrived headers
  as progress) has no counterpart in EdgeZero's batch and is not ported, nor
  is its doc-only change to `select_until`.
- Async KV: `ec/kv.rs`, `ec/mod.rs` and `ec/finalize.rs` take PR #1487's
  versions converted to PR #1526's async store; the page-load and
  after-response writes, the write planner, after-response commit, batch
  sync's provider-owned check and their tests await the store. Unchanged
  tests take PR #1526's async versions, and PR #1526's non-Send yielding
  store test is kept. `generate_if_needed` stays async as on the base.
- Fastly entry point: `main.rs` takes the base's lifecycle. `finalize`
  collects the navigation's page-phase lookups inside the async EC finalize
  under its one `block_on`, using the partner registry built once in
  `AppState`. After-response lookups run in post-delivery work, after pull
  sync, on the post-delivery client and clock. Lookups still travel through
  the request-local slot, because an in-flight batch is not `Send`; the slot
  is now cleared before each request and after delivery.
- Fastly app: keeps the base's local 501/405 identify and batch-sync answers,
  publisher origin options and response-body scope owner, with PR #1487's
  partner registry, identity coordinators, navigation permit and dispatch.
- Other adapters, core call sites and tests: kept both sides; PR #1487's
  tests that used the removed platform HTTP client or a custom auction
  provider now use the scripted outbound client, and the Fastly identity
  tests route with the host geo lookup the entry point's `prepare` runs.
- Cargo: the lockfile adds only the identity-providers crate to the base's.
- Docs and CHANGELOG: auction collection text follows the base; PR #1487's
  identity entries and spec now describe the EdgeZero batch transport.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Answer identify and batch sync locally on every adapter

3 participants