Skip to content

Bring the identity stack onto the native EC stack - #1566

Open
dhruv8sh wants to merge 10 commits into
baseline/aws-examples-20261110from
feat/axum-identity-providers
Open

dhruv8sh wants to merge 10 commits into
baseline/aws-examples-20261110from
feat/axum-identity-providers

Conversation

@dhruv8sh

@dhruv8sh dhruv8sh commented Nov 12, 2026 •

Copy link
Copy Markdown
Collaborator

Stack order. This chain merges in this order (into rc/identity under DEC-20 and DEC-21 of #1559):

  1. Cut Trusted Server over to EdgeZero PR 275 #1506 < Answer identify and batch sync locally on every adapter #1614 < Allow explicit per-provider HTTP auction endpoints #1525 < Make shared EC persistence awaitable #1526 < Collect server-side identity provider EIDs in a single page-load write #1487 < Add identity providers for ID5, LiveIntent, LiveRamp, ConnectID and UID2 #1488 < Mirror provider EIDs to a cookie and run pull sync as an identity provider #1489
  2. prk-Jr's Say certificate checks cover every outbound HTTPS request #1616 < Standardize health and startup-failure responses across adapters #1617 < Restrict creative proxy identity disclosure #1615, and Add identity-only mode to the Prebid integration #1620, all on Mirror provider EIDs to a cookie and run pull sync as an identity provider #1489
  3. Stop pull sync from re-calling a partner for a full row #1639 < Send an EC to bidders only when its consented row was read #1640 < Send EIDs in OpenRTB 2.6 user.eids #1612 < Read SharedID from Prebid's configured cookie and send it once as pubcid.org #1628 < Build identity state once in a core identity app #1623, landing after item 2: Stop pull sync from re-calling a partner for a full row #1639 merges Restrict creative proxy identity disclosure #1615 and Add identity-only mode to the Prebid integration #1620, and Build identity state once in a core identity app #1623 carries the fix for its conflict with Standardize health and startup-failure responses across adapters #1617's health contract (not yet done)
  4. Christian's Extract native prepared startup on the demo baseline #1531 < Extract the protected CloudFront Geo baseline #1532 < Retain provider-local HTTP opt-in in the demo baseline #1533 < Separate shared async EC on the compatible demo baseline #1534 < Extract native EC lifecycle and DynamoDB persistence #1536 < Extract existing Sourcepoint consent synchronization #1535 < Extract explicit EC storage selection and celld #1537 < Separate the source-built native image #1539 < Separate the native AWS runtime wrapper #1538 < Unify the AWS example with CloudFront and durable Edge Cookie storage #1540, on Build identity state once in a core identity app #1623
  5. Bring the identity stack onto the native EC stack #1566 < Create and write back SharedID from the EC record #1624 < Move the lazily opened EC record store into core #1641 < Verify awaited identity lookups and save /auction body EIDs #1642, on Unify the AWS example with CloudFront and durable Edge Cookie storage #1540

Summary

This replaces the earlier content of this PR (an in-memory Axum ec_kv.rs and Axum identity providers). Providers on Axum are tracked in #1581.

Changes

File Change
Merge commits up Christian's chain (#1531 → #1540) Forward-merge of #1623. Kept from both sides: identify and batch-sync claim every method; managed EdgeZero logger; EdgeZero pinned to 9d02d7b3 for all crates (celld added); geo-kv; Axum grants the navigation permit and mints only for permitted browser navigations; startup runs both ec.storage resolution and the enabled-provider check.
crates/trusted-server-adapter-fastly/src/app.rs, ec_record_store.rs, main.rs Identity app's record store selected through ec.storage (legacy ec_store still works) and opened lazily; configure declares trusted_server_kv; tests use the registry-aware EdgeZero test runner.
crates/trusted-server-core/src/ec/finalize.rs, ec/kv.rs, ec/page_load.rs, ec/consent_update.rs Christian's withdrawal, collision and navigation rules restored; his tests restored, ours updated.
crates/trusted-server-adapter-axum/src/app.rs, tests/native_ec.rs Navigation permit wiring; his fixture restored.
docs/superpowers/specs/2026-10-30-s2s-eid-collection-design.md, docs/guide/edge-cookies.md State that from #1566 up Christian's shared async EC rules apply. Christian's own specs and guides are unchanged.

Closes

Refs #1567. Sits on #1540 (Christian's stack); #1624 stacks on top.

Spec

Christian's design docs: docs/superpowers/specs/2026-11-10-shared-async-ec-design.md, docs/superpowers/specs/2026-11-03-ec-storage-provider-configuration-design.md, docs/guide/native-ec-persistence.md; ours: docs/superpowers/specs/2026-10-30-s2s-eid-collection-design.md.

Test plan

  • cargo test-fastly && cargo test-axum (also test-fastly-reuse, test-cloudflare, test-spin, parity, build-digest, CLI tests)
  • cargo clippy-fastly && cargo clippy-axum (all clippy aliases)
  • cargo fmt --all -- --check
  • JS tests: cd crates/trusted-server-js/lib && npx vitest run
  • JS format: cd crates/trusted-server-js/lib && npm run format
  • Docs format: cd docs && npm run format
  • WASM build: cargo build --package trusted-server-adapter-fastly --release --target wasm32-wasip1
  • Manual testing: scripts/smoke-axum.sh and scripts/smoke-fastly.sh (Viceroy) pass
  • Other: scripts/lint-wasm-clock.sh

Checklist

  • Changes follow AGENTS.md conventions
  • No unwrap() in production code — use expect("should ...")
  • Uses log macros (not println!)
  • New code has tests
  • No secrets or credentials committed

@dhruv8sh dhruv8sh self-assigned this Nov 12, 2026
@dhruv8sh
dhruv8sh force-pushed the feat/axum-identity-providers branch from 53edc2f to 117a58e Compare November 12, 2026 14:43
@dhruv8sh
dhruv8sh changed the base branch from fix/1488-sharedid-prebid-alignment to feat/s2s-eid-vendors November 12, 2026 14:43

@jevansnyc jevansnyc left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed the Axum identity lifecycle, process-local KV/CAS behavior, auction wiring, and included SharedID changes. No blocking runtime issues found. One non-blocking documentation correction below. All reported CI checks pass. Local verification: cargo test-axum passed all 73 tests (including all six new identity tests); one documentation example is ignored.

Comment thread docs/guide/configuration.md Resolved

@ChristianPavilonis ChristianPavilonis left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Summary

Adds server-side identity collection to the Axum dev server with process-local generation-based CAS storage, and updates SharedID defaults and retention. Approved with one non-blocking documentation correction.

Non-blocking

🏕 camp site

  • Update contradictory provider documentation — see inline at docs/guide/configuration.md:926–929.

The inline comment is prose-only because the complete correction spans multiple files and lines outside the diff.

Verification

  • Focused configuration-validation scratch probe: PASS.
  • git diff --check: PASS.
  • Passing remote CI results reused; full local suites and deployed-runtime testing were not rerun.

CI Status

  • integration tests (Fastly EC lifecycle): PASS
  • integration tests: PASS
  • browser integration tests: PASS
  • trace runtime and browser acceptance: PASS
  • cargo test (ts CLI, native) (macos-latest): PASS
  • cargo test (ts CLI, native) (ubuntu-latest): PASS
  • cargo check (cloudflare native + wasm32-unknown-unknown): PASS
  • cargo test (cross-adapter parity): PASS
  • worker-deadlines (cloudflare): PASS
  • cargo test (axum native): PASS
  • ts dev lint domains (changed lines): PASS
  • cargo check/build/test (spin native + wasm32-wasip1): PASS
  • HTML EOF runtime (Fastly): PASS
  • CI Node pin guard: PASS
  • format-typescript: PASS
  • prepare integration artifacts: PASS
  • cargo test: PASS
  • vitest: PASS
  • cargo fmt: PASS
  • release scripts: PASS
  • CLAUDE.md symlink guard: PASS
  • docs-proposal scripts: PASS
  • format-docs: PASS
  • worker-deadlines (spin): PASS

GitHub reports no required checks for this branch.

Comment thread docs/guide/configuration.md Resolved
aram356 added a commit that referenced this pull request Nov 12, 2026


Per the user, every feature is built once and works on every adapter in
platform order, with platform gaps sent to EdgeZero (DEC-16, SH-5). T21
moves the identity routes, EC lifecycle, providers and write-back into one
router in core that each adapter mounts; A2, A10, T8, T9, T17 and T18
shrink to turning it on per adapter and passing the conformance suite, and
T7, T14 and A11 fold into A2 and A10. EZI-2a adds EdgeZero's local store
for Axum development and EZI-4 binds Axum's geo provider from the manifest.

Restack the Axum RC units on #1506: #1531 carries A1 in place of #1528,
#1533 and #1534 close as copies of #1525 and #1526, and the image and
Terraform units sit directly on #1532 so A6 and A7 stop waiting on A2.
Record #1566, refresh the provider stack's fixes at their current heads,
and flag #1535 as unplanned.
@aram356

aram356 commented Nov 13, 2026

Copy link
Copy Markdown
Collaborator

CREATOR: From the server-side identity plan (#1559, c7dc9de77): Move 0a2a00aaf (SharedID name, source and retention) and 0c13d4272 (drops the blanket provider rejection) into #1488. Rebuild the Axum part on T21's shared app and EdgeZero capabilities: no Axum-only EcKvStore (ec_kv.rs:87; ID-2, DEC-3), no Trusted Server rate limiter (none on Axum, DEC-11), no User-Agent-only gate (EZI-3, ID-7), and no detached block_on task (EZI-1). Local development uses EdgeZero's local store (EZI-2a). It then serves A2 and A10.

Part of #1574

@dhruv8sh dhruv8sh linked an issue Nov 13, 2026 that may be closed by this pull request
@dhruv8sh
dhruv8sh force-pushed the feat/axum-identity-providers branch from 117a58e to 3d62f4d Compare November 13, 2026 17:36
@dhruv8sh

Copy link
Copy Markdown
Collaborator Author

Moved 0a2a00aaf and 0c13d4272 into #1488 (now ecaeb0a44, f500c5249). #1488 had already dropped the blanket rejection, so the second is small. The SharedID "stored ID wins" rule is reverted there (518f88984): Prebid's cookie replaces a stored SharedID again.

#1566 now holds only the Axum work, at full EC-lifecycle parity with Fastly. The remaining differences (process-local store, UA-only bot gate, detached post-response task) are what T21/EdgeZero replaces, so this stays open until the rebuild on T21.

The merged identity app opened the record store when the sandbox started, so a missing or failing store turned every route into an assembly error. A lazy store keeps one store per sandbox but opens it on the first EC operation, so only EC work fails.

Without an identity graph a withdrawal expires the cookie because no row can exist; drop the redundant failure mark, since the consent-update endpoint already answers 503 without a graph.

Signed-off-by: dhruv8sh <dhruv8sh@proton.me>
Native (Axum) EC persistence now applies the page-load write gate and withdrawal rule, and refuses to start with an enabled provider.

Signed-off-by: dhruv8sh <dhruv8sh@proton.me>
Explicit withdrawal expires the EC cookie and strips identity headers even when a tombstone fails or no identity graph is available; the failure is logged and marks the consent-update response failed.

New issuance and orphan recovery retry an ID collision with a fresh ID, up to five Add attempts, inside the navigation's single page-load write, so a page load still writes the row once and the row is still written before the cookie.

The page-load permit now follows the shared navigation rule (Sec-Fetch-Dest document with the Accept fallback), so Axum and Fastly mint an EC on the same navigations as before the merge. Restore the affected pre-merge tests.

Signed-off-by: dhruv8sh <dhruv8sh@proton.me>
… spec

Native and Fastly EC follow the shared async EC minting, collision and withdrawal rules from 1566 up, not the page-load gate and withdrawal rule this spec describes for PR A.

Signed-off-by: dhruv8sh <dhruv8sh@proton.me>
From #1566 up an EC is minted, cookies persist and pull sync runs on any browser document navigation (Sec-Fetch-Dest: document, or an Accept header naming text/html without Fetch Metadata), an ID collision retries inside the navigation's one write, and an explicit withdrawal expires ts-ec even when its tombstone fails. The edge cookies guide still described the Fetch Metadata-only rule and the kept cookie.

Signed-off-by: dhruv8sh <dhruv8sh@proton.me>
@dhruv8sh
dhruv8sh force-pushed the feat/axum-identity-providers branch from 3d62f4d to 32d83a9 Compare November 20, 2026 01:48
The workspace pins EdgeZero by rev, so the job's tag-only lookup failed. A full commit SHA is as immutable as a tag; branches are still rejected.

Signed-off-by: dhruv8sh <dhruv8sh@proton.me>
@dhruv8sh
dhruv8sh removed this pull request from stack #1629 November 20, 2026 01:58
@dhruv8sh dhruv8sh changed the title Run server-side identity collection on the Axum dev server Bring the identity stack onto the native EC stack Nov 20, 2026
@dhruv8sh
dhruv8sh changed the base branch from feat/s2s-eid-vendors to baseline/aws-examples-20261110 November 20, 2026 01:58
@dhruv8sh
dhruv8sh added this pull request to stack #1645 November 20, 2026 01:58
The native EC stack wires the Axum EC lifecycle, identify and partner registry directly, as Fastly does. List them as pending lifecycle moves so the guard tracks them until the core identity app owns those routes.

Signed-off-by: dhruv8sh <dhruv8sh@proton.me>
Signed-off-by: dhruv8sh <dhruv8sh@proton.me>
Signed-off-by: dhruv8sh <dhruv8sh@proton.me>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Collect identity provider EIDs server-side for the server-side auction Write Lockr IDs to KV store

4 participants