Skip to content

Allow explicit per-provider HTTP auction endpoints - #1525

Open
ChristianPavilonis wants to merge 6 commits into
feat/identify-batch-sync-local-all-adaptersfrom
review/http-auction-provider
Open

ChristianPavilonis wants to merge 6 commits into
feat/identify-batch-sync-local-all-adaptersfrom
review/http-auction-provider

Conversation

@ChristianPavilonis

@ChristianPavilonis ChristianPavilonis commented Nov 11, 2026 •

Copy link
Copy Markdown
Collaborator

Summary

  • Extract the existing demo's provider-local HTTP opt-in without importing relay ancestry.
  • Keep HTTPS as the default and retain credentials, fragment, scheme and provider-isolation checks.
  • Preserve the completed two independent contract reviews; this is not production plaintext approval.

Ready for feature review. All29 current-head hosted checks passed at cd8977c. This is not deployment or production readiness.

Stack and exact input

Head: cd8977cad39d8388d99f6ff3f93083ba880deb3e. Base: main.

Retain the original demo RC #1424 and its dirty worktrees. No source from excluded #1499 or the independent #1506 implementation is imported or coordinated. Relay/privacy work remains separate. No deployment, live input change, main merge, old-RC retirement or recovery deletion is part of this PR.

Changes

Area Change
Core auction plan/configuration and provider literals Add the default-false per-provider flag and scoped validation regressions.
Affected adapter/integration callers Retain explicit false defaults.
Guide/spec Document the exception and original reviewed inputs.

Closes

Closes #1521.

Spec

HTTP auction opt-in contract and completed review log. The two reviews examined the original immutable delta, after implementation and before readiness; they do not assert execution of this replayed head.

Test plan

The inherited HTTP fixture now uses pbs.example.com instead of a TEST-NET IP rejected by the repository domain linter. All27 focused auction-plan tests and strict Core Clippy passed on that test-only correction; no allowlist/suppression, behavior or assertion was relaxed. Other unchanged-input results below remain bounded to their recorded source.

Existing focused plan/adapter receipts and full source reviews belong to their recorded earlier inputs. The feature was cleanly replayed onto current main6fbe64aa. All29 current-head hosted checks passed. These results establish this isolated feature's review eligibility, not the later native combination or deployment qualification.

  • Required current-head hosted gates passed; raw eligibility retained.
  • Initial domain-lint failure and exact test-only correction are retained; ignored tests are not passes.

Checklist

  • Scope is the existing demo extraction and minimum caller adaptation.
  • Feature contract/review history is linked where applicable, without backdating approval.
  • Synthetic test/operator examples only; no credentials, state or licensed Geo inputs added.
  • Original work, PR identities and parked experiments remain preserved.
  • Isolated feature eligibility and affected-delta review complete; no whole-demo claim.

@ChristianPavilonis
ChristianPavilonis marked this pull request as ready for review November 11, 2026 01:13
@aram356
aram356 added this pull request to stack #1542 November 11, 2026 06:48

@prk-Jr prk-Jr left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

APPROVE — no actionable findings at cd8977cad39d8388d99f6ff3f93083ba880deb3e, reviewed against 6fbe64aa056a17d32b272d67df3dd89b21ce895f. Both revisions remained unchanged at the final check.

The HTTP exception stays local to the configured provider. HTTPS remains the default; credentials, fragments, and unsupported schemes remain rejected. Configuration loading preserves the flag, and dispatch preserves the validated scheme across all four adapters.

Evidence and limits:

  • All 29 current-head hosted checks pass.
  • Hosted logs confirm all 27 auction-plan tests passed, including the three new regressions.
  • Local git diff --check passed.
  • No local build or live HTTP probe was run. Large unrelated, unchanged test bodies were sampled.

CI Status

@aram356 aram356 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Summary

The validation change in canonicalize_endpoint is correct and stays scoped to one provider: an opted-in provider's flag doesn't leak to others, and credentials, fragments and other schemes stay rejected. I confirmed it works end to end on Axum and on Fastly (Viceroy). What's missing is around the change. Operators get no signal that a provider is sending in plaintext, the tests don't cover the transport, and the CHANGELOG, configuration reference and spec provenance don't match the change.

5 of the inline comments below carry a one-click GitHub suggestion. Use Commit suggestion (or Add suggestion to batch for several at once) to apply them as commits on the PR branch. The remaining comments describe the fix in prose because the change touches lines outside the diff or other files.

Blocking

🔧 wrench

  • Nothing tells the operator a provider is running in plaintext: see inline at crates/trusted-server-core/src/auction/plan.rs:326
  • Tests don't check how an opted-in provider is dialed, or that HTTPS isn't downgraded: see inline at crates/trusted-server-core/src/auction/plan.rs:1156
  • Spec provenance cites commits that don't exist in this repository: see inline at docs/superpowers/specs/2026-11-10-http-auction-provider-opt-in-design.md:7
  • No CHANGELOG entry, and pushing the flag breaks rollback to older binaries: see body-level below
  • Configuration reference and three other docs still say HTTPS-only: see body-level below

Non-blocking

♻️ refactor / ⛏ nitpick

  • Field doc comment: summary/details split and rollback rationale: see inline at crates/trusted-server-core/src/auction/plan.rs:139
  • Guide table understates plaintext exposure: see inline at docs/guide/auction-orchestration.md:786
  • is_false duplicates settings::is_false: see inline at crates/trusted-server-core/src/auction/plan.rs:611
  • Scheme error message says HTTPS for opted-in providers: see inline at crates/trusted-server-core/src/auction/plan.rs:585

Cross-cutting / body-level findings

  • 🔧 No CHANGELOG entry, and pushing the flag breaks rollback to older binaries. ProviderConfig has #[serde(deny_unknown_fields)], so a pushed blob with allow_insecure_http = true won't load on any binary that predates the field. That needs the same upgrade/rollback ordering the CHANGELOG gives sanitize_creatives. The Unreleased breaking config-first entry also still says "Provider endpoints must be absolute HTTPS URLs." Proposed ### Added entry:

    - Added per-provider `allow_insecure_http` (default `false`) under `[auction.providers.<id>]`, which lets an explicitly approved demo or test provider use a plain `http://` endpoint. Other providers still require HTTPS, and credentials, fragments and other schemes remain rejected. Requests to an opted-in provider carry client IP, User-Agent, geo, consent signals and identifiers in cleartext, and its responses can be altered in transit. Upgrading: binaries that predate the field reject a blob that sets it, so upgrade the binary first, then push the config. Rollback: remove the flag, push the default-compatible blob, then roll back.

    Also amend the breaking entry's sentence to "Provider endpoints must be absolute HTTPS URLs unless the provider sets allow_insecure_http."

  • 🔧 Configuration reference and three other docs still say HTTPS-only. The PR updates only docs/guide/auction-orchestration.md. Operators configuring providers read these too:

    • docs/guide/configuration.md:2444-2452: the provider-field table has no allow_insecure_http row, and its endpoint row says "Absolute HTTPS URL with host and no credentials or fragment". Proposed rows (then run npm run format:write in docs/):

      | `endpoint`            | Yes      | None            | Absolute HTTPS URL with host and no credentials or fragment; plain HTTP only with `allow_insecure_http` |
      | `allow_insecure_http` | No       | `false`         | Accept a plain `http://` endpoint for this provider only; approved demo/test endpoints only             |
    • docs/guide/configuration.md:2386: "All provider endpoints must be absolute HTTPS URLs." Suggest: "All provider endpoints must be absolute HTTPS URLs, except a provider that sets allow_insecure_http for an approved demo or test endpoint."

    • crates/trusted-server-core/src/auction/README.md:160: same sentence. Suggest: "Provider endpoints must be absolute HTTPS URLs unless the provider sets allow_insecure_http for an approved demo or test endpoint."

    • docs/guide/error-reference.md:92: the solution for this error. Add: "Plain HTTP is accepted only when that provider sets allow_insecure_http = true, for an approved demo or test endpoint."

  • 📌 The CLI drops core warnings. trusted-server-cli depends on log but installs no logger, so anything core logs during ts config validate / ts config push is discarded. That includes the existing publisher.public_origin HTTP warning and the warning proposed at plan.rs:326. With the real ts binary, validate and push printed only their success lines for an opted-in HTTP provider. Worth a follow-up issue: either install a stderr logger at warn, or return warnings from validation.

  • 🤔 The flag accepts any host, including public ones, so plaintext auction traffic can cross the internet. The spec deliberately leaves allowlists out of scope. If demos only need a local or private-network PBS, restricting http to loopback or private addresses would bound the exposure without losing that use case. The public_origin warning already distinguishes loopback hosts the same way.

  • 📝 PR description. It has typos ("All29", "main6fbe64aa") and internal process notes (relay ancestry, retained dirty worktrees, excluded PR identities) that a reviewer can't act on. A plain summary of the behavior change, the default and the rollback note would serve readers better.

Verification

  • Real binaries at cd8977cad: ts config validate rejects an http:// provider endpoint without the flag and accepts it with the flag, and ts config push keeps "allow_insecure_http":true in the blob. A live POST /auction on Axum and on Fastly (Viceroy) reached a stub bidder over plaintext and returned its bid. Fastly registered a non-TLS dynamic backend (backend_http_127_0_0_1_…_p_pbs-main_…).
  • Mutation testing: 9 mutants of plan.rs against the full trusted-server-core lib suite. 7 were killed and 2 survived (see inline at plan.rs:1156). With that suggestion applied, both are killed.
  • Each suggestion was verified alone (fmt, clippy-fastly, core lib tests, prettier, domain lint) and all together through every clippy and test alias plus the parity suite.

CI Status

  • Analyze (actions): PASS
  • Analyze (javascript-typescript): PASS
  • Analyze (javascript-typescript): PASS
  • Analyze (python): PASS
  • Analyze (rust): PASS
  • CI Node pin guard: PASS
  • CLAUDE.md symlink guard: PASS
  • CodeQL: PASS
  • HTML EOF runtime (Fastly): PASS
  • browser integration tests: PASS
  • cargo check (cloudflare native + wasm32-unknown-unknown): PASS
  • cargo check/build/test (spin native + wasm32-wasip1): PASS
  • cargo fmt: PASS (required)
  • cargo test: PASS (required)
  • cargo test (axum native): PASS
  • cargo test (cross-adapter parity): PASS
  • cargo test (ts CLI, native) (macos-latest): PASS
  • cargo test (ts CLI, native) (ubuntu-latest): PASS
  • docs-proposal scripts: PASS
  • format-docs: PASS (required)
  • format-typescript: PASS (required)
  • integration tests: PASS
  • integration tests (Fastly EC lifecycle): PASS
  • prepare integration artifacts: PASS
  • trace runtime and browser acceptance: PASS
  • ts dev lint domains (changed lines): PASS
  • vitest: PASS
  • worker-deadlines (cloudflare): PASS
  • worker-deadlines (spin): PASS

Comment on lines +321 to +326
let endpoint = canonicalize_endpoint(
&id,
registration.id,
&raw.endpoint,
raw.allow_insecure_http,
)?;

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔧 wrench: Nothing tells the operator that a provider is running in plaintext.

An opted-in provider's requests carry the client IP (device.ip, plus X-Forwarded-For for the prebid-server profile), geo (lat/lon for PBS), User-Agent, the TCF string and consent cookies, and the EC ID and EIDs when present. Its responses (price, adm, nurl/burl) can be rewritten in transit, and sanitize_creatives defaults to off. I ran opted-in configs end to end on Axum and on Fastly under Viceroy. ts config validate, ts config push and the adapter runtime all stayed silent while the stub bidder received a plaintext POST /openrtb2/auction carrying those fields.

The repo already warns for the equivalent publisher-origin case (publisher_origin_diagnostics, config.rs:89). Logging at plan compile covers deploy validation and every adapter's startup from one place. On Fastly it fires once per instance, like the public_origin warning. CodeQL may flag the new log site as rust/cleartext-logging (Settings taint); the comment follows the convention at the orchestrator's provider-ID log sites. The CLI installs no logger, so this reaches adapter logs only (see the 📌 in the review body).

Suggested change
let endpoint = canonicalize_endpoint(
&id,
registration.id,
&raw.endpoint,
raw.allow_insecure_http,
)?;
let endpoint = canonicalize_endpoint(
&id,
registration.id,
&raw.endpoint,
raw.allow_insecure_http,
)?;
if endpoint.url().scheme() == "http" {
// This logs a configured provider identifier, not request data or secrets.
log::warn!(
"provider `{id}` uses unencrypted HTTP; its requests carry client IP, geo, consent signals and identifiers in cleartext, and its responses can be altered in transit"
);
}

Comment on lines +1152 to +1156
assert_eq!(
plan.providers()[0].endpoint.as_str(),
"http://pbs.example.com:8000/openrtb2/auction",
"should normalize the opted-in Prebid Server endpoint"
);

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔧 wrench: The tests don't check how an opted-in provider is dialed, or that the flag leaves HTTPS alone.

I ran 9 mutants of plan.rs against the full core lib suite. The scope, scheme, credential, fragment and serde mutants are all killed, but two survive:

  • backend_spec_with_transport_timeout hard-coding scheme: "https". Every opted-in HTTP provider would then dial TLS (Fastly registers an https_… backend with enable_ssl), and CI stays green.
  • canonicalize_endpoint calling set_scheme("http") when the flag is set. The spec's "HTTPS is not downgraded when the flag is true" has no test behind it.

With this suggestion applied, both mutants fail this test (should dial the opted-in provider without TLS on its configured port and should not downgrade HTTPS when the provider opts in to HTTP).

Suggested change
assert_eq!(
plan.providers()[0].endpoint.as_str(),
"http://pbs.example.com:8000/openrtb2/auction",
"should normalize the opted-in Prebid Server endpoint"
);
assert_eq!(
plan.providers()[0].endpoint.as_str(),
"http://pbs.example.com:8000/openrtb2/auction",
"should normalize the opted-in Prebid Server endpoint"
);
let spec = plan.providers()[0].backend_spec();
assert_eq!(
(spec.scheme.as_str(), spec.port),
("http", Some(8000)),
"should dial the opted-in provider without TLS on its configured port"
);
plan.validate_for_target(AuctionTargetId::Fastly)
.expect("should predict a Fastly backend for the opted-in HTTP provider");
let mut secure = demo.clone();
secure.endpoint = "https://pbs.example.com".to_string();
let secure_plan = AuctionPlan::compile(config(BTreeMap::from([(id("demo"), secure)])))
.expect("should compile an opted-in HTTPS provider");
assert_eq!(
secure_plan.providers()[0].endpoint.as_str(),
"https://pbs.example.com/openrtb2/auction",
"should not downgrade HTTPS when the provider opts in to HTTP"
);


Permit an explicitly configured auction provider to use unencrypted HTTP for an approved demo or test endpoint. Keep HTTPS as the default. Do not turn one provider's exception into permission for another provider.

This documents the existing reconstructed feature at `2c79ea59`. Its formal design reviews occur before publication readiness, not before the earlier implementation. It does not authorize deployment or production use of unencrypted auction traffic.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔧 wrench: The provenance cites commits that don't exist in this repository.

2c79ea59 here, and dd2ca144..2c79ea59 in the review log (line 65), return 422 from the GitHub commits API and aren't on any ref. The "Contract SHA256" hashes a document that isn't published either. A reader can't check what the two reviews examined. Either push those commits to a ref in this repository, or describe the reviewed inputs without SHAs. The review-log rows and the closing paragraph at line 68 ("relay ancestry", "replays") need the same treatment; that spans the table, so it can't be a single suggestion. For line 7:

Suggested change
This documents the existing reconstructed feature at `2c79ea59`. Its formal design reviews occur before publication readiness, not before the earlier implementation. It does not authorize deployment or production use of unencrypted auction traffic.
This documents the provider-local HTTP opt-in added by this change. It does not authorize deployment or production use of unencrypted auction traffic.

Comment on lines +138 to +139
/// Permit unencrypted HTTP for this provider only. Defaults to false.
/// Use only for explicitly approved demo/test endpoints.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

♻️ refactor: Split the summary from the details per AGENTS.md, and say why the field is skipped while false. The sibling fields that use the same pattern (Publisher::allow_async_html_finish and DebugConfig::sandbox_metrics_enabled in settings.rs) document the rollback consequence. It applies here too, because ProviderConfig denies unknown fields.

Suggested change
/// Permit unencrypted HTTP for this provider only. Defaults to false.
/// Use only for explicitly approved demo/test endpoints.
/// Permit an unencrypted `http` endpoint for this provider only.
///
/// Defaults to false. Use only for explicitly approved demo/test endpoints:
/// requests to this provider carry client IP, geo, consent signals and
/// identifiers in cleartext, and its responses can be altered in transit.
///
/// Skipped from serialization while false: [`ProviderConfig`] denies
/// unknown fields, so a default blob stays readable by a binary built
/// before this field existed. A blob with it enabled requires restoring a
/// compatible blob before rolling back.

Comment on lines +780 to 786
| `endpoint` | Required | Fixed absolute HTTPS endpoint |
| `allow_insecure_http` | `false` | Explicit per-provider HTTP opt-in for approved demos/tests; traffic is unencrypted |
| `timeout_ms` | Profile default | PBS 1000 ms, APS 800 ms, standard inherits auction timeout |
| `routing` | `explicit` | `explicit`, or `all_eligible` for non-PBS profiles |
| `profile_config` | `{}` | Profile-owned typed settings |
| `notifications` | No suppression | Common `nurl`/`burl` suppression by all bids or returned seats |

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

♻️ refactor: The endpoint row still reads as HTTPS-only, and "traffic is unencrypted" understates what an operator is opting into: client IP, geo, consent and identifiers leave in cleartext, and responses can be altered. A short warning block keeps the table cell readable. The cell widths are sized so prettier leaves the other rows alone.

Suggested change
| `endpoint` | Required | Fixed absolute HTTPS endpoint |
| `allow_insecure_http` | `false` | Explicit per-provider HTTP opt-in for approved demos/tests; traffic is unencrypted |
| `timeout_ms` | Profile default | PBS 1000 ms, APS 800 ms, standard inherits auction timeout |
| `routing` | `explicit` | `explicit`, or `all_eligible` for non-PBS profiles |
| `profile_config` | `{}` | Profile-owned typed settings |
| `notifications` | No suppression | Common `nurl`/`burl` suppression by all bids or returned seats |
| `endpoint` | Required | Fixed absolute HTTPS endpoint; plain HTTP only with `allow_insecure_http` |
| `allow_insecure_http` | `false` | Per-provider plain HTTP opt-in for approved demo/test endpoints; see warning below |
| `timeout_ms` | Profile default | PBS 1000 ms, APS 800 ms, standard inherits auction timeout |
| `routing` | `explicit` | `explicit`, or `all_eligible` for non-PBS profiles |
| `profile_config` | `{}` | Profile-owned typed settings |
| `notifications` | No suppression | Common `nurl`/`burl` suppression by all bids or returned seats |
::: warning Plaintext provider traffic
`allow_insecure_http = true` sends that provider's requests without TLS. Their
headers and OpenRTB bodies, including client IP, User-Agent, geo, consent
signals and identifiers such as the EC ID and EIDs, are readable on the
network, and bid prices, creatives and notification URLs in its responses can
be altered in transit. The flag does not restrict which hosts are reachable.
:::

Ok(CanonicalProviderEndpoint(endpoint))
}

fn is_false(value: &bool) -> bool {

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

♻️ refactor: settings.rs:2822 already defines the same is_false serde predicate, with a doc comment. Make that one pub(crate) and point this field at it, so the two can't drift:

// settings.rs
/// Serde predicate for omitting `false` flags from serialized config blobs.
pub(crate) fn is_false(value: &bool) -> bool {
    !*value
}
// plan.rs
#[serde(default, skip_serializing_if = "crate::settings::is_false")]
pub allow_insecure_http: bool,

Then delete this copy. Apply manually: the fix touches settings.rs, which is outside this diff.

))
})?;
if endpoint.scheme() != "https"
if !(endpoint.scheme() == "https" || (allow_insecure_http && endpoint.scheme() == "http"))

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⛏ nitpick: For an opted-in provider, a rejected http:// endpoint (for example http://user:pass@pbs.example.com/bid) still gets endpoint must be absolute HTTPS with a host and no credentials or fragment from line 592. That reads as if the scheme were the problem. Consider wording that covers both schemes, for example:

"provider `{provider_id}` endpoint must be an absolute HTTPS URL, or HTTP when allow_insecure_http is set, with a host and no credentials or fragment"

docs/guide/error-reference.md documents this error family, so update it alongside. Apply manually: line 592 is outside the diff hunk.

@dhruv8sh dhruv8sh left a comment •

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Superseded by a follow-up review with inline comments: #1525 (review)

@dhruv8sh dhruv8sh left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Summary

Adds a default-false, per-provider allow_insecure_http flag that lets one provider use a plain http:// endpoint while HTTPS stays the default. Credential, fragment, host and other-scheme checks still apply, and the opt-in doesn't leak to other providers. The scheme passes through backend_spec_with_transport_timeout unchanged, so dispatch works without other changes. Tests cover opt-in isolation, rejected shapes and unchanged wire shape. Approving; the items below are non-blocking.

2 of the inline comments below carry a one-click GitHub suggestion. Use Commit suggestion (or Add suggestion to batch) to apply them.

Non-blocking

⛏ nitpick

  • Endpoint errors still say HTTPS-only — see inline at crates/trusted-server-core/src/auction/plan.rs:582 (also covers :592)
  • Guide endpoint row still says HTTPS — see inline at docs/guide/auction-orchestration.md:780

👍 praise

  • Wire shape kept for default providers — see inline at crates/trusted-server-core/src/auction/plan.rs:140

Cross-cutting / body-level findings

  • 🌱 Make plaintext providers visible in logs — consider a log::warn! (once per compiled plan) naming each provider whose endpoint compiled with http, so a demo opt-in left in a production config is easy to spot.

CI Status

All 29 reported checks PASS (cargo fmt, cargo test, cargo test (axum native), cargo test (cross-adapter parity), cargo check (cloudflare native + wasm32-unknown-unknown), cargo check/build/test (spin native + wasm32-wasip1), cargo test (ts CLI, native) on ubuntu-latest and macos-latest, integration tests, integration tests (Fastly EC lifecycle), browser integration tests, HTML EOF runtime (Fastly), worker-deadlines (spin/cloudflare), vitest, format-typescript, format-docs, ts dev lint domains (changed lines), CodeQL, Analyze (rust/python/actions/javascript-typescript), docs-proposal scripts, CI Node pin guard, CLAUDE.md symlink guard, prepare integration artifacts, trace runtime and browser acceptance).

) -> Result<CanonicalProviderEndpoint, Report<TrustedServerError>> {
let mut endpoint = Url::parse(value).map_err(|error| {
configuration_error(format!(
"provider `{provider_id}` endpoint must be an absolute HTTPS URL: {error}"

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⛏ nitpick — Both endpoint errors still say HTTPS is required, which misleads an operator who set allow_insecure_http = true and then hits a parse or credentials/fragment failure. This first message fires on parse failure only, so the scheme isn't the issue there:

Suggested change
"provider `{provider_id}` endpoint must be an absolute HTTPS URL: {error}"
"provider `{provider_id}` endpoint must be an absolute URL: {error}"

The second message at line 592 (outside the diff hunk, so no one-click suggestion) could become something like "provider {provider_id} endpoint must be absolute HTTPS (or HTTP when allow_insecure_http is set) with a host and no credentials or fragment". Apply manually.

| --------------------- | --------------- | ---------------------------------------------------------------------------------- |
| `protocol` | Required | `openrtb-2.6` |
| `profile` | `standard` | Typed OpenRTB behavior |
| `endpoint` | Required | Fixed absolute HTTPS endpoint |

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⛏ nitpick — The endpoint row still says HTTPS-only, which contradicts the new row below it:

Suggested change
| `endpoint` | Required | Fixed absolute HTTPS endpoint |
| `endpoint` | Required | Fixed absolute HTTPS endpoint; plain HTTP only with `allow_insecure_http` |

pub endpoint: String,
/// Permit unencrypted HTTP for this provider only. Defaults to false.
/// Use only for explicitly approved demo/test endpoints.
#[serde(default, skip_serializing_if = "is_false")]

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍 praise — skip_serializing_if = "is_false" keeps the serialized config byte-identical for every existing provider, and insecure_http_default_preserves_existing_provider_wire_shape pins that. Nice care for published config blobs.

aram356 added a commit that referenced this pull request Nov 12, 2026


Per the user, every feature is built once and works on every adapter in
platform order, with platform gaps sent to EdgeZero (DEC-16, SH-5). T21
moves the identity routes, EC lifecycle, providers and write-back into one
router in core that each adapter mounts; A2, A10, T8, T9, T17 and T18
shrink to turning it on per adapter and passing the conformance suite, and
T7, T14 and A11 fold into A2 and A10. EZI-2a adds EdgeZero's local store
for Axum development and EZI-4 binds Axum's geo provider from the manifest.

Restack the Axum RC units on #1506: #1531 carries A1 in place of #1528,
#1533 and #1534 close as copies of #1525 and #1526, and the image and
Terraform units sit directly on #1532 so A6 and A7 stop waiting on A2.
Record #1566, refresh the provider stack's fixes at their current heads,
and flag #1535 as unplanned.
@aram356

aram356 commented Nov 13, 2026

Copy link
Copy Markdown
Collaborator

CREATOR: From the server-side identity plan (#1559, c7dc9de77): Address its round-1 review.

#1614 now carries #1506's EdgeZero cutover, so #1525's per-provider HTTP
opt-in meets the cutover here, resolved the way rc/identity resolved it in
30a74de:

- The endpoint rule is the union: https, plain http to loopback (#1506), or
  any http with allow_insecure_http (#1525).
- Cloudflare keeps #1506's startup_loads_multi_provider_plan; the dormant
  multi-provider test is dropped because the cutover loads every plan.
- #1506's test ProviderConfig literals in the Fastly app and the auction
  orchestrator gain allow_insecure_http: false.
@aram356
aram356 removed this pull request from stack #1542 November 17, 2026 18:05
@aram356
aram356 changed the base branch from main to feat/identify-batch-sync-local-all-adapters November 17, 2026 18:05
@aram356
aram356 added this pull request to stack #1621 November 17, 2026 18:05
aram356 added a commit that referenced this pull request Nov 17, 2026
#1525 now carries #1506's EdgeZero cutover and #1614, so #1526's awaitable EC
persistence meets them here. The resolution is the one rc/identity made in
30a74de and 4a414a0:

- Fastly runs the async EC finalize from finalize_response, which EdgeZero
  calls after its dispatch executor returns, and drops edgezero_main.
- Pull sync and batch sync keep #1506's EdgeZero send paths with #1526's
  awaits, and their KV-touching tests become async, including #1506's
  pull-sync group failure test.
- The post-delivery test blocks on the now-async snapshot load.

#1614's identify handler awaits #1526's async handle_identify.
@aram356

aram356 commented Nov 17, 2026

Copy link
Copy Markdown
Collaborator

For the identity integration chain (#1608), this PR now stacks on #1614, and through it on #1506's EdgeZero cutover. 45cdb1ee3 merges #1614's head 9627ca12a and resolves this PR against the cutover the way rc/identity resolved it in 30a74def1:

Area Resolution
Endpoint rule (auction/plan.rs) The union: https, plain http to loopback (#1506), or any http with allow_insecure_http (this PR)
Cloudflare startup test Keeps #1506's startup_loads_multi_provider_plan; the dormant multi-provider test is dropped, because the cutover loads every plan on Cloudflare
Test ProviderConfig literals #1506's literals in the Fastly app and the auction orchestrator gain allow_insecure_http: false

The base is now feat/identify-batch-sync-local-all-adapters (#1614), so the diff still shows only this PR's changes; it lands after #1506 and #1614.

The chain is GitHub stack 1621: #1506, #1614, #1525, #1526, with the provider stack (#1487 to #1489) to join its top once it is ported. GitHub does not let a stacked PR change its base, so stack 1542 (#1525 to #1528) was dissolved first; #1527 and #1528 stay open on their current bases.

Checked locally at 45cdb1ee3 and at #1526's f87078412: cargo fmt, the six adapter clippy aliases, clippy-cli, scripts/lint-wasm-clock.sh, test-axum (63), test-cloudflare (65), test-spin (88), test-fastly (3,732 and 3,733), test-fastly-reuse (226) and the parity tests (33), all passing.

Takes #1614's 9552a89, which moves its ported Fastly partner API tests onto allowlisted example hosts.
aram356 added a commit that referenced this pull request Nov 17, 2026
Takes #1525's f5d8b7f, which brings #1614's move of its ported Fastly partner API tests onto allowlisted example hosts.
aram356 added a commit that referenced this pull request Nov 17, 2026
… rc/identity

#1526's head b53c6e2 now stacks on #1525 (f5d8b7f), which stacks on #1614 (9552a89), which stacks on #1506. Each PR merged the one below it and resolved its own conflicts, so this merge needs no resolution. #1525's and #1526's resolutions against #1506 are the ones this branch made in 30a74de and 4a414a0, and #1614 (T1) is new here.
aram356 added a commit that referenced this pull request Nov 17, 2026
Brings PR #1487 onto PR #1526's base (PR #1506's EdgeZero cutover, PR #1614,
PR #1525 and PR #1526's awaitable EC persistence) and keeps PR #1487's
behavior in the new base's structure.

How each area was resolved:

- Provider lookups: the lookup transport now sends through EdgeZero's
  outbound client in the auction fan-out's shape. `LookupTransport::start`
  takes every built request and starts them as one batch with
  `HttpClient::start_batch_until` and a cutoff of the identity budget,
  taken on the client's own clock;
  `gather` reads the batch until it finishes, maps slots back to providers
  and reports unresolved slots as timeouts. `PlatformLookupTransport` and the
  coordinator's web-time `Deadline` are replaced by `OutboundLookupTransport`;
  requests keep the 64 KiB response cap, cache bypass and strict TLS.
- Deleted modules: `platform/http.rs` and the Fastly `platform/deadline.rs`
  stay deleted. PR #1487's final-snapshot change to the Fastly deadline
  select (a 1 ms surfacing wait per pending body and counting arrived headers
  as progress) has no counterpart in EdgeZero's batch and is not ported, nor
  is its doc-only change to `select_until`.
- Async KV: `ec/kv.rs`, `ec/mod.rs` and `ec/finalize.rs` take PR #1487's
  versions converted to PR #1526's async store; the page-load and
  after-response writes, the write planner, after-response commit, batch
  sync's provider-owned check and their tests await the store. Unchanged
  tests take PR #1526's async versions, and PR #1526's non-Send yielding
  store test is kept. `generate_if_needed` stays async as on the base.
- Fastly entry point: `main.rs` takes the base's lifecycle. `finalize`
  collects the navigation's page-phase lookups inside the async EC finalize
  under its one `block_on`, using the partner registry built once in
  `AppState`. After-response lookups run in post-delivery work, after pull
  sync, on the post-delivery client and clock. Lookups still travel through
  the request-local slot, because an in-flight batch is not `Send`; the slot
  is now cleared before each request and after delivery.
- Fastly app: keeps the base's local 501/405 identify and batch-sync answers,
  publisher origin options and response-body scope owner, with PR #1487's
  partner registry, identity coordinators, navigation permit and dispatch.
- Other adapters, core call sites and tests: kept both sides; PR #1487's
  tests that used the removed platform HTTP client or a custom auction
  provider now use the scripted outbound client, and the Fastly identity
  tests route with the host geo lookup the entry point's `prepare` runs.
- Cargo: the lockfile adds only the identity-providers crate to the base's.
- Docs and CHANGELOG: auction collection text follows the base; PR #1487's
  identity entries and spec now describe the EdgeZero batch transport.
aram356 added a commit that referenced this pull request Nov 17, 2026
…ity plan

Both integration branches now carry every in-review issue's non-draft pull
request. DEC-20 records the new rule that a conflict is fixed in the pull
request that lands second, and DEC-21 the merge order the user chose: the
Trusted Server pull requests stack linearly on #1506 (stack 1621), so T1,
#1525 and #1526 land after the cutover. The spec's Integration branches
section and the plan's State section list the new heads, and the wave rows
for T1, #1525, #1526, #1506 and #1487 follow the new order.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Allow explicit per-provider HTTP auction endpoints

4 participants