feat(agent): Turnstile on the public chat and site feedback

agents/README.md

@@ -90,7 +90,7 @@ Three rules hold for everything here:
 | `AGENT_ADMIN_USER_MAX_RUNS` | `5` | The same for an admin (a request with `X-Anyplot-Tier: admin` from an allowed caller, the BFF for its admins), whose runs share the concurrency and the rate with everyone's; `GET /v1/status` reports it to an admin as `user_max_runs`, and `.plot_all()` starts that many plots at once |
 | `AGENT_IP_MAX_RUNS` | `4` | The same for the anonymous users of one network (an IPv4 address or an IPv6 /64, from the public BFF's `X-Anyplot-Network`), together; one more ends its turn with `error{code:"run_limit"}` |
 | `AGENT_IP_MAX_SESSIONS` | `8` | Live sessions the anonymous users of one network may hold at once; one more is `429 session_limit` |
-| `AGENT_IP_DATASET_BYTES` | `67108864` | Bytes of the dataset store (64 MiB of its 256 MiB) the sessions of one network may hold; past it a paste is refused like a full store |
+| `AGENT_IP_DATASET_BYTES` | `33554432` | Bytes of the dataset store (32 MiB of its 256 MiB) the sessions of one network may hold; past it a paste is refused like a full store |
 | `AGENT_PUBLIC_QUEUE_MAX_WAITING` | `10` | Waiting entries the public lane may hold in the run queue (of its 60); one more ends its turn with `error{code:"capacity"}`, so open public streams keep to a share of anyplot-api's request slots |
 | `AGENT_USER_MESSAGES_PER_MINUTE` | `10` | Chat turns (typed messages and actions) per user within any 60 seconds; one more gets the fixed `rate_limited` refusal before the scope judge, so it costs no model call. An admin is exempt |
 | `AGENT_USER_MESSAGES_PER_DAY` | `200` | Chat turns per user and UTC day; one more gets the fixed `daily_limit` refusal before the scope judge. An admin is exempt |

agents/anyplot/settings.py

@@ -43,7 +43,7 @@
 | `AGENT_ADMIN_USER_MAX_RUNS` | `5` | The same for an admin (`X-Anyplot-Tier: admin` from an allowed caller), whose runs share the concurrency and the rate with everyone's; `GET /v1/status` reports it to an admin as `user_max_runs`, which the fan-out keeps to |
 | `AGENT_IP_MAX_RUNS` | `4` | The same for the anonymous users of one network (an IPv4 address or an IPv6 /64, `X-Anyplot-Network` from the public BFF), together; one more ends its turn with `error{code:"run_limit"}` |
 | `AGENT_IP_MAX_SESSIONS` | `8` | Live sessions the anonymous users of one network may hold at once; one more is `429 session_limit` |
-| `AGENT_IP_DATASET_BYTES` | `67108864` | Bytes of the dataset store (64 MiB of its 256 MiB) the sessions of one network may hold; past it a paste is refused like a full store |
+| `AGENT_IP_DATASET_BYTES` | `33554432` | Bytes of the dataset store (32 MiB of its 256 MiB) the sessions of one network may hold; past it a paste is refused like a full store |
 | `AGENT_PUBLIC_QUEUE_MAX_WAITING` | `10` | Waiting entries the public lane may hold in the run queue; one more ends its turn with `error{code:"capacity"}`, so open public streams keep to a share of anyplot-api's request slots |
 | `AGENT_USER_MESSAGES_PER_MINUTE` | `10` | Chat turns (typed messages and actions) per user within any 60 seconds; more get the `rate_limited` refusal before the scope judge runs. An admin is exempt |
 | `AGENT_USER_MESSAGES_PER_DAY` | `200` | Chat turns per user and UTC day; more get the `daily_limit` refusal before the scope judge runs. An admin is exempt |
@@ -370,11 +370,11 @@ class AgentSettings(BaseSettings):
     (`AGENT_SESSION_IDLE_S`) drops it, so without this one machine minting tab ids could fill the instance's memory
     (Copilot and Fable reviews of the launch package, 2026-11-02)."""
 
-    ip_dataset_bytes: PositiveInt = 64 * 1024 * 1024
+    ip_dataset_bytes: PositiveInt = 32 * 1024 * 1024
     """The bytes of the dataset store (`DatasetStore`, 256 MiB in all) the sessions of one network may hold
-    (`AGENT_IP_DATASET_BYTES`, 64 MiB): past it a paste or a data change is refused like a full store, so one machine
-    cannot fill the store every user's data shares. A quarter of the store, about eight sessions with a 1 MB paste and
-    its five undo versions each."""
+    (`AGENT_IP_DATASET_BYTES`, 32 MiB): past it a paste or a data change is refused like a full store, so one machine
+    cannot fill the store every user's data shares. An eighth of the store, about four sessions with a 1 MB paste and
+    its five undo versions each; at a quarter, four networks together filled the store for everyone else."""
 
     public_queue_max_waiting: PositiveInt = 10
     """Waiting entries the public lane (`normal`) may hold in the run queue (`AGENT_PUBLIC_QUEUE_MAX_WAITING`); one

api/cloudbuild.yaml

@@ -24,8 +24,9 @@ substitutions:
   # The runtime identity, a dedicated least-privilege account since 2026-11-02 (before:
   # the Compute Engine default service account, which holds roles/editor). Its grants:
   # roles/cloudsql.client and roles/bigquery.jobUser on the project, secretAccessor on
-  # the six secrets the service reads (DATABASE_URL, CACHE_INVALIDATE_TOKEN, ADMIN_TOKEN,
-  # PLAUSIBLE_API_KEY, AGENT_USER_ID_KEY and the hand-attached ORIGIN_SECRET),
+  # the seven secrets the service reads (DATABASE_URL, CACHE_INVALIDATE_TOKEN, ADMIN_TOKEN,
+  # PLAUSIBLE_API_KEY, AGENT_USER_ID_KEY, TURNSTILE_SECRET_KEY and the hand-attached
+  # ORIGIN_SECRET),
   # objectCreator and objectViewer on gs://anyplot-agent-cases, objectViewer on
   # gs://anyplot-static (fonts), READER on the BigQuery datasets agent_telemetry and
   # agent_telemetry_logs, and run.invoker on anyplot-agents. The build account (the
@@ -132,7 +133,13 @@ steps:
       # that creates it. The cost of `--update-secrets` is that a binding
       # dropped from this line is no longer removed automatically — worth it
       # against a gate that turns itself off.
-      - "--update-secrets=DATABASE_URL=DATABASE_URL:latest,CACHE_INVALIDATE_TOKEN=CACHE_INVALIDATE_TOKEN:latest,ADMIN_TOKEN=ADMIN_TOKEN:latest,PLAUSIBLE_API_KEY=PLAUSIBLE_API_KEY:latest,AGENT_USER_ID_KEY=AGENT_USER_ID_KEY:latest"
+      #
+      # TURNSTILE_SECRET_KEY (api/turnstile.py) is listed, unlike ORIGIN_SECRET:
+      # production refuses the bot-checked calls and keeps the public chat shut
+      # without it, so it has to arrive with every revision. Its Secret Manager
+      # entry and the runtime account's secretAccessor on it must exist before
+      # the first deploy that names it, or that deploy fails.
+      - "--update-secrets=DATABASE_URL=DATABASE_URL:latest,CACHE_INVALIDATE_TOKEN=CACHE_INVALIDATE_TOKEN:latest,ADMIN_TOKEN=ADMIN_TOKEN:latest,PLAUSIBLE_API_KEY=PLAUSIBLE_API_KEY:latest,AGENT_USER_ID_KEY=AGENT_USER_ID_KEY:latest,TURNSTILE_SECRET_KEY=TURNSTILE_SECRET_KEY:latest"
       - "--execution-environment=gen2"
       # ^|^ alt delimiter: values contain @ (emails) and may contain , (multi-email lists)
       #
@@ -285,23 +292,10 @@ steps:
           grep -qF '"libraries"' body.out || { echo "agent BFF status did not relay the agents service's status"; exit 1; }
           echo "OK: /debug/agent/status reaches anyplot-agents"
         fi
-        # The public door answers exactly as the trigger set it: 404 while
-        # closed, so a revision never opens it by accident, and while open
-        # `{"open": true}` plus a public status call that reaches the agents
-        # service as an anonymous caller (it books nothing, and no login is
-        # involved). The tab id is a fixed smoke value, never a real visitor's.
-        code=$$(curl -s $$RETRY "$${HDR[@]}" -o body.out -w '%{http_code}' "$$URL/agent/lane")
-        if [ "${_AGENT_ENABLED}" = "true" ] && [ "${_AGENT_PUBLIC_ENABLED}" = "true" ]; then
-          test "$$code" = "200" && grep -qF '"open":true' body.out \
-            || { echo "public agent lane expected open (200), got $$code (in production it opens only while ORIGIN_SECRET arms the origin gate)"; exit 1; }
-          code=$$(curl -s $$RETRY "$${HDR[@]}" -H "X-Anyplot-Anon: deploy-smoke-0000000000000000" -o body.out -w '%{http_code}' "$$URL/agent/status")
-          test "$$code" = "200" || { echo "public agent status expected 200, got $$code"; exit 1; }
-          grep -qF '"libraries"' body.out || { echo "public agent status did not relay the agents service's status"; exit 1; }
-          echo "OK: /agent/* is open and reaches anyplot-agents"
-        else
-          test "$$code" = "404" || { echo "public agent lane expected closed (404), got $$code"; exit 1; }
-          echo "OK: /agent/* is closed"
-        fi
+        # The public door's probes (the lane, a public status call and the bot
+        # check) live in their own script: Cloud Build refuses a step argument
+        # over 10,000 characters. The origin secret goes through the environment.
+        ORIGIN_SECRET="$$ORIGIN_SECRET" bash api/deploy_smoke_public.sh "$$URL" "${_AGENT_ENABLED}" "${_AGENT_PUBLIC_ENABLED}"
         # Re-assert the tag AFTER the probes. `candidate` is a shared tag, so a
         # concurrent build could move it between the check above and the last
         # probe, and this build would then have smoked someone else's revision

api/deploy_smoke_public.sh

@@ -0,0 +1,52 @@
+#!/bin/bash
+# The API deploy's probes of the public agent chat (`/agent/*`, api/routers/agent_public.py) against the
+# CANDIDATE revision (api/cloudbuild.yaml, step "smoke", after its admin-gate probes). Each probe gates the
+# promotion: a failure exits non-zero, and the revision serving now keeps serving.
+#
+# They live in this file because Cloud Build refuses a step argument longer than 10,000 characters;
+# tests/unit/test_cloudbuild_configs.py holds every config's arguments under that limit.
+#
+# Usage, from the repository root (Cloud Build's /workspace):
+#   ORIGIN_SECRET=<secret or empty> bash api/deploy_smoke_public.sh <candidate URL> <_AGENT_ENABLED> <_AGENT_PUBLIC_ENABLED>
+# The secret comes through the environment, never as an argument, and is never echoed. The probes write their
+# response into body.out in the working directory.
+set -eu
+URL="$1"
+AGENT_ENABLED="$2"
+PUBLIC_ENABLED="$3"
+# Every probe retries: a freshly deployed candidate starts cold.
+RETRY=(--retry 5 --retry-delay 5 --retry-all-errors)
+HDR=()
+if [ -n "${ORIGIN_SECRET:-}" ]; then HDR=(-H "X-Origin-Secret: $ORIGIN_SECRET"); fi
+# A fixed smoke value, never a real visitor's tab id.
+ANON="X-Anyplot-Anon: deploy-smoke-0000000000000000"
+
+# The public door answers exactly as the trigger set it: 404 while closed, so a revision never opens it by
+# accident, and while open `{"open": true}`.
+code=$(curl -s "${RETRY[@]}" "${HDR[@]}" -o body.out -w '%{http_code}' "$URL/agent/lane")
+if [ "$AGENT_ENABLED" != "true" ] || [ "$PUBLIC_ENABLED" != "true" ]; then
+  test "$code" = "404" || { echo "public agent lane expected closed (404), got $code"; exit 1; }
+  echo "OK: /agent/* is closed"
+  exit 0
+fi
+test "$code" = "200" && grep -qF '"open":true' body.out \
+  || { echo "public agent lane expected open (200), got $code (in production it opens only while ORIGIN_SECRET arms the origin gate and TURNSTILE_SECRET_KEY is bound)"; exit 1; }
+
+# A public status call that reaches the agents service as an anonymous caller: it books nothing, and no login is
+# involved.
+code=$(curl -s "${RETRY[@]}" "${HDR[@]}" -H "$ANON" -o body.out -w '%{http_code}' "$URL/agent/status")
+test "$code" = "200" || { echo "public agent status expected 200, got $code"; exit 1; }
+grep -qF '"libraries"' body.out || { echo "public agent status did not relay the agents service's status"; exit 1; }
+echo "OK: /agent/* is open and reaches anyplot-agents"
+
+# The bot check (api/turnstile.py), end to end: Cloudflare's dummy token, which every production secret refuses,
+# must read 403 bot_check_failed. That proves the secret arrived, Cloudflare accepted it and siteverify is
+# reachable from the service. A 503 bot_check_unavailable is a wrong or test secret or no route to Cloudflare; a
+# 200 means a session opened unchecked. The token is checked before the agents service is asked, so on the 403
+# path no session opens and nothing is booked.
+code=$(curl -s "${RETRY[@]}" "${HDR[@]}" -X POST -H "X-Anyplot-Client: agent-chat/1" \
+  -H "Content-Type: application/json" -H "$ANON" -H "X-Turnstile-Token: XXXX.DUMMY.TOKEN.XXXX" \
+  -d '{"spec_id":"scatter-basic","library":"matplotlib","locale":"en"}' -o body.out -w '%{http_code}' "$URL/agent/sessions")
+test "$code" = "403" && grep -qF '"bot_check_failed"' body.out \
+  || { echo "the bot check expected 403 bot_check_failed for a dummy token, got $code: $(head -c 200 body.out)"; exit 1; }
+echo "OK: /agent/sessions refuses a dummy Turnstile token"

api/routers/agent_public.py

@@ -7,14 +7,16 @@
   `X-Anyplot-Anon`; `require_public_identity` turns a request into that id plus the client's network (its IPv4
   address, or the /64 of an IPv6 one). Both go to the agents service, which salts them with its own per-process
   salt and keys the per-user and per-IP limits on the result (`agents/main.py` `require_user`), so nothing
-  durable links usage to an address, and every anyplot-api instance reaches the same user. That dependency is
-  the one seam where a bot check (Turnstile) slots in later.
+  durable links usage to an address, and every anyplot-api instance reaches the same user.
+* **Bot check.** Opening a session and storing a feedback case need a Cloudflare Turnstile token in
+  `X-Turnstile-Token` (`api/turnstile.py`); every other route acts on a session such a call opened.
 * **Tier.** Every request asserts `X-Anyplot-Tier: public`: the run queue's `normal` lane with its per-user cap,
   and every day cap.
 * **No session id in a URL.** Cloud Run's request log keeps a URL together with the client address for 30 days, so
   the session id travels in `X-Anyplot-Session` instead of the path.
 * **No admin-only route.** The fan-out's data copy (`/dataset/copy`) and `source: fanout`, the day budget
-  (`/budget`) and the feedback triage (`/cases*`) exist only under `/debug/agent`.
+  (`/budget`), the feedback triage (`/cases*`) and `.save()`'s session file (`/bundle`) exist only under
+  `/debug/agent`.
 
 Every route answers 404 until `AGENT_PUBLIC_ENABLED` is on, on top of `AGENT_ENABLED` and its configuration, which
 stays the master switch: closing the public door leaves the admin lane open. `GET /agent/lane` answers
@@ -46,6 +48,7 @@
 from pydantic import BaseModel, ConfigDict
 from sqlalchemy.ext.asyncio import AsyncSession
 
+from api import turnstile
 from api.dependencies import require_db
 from api.origin_gate import gate_is_armed
 from api.request_context import client_ip
@@ -136,12 +139,13 @@ def require_public_enabled(config: AgentConfig = Depends(require_agent_enabled))
 
     In production it also stays shut while the origin gate is unarmed (`ORIGIN_SECRET`, `api/origin_gate.py`): the
     per-IP limits key on `CF-Connecting-IP`, which only a request that came through Cloudflare carries honestly, and
-    without the gate a caller on the direct `run.app` URL could forge a fresh address per request. The deploy smoke
-    then fails on `/agent/lane`, which says so before the switch seems to work.
+    without the gate a caller on the direct `run.app` URL could forge a fresh address per request. And it stays
+    shut without the Turnstile secret (`TURNSTILE_SECRET_KEY`, `api/turnstile.py` `enforced`) or with one of
+    Cloudflare's test secrets, either of which would refuse every new session. The deploy smoke then fails on `/agent/lane`, which says so before the switch seems to work.
     """
     if not settings.agent_public_enabled:
         raise AgentHTTPError(404, NOT_ENABLED)
-    if settings.is_production and not gate_is_armed():
+    if settings.is_production and not (gate_is_armed() and turnstile.enforced()):
         raise AgentHTTPError(404, NOT_ENABLED)
     return config
 
@@ -175,10 +179,11 @@ class PublicIdentity:
 
 
 def require_public_identity(request: Request, x_anyplot_anon: Annotated[str | None, Header()] = None) -> PublicIdentity:
-    """Turn a request into the public identity: the one seam a bot check such as Turnstile goes into.
+    """Turn a request into the public identity.
 
     `400 anon_required` without a tab id of `ANON_ID_PATTERN`. Nothing is stored and nothing is logged: the agents
-    service salts both values with its per-process salt before it keys anything on them.
+    service salts both values with its per-process salt before it keys anything on them. The bot check is not part
+    of it: a token is single-use, so only the calls that open or store something take one (`require_human`).
     """
     if x_anyplot_anon is None or not _ANON_ID.fullmatch(x_anyplot_anon):
         raise AgentHTTPError(400, "anon_required")
@@ -282,6 +287,20 @@ def get_feedback_limiter() -> FeedbackLimiter:
 Db = Annotated[AsyncSession, Depends(require_db)]
 SessionId = Annotated[str, Header(alias="X-Anyplot-Session", pattern=SESSION_ID_PATTERN)]
 """The session id, in a header: Cloud Run logs every URL with the client's address."""
+HumanToken = Annotated[str | None, Header(alias=turnstile.TOKEN_HEADER)]
+"""The page's Turnstile token for this one call; checked by `require_human`."""
+Siteverify = Annotated[httpx.AsyncClient, Depends(turnstile.get_turnstile_client)]
+
+
+async def require_human(token: str | None, client: httpx.AsyncClient, ctx: AgentContext) -> None:
+    """Redeem the call's Turnstile token: `403 bot_check_failed` or `503 bot_check_unavailable` unless it passes.
+
+    Called inside the route, after its body and identity are validated, so a malformed request costs no
+    siteverify call.
+    """
+    refused = turnstile.refusal(await turnstile.verify(token, client))
+    if refused is not None:
+        raise AgentHTTPError(refused[0], refused[1], ctx.request_id)
 
 
 class PublicCreateSessionBody(BaseModel):
@@ -334,9 +353,15 @@ async def eligibility(
 
 
 @router.post("/sessions")
-async def create_session(body: PublicCreateSessionBody, ctx: Ctx, client: Client, db: Db) -> Any:
+async def create_session(
+    body: PublicCreateSessionBody, ctx: Ctx, client: Client, db: Db, siteverify: Siteverify, human: HumanToken = None
+) -> Any:
     """Open a chat session: `{session_id, eligibility, roles}`. The page keeps the id and sends it as
-    `X-Anyplot-Session`."""
+    `X-Anyplot-Session`.
+
+    Needs a Turnstile token (`require_human`), checked before the agents service is asked.
+    """
+    await require_human(human, siteverify, ctx)
     admin_body = CreateSessionBody(spec_id=body.spec_id, library=body.library, locale=body.locale)
     return await agent.create_session(admin_body, ctx, client, db)
 
@@ -485,19 +510,6 @@ async def get_artifact(
     return await agent.get_artifact(sid, name, ctx, client, v, theme)
 
 
-@router.get("/session/bundle")
-async def get_bundle(
-    sid: SessionId,
-    request: Request,
-    ctx: Ctx,
-    client: Client,
-    version: Annotated[int | None, Query(ge=0, le=MAX_AGENT_VERSION)] = None,
-    include_data: bool = False,
-) -> Response:
-    """`.save()`: the session as a JSON file to keep, as the admin lane's route; nothing is stored."""
-    return await agent.get_bundle(sid, request, ctx, client, version, include_data)
-
-
 @router.post("/session/feedback", status_code=201)
 async def submit_feedback(
     sid: SessionId,
@@ -507,12 +519,17 @@ async def submit_feedback(
     cases: Cases,
     db: Db,
     limiter: Annotated[FeedbackLimiter, Depends(get_feedback_limiter)],
+    siteverify: Siteverify,
+    human: HumanToken = None,
 ) -> dict[str, Any]:
     """Store the feedback case the user sent, as the admin lane does: `201 {case_id, stored}`.
 
-    `include_data` carries the user's consent to store the data with the case; the page leaves it unticked. A
-    network past `AGENT_PUBLIC_FEEDBACK_PER_HOUR` cases gets `429 rate_limited` before the bundle is fetched.
+    `include_data` carries the user's consent to store the data with the case; the page leaves it unticked. The
+    token comes first (`require_human`), so a script without one cannot spend the hourly cases of a network it
+    shares with real visitors; then a network past `AGENT_PUBLIC_FEEDBACK_PER_HOUR` cases gets `429 rate_limited`
+    before the bundle is fetched.
     """
+    await require_human(human, siteverify, ctx)
     if ctx.network is None or not limiter.allow(ctx.network):
         raise AgentHTTPError(429, "rate_limited", ctx.request_id)
     return await agent.submit_feedback(sid, body, ctx, client, cases, db)