Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 7 additions & 1 deletion agents/anyplot/render/backends/local.py
Original file line number Diff line number Diff line change
Expand Up @@ -409,7 +409,13 @@ async def render(self, job: RenderJob) -> RenderResult:
(run_dir / "data.csv").write_text(job.data_csv, encoding="utf-8")
if job.data_json is not None:
(run_dir / DATA_JSON_FILE).write_text(job.data_json, encoding="utf-8")
outputs = await asyncio.gather(*(self._run_theme(job, theme, run_dir) for theme in job.themes))
runs = [asyncio.create_task(self._run_theme(job, theme, run_dir)) for theme in job.themes]
try:
outputs = await asyncio.gather(*runs)
except asyncio.CancelledError:
# `gather` raises when the first theme ends; the other's container may still be dying, so wait for it.
await asyncio.gather(*runs, return_exceptions=True)
raise
return RenderResult(job.job_id, {output.theme: output for output in outputs})
finally:
shutil.rmtree(run_dir, ignore_errors=True)
Expand Down
8 changes: 6 additions & 2 deletions agents/cloudbuild.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -116,8 +116,12 @@ substitutions:
# public release instead. AGENT_LIBRARIES' default stays matplotlib,seaborn.
_LIBRARIES: matplotlib,seaborn,plotnine,altair,pygal,letsplot,plotly,bokeh,ggplot2,makie,chartjs,d3,echarts,highcharts,muix
# "1" keeps an instance warm; "0" scales to zero between sessions (and so
# forgets every in-memory session, which the client shows as "session expired").
_MIN_INSTANCES: "0"
# forgets every in-memory session, which the client shows as "session expired",
# and resets the in-memory day counters). "1" since 2026-11-02 (owner decision):
# the idle instance costs about 0.44 CHF a day at list price, which the account's
# unused Compute Flexible commitment absorbs until its first part ends on
# 2027-01-15; revisit before then.
_MIN_INSTANCES: "1"
Comment thread
MarkusNeusinger marked this conversation as resolved.
Resolved
# "true": the smoke step calls /v1/status with an ID token minted for
# _SERVICE_ACCOUNT and checks the configuration it reports. That needs the
# service URL in AGENT_SERVICE_URLS, roles/run.invoker on the service for
Expand Down
7 changes: 7 additions & 0 deletions changelog.d/agents-warm-instance.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
### Fixed

- **A cancelled local render waits for both theme containers.** It used to return, and remove the run directory, while the second container was still being killed, which also made a backend test flaky (#12511).

### Changed

- **The agents service keeps one instance warm.** `anyplot-agents` deploys with one minimum instance, so the first chat message after a quiet period no longer waits for a cold start, and sessions and day counters no longer vanish when Cloud Run scales the service to zero; a session still ends after an hour without activity (`AGENT_SESSION_IDLE_S`), and a deploy still resets both. Its idle cost of about 0.44 CHF a day falls inside the account's unused Compute Flexible commitment until 2027-01-15 (#12511).
4 changes: 2 additions & 2 deletions docs/concepts/agent-network.md
Original file line number Diff line number Diff line change
Expand Up @@ -498,7 +498,7 @@ Built on 2026-10-10: the harness, the two fixture generators and the 146 fixture

## Serving and infrastructure

- **`anyplot-agents`**, a new Cloud Run service built from `agents/Dockerfile` and deployed by `agents/cloudbuild.yaml` (both built on 2026-10-10; the service has run at `https://anyplot-agents-r3tvmejsmq-ez.a.run.app` since 2026-10-11, deployed by the `deploy-agents` trigger on every push to `main` that touches it) with the renderer's candidate-then-smoke-then-promote steps and `--update-*` flags only. Like the renderer's, its deploy step creates the service on the first build without `--no-traffic`, because gcloud rejects `--no-traffic` on a new service, and only when `services describe` reports the service as not found; the API side stayed dark through `AGENT_ENABLED=false` until the switch-on of 2026-10-11. Flags: `gcloud run deploy anyplot-agents --region=europe-west4 --execution-environment=gen2 --no-allow-unauthenticated --service-account=anyplot-agents@anyplot.iam.gserviceaccount.com --cpu=1 --memory=2Gi --min-instances=${_MIN_INSTANCES} --max-instances=1 --concurrency=20 --timeout=1200 --update-env-vars=^|^... --no-traffic --tag=candidate`, where `_MIN_INSTANCES` is 0 unless a build sets 1. No secrets and no Cloud SQL in phase 1. The smoke step checks that Cloud Run IAM refuses an anonymous `GET /v1/status` with exactly `403` (a `401` would come from the container's own caller check, past IAM) and, with `_SMOKE_AUTH=true`, calls `/v1/status` with an ID token impersonated for `anyplot-agents@` (the renderer's token recipe) and checks that it names the provider, model, location and libraries the build configured; the bootstrap build runs with `_SMOKE_AUTH=false`, because the service has no URL and no invoker binding before it exists. The image installs `--extra agents` without plotting extras, copies the source tree with the prompts, the fonts and the catalogue prompt sources at their repository paths, runs as uid 1000, and fails its build when a prompt source or a font is missing: the build imports `agents.main` and reads the two judge rubrics, which the service otherwise reads on first use. The root `.dockerignore` keeps `agents/evals` and ADK's local `.adk/` store, both of which can hold users' data, out of the image. `.github/workflows/ci-image.yml` builds it before merge and checks `/v1/status`, the production default's `401` for an anonymous caller, the files in the image and the uid. The concurrency and the timeout follow the run queue, not the memory: the queue, not Cloud Run, limits the runs in memory, but every queued run holds an open stream, so the instance must accept the queue's 10 waiting streams, the run in flight and the other routes (`--concurrency=20`), and one turn can last the 600 s maximum wait plus the 540 s deadline (`--timeout=1200` since 2026-10-13, 900 before). The FastAPI app in `agents/main.py` sets `docs_url=None`. The service runs no code of its own users: it has no `--sandbox-launcher`, and every render goes to `anyplot-renderer` through the `remote` backend. Its 2 vCPU and 4 GiB were sized when it still hosted the sandboxes; since 2026-10-29 it runs on 1 vCPU and 2 GiB (owner decision): over the 14 days before, its hourly p99 peaked at 0.46 of a core and 370 MiB, and the worst case of about 1.1 GiB (the dataset and render stores' 256 MiB caps, the process, a 1 MB paste's parse and the PNG gates) keeps headroom, because an out-of-memory kill would lose every session in memory. The renderer keeps 2 vCPU and 4 GiB: its renders peaked at 1.2 cores (the sandbox's gVisor and the host share the CPU with the render), a Makie render needs up to 1.5 GiB, and a single render is no faster with more CPUs (a Makie heatmap took 8.9 s locally at 1, 2 and 4 CPUs alike: Julia compiles on one thread). Both services scale to zero and bill only while they serve a request.
- **`anyplot-agents`**, a new Cloud Run service built from `agents/Dockerfile` and deployed by `agents/cloudbuild.yaml` (both built on 2026-10-10; the service has run at `https://anyplot-agents-r3tvmejsmq-ez.a.run.app` since 2026-10-11, deployed by the `deploy-agents` trigger on every push to `main` that touches it) with the renderer's candidate-then-smoke-then-promote steps and `--update-*` flags only. Like the renderer's, its deploy step creates the service on the first build without `--no-traffic`, because gcloud rejects `--no-traffic` on a new service, and only when `services describe` reports the service as not found; the API side stayed dark through `AGENT_ENABLED=false` until the switch-on of 2026-10-11. Flags: `gcloud run deploy anyplot-agents --region=europe-west4 --execution-environment=gen2 --no-allow-unauthenticated --service-account=anyplot-agents@anyplot.iam.gserviceaccount.com --cpu=1 --memory=2Gi --min-instances=${_MIN_INSTANCES} --max-instances=1 --concurrency=20 --timeout=1200 --update-env-vars=^|^... --no-traffic --tag=candidate`, where `_MIN_INSTANCES` is 1 since 2026-11-02 (0 before): a warm instance keeps the in-memory sessions and day counters through idle periods and spares the first message a cold start, and the account's unused Compute Flexible commitment absorbs its idle cost (about 0.44 CHF a day at list price) until the commitment's first part ends on 2027-01-15. No secrets and no Cloud SQL in phase 1. The smoke step checks that Cloud Run IAM refuses an anonymous `GET /v1/status` with exactly `403` (a `401` would come from the container's own caller check, past IAM) and, with `_SMOKE_AUTH=true`, calls `/v1/status` with an ID token impersonated for `anyplot-agents@` (the renderer's token recipe) and checks that it names the provider, model, location and libraries the build configured; the bootstrap build runs with `_SMOKE_AUTH=false`, because the service has no URL and no invoker binding before it exists. The image installs `--extra agents` without plotting extras, copies the source tree with the prompts, the fonts and the catalogue prompt sources at their repository paths, runs as uid 1000, and fails its build when a prompt source or a font is missing: the build imports `agents.main` and reads the two judge rubrics, which the service otherwise reads on first use. The root `.dockerignore` keeps `agents/evals` and ADK's local `.adk/` store, both of which can hold users' data, out of the image. `.github/workflows/ci-image.yml` builds it before merge and checks `/v1/status`, the production default's `401` for an anonymous caller, the files in the image and the uid. The concurrency and the timeout follow the run queue, not the memory: the queue, not Cloud Run, limits the runs in memory, but every queued run holds an open stream, so the instance must accept the queue's 10 waiting streams, the run in flight and the other routes (`--concurrency=20`), and one turn can last the 600 s maximum wait plus the 540 s deadline (`--timeout=1200` since 2026-10-13, 900 before). The FastAPI app in `agents/main.py` sets `docs_url=None`. The service runs no code of its own users: it has no `--sandbox-launcher`, and every render goes to `anyplot-renderer` through the `remote` backend. Its 2 vCPU and 4 GiB were sized when it still hosted the sandboxes; since 2026-10-29 it runs on 1 vCPU and 2 GiB (owner decision): over the 14 days before, its hourly p99 peaked at 0.46 of a core and 370 MiB, and the worst case of about 1.1 GiB (the dataset and render stores' 256 MiB caps, the process, a 1 MB paste's parse and the PNG gates) keeps headroom, because an out-of-memory kill would lose every session in memory. The renderer keeps 2 vCPU and 4 GiB: its renders peaked at 1.2 cores (the sandbox's gVisor and the host share the CPU with the render), a Makie render needs up to 1.5 GiB, and a single render is no faster with more CPUs (a Makie heatmap took 8.9 s locally at 1, 2 and 4 CPUs alike: Julia compiles on one thread). The renderer scales to zero and bills only while it serves a request; the agents service keeps one instance warm (see its `_MIN_INSTANCES` above).
- **`anyplot-renderer`**, a phase-1 Cloud Run service that runs the code (`agents/renderer/`), deployed by `agents/renderer/cloudbuild.yaml` with the same candidate, smoke and promote steps. Since 2026-10-29 its image builds on the toolchain image `anyplot-toolchain` (`agents/renderer/toolchain.Dockerfile`: `python:3.14-slim` pinned by digest, R 4.6.1 with the pipeline's package set, and the exec filter's preload library for the exec-mode rows; step 2a of `agentic/specs/261024-agent-all-libraries.md`). The deploy hashes the toolchain's inputs (`agents/renderer/toolchain.py`), builds and pushes `anyplot-toolchain:<hash>` to the private Artifact Registry repository `anyplot` only when that tag is missing, and builds the renderer from it by digest; the CI job pulls the same tag through Workload Identity Federation (`roles/artifactregistry.reader` on the repository) or builds it. Its deploy step creates the service on the first build, because gcloud refuses `--no-traffic` for a service that does not exist yet; it does so only when `services describe` reports the service as not found, and stops on any other describe error, so a permission or API error never sends all traffic to an unsmoked revision. Flags: `gcloud beta run deploy anyplot-renderer --region=europe-west4 --execution-environment=gen2 --sandbox-launcher --no-allow-unauthenticated --service-account=anyplot-renderer@anyplot.iam.gserviceaccount.com --cpu=2 --memory=4Gi --min-instances=${_MIN_INSTANCES} --max-instances=1 --concurrency=4 --timeout=300 --add-volume=name=runs,type=in-memory,size-limit=512Mi --add-volume-mount=volume=runs,mount-path=/tmp/runs --no-traffic --tag=candidate --update-env-vars=^|^ENVIRONMENT=production|RENDERER_ALLOWED_CALLERS=...|RENDERER_AUDIENCES=...`, where `_MIN_INSTANCES` is 0 unless a build sets 1 to keep an instance warm. One instance with one render slot, because renders are serial; `--concurrency=4` so a second caller gets the slot's `503 busy` after 30 s instead of a Cloud Run 429, and a cancel reaches the instance while a render holds the slot; the run directories on the in-memory volume (see [Render](#render)). No public domain, no secrets, no Cloud SQL and no role: the service account exists only to be an identity. The process runs as root, because `sandbox do` creates the gVisor sandboxes; the model-written code runs inside them. The smoke step checks that IAM refuses an anonymous caller (403) and, with `_SMOKE_RENDER=true`, checks that `/status` reports the launcher and the volume and renders a matplotlib plot through the candidate as the service's own account, which proves the launcher, the volume and the harness together; a Makie, a Chart.js and a plotly render and a data run follow, and each one gates the promotion too. These render probes live in `agents/renderer/deploy_smoke.sh`, which the step calls with the token in the environment, because Cloud Build refuses the whole build when one step argument is longer than 10,000 characters: the inline step reached 11,347 with the plotly probe, and PB-2a's renderer did not deploy (build `dc1cf3ae` of 2026-11-01). `tests/unit/test_cloudbuild_configs.py` keeps every Cloud Build config of the repository under that limit. The build impersonates `anyplot-renderer@` for that token (`gcloud auth print-identity-token --impersonate-service-account`), because a build cannot mint an ID token for its own identity: gcloud refuses `--include-email` for it and Cloud Build's metadata server has no `identity` endpoint (builds `da6f53f5` and `a9d3ac7f` of 2026-10-10). The token is minted for the service URL while the requests go to the `candidate` tag URL, because Cloud Run accepts no other audience. `.github/workflows/ci-image.yml` builds the same image before merge and renders a seaborn plot through its harness.

| Service | Runs | Called by | Identity and roles |
Expand Down Expand Up @@ -651,7 +651,7 @@ Defaults apply until the owner decides otherwise.
- A footer strip on the served PNGs: decided on 2026-10-10, yes, at image level only (variant `s1c-jb-sq2`: JetBrains Mono, the dot as a brand-green square); the exported code stays without it, and `AGENT_WATERMARK=false` switches it off. See [Footer strip](#footer-strip).
- Disabling the Vertex 24 h in-memory cache (needs `roles/aiplatform.admin` and applies to all regions): no in phase 1.
- Retention periods for the legal page (phase 2).
- Scale-to-zero "session expired" versus `min-instances=1`: accept in phase 1.
- Scale-to-zero "session expired" versus `min-instances=1`: accepted until 2026-11-02, when the agents service moved to `_MIN_INSTANCES=1`. A deploy, the one-hour idle sweep (`AGENT_SESSION_IDLE_S`) and a rare platform restart still end the in-memory sessions and reset the day counters; persisting the counters is planned before the public launch.
- A named fallback model on `eu` for capacity failover, which deviates from one pinned model everywhere: none.
- The request timeout for a queued turn: a run at the back of a full queue needs up to 780 s (600 s in the queue, 180 s to run). Decided on 2026-10-10: anyplot-api runs with `--timeout=900` and the BFF caps a turn at `AGENT_TURN_MAX_S` (890 s), so the full wait is reachable through the BFF. Raised on 2026-10-13 with the 540 s deadline (owner: a complicated plot must not stop at 80 %): up to 1,140 s, `--timeout=1200` on both services and a 1,190 s turn cap.
- The queue length: the owner's formula (rate x maximum wait / 60) assumes runs within the 60 s rate window, so with longer runs the back of a full queue waits the full maximum and gets `capacity`. Decided on 2026-10-10: the formula stays; admitting by an estimated start time (position times the larger of 60 s per start and the recent run time, divided by the concurrency) is the fallback if the live service shows many expiries.
Expand Down
3 changes: 2 additions & 1 deletion tests/unit/agents/test_image.py
Original file line number Diff line number Diff line change
Expand Up @@ -517,7 +517,8 @@ def test_substitution_defaults(self) -> None:
"matplotlib,seaborn,plotnine,altair,pygal,letsplot,plotly,bokeh,ggplot2,makie,chartjs,d3,echarts,"
"highcharts,muix"
),
"_MIN_INSTANCES": "0",
# One warm instance since 2026-11-02 (owner): the unused Compute Flexible commitment absorbs it.
"_MIN_INSTANCES": "1",
"_SMOKE_AUTH": "true",
}
deployed = _config(CLOUDBUILD)["substitutions"]["_LIBRARIES"].split(",")
Expand Down
Loading