Repository navigation
Sync upstream main through 94e0a26ac - #137
Merged
Merged
Conversation
Part of agent-substrate#3101. When local snapshots fill a node's disk, the kubelet evicts atelet and rejects every replacement while the node has DiskPressure. Uploads and detaches stop, so nothing frees the space. Only atelet frees what substrate writes under `/var/lib/ate`, because the kubelet doesn't count that hostPath toward any pod and its image GC doesn't reach it. The kubelet never evicts a pod at priority 2000000000 or above, and it admits one under DiskPressure. So atelet now runs at `system-cluster-critical`. In test runs based off of agent-substrate#3101 atelet was never evicted for either `system-node-critical` or `system-cluster-critical` and uploaded on schedule. `system-node-critical` would also give atelet `oom_score_adj` -997. With no memory limits on atelet and objectstore-plugin, a leak in either would then make the kernel kill workers first. Per discussion in this PR the manifest comment says to switch once both containers have memory limits. GKE admits critical pods outside kube-system only in a namespace with a `ResourceQuota` scoped to their class. The manifest adds one that allows both classes, so switching later needs only the `priorityClassName` change. Its `pods: 1G` limit matches GKE's own `gcp-critical-pods` quota. Clusters that don't restrict the critical classes, like kind, admit atelet without it. The kubelet enforces an emptyDir `sizeLimit` only by evicting the pod, so the `asset-staging` limit no longer applies. I removed it and the test that checked `--asset-staging-capacity` fits under it. An install upgraded across this change keeps the old `ate-node-critical` PriorityClass. Delete it with `kubectl delete priorityclass ate-node-critical`. I used an AI agent (Claude Code) to run the agent-substrate#3101 tests, make this change and draft this description. --------- Signed-off-by: Ivy <igooch@google.com>
…e#3457) Part of agent-substrate#3226. The API guide says a `durableDir` is created `0777`, like an emptyDir. On gVisor its root is `01777`. The volume is a tmpfs inside the sandbox, and its mount hint sets no mode, so it gets the tmpfs default. With the sticky bit, a user can't delete or rename another user's files in the volume's root. A durable dir created before this keeps `01777` when it is restored from a VOLUMES or ROOTFS snapshot, since the snapshot records its mode. Breaking Change: A gVisor actor whose template has a `durableDir` can't restore a MEMORY snapshot taken before this change. On a MEMORY restore, runsc checks that the sandbox spec's mount hints match the checkpoint, except `.source`. The new `dev.gvisor.spec.mount.<volume>.options` key doesn't match, so `runsc restore` fails and the actor goes to `CRASHED`. This includes MEMORY golden snapshots of these templates, and actors suspended while their worker rolls during the upgrade. No API restores such a snapshot without its memory. On upgrade: - Recreate affected templates so they take new golden snapshots. - Delete and recreate affected actors that hold a MEMORY snapshot from before the upgrade. A rollback breaks the same way. MEMORY snapshots taken after the upgrade fail to restore on the old worker. From v1.0.0, changes must keep upgrades and rollbacks working, so this has to land before then. AI use: AI-assisted, I reviewed and tested it. - [x] Tests pass - [ ] Docs updated Signed-off-by: Nayana Bidari <nybidari@google.com>
This adds an e2e suite and a workflow that upgrade a running install with actors on it, following `docs/upgrade.md` step by step. The workflow installs the newest earlier release on the branch with that release's own installer, and the suite starts counter actors that are running, paused, or suspended. One job runs them on gVisor and one on microVM. It then runs each runbook step with this checkout's `ate-setup` and a copy of the install record: podcertificate-controller, atelet, the WorkerPool's `workerImage`, the API server, ate-controller, atenet, the SandboxConfig, then `deploy ate-system --setup-csi=none`. While it runs, it acts as the actors' harness and suspends an actor as soon as its worker pod starts terminating. After each step, the step's component has to run new images and no actor may be `CRASHED`. Only the pool roll may roll the workers and evict actors, plus ate-controller if its pod template changed. At the end, every actor, plus one created after the upgrade, has to resume on a new worker with its counter intact. ----- ### Cadence Patch releases are cut by merging fixes into `release-X.Y` and tagging there. The workflow runs on every push to a `release-X.Y` branch and upgrades from the newest earlier tag on that branch, so the commit that becomes v0.6.1 is tested as `v0.6.0` → `v0.6.1` before it is tagged. A push with no earlier tag yet, such as the branch cut itself, is skipped with a notice. It also runs on demand. It doesn't run on pull requests or on pushes to main. After GA 1.0 we support upgrading to the next minor, for example 1.1.x → 1.2.y, so a push to `release-1.2` should also upgrade from the newest 1.1 release. That second lane comes with 1.0. -------- #### Tested - main → this PR, on GitHub runners in my fork: both lanes pass all 8 steps, in about 15 and 18 minutes. The microVM lane needs KVM, so I only runs it on GitHub runners. Not covered yet: rollback. I used Claude Code to help write the suite and to run the tests above. Part of agent-substrate#1784 - [x] Tests pass - [x] Appropriate changes to documentation are included in the PR --------- Signed-off-by: Haven Xia <haoyuxia@google.com>
…e#3591) The `apitool generate-docs` command now adds a section with CRD types as well. Signed-off-by: Julian Gutierrez Oschmann <juliangut@google.com>
…can run the actor checks (agent-substrate#3358) Step 2 of agent-substrate#3095. A provider in its own repository cannot import the actor checks under `internal/`. - `New(ctx, Config)` builds the verifier through `internal/objectstoreplugin/actorauthz.StartVerifier`, as the in-tree exchange does. - A `.svc` issuer is trusted only through `IssuerCAFile`, in-tree too. An install whose `.svc` issuer certificate verified only through the system roots now fails at sidecar start. - `Authorize(ctx, jwt)` returns `ActorUID`, `Grants` and `Expiry`, or a gRPC status error. A grant access with no public `Access` value gets `Internal`. - `Close` closes the ate-api-server connection and idle issuer connections. - `ateapiauth.ClientConfig` loses `K8sClient`. Its four users add the `k8sresolver` option themselves, so the package does not link client-go. - `hack/verify/package-deps.sh` bans cloud SDKs and client-go here. `hack/verify/pkg-external-import.sh` builds the package from an outside module. Tested: `hack/verify-all.sh` passes. `go test -race ./...` fails only in `internal/atunnel/egress` and `internal/netns`, as at the merge base. No e2e. - [ ] Tests pass - [x] Docs updated --------- Signed-off-by: Anish Ramasekar <anish.ramasekar@gmail.com>
…#3647) The `glutton` template stored `WriteDisk` files under `/tmp`, which is a `tmpfs` on both sandbox classes (agent-substrate#3533), so they counted as memory and no benchmark wrote to the rootfs upper layer. Use `/var/lib/glutton` with no volume mounted instead, so `agentsession`'s files land in the rootfs upper layer and in its snapshot. On gVisor, `agentsession` now writes to the rootfs for the first time, so its numbers will change. On microVM, its files go back to the host-backed rootfs upper layer, as before agent-substrate#3533. AI use: AI-assisted, I reviewed and tested it. - [x] Tests pass - [x] Docs updated Signed-off-by: Chenyi Wang <chenyiwang@google.com>
…gent-substrate#3599) The website templates only allowed `experimental`, `planned`, and `deprecated` page statuses, while Substrate feature gates use `preview` as the pre-GA state. On the CSI guides, API guide, and counter demo, the `--preview` gate and `kubectl ate --preview` / `Ate-Preview-Allowed: true` opt-in requirements were also written as plain body prose or a `Note` block that was easy to miss. ## Summary ```diff site/ ├── layouts/_partials/page-params.html # allow status: preview ├── layouts/_partials/status-text.html # default preview banner text ├── layouts/docs/single.html # preview banner icon (fa-eye) └── assets/scss/_styles_project.scss # .as-status-banner--preview, .as-pill--preview docs/ & demos/ ├── guides/csi-volumes.md # + status: preview, status_note, > [!IMPORTANT] ├── guides/csi-deployment.md # + status: preview, status_note, > [!IMPORTANT] ├── reference/api-guide.md # > [!IMPORTANT] on goldenEgressPolicy & storageLocation; bold **Preview** in table └── demos/counter/README.md # > [!IMPORTANT] on ExternalVolumes section ``` ## Evidence - **Before:** Setting `status: preview` in doc front matter failed the Hugo build in `page-params.html` (`status "preview" is not one of: experimental, deprecated, planned`). `docs/guides/csi-volumes.md` and `docs/guides/csi-deployment.md` had no `Preview` pill in the left sidebar or top status banner, and the `--preview=ExternalVolumes` and `--preview=GoldenEgressPolicy` opt-in instructions rendered as plain paragraphs. - **After:** `hack/site.sh build` and `hack/test-site-templates.sh` pass with `status: preview` rendering `.as-status-banner--preview` above the page header, `.as-pill--preview` in the left sidebar and section cards, and `[Status: Preview]` in `llms.txt`, per-page `index.md`, and "Copy as Markdown", while `> [!IMPORTANT]` highlights the preview gate and client opt-in requirements in the body on both `ate.dev` and GitHub. ## Merge Danger **Door:** Two-way **Blast Radius:** Small (documentation Markdown and Hugo website templates/styles only; no runtime, API, or CRD changes) AI use: AI-assisted, I reviewed and tested it. - [x] Tests pass - [x] Docs updated Signed-off-by: Mofi Rahman <mofi@google.com>
Agent Substrate is preparing for the `v1.0.0` release, so the site-wide `Pre-1.0` warning bar above the top navbar on `ate.dev` is obsolete. ## Summary ```diff site/ ├── layouts/_partials/navbar.html # - .as-pre-banner └── assets/scss/_styles_project.scss # - .as-pre-banner styles ``` ## Evidence - **Before:** Every page on the website rendered `.as-pre-banner` (`Pre-1.0` / `APIs and behavior may still change significantly.` on desktop, `APIs may change.` on mobile) above `.td-navbar.as-navbar`. - **After:** `hack/site.sh build` and `hack/test-site-templates.sh` pass with `.td-navbar.as-navbar` flush at the top of the viewport (`top: 0px`) across the home page, documentation pages, and generated `/docs/reference/api/` reference, with zero `Pre-1.0` or `.as-pre-banner` matches in `site/public/`. ## Merge Danger **Door:** Two-way **Blast Radius:** Small (Hugo website navbar partial and SCSS only; no runtime, API, or CRD changes) AI use: AI-assisted, I reviewed and tested it. - [x] Tests pass - [x] Docs updated Signed-off-by: Mofi Rahman <mofi@google.com>
…substrate#3642) The README's GKE Quickstart was a five-step manual setup (env file, ADC, `tools/setup-gcp`, IAM, `ate-setup` flags) plus GCP custom setup and teardown sections, all duplicating what the substrate-gke installer already does. Replace it with a link to the official GKE Installation Guide, which has the prerequisites and the one-line installer, matching how ate.dev links out. Links to the old `#gke-quickstart-development` anchor are updated. AI use: AI-assisted, I reviewed it. - [ ] Tests pass - [x] Docs updated Signed-off-by: Grant McCloskey <gmccloskey@google.com>
…BUTING.md, move it back to repo root (agent-substrate#3643) <!-- The PR title becomes the release note. Write it for users. --> Why is this needed? (Or link the issue. Open one first for features, API, install or CI changes.) - Github treats CONTRIBUTING.md specially, it will point contributors to that file. - We weren't serving this content in the website anyhow. - These two files had a lot of overlap, consolidate the expectations to one place <!-- e.g. "AI-assisted, I reviewed and tested it." No AI co-author trailers. --> AI use: reviewed with AI. Largely manual. <!-- Breaking change? Describe what users must do on upgrade and ask for the `breaking-change` label. --> - [x] Tests pass - [x] Docs updated --------- Signed-off-by: Benjamin Elder <bentheelder@google.com>
…nt Substrate Authors" going forward (agent-substrate#3644) <!-- The PR title becomes the release note. Write it for users. --> Why is this needed? (Or link the issue. Open one first for features, API, install or CI changes.) This project was donated to the CNCF sandbox, for some time now we've accepted either Google LLC or The Agent Substrate Authors. Consolidate on the latter, and stop accepting new files with the former. <!-- e.g. "AI-assisted, I reviewed and tested it." No AI co-author trailers. --> AI use: .... technically to help vet the bash command, but this is a mechanical change. The bash fu is included in the commit message. <!-- Breaking change? Describe what users must do on upgrade and ask for the `breaking-change` label. --> - [x] Tests pass - [x] Docs updated --------- Signed-off-by: Benjamin Elder <bentheelder@google.com>
Also reorder some sections, tweak words, and document release numbers and branches. Builds on @BenTheElder agent-substrate#3643 Signed-off-by: Tim Hockin <thockin@google.com> Signed-off-by: Tim Hockin <thockin@google.com>
Signed-off-by: Eitan Yarmush <eitan.yarmush@solo.io>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Sync
mainwith 12 upstream commits through94e0a26ac, preserving the fork changes and both histories. The branch adds one merge commit onc56cec68c; most changed files come from upstream's copyright update.The integration keeps atelet's local insecure option while adapting its Kubernetes resolver setup. Both atelet charts now use upstream's critical priority and namespace quota, and remove the staging volume's eviction limit. The initial SQL migration stays unchanged, with a boilerplate exception for its original header.
Passed repository verifiers, manifest and CRD checks, migration immutability, Helm lint, and 53 Helm unit tests. All four Helm E2E lanes passed: gVisor, microVM, credential injection, and restored actor identity. They reported 294 cases with six expected skips for GCS, Envoy, or runtime-specific coverage.
The full race suite has one failure:
TestUnpackLayer_UnprivilegedKeepsUnpackingUserexpects GID 1003 but gets 1000. The same failure reproduces three times on the unchanged fork base. Independent integration review has no unresolved findings.Merge with a merge commit to preserve upstream ancestry.
AI use: Codex prepared the merge, resolved integration issues, ran local validation, and performed an independent review.