feat: adopt mode for impl-generate (source_ref)

.claude/skills/babysit-pipeline/SKILL.md

@@ -42,6 +42,11 @@ Reviews always run on Opus whatever `model` says (only a manual
 `impl-review.yml` dispatch with `review_model` changes that).
 impl-generate auto-closes any existing open PR for the same spec/lib.
 
+`-f source_ref=<branch|tag|sha>` (adopt mode, either workflow) takes each
+implementation file from that ref instead of generating it, always as
+`regen:forced`. An "Adoption Failed" issue comment is never retried: fix
+the file on the ref, then re-dispatch.
+
 ## 2 · Monitor — use the bundled scripts, don't hand-roll
 
 Completion signal = the spec issue carries all 15 `impl:{lib}:done`

.github/workflows/bulk-generate.yml

@@ -62,6 +62,11 @@ on:
         required: false
         type: boolean
         default: true
+      source_ref:
+        description: "Adopt mode for every pair: take each implementation file from this branch, tag or SHA of origin (e.g. gemini/muix-9) instead of generating it. Always a forced regeneration. Empty = generate."
+        required: false
+        type: string
+        default: ''
 
 env:
   ALL_LIBRARIES: "matplotlib seaborn plotly bokeh altair plotnine pygal highcharts letsplot ggplot2 makie chartjs d3 echarts muix"
@@ -157,12 +162,16 @@ jobs:
           COUNT: ${{ needs.build-matrix.outputs.count }}
           DRY_RUN: ${{ inputs.dry_run }}
           PACE_SECONDS: ${{ inputs.pace_seconds || ((inputs.model == 'sonnet' || inputs.model == 'haiku') && '120' || '180') }}
+          SOURCE_REF: ${{ inputs.source_ref }}
         run: |
           echo "## Bulk Generation Plan"
           echo ""
           echo "**Total items:** $COUNT"
           echo "**Pacing:** ${PACE_SECONDS}s between dispatches (0 = fire all)"
           echo "**Dry run:** $DRY_RUN"
+          if [ -n "$SOURCE_REF" ]; then
+            echo "**Adopt mode:** every file is taken from \`${SOURCE_REF}\` instead of being generated"
+          fi
           echo ""
           echo "### Items to generate:"
           echo "$MATRIX" | jq -r '.include[] | "- \(.specification_id) / \(.library)"'
@@ -203,6 +212,9 @@ jobs:
           MODEL: ${{ inputs.model || 'auto' }}
           CHANGE_REQUESTS: ${{ inputs.change_requests || '{}' }}
           REGEN_GATE: ${{ inputs.regen_gate == false && 'false' || 'true' }}
+          # Passed through verbatim; impl-generate validates it and forces the
+          # regeneration whenever it is set.
+          SOURCE_REF: ${{ inputs.source_ref }}
         run: |
           set -u
 
@@ -217,7 +229,7 @@ jobs:
             CHANGE_REQUESTS='{}'
           fi
           flagged_count=$(echo "$CHANGE_REQUESTS" | jq 'length')
-          echo "::notice::Dispatching $total item(s) with ${pace}s pacing between each (model=${MODEL}, change_requests for ${flagged_count} libs)"
+          echo "::notice::Dispatching $total item(s) with ${pace}s pacing between each (model=${MODEL}, change_requests for ${flagged_count} libs, source_ref=${SOURCE_REF:-none})"
 
           i=0
           failed=0
@@ -256,14 +268,16 @@ jobs:
                   -f issue_number="${ISSUE}" \
                   -f model="${MODEL}" \
                   -f change_request="${HINT}" \
-                  -f regen_gate="${REGEN_GATE}" && dispatched=1 && break
+                  -f regen_gate="${REGEN_GATE}" \
+                  -f source_ref="${SOURCE_REF}" && dispatched=1 && break
               else
                 gh workflow run impl-generate.yml --repo "${{ github.repository }}" \
                   -f specification_id="${SPEC_ID}" \
                   -f library="${LIBRARY}" \
                   -f model="${MODEL}" \
                   -f change_request="${HINT}" \
-                  -f regen_gate="${REGEN_GATE}" && dispatched=1 && break
+                  -f regen_gate="${REGEN_GATE}" \
+                  -f source_ref="${SOURCE_REF}" && dispatched=1 && break
               fi
               echo "::warning::Dispatch attempt $attempt failed for ${SPEC_ID}/${LIBRARY}, retrying in 10s"
               sleep 10

agentic/docs/project-guide.md

@@ -793,6 +793,7 @@ The new architecture separates specification and implementation processes:
 **Models:**
 - **Generation model.** `impl-generate.yml` routes each (spec, library) pair: Opus for its first implementation (no implementation file on `origin/main` yet), Sonnet for a regeneration, Opus again for a forced regeneration (`regen_gate=false`). The resolved model is threaded into that PR's repairs and recorded in the PR body (`**Model:** opus`). When a rescue (impl-review-retry, the watchdog, a manual rerun) dispatches `impl-review.yml` or `impl-repair.yml` without a model, they read that line back, so an explicit pin survives; only a PR without it is routed. `impl-review.yml` resolves this value only to forward it to repair. If `origin/main` is unavailable, impl-generate fails instead of guessing, while review and repair warn and assume a first run (Opus).
 - The `model` input (default `auto`) on `impl-generate.yml`, `bulk-generate.yml`, `daily-regen.yml`, `impl-review.yml` and `impl-repair.yml` overrides the generation routing when set to `haiku`, `sonnet` or `opus`; label-triggered runs always route. A pin covers generation and repair, never the review.
+- **Adopt mode.** The `source_ref` input (default empty) on `impl-generate.yml` and `bulk-generate.yml` takes each implementation file from that branch, tag or SHA instead of running Claude, for code written outside the pipeline (a `gemini/*` branch). The run renders the file with the generator's commands, checks the canvas with `core.canvas`, and is always a forced regeneration (`regen:forced`); the routed model only runs repair, the PR body adds a `**Generation:** adopted from …` line above `**Model:**`, and the metadata records `generated_by: adopted:{source_ref}`. Python and JavaScript only. A missing file or a failed render is never retried and never counts toward the 3-attempt cap. See `docs/workflows/overview.md#adopt-implementations-from-a-branch`.
 - **Review model.** Every quality review runs on Opus, whatever model generated the implementation: first implementations, reviews after a repair, and the regen gate's session (the predecessor's re-score and the new render's review). Only `impl-review.yml` chooses it; its `review_model` input (default `opus`) pins another model for one manual review and that run's own auto-retry, and no other workflow forwards one. The retest harness's `models=production` runs every cell on Opus to match (`review_retest.PRODUCTION_MODELS`, kept equal by `tests/unit/workflows/test_model_routing.py`).
 - Fixed models: `spec-create.yml` runs on Opus; the spec polish and cross-library similarity audit in `daily-regen.yml` run on Sonnet. See `docs/workflows/overview.md#pipeline-models`.
 
@@ -847,7 +848,7 @@ uv run python -m automation.scripts.label_manager list
 - **Failure**: < 50 after 4 repairs -> close PR and regenerate
 - **Nothing to repair**: a review below its threshold with a score >= 80, no defect line and every technical criterion (VQ, SC, DQ, CQ) at its maximum -> ai-approved as it stands
 - **During a review** the stored score and review are out of the working tree (the header reads `Quality: pending`, the metadata file is moved aside); the branch history and the pull request still hold them (see `docs/workflows/overview.md`, "Quality workflow")
-- **Regenerations** skip the cascade: one review, and the regen gate (`automation/scripts/regen_gate.py`) replaces the live implementation only when the new score >= the predecessor re-scored in the same review - 1, at least one improvement is visible, and nothing regressed. `regen_gate=false` on `impl-generate.yml` / `bulk-generate.yml` forces the old path (`regen:forced`).
+- **Regenerations** skip the cascade: one review, and the regen gate (`automation/scripts/regen_gate.py`) replaces the live implementation only when the new score >= the predecessor re-scored in the same review - 1, at least one improvement is visible, and nothing regressed. `regen_gate=false` on `impl-generate.yml` / `bulk-generate.yml` forces the old path (`regen:forced`), and so does adopt mode (`source_ref`).
 
 ### Quality Score Labels
 

changelog.d/impl-generate-adopt.md

@@ -0,0 +1,16 @@
+### Added
+
+- **Adopt mode for the implementation pipeline: `source_ref` on
+  `impl-generate.yml` and `bulk-generate.yml`.** With it set, impl-generate
+  takes the implementation file from that branch, tag or commit SHA instead of
+  running Claude, renders it in both themes with the generator's commands,
+  checks the canvas with `core.canvas`, and runs every later step unchanged:
+  metadata, PR, staging upload, review, repair and merge. It exists so code
+  written outside the pipeline, such as a `gemini/*` branch that migrates the
+  323 MUI X implementations to MUI X 9, reaches main only through the Opus
+  review, with metadata, quality score, GCS promotion and labels still owned by
+  the pipeline. An adopted run is always a forced regeneration (`regen:forced`),
+  the routed model only runs repair, the PR body and the metadata
+  (`generated_by: adopted:<ref>`) name the source, and a missing file or a
+  failed render is reported once without a retry, because a retry would read
+  the same file. Python and JavaScript only for now. (#12274)

docs/workflows/overview.md

@@ -124,6 +124,7 @@ Issue + [generate:{library}] label  OR  workflow_dispatch
 impl-generate.yml
   |-- Creates branch: implementation/{spec-id}/{library}
   |-- AI generates code (Python: adaptability self-check, Step 3c)
+  |     or, with source_ref, takes the file from that ref and renders it (adopt mode)
   |-- Fails if the commit changes a dependency file
   |-- Creates metadata/python/{library}.yaml (initial)
   |-- Python: adaptability smoke run, plus the predecessor's render on a regeneration (record-only)
@@ -227,7 +228,7 @@ PR. The same test checks that every pin resolves.
 | `generate:{library}` | Trigger generation for library | User |
 | `impl:{library}:pending` | Generation in progress | Workflow |
 | `impl:{library}:done` | Implementation merged to main | Workflow |
-| `impl:{library}:failed` | Set by `impl-generate.yml` after three failed generation attempts for the pair within a 12-hour window, or by `impl-review.yml` when the PR still scores below 50 after four repair attempts. Infrastructure failures (provider incident, cloud auth, GitHub API) are retried on a separate cap of 5 and never set this label; a pair paused that way carries no label. | Workflow |
+| `impl:{library}:failed` | Set by `impl-generate.yml` after three failed generation attempts for the pair within a 12-hour window, or by `impl-review.yml` when the PR still scores below 50 after four repair attempts. Infrastructure failures (provider incident, cloud auth, GitHub API) are retried on a separate cap of 5 and never set this label; a pair paused that way carries no label. An adopt-mode source failure never sets it either (see [Adopt implementations from a branch](#adopt-implementations-from-a-branch)). | Workflow |
 
 ### PR labels (on Pull Requests)
 
@@ -239,7 +240,7 @@ PR. The same test checks that every pin resolves.
 | `quality:XX` | Quality score (e.g., quality:92) | Workflow |
 | `quality-poor` | Score < 50, needs fundamental fixes | Workflow |
 | `regen` | Regeneration of an implementation that is live on main (marker for humans and the watchdog; `impl-review.yml` decides "regeneration" from `origin/main`) | Workflow |
-| `regen:forced` | Regeneration dispatched with `regen_gate=false`: bypasses the regen gate and takes the fresh-generation path | Workflow |
+| `regen:forced` | Regeneration dispatched with `regen_gate=false` or in adopt mode (`source_ref`): bypasses the regen gate and takes the fresh-generation path | Workflow |
 | `regen:improved` | The regen gate replaced the live implementation (added before `ai-approved`) | Workflow |
 | `regen:kept` | The regen gate kept the live implementation; the PR is closed unmerged | Workflow |
 | `regen:adapt` | The regen gate replaced the live implementation on the adapt route (added after `regen:improved`); one of the two counts the owner reads before turning on `ADAPT_REVIEW_FREE` | Workflow |
@@ -368,7 +369,7 @@ Any failure leaves the stored review as it was and closes the write-back PR; the
 
 So a stored weakness list in the older format lasts only until the pair's first gated regeneration: a merge stores the new render's review, and a keep stores the re-score, both in the defect and suggestion format.
 
-To replace an implementation without the gate, dispatch with `regen_gate=false` (`impl-generate.yml` or `bulk-generate.yml`): the PR is labelled `regen:forced` and takes the fresh-generation path, including the repair loop. If the repairs are exhausted, the PR is closed and the live implementation stays on main; nothing is deleted.
+To replace an implementation without the gate, dispatch with `regen_gate=false` (`impl-generate.yml` or `bulk-generate.yml`): the PR is labelled `regen:forced` and takes the fresh-generation path, including the repair loop. If the repairs are exhausted, the PR is closed and the live implementation stays on main; nothing is deleted. A run in adopt mode (`source_ref`) is always forced this way; see [Adopt implementations from a branch](#adopt-implementations-from-a-branch).
 
 The local `/regen` command (`agentic/commands/regen.md`) is an owner override without the review gate; it only withholds `ai-approved` when the new score is more than one point below the stored score.
 
@@ -391,7 +392,7 @@ Located in `.github/workflows/`:
 | Workflow | Purpose |
 |----------|---------|
 | `spec-create.yml` | Creates new specifications |
-| `impl-generate.yml` | Generates single implementation |
+| `impl-generate.yml` | Generates single implementation, or adopts one from a ref with `source_ref` |
 | `impl-review.yml` | AI quality review |
 | `impl-repair.yml` | Fixes rejected implementations |
 | `impl-merge.yml` | Merges approved PRs, and the review write-back PRs a kept regeneration opens (job `writeback`) |
@@ -426,6 +427,61 @@ gh workflow run bulk-generate.yml -f specification_id=all -f library=matplotlib
 
 **Concurrency limit**: Max 3 parallel implementations globally.
 
+### Adopt implementations from a branch
+
+Adopt mode takes each implementation file from a branch, tag, or commit SHA of
+the repository instead of generating it. Use it for code written outside the
+pipeline, for example a `gemini/*` branch that migrated every MUI X
+implementation to a new major version. The pipeline still owns everything
+else: render, metadata, review, repair, merge, GCS promotion, and labels.
+Nothing from the source branch is merged directly.
+
+To adopt, pass `source_ref` to `bulk-generate.yml` or `impl-generate.yml`:
+
+```bash
+gh workflow run bulk-generate.yml -f specification_id=all -f library=muix -f source_ref=gemini/muix-9
+```
+
+Name the ref as the repository knows it, such as `main` or `gemini/muix-9`. A
+leading `origin/` is dropped. An adopted run differs from a generation in these
+ways:
+
+- **The file comes from the ref.** The step "Adopt implementation from source
+  ref" fetches the ref and takes exactly
+  `plots/{spec-id}/implementations/{language}/{library}{ext}`, the file the
+  generator would write. The Claude steps and the prompts overlay are skipped,
+  and every later step runs unchanged.
+- **It renders the file the way the generator does.** Both themes run with the
+  generator's commands: `python -P` with `MPLBACKEND=Agg` for Python, and the
+  browser render harness for JavaScript. `core.canvas` then checks both PNGs.
+  A drift from the canvas only warns, because the canvas gate in
+  `impl-review.yml` sends it to repair. Python and JavaScript only: an R or
+  Julia run fails with an error.
+- **It's always a forced regeneration.** The PR is labelled `regen:forced`
+  whatever `regen_gate` says, so it takes the fresh-generation path with the
+  repair loop and never the regen gate.
+- **No model writes the code.** The routed model, Opus under `model=auto` as
+  for any forced regeneration, still runs repair. The PR body names the source
+  on a `**Generation:**` line and keeps the repair model on its `**Model:**`
+  line, which review and repair rescues read. The metadata records
+  `generated_by: adopted:{source_ref}`.
+- **A source failure is not retried.** When the file or the ref is missing, or
+  the file doesn't render, the run comments on the spec issue and stops. It
+  isn't retried, it doesn't count toward the 3-attempt cap, and it never sets
+  `impl:{library}:failed`, because every retry would read the same file. Fix
+  the file on the source branch, then dispatch again. A failure of the runner,
+  such as an unreachable origin or a canvas check that can't run, and a failure
+  in a later step, such as the push, the metadata, the upload, or the PR, are
+  retried as usual, and the retry keeps `source_ref`.
+
+Each adopted PR still gets an Opus review. At the default pace of 180 seconds,
+one `bulk-generate.yml` run dispatches about 115 pairs before its 6-hour job
+timeout, so a library across the whole catalogue needs a lower `pace_seconds`
+or several runs. With `specification_id=all`, every spec whose file is missing
+on the source ref gets an "Adoption Failed" comment. The watchdog's generation
+rescues don't carry `source_ref`: a pair the watchdog re-dispatches is
+generated by Claude.
+
 ---
 
 ## Pipeline models
@@ -446,6 +502,7 @@ and threads it into every repair of that pair's PR:
 | First implementation: the pair has no implementation file on `origin/main` yet | Opus |
 | Regeneration: the pair already has an implementation on `origin/main` | Sonnet |
 | Forced regeneration: a regeneration dispatched with `regen_gate=false` (see [Regen gate](#regen-gate-regenerations)) | Opus |
+| Adopt mode (`source_ref`): the file comes from that ref (see [Adopt implementations from a branch](#adopt-implementations-from-a-branch)) | None generates; the routed model runs repair (Opus, as for a forced regeneration) |
 | Explicit `model` input (`haiku`, `sonnet`, or `opus`) | That model, for every pair |
 
 The routing applies whenever nobody chooses a model: a `generate:{library}`