Repository navigation
feat(ci): enable engineer bot mention requests - #511
Conversation
Signed-off-by: Vu Anh Phung <vu.phung@databricks.com>
Signed-off-by: Vu Anh Phung <vu.phung@databricks.com>
There was a problem hiding this comment.
Verdict: 1 Medium
Looks solid overall — the two-job authorize→mention split keeps secrets behind a write-access + non-fork check, reuses the trusted default-branch .bot, and the shared engineer-bot-comment-pr-<N> concurrency group correctly serializes with the follow-up workflow. One medium reliability concern: the local PR-diff step can fail on PRs whose base branch has moved because base.sha isn't fetched into the branch checkout.
Signed-off-by: Vu Anh Phung <vu.phung@databricks.com>
Signed-off-by: Vu Anh Phung <vu.phung@databricks.com>
Signed-off-by: Vu Anh Phung <vu.phung@databricks.com>
Signed-off-by: Vu Anh Phung <vu.phung@databricks.com>
Signed-off-by: Vu Anh Phung <vu.phung@databricks.com>
Configure all six bot workflows to use `databricks-claude-opus-5-5` through the supported per-bot overrides. Pin the shared engine upgrade and align the local SDK/CLI defaults to `0.2.167` / `2.1.300`, with Node 22 for the npm CLI. The engine adds the `[1m]` suffix automatically. Pins merged databricks/databricks-bot-engine#166 (`ac3b08eab993ef14bf997a9bf4c776fa5acb16d8`). This PR is also stacked on #511. Validation: all six workflow callers, action YAML/shell syntax, and engine/SDK/CLI/Node pin consistency passed. The engine upgrade passed CI, including the JFrog SDK install and bundled CLI version check, plus 1,095 local tests (14 skipped).
There was a problem hiding this comment.
Verdict: 3 Low
Looks reasonable overall, with 3 low-severity hardening/UX notes on the new mention workflow. Authorization (write-permission API check, same-repo/open PR check, User-type comment) and concurrency sharing with the follow-up workflow look correct. I couldn't verify engine-side flags (--phase mention, --bot) or the bumped SDK/CLI versions from this repo.
85239b4 to
89fd6cc
Compare
There was a problem hiding this comment.
Verdict: 1 Medium · 1 Low
The mention workflow mostly follows the existing follow-up workflow's security pattern: default-branch bot setup, a fork check, a write-permission gate, and a JFrog scrub. One medium issue: the shared concurrency group can silently drop queued requests. One low hardening point: the untrusted diff is concatenated into the system prompt. Nit, scope: the PR also moves every bot workflow to databricks-claude-opus-5-5 via new *_MODEL_OVERRIDE vars, and bumps Node 20→22 and the SDK/CLI versions. The description doesn't mention any of this, so please call it out for reviewers.
Signed-off-by: Vu Anh Phung <vu.phung@databricks.com>
There was a problem hiding this comment.
Verdict: 2 Low
Looks good overall, with 2 low-severity concerns about how the mention workflow builds the diff and its offline module cache. The security setup holds up: write access is checked before any secrets are requested, the PR must come from this repo, bot setup and caches come from the default branch, and JFrog/OIDC credentials are scrubbed before the agent runs.
Signed-off-by: Vu Anh Phung <vu.phung@databricks.com>
There was a problem hiding this comment.
Verdict: 1 Low
Looks good overall, with one low concern about the PR head moving while a request waits in the queue. The security setup is sound: write-access check before any secrets are used, same-repo PRs only, prompts and tool binaries from the trusted default branch, JFrog/OIDC credentials removed before the agent runs, and the diff reference file created exclusively, made read-only and git-excluded. The model, engine and SDK/CLI version bumps are applied consistently across all six workflows.
Signed-off-by: Vu Anh Phung <vu.phung@databricks.com>
There was a problem hiding this comment.
Verdict: 1 Low
Looks good, with one low-severity concern: the PR state isn't re-checked when a queued request finally runs. The setup is well isolated: prompts and cache warming come from the default branch, tools come from the trusted checkout, bin/ and test-results/ are gitignored, and the trust model matches the existing follow-up workflow. queue: max and the engine's --phase mention support at the pinned SHA could not be verified from this checkout.
Signed-off-by: Vu Anh Phung <vu.phung@databricks.com>
There was a problem hiding this comment.
Verdict: 1 Low
Looks good, with one low-severity robustness issue. The trust boundaries are handled well: the bot loads its setup from the default branch, removes registry credentials before the agent runs, rechecks the PR after queueing, keeps the diff as a read-only file outside git, and copies tool binaries with symlink checks.
Signed-off-by: Vu Anh Phung <vu.phung@databricks.com>
There was a problem hiding this comment.
Verdict: 2 Low
Looks good overall, with two low-severity concerns. The mention workflow gates on write access, runs only on same-repo PRs, loads its prompts, tools and cache from the trusted default branch, and removes the JFrog credentials before the agent runs. The model, engine and SDK bumps are applied the same way across all six workflows.
| echo "ACTIONS_ID_TOKEN_REQUEST_TOKEN=" | ||
| echo "ACTIONS_ID_TOKEN_REQUEST_URL=" | ||
| } >> "$GITHUB_ENV" | ||
|
|
There was a problem hiding this comment.
🔵 Low — Recheck PR eligibility re-reads the PR's state, head repo and head ref after the queue wait, but it doesn't re-check the commenter's write permission. Requests can sit in the shared queue for a long time (up to 100 pending, 30–45 min each), so a request from someone whose access was revoked meanwhile will still run with the App token and model secrets. The PR description says "Eligibility is rechecked before PR checkout", which suggests the full gate. Consider repeating the collaborators/$COMMENTER/permission lookup here (pass github.event.comment.user.login through) and failing closed, as authorize does. Separately, the .head.ref == $ref comparison is effectively a no-op because GitHub doesn't allow a PR's head branch to change.
Allow maintainers with write access to invoke the engineer bot with a top-level
@botcomment on an open, same-repository PR. Review requests inspect the diff without edits; change requests can push fixes and reply. Mention and follow-up writers share a queue of up to 100 pending requests. Eligibility is rechecked before PR checkout. Prompts and cache warming use the trusted default branch; PR diffs are separate read-only reference files, with the base refreshed after queueing and the checked-out head SHA included in replies. The trigger activates once the workflow lands onmain.This PR includes merged #512: all six bot workflows use
databricks-claude-opus-5-5, with engine commitac3b08eab993ef14bf997a9bf4c776fa5acb16d8from databricks/databricks-bot-engine#166. SDK/CLI defaults are0.2.167/2.1.300, with Node 22 for the npm CLI. The engine adds[1m]automatically.Validation: 39 local workflow scenarios and shell checks passed, including permission lookup failures, closed/merged PRs, and head changes after queueing. actionlint 1.7.12 passed with a temporary compatibility filter for the new
queuekey;queue: maxwas verified against GitHub's current syntax documentation. Diff checks passed.