Repository navigation
FEAT Configure scan targets before launch - #3794
Merged
Richard Lundeen (richlundeen) merged 12 commits intoNov 10, 2026
Merged
Richard Lundeen (richlundeen) merged 12 commits into
Richard Lundeen (richlundeen) merged 12 commits into
Conversation
Decouple scenario discovery from runtime roles, validate selected dependencies before admission, and preserve explicit setup and saved identities across Python, API, CLI, GUI, and suites. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Preserve explicit API connection-field validation in the imported HuggingFace event-loop regression by supplying its test-only token. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Bind benchmark and converter dependencies, align effective factories and actual consumer capabilities, preserve scorer rubrics, and repair launch correction/confirmation UX. Cover frozen suite references at real registry admission. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Retain positive reuse and deliberate fresh edit-reruns, assert earlier drift rejection before datasets or new results, and configure the helper explicitly when constructing synthetic old saved runs. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Check every actual shared-helper consumer before work, including accepted simulated seeds with uses_adversarial=False. Preserve native helper success, objective-incompatible SKIP behavior and configured translation adaptation. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Defer packaged compatibility reads until fixture execution. Compare complete target registry identity deltas and persisted offline run dependencies instead of prohibiting other specs' explicit OpenAI-shaped registrations. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Keep setup focused on selecting only needed scan roles and preserving environment-based initialization. Remove implementation and review-detail exposition. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Preserve explicit scan roles through launch-form extraction and scenario run guards. Update readiness consumer mocks without changing production fallbacks or presentation. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Update paired documentation, suite contracts and readiness fixtures to the public import policy without behavior or notebook output changes. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Assert disabled launch and actionable missing-target readiness before selecting the objective target. Preserve target-free estimates, direct API rejection and exact dataset execution checks. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Richard Lundeen (richlundeen)
approved these changes
Nov 10, 2026
Richard Lundeen (richlundeen)
left a comment
Contributor
There was a problem hiding this comment.
Automated review completed. No blocking concerns remain. This review records source coverage; merging is handled separately.
Richard Lundeen (richlundeen)
force-pushed
the
fix/3703-credential-free-scenario-catalog
branch
from
November 10, 2026 18:53
c798414 to
99d7997
Compare
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Richard Lundeen (richlundeen)
enabled auto-merge
November 10, 2026 19:41
Julio César Suástegui (juliosuas)
added a commit
to juliosuas/PyRIT
that referenced
this pull request
Nov 10, 2026
Resolve conflicts with microsoft#3794 (configure scan targets before launch): - scenario_configuration_resolver: keep the exact-selection imports and the readiness/adversarial-override imports; resolve_configuration_async now accepts and forwards scoring_target_name so launch, estimate and dataset load keep the explicit scoring target with an exact selection. - scenario_run_service: optional launch fields keep both scoring_target_name and example_selection. - tests: patch resolve_target on the resolver class, as main's tests do, and cover scoring_target_name together with an exact selection.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Let users configure a scan before connecting a model. Browsing scenarios and selecting techniques no longer requires
OPENAI_CHAT_MODELor creates an OpenAI target behind the scenes.Scanner with
OPENAI_CHAT_MODELunsetBoth screenshots are real production builds with the variable unset and no configured targets or scorers. Before is the pre-change baseline (
80f28af); after is this PR (99d7997). No scan or provider request was submitted.Credit
Fixes #3703. Thank you Aditya Malkar (@adimalkar). You found and pinned down this bug, and your original fix in this PR came first and was in good shape. I had made overlapping changes for the scan-target configuration work in #3926, so I pushed that branch here so both land as one change. Your diagnosis shaped the lazy default-scorer approach this uses.