Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 3 additions & 1 deletion doc/code/scenarios/2_custom_scenario_parameters.ipynb
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,8 @@
"scorers, baseline). This guide is about parameters that scenario authors add\n",
"on their own classes.\n",
"\n",
"Metadata inspection below needs no model configuration. Recorded \"fallback default objective scorer\" logs are historical; execution now requires explicitly configured scoring and generation roles. The CLI examples deliberately run the target initializer with those role-specific environment settings.\n",
"\n",
"## Declaring a parameter\n",
"\n",
"`Parameter` is the unified declaration shared by initializers and scenarios.\n",
Expand Down Expand Up @@ -77,7 +79,7 @@
}
],
"source": [
"from pyrit.scenario.airt.scam import Scam\n",
"from pyrit.scenario.scenarios.airt import Scam\n",
"from pyrit.setup import initialize_pyrit_async\n",
"from pyrit.setup.initializers import TechniqueInitializer\n",
"\n",
Expand Down
4 changes: 3 additions & 1 deletion doc/code/scenarios/2_custom_scenario_parameters.py
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,8 @@
# scorers, baseline). This guide is about parameters that scenario authors add
# on their own classes.
#
# Metadata inspection below needs no model configuration. Recorded "fallback default objective scorer" logs are historical; execution now requires explicitly configured scoring and generation roles. The CLI examples deliberately run the target initializer with those role-specific environment settings.
#
# ## Declaring a parameter
#
# `Parameter` is the unified declaration shared by initializers and scenarios.
Expand Down Expand Up @@ -49,7 +51,7 @@
# scorers):

# %%
from pyrit.scenario.airt.scam import Scam
from pyrit.scenario.scenarios.airt import Scam
from pyrit.setup import initialize_pyrit_async
from pyrit.setup.initializers import TechniqueInitializer

Expand Down
15 changes: 14 additions & 1 deletion doc/code/setup/0_setup.md
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,20 @@ from pyrit.setup.initializers import ScorerInitializer, TargetInitializer
await initialize_pyrit_async(memory_db_type="InMemory", initializers=[TargetInitializer(), ScorerInitializer()])
```

This configuration allows you to run most PyRIT notebooks immediately.
This registers targets and scorers from your environment configuration. Custom
initializers and directly constructed targets are also supported.

## Scan targets

You can browse and configure scenarios without model credentials. Before launching,
select the target to test and any scoring or adversarial targets the chosen technique
needs, or configure them through an initializer. Deterministic scorers do not need a
scoring model, and techniques without generation do not need an adversarial model.

In CoPyRIT, you can create targets and scorers from the launch form without losing
your draft. Missing roles are shown before a scan starts rather than filled with an
implicit OpenAI target. To use the same target for multiple roles, select it for each.
See the [scanner options](../../scanner/0_scanner.md#running-scans) for CLI selections.

## Configuration Options

Expand Down
4 changes: 3 additions & 1 deletion doc/code/setup/1_configuration.ipynb
Original file line number Diff line number Diff line change
Expand Up @@ -121,7 +121,9 @@
"\n",
"The recommended step to setup PyRIT is that it needs access to secrets and endpoints. These can be loaded in environment variables or put in a `.env` file. See `.env_example` for how this file is formatted.\n",
"\n",
"Each target has default environment variables to look for. For example, `OpenAIChatTarget` looks for the `OPENAI_CHAT_ENDPOINT` for its endpoint, `OPENAI_CHAT_MODEL` for its model name, and `OPENAI_CHAT_KEY` for its key. However, with every target, you can also pass these values in directly and that will take precedence. For Azure endpoints with Entra ID authentication, pass a token provider from `pyrit.auth` as the `api_key`."
"Each target has default environment variables to look for. For example, `OpenAIChatTarget` looks for the `OPENAI_CHAT_ENDPOINT` for its endpoint, `OPENAI_CHAT_MODEL` for its model name, and `OPENAI_CHAT_KEY` for its key. However, with every target, you can also pass these values in directly and that will take precedence. For Azure endpoints with Entra ID authentication, pass a token provider from `pyrit.auth` as the `api_key`.\n",
"\n",
"Environment resolution happens only in deliberately invoked constructors or initializers. Scenarios never construct an implicit OpenAI helper: select/register each required scoring and generation role before scanning. See [explicit scan targets and migration](0_setup.md#explicit-scan-targets-and-migration)."
]
},
{
Expand Down
2 changes: 2 additions & 0 deletions doc/code/setup/1_configuration.py
Original file line number Diff line number Diff line change
Expand Up @@ -52,6 +52,8 @@
# The recommended step to setup PyRIT is that it needs access to secrets and endpoints. These can be loaded in environment variables or put in a `.env` file. See `.env_example` for how this file is formatted.
#
# Each target has default environment variables to look for. For example, `OpenAIChatTarget` looks for the `OPENAI_CHAT_ENDPOINT` for its endpoint, `OPENAI_CHAT_MODEL` for its model name, and `OPENAI_CHAT_KEY` for its key. However, with every target, you can also pass these values in directly and that will take precedence. For Azure endpoints with Entra ID authentication, pass a token provider from `pyrit.auth` as the `api_key`.
#
# Environment resolution happens only in deliberately invoked constructors or initializers. Scenarios never construct an implicit OpenAI helper: select/register each required scoring and generation role before scanning. See [explicit scan targets and migration](0_setup.md#explicit-scan-targets-and-migration).

# %%
import os
Expand Down
7 changes: 6 additions & 1 deletion doc/code/setup/2_resiliency.ipynb
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,8 @@
"\n",
"PyRIT provides multiple layers of retry and resiliency mechanisms to handle failures gracefully during security testing. This notebook explains the different retry mechanisms, how they work together, and how to configure them for your use case.\n",
"\n",
"Recorded output below predates explicit scan roles. Historical \"fallback default objective scorer\" messages describe a scoring rubric, not automatic target provisioning. Configure each required model role explicitly, as in the example below.\n",
"\n",
"## Overview: Different Levels of Retry Mechanisms\n",
"\n",
"PyRIT implements retries at multiple levels, each serving a different purpose:\n",
Expand Down Expand Up @@ -202,7 +204,7 @@
],
"source": [
"from pyrit.prompt_target import OpenAIChatTarget\n",
"from pyrit.scenario.foundry import FoundryTechnique, RedTeamAgent\n",
"from pyrit.scenario.scenarios.foundry import FoundryTechnique, RedTeamAgent\n",
"from pyrit.setup import IN_MEMORY, initialize_pyrit_async\n",
"from pyrit.setup.initializers import LoadDefaultDatasets, TechniqueInitializer\n",
"\n",
Expand All @@ -214,13 +216,15 @@
") # type: ignore\n",
"\n",
"objective_target = OpenAIChatTarget()\n",
"scoring_target = OpenAIChatTarget()\n",
"\n",
"# Create a scenario with retry configuration\n",
"scenario = RedTeamAgent()\n",
"\n",
"scenario.set_params_from_args( # type: ignore\n",
" args={\n",
" \"objective_target\": objective_target,\n",
" \"scoring_target\": scoring_target,\n",
" \"max_concurrency\": 5,\n",
" \"max_retries\": 3,\n",
" \"scenario_techniques\": [FoundryTechnique.Base64],\n",
Expand Down Expand Up @@ -300,6 +304,7 @@
"resumed_scenario.set_params_from_args(\n",
" args={\n",
" \"objective_target\": objective_target,\n",
" \"scoring_target\": scoring_target,\n",
" \"scenario_techniques\": [FoundryTechnique.Base64],\n",
" }\n",
")\n",
Expand Down
7 changes: 6 additions & 1 deletion doc/code/setup/2_resiliency.py
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,8 @@
#
# PyRIT provides multiple layers of retry and resiliency mechanisms to handle failures gracefully during security testing. This notebook explains the different retry mechanisms, how they work together, and how to configure them for your use case.
#
# Recorded output below predates explicit scan roles. Historical "fallback default objective scorer" messages describe a scoring rubric, not automatic target provisioning. Configure each required model role explicitly, as in the example below.
#
# ## Overview: Different Levels of Retry Mechanisms
#
# PyRIT implements retries at multiple levels, each serving a different purpose:
Expand Down Expand Up @@ -122,7 +124,7 @@

# %%
from pyrit.prompt_target import OpenAIChatTarget
from pyrit.scenario.foundry import FoundryTechnique, RedTeamAgent
from pyrit.scenario.scenarios.foundry import FoundryTechnique, RedTeamAgent
from pyrit.setup import IN_MEMORY, initialize_pyrit_async
from pyrit.setup.initializers import LoadDefaultDatasets, TechniqueInitializer

Expand All @@ -134,13 +136,15 @@
) # type: ignore

objective_target = OpenAIChatTarget()
scoring_target = OpenAIChatTarget()

# Create a scenario with retry configuration
scenario = RedTeamAgent()

scenario.set_params_from_args( # type: ignore
args={
"objective_target": objective_target,
"scoring_target": scoring_target,
"max_concurrency": 5,
"max_retries": 3,
"scenario_techniques": [FoundryTechnique.Base64],
Expand Down Expand Up @@ -210,6 +214,7 @@
# resumed_scenario.set_params_from_args(
# args={
# "objective_target": objective_target,
# "scoring_target": scoring_target,
# "scenario_techniques": [FoundryTechnique.Base64],
# }
# )
Expand Down
15 changes: 15 additions & 0 deletions doc/scanner/0_scanner.md
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,21 @@ PyRIT provides two command-line interfaces:
pyrit_scan run foundry.red_team_agent --target openai_chat --initializers target --techniques base64
```

Targets must already be registered or be created by an explicitly requested
initializer. Select helper roles separately when needed:

```bash
pyrit_scan run technique.execute --target model_under_test --techniques prompt_sending \
--scoring-target scoring_model
```

`--adversarial-target generation_model` selects the generation role for generative
techniques and simulated conversations. These same options work in `pyrit_shell`.
They do not automatically reuse `--target`, and no missing role constructs OpenAI
from ambient environment variables. A configured deterministic objective scorer
needs no scoring model; techniques without generation need no adversarial model.
See [scan targets](../code/setup/0_setup.md#scan-targets) for setup options.

### Connecting to CoPyRIT

Point a local configuration file at the remote backend:
Expand Down
Loading
Loading