Repository navigation
Description
ty's unsound-assignment rule is switched off in pyproject.toml (unsound-assignment = "ignore" under [tool.ty.rules], where everything else runs at all = "error"). This issue tracks turning it on.
What the rule catches
It flags a variable annotated with a concrete type that is assigned a value the checker only knows as Any:
# pyrit/common/cli_helpers.py: getattr returns Any, so nothing checks that this is an int
level_value: int = getattr(logging, level_upper)The annotation states a type that nothing verified. If the value is something else, type checking still passes and the failure shows up later at runtime, far from where the wrong value came in. With the rule on, every such annotation becomes a checked boundary.
Current state
Measured at fdeb9c5 with ty 0.0.99 on Python 3.14: 267 findings in 45 files.
uv run --frozen --extra all ty check pyrit --error unsound-assignment --output-format concise
| Group | Findings | Files | What they are |
|---|---|---|---|
pyrit/memory/memory_models.py |
196 | 1 | The standard SQLAlchemy declarative idiom: 155 x col: Mapped[T] = mapped_column(...) and 41 x rel: Mapped[T] = relationship(...). One root cause, not 196 separate problems. |
| Everything else | 71 | 44 | Individual Any values (JSON, getattr, config and environment lookups, untyped third-party returns) assigned to annotated variables. |
One PR or several? Several.
- There are two different kinds of work here.
memory_models.pyis a tooling decision:x: Mapped[T] = mapped_column(...)is how SQLAlchemy 2.0 models are meant to be written, so the right answer is one scoped override, not rewriting 196 declarations in the file our migrations depend on. The other 71 findings each need a local judgment about whether the value really has the annotated type, and some of them may turn out to be real unchecked assumptions. - Reviewability. The 71 findings are spread over 44 files in 14 packages. A fix that adds validation or narrowing changes behavior on bad input and needs a test next to the code. That cannot be reviewed carefully in one sweep.
- Merge conflicts. Small per-package PRs rebase easily against the many PRs that are open at any time; one PR touching 45 files does not.
- No backsliding while the work is in progress. Step 1 below turns the rule on immediately, so new code is checked from day one and the remaining findings can only go down.
Plan
Step 1 (UA-0), one PR, must merge before anything else starts. Configuration only, no source changes:
- Delete
unsound-assignment = "ignore". - Add a permanent
[[tool.ty.overrides]]entry forpyrit/memory/memory_models.pythat ignores the rule, with a comment explaining the SQLAlchemy idiom. - Add a second, temporary
[[tool.ty.overrides]]entry that ignores the rule for the 44 remaining files listed below, with a comment pointing to this issue. - Result:
ty check pyritis clean and every file not on the temporary list is checked.
Step 2, in parallel, one PR per chunk. Fix the findings in the chunk and remove its files from the temporary override.
| ID | Scope | Findings |
|---|---|---|
| UA-1 | pyrit/executor/promptgen/gcg/attack/base/attack_manager.py |
12 |
| UA-2 | rest of pyrit/executor, plus pyrit/scenario, pyrit/converter, pyrit/prompt_normalizer |
7 |
| UA-3 | pyrit/prompt_target, pyrit/embedding, pyrit/auth |
13 |
| UA-4 | pyrit/backend |
9 |
| UA-5 | pyrit/score |
7 |
| UA-6 | pyrit/models, pyrit/registry |
11 |
| UA-7 | pyrit/memory (without memory_models.py), pyrit/cli, pyrit/common |
12 |
Step 3. Whoever empties the temporary override deletes it, and this issue can be closed.
How to fix a finding
In order of preference:
- Fix the type at the source. If the
Anycomes from one of our own functions, give that function a real return type. - Check the value when it comes from outside the program (JSON, HTTP responses, environment, files): narrow with
isinstanceor validate, and raise a clear error otherwise. Add a test for the rejected case. cast(...)only when the type is guaranteed by construction, with a short comment saying why.
Do not add # type: ignore, and do not weaken the annotation to Any to silence the rule. If a finding turns out to be a real bug, say so in the PR description.
The 44 files outside memory_models.py, with finding counts
pyrit/auth/browser_session_copilot_authenticator.py(1)pyrit/backend/middleware/runtime.py(1)pyrit/backend/routes/targets.py(1)pyrit/backend/routes/version.py(3)pyrit/backend/services/attack_service.py(2)pyrit/backend/services/finding_export.py(1)pyrit/backend/services/scenario_service.py(1)pyrit/cli/api_client.py(2)pyrit/cli/pyrit_scan.py(2)pyrit/common/cli_helpers.py(1)pyrit/common/yaml_loadable.py(1)pyrit/converter/decomposition_converter.py(1)pyrit/embedding/openai_text_embedding.py(2)pyrit/executor/attack/multi_turn/tree_of_attacks.py(1)pyrit/executor/attack/single_turn/many_shot_jailbreak.py(1)pyrit/executor/promptgen/gcg/attack/base/attack_manager.py(12)pyrit/memory/alembic/env.py(1)pyrit/memory/attack_analytics.py(1)pyrit/memory/azure_sql_memory.py(2)pyrit/memory/memory_interface.py(2)pyrit/models/harm_category.py(2)pyrit/models/messages/conversations.py(1)pyrit/models/parameter.py(1)pyrit/models/score/observation.py(1)pyrit/models/score/scorable.py(1)pyrit/prompt_normalizer/json_retry.py(1)pyrit/prompt_normalizer/prompt_normalizer.py(1)pyrit/prompt_target/azure_blob_storage_target.py(1)pyrit/prompt_target/common/discover_target_capabilities.py(1)pyrit/prompt_target/openai/openai_error_handling.py(3)pyrit/prompt_target/openai/openai_response_target.py(1)pyrit/prompt_target/prompt_shield_target.py(4)pyrit/registry/components/target_registry.py(1)pyrit/registry/registry_recipe_restore.py(1)pyrit/registry/registry_recipe_rules.py(2)pyrit/registry/registry_recipe_save.py(1)pyrit/scenario/core/attack_technique_factory.py(1)pyrit/scenario/scenarios/foundry/assert_policy.py(1)pyrit/score/float_scale/local_violence_classifier_scorer.py(1)pyrit/score/llm_scoring.py(1)pyrit/score/observation/otel_execution_trace_source.py(1)pyrit/score/scorer.py(2)pyrit/score/scorer_prompt_validator.py(1)pyrit/score/true_false/prompt_shield_scorer.py(1)
Status — who's doing what
(Maintained automatically; last updated 2026-11-20. Claim rules are below — first comment wins, no confirmation needed.)
| ID | Scope | Claimed by | Since | PR | State |
|---|---|---|---|---|---|
| UA-0 | configuration (gates all others) | Mohammad Saeed (@mss06s) | 2026-11-18 | #4407 | open |
| UA-1 | gcg/attack/base/attack_manager.py |
— | available | ||
| UA-2 | rest of executor, scenario, converter, prompt_normalizer |
— | available | ||
| UA-3 | prompt_target, embedding, auth |
Mehreen Munir (@clossal) | 2026-11-19 | waiting on UA-0 | |
| UA-4 | backend |
SteeveCrypto (@Steeve-Crypto) | 2026-11-20 | waiting on UA-0 | |
| UA-5 | score |
a12san (@sainimal1ba-hue) | 2026-11-20 | waiting on UA-0 | |
| UA-6 | models, registry |
Shawn Russ (@shawnrruss) | 2026-11-19 | waiting on UA-0 | |
| UA-7 | memory (minus memory_models.py), cli, common |
— | available |
How to contribute
- Claim one chunk at a time, by its ID. Before you start, comment on this issue with the ID you are taking (for example
UA-4). - First comment wins. If yours is the first comment claiming that ID, go ahead and open a PR; you do not need to wait for a reply. If someone has already claimed it, pick a different ID.
- UA-0 goes first. UA-1 to UA-7 can be claimed at any time, but their PRs can only be opened once the UA-0 PR is merged.
- Link this issue and the ID in your PR description.
Activity
mss06s commentedon Nov 18, 2026
I’ll take UA-0.
MAINT: Keep unsound-assignment off for unit tests (UA-0)
clossal commentedon Nov 19, 2026
Hi, I'd like to take UA-3 (
pyrit/prompt_target,pyrit/embedding, andpyrit/auth) if it's still available.I've reproduced all 13
unsound-assignmentfindings locally and started investigating the affected code and existing tests.Could you assign UA-3 to me? I'll keep the changes focused on the assigned scope and add regression tests where appropriate.
shawnrruss commentedon Nov 19, 2026
I’ll take UA-6 (
pyrit/modelsandpyrit/registry). I’ll keep the changes scoped to that chunk and wait for UA-0 to merge before opening a PR.Steeve-Crypto commentedon Nov 20, 2026
I'll take UA-4 (
pyrit/backend). I'll keep it scoped to that chunk and wait for UA-0 to merge before opening a PR.sainimal1ba-hue commentedon Nov 20, 2026
I will take UA-5