Repository navigation
Description
Is your feature request related to a problem? Please describe.
PyRIT now requires Python >=3.11 (currently <3.15), and Ruff targets py311. We can use Python 3.11 features consistently across supported versions, simplify older compatibility patterns, and improve typing, diagnostics, and async lifecycle management. Some features are already partially adopted; the opportunities below are investigations, not instructions to replace every existing pattern.
Describe the solution you'd like
How to participate: ONE opportunity at a time
Claim exactly ONE checklist item at a time by commenting on this issue thread with its ID (for example, P311-01) and a short description of what you intend to investigate. Do not start multiple items or claim a bundle. Wait for confirmation here before treating the item as reserved. Claims are processed in comment order, and this issue is checked approximately every three hours.
- You must open a PR within five days (120 hours) of your claim comment. The table below records the due date. If no PR is opened by then, the claim expires, the item becomes available, and your issue assignment is removed if you have no other active contribution on this issue. You can claim it again if it is still available.
- Opening a PR within the deadline retains your claim while that PR is active. You cannot claim another item until your current PR is merged or the current contribution is explicitly released/closed.
- Link this issue and the item ID in your PR description. Keep the PR focused on that one opportunity. Follow repository instructions and add relevant behavior/type-checking coverage.
- Investigate first. Preserve public contracts and intended behavior. If a feature is not a good fit, explain why on this thread rather than forcing a change. An investigation can conclude that no code change is appropriate.
- A PR opened without a prior claim will be highlighted to Roman Lutz (@romanlutz). If the item is unclaimed, we can link the PR and assign the item to its author. If someone else has already claimed it or another PR overlaps, Roman Lutz (@romanlutz) will assess how to proceed; no existing claim will be displaced automatically.
- Checkboxes mean completed, not claimed. Check an item only after its PR is merged, or after a maintainer explicitly accepts a no-change investigation. Ownership and PR links are tracked separately below.
Opportunity checklist
Typing
- P311-01: Extend
typing.Selfadoption. Investigate older same-concrete-class factories and fluent APIs, such aspyrit/common/yaml_loadable.py:YamlLoadable.from_yaml_file. Replace bound-TypeVar boilerplate only where it really modelsSelf; retain independent generic parameters. Merged scope: #3194 covers YAML/seed and scorer-metrics loaders, strategy-context copying, and technique collections, with type-regression coverage. Additional merged scope: #3201 covers the threeMessagefactories andAtomicAttackIdentifier.build, including subclass type-regression tests. Additional merged scope: #3266 coversSeedDataset.from_dict,MessageScorable.from_message, andContentScorable.from_message, with subclass runtime andassert_typecoverage and unchanged validation. Additional merged scope: #3383 coversAttackTechniqueSeedGroup.from_system_promptandfrom_messages, with unchangedclsconstruction/validation and subclass runtime/assert_typecoverage. Additional merged scope: #3411 coversMessagePiece.adversarial_placeholderwith unchangedclsconstruction and subclass runtime/assert_typecoverage. Additional merged scope: #3470 coversTagQuery.all,any_of, andnone_of, with subclass runtime/assert_typecoverage and unchanged predicates/validation. Additional merged scope: #3520 coversScenarioRunSizeEstimate.unavailable, preservingclsconstruction, unavailable status, defaults, dataset/limit handling and validation, with subclass runtime/assert_typeand input/default coverage. Additional merged scope: #3529 coversHarmDefinition.from_yaml, with subclass runtime/assert_typecoverage and unchanged YAML loading, path resolution, field conversion, source-path recording and validation. Additional merged scope: #3577 annotatesJsonResponseConfig.from_metadatawithSelfand adds subclass runtime/assert_typecoverage; the factory runtime body, schema parsing, defaults, validation and metadata round trips are unchanged. Additional merged scope: #3626 annotatesWebSearchConfig.from_metadatawithSelf, with subclass runtime/assert_typecoverage of absent and explicit policies. Construction, read-onlyMappinginputs, default/disabled/required modes, frozen/extra-forbid validation, errors, non-mutation and metadata/JSON round trips are unchanged. Additional merged scope: #3779 annotatesAdversarialPrompts.from_metadatawithSelf | Noneand adds base/subclass runtime andassert_typechecks, absent-association cases, read-only mapping, non-mutation, JSON round-trip and validation coverage. The factory body and seed semantics are unchanged. Additional merged scope: #3823 annotatesAttackResultMetadata.from_metadatawithSelfand adds subclass runtime/assert_typecoverage. Its factory body, defaults, unknown-role fallback, invalid-position handling and parent-linkage separation are unchanged. Additional merged scope: #3884 annotatesTokenUsage.from_metadatawithSelf | Noneand checks subclass construction and static optional narrowing. Its factory body, absent-metadataNone, counter filtering, extra counters and metadata round trips are unchanged. Additional merged scope: #3897 annotatesRequestTraceContext.from_metadatawithSelf | Noneand checks subclass construction, optional narrowing, absent metadata and input non-mutation. Its runtime factory body, trace validation and metadata semantics are unchanged. Further merged contribution: #3992 completes subclass-preservingSelf | Nonetyping forToolExecutionMetadata.from_metadata; runtime factory behavior is unchanged, with focused subclass/type and non-mutation coverage. Additional merged scope: #3999 annotatesSeedIdentifier.from_seedwithSelfand adds base/subclass runtime andassert_typecoverage. The final merged factory still delegates tocls.of(...); its executable body, behavioral parameter projection, hashes, promoted fields and serialization are unchanged. Slai.life (@huong8373tt-beep)'s reservation is released. No further focused case is identified here; propose a concrete subclass-preserving factory or fluent-API gap before claiming more work. The broad opportunity remains unchecked pending maintainer acceptance. Preserve fixed-baseTagQuery.__and__/__or__andComponentIdentifier.with_eval_hash, independent generics and factories such asSeedPrompt.from_value_with_required_parametersthat can return an existing base input. - P311-02: Improve dictionary contracts with
TypedDict,Required, andNotRequired. Investigate internal metadata/provider payloads currently typed asdict[str, Any]. Distinguish absent keys from nullable values.RawAnalyticsProfilealready provides prior art. Do not replace Pydantic runtime validation. - P311-03: Add exhaustive dispatch checks with
assert_never/Never. Investigate closed enum/union dispatch for outcomes, states, modalities, or capabilities so added cases surface during type checking. Apply only where handling is genuinely exhaustive, not to intentionally open extension points. Merged scope: #3441 makesAttackStrategy._log_attack_outcomeexhaustive acrossAttackOutcome, preserving existing log messages and rejecting unknown values. Additional merged scope: #3456 explicitly handlesScenarioResultView.CONVERSATIONSand guards both CLI dispatchers withassert_never, preserving HTML full-report override, default view, limits and lazy imports. Additional merged scope: #3472 makesAttackResultAnalytics._optionexhaustive for all fourAttackAnalyticsValueKindmembers, preserving display labels and stored keys, with an actual-source enum-extension type regression. Additional merged scope: #3504 explicitly handles all threePuzzleTypemembers inPuzzledConverter.convert_asyncand adds anassert_neverguard with a runtime regression. Constructor validation, crossword fallback, RNG behavior, Unicode handling, clues and output are preserved. Additional merged scope: #3601 covers all fiveCodeAttackConverter._encodeencodings with a type-onlyassert_neverguard, preserving the runtime invalid-stateValueError, constructor validation, template pairing and output. Per-encoding, invalid-state and actual-source type checks cover the change. Additional merged scope: #3654 adds an exhaustiveassert_neverguard for the validatedRepeatTokenConvertermode dispatch, preserving its string-Literal API,Nonedefaulting tosplit, validation errors/messages, punctuation, spacing, output and identifiers. The unreachable fallback uses the existing coverage exclusion pattern; changed-line coverage passed at 100%. Additional merged scope: #3797 makesPromptShieldTarget._input_parserexhaustive for its three declared Literal values with aTYPE_CHECKING-onlyassert_never. Shared parsing preserves forced-field early returns,Noneand unknown-runtime-field fallback, and trailing document text; a regression covers the fallback. Additional merged scope: #3830 makesHumanLabeledDataset.from_csventry construction exhaustive forMetricsType.HARMandMetricsType.OBJECTIVE, with a type-onlyassert_never. CSV parsing, validation and the unknown-runtime-value objective fallback are preserved. Additional merged scope: #3924 makesScorerEvaluator.from_scorerconstructor selection exhaustive for bothMetricsTypemembers with a type-only guard. Omitted-type inference, explicit overrides and the historical unknown-runtime-value harm fallback are preserved. The final merged tests include the actual factory source checked against current and extended enums, addressing the coverage gap. Additional merged scope: #3998 makesTargetConfiguration.ensure_can_handleexhaustive forUnsupportedCapabilityBehaviorRAISE/ADAPTwith a type-only guard. Native-support and configured-normalizer early returns, missing-policy handling, error messages and the historical runtime fallback are preserved. The final tests check the actual method source against current and extended enums for Python 3.11/3.12, so removing the guard is detected. B Sainath Reddy (@Sainath-Reddy7)'s reservation is released. No further focused case is identified here; propose a concrete closed-dispatch gap before claiming more work. The broad opportunity remains unchecked pending maintainer acceptance. Keep open string, custom-callable and provider-extension dispatch unchanged. - P311-04: Add type-regression checks with
assert_type. Investigate registry factories, generic results, decorators, and subclass-returning APIs. Ensure checks run through the type checker;assert_typeis not a runtime assertion. Additional merged scope: #3625 addsassert_type(results, list[int])forgather_with_cleanup_asyncand checks ordered values intests/unit/test_async_utils.py. The canonicalmake tycheckstests/unit; the generic helper, public signature and cancellation/cleanup behavior are unchanged. Additional merged scope: #3686 adds concrete and protocol-consumerassert_typechecks forDefaultInstanceRegistry[T].get,get_entryandget_all_instances, retainingT | None,RegistryEntry[T] | Noneandlist[RegistryEntry[T]]. Present/missing retrieval, identity and sorted order are checked; the protocol consumer uses a helper parameter to avoid concrete assignment narrowing. Production signatures and runtime behavior are unchanged. Additional merged scope: #3770 adds fourassert_typechecks for the raw singleton-factory results ofSlowConstructingRegistryandOtherSlowConstructingRegistrybefore runtime narrowing. Identity, subclass isolation, synchronization, timeouts and cleanup are preserved; production signatures and behavior are unchanged. Assessment: sufficiently addressed for this selective opportunity. Existing CI-checked coverage now protects generic gather results, concrete and protocol instance-registry retrieval, and subclass-returning factories including this singleton case. Controlled annotation regressions fail these checks. The bounded assessment found no additional worthwhile typing case to prescribe; this is not an exhaustive audit or a requirement to add assertions to every registry/generic API. The separate backend timing-test change in TEST: Assert Registry.get_registry_singleton return types (P311-04) #3770 is CI-reliability work, not P311-04 scope. Its traversal-oracle caveat does not reopen the completed typing case. - P311-05: Investigate
LiteralStringat static-template boundaries. Consider developer-authored SQL or command templates where the type checker can enforce the distinction. Confirm checker support. This is not runtime sanitization and never replaces parameterized SQL. - P311-06: Investigate
TypeVarTuple/Unpackfor variadic types. Look for a concrete heterogeneous-tuple or numerical/multimodal API that benefits. Do not add variadic complexity without a real need or assume this automatically checks NumPy shapes. Features standardized after 3.11 still need compatible backports. - P311-07: Investigate generic
TypedDict/NamedTuplecontainers. Identify payload containers whose content type should be parameterized. GenericTypedDictin analytics result analysis provides existing prior art.
Async execution and diagnostics
- P311-08: Enrich propagated exceptions with
BaseException.add_note. Add useful execution, technique, dataset, or conversation context while preserving exception types and tracebacks. Notes appear in formatted tracebacks, notstr(error). Exclude secrets and sensitive prompt content. Merged scope: #3175 adds dataset/provider notes to concurrent provider-fetch failures. Additional merged scope: #3277 adds scenario and atomic-attack names to caught worker-pool exceptions inScenario._execute_atomic_attacks_parallel_async, preserving the original exceptions and existing single/grouped failure behavior. Regression coverage checks grouped notes and single-failure identity. Additional merged scope: #3281 labels cached-backend and analytics close failures inclose_services_async, preserving the original exceptions, cancellation and single/grouped behavior. Additional merged scope: #3333 adds safe static phase notes for scenario-service shutdown and memory disposal inRuntimeLifecycle._shutdown_runtime_async. Existing notes, original exception identity/type, cleanup order, cancellation and single/grouped behavior remain intact. Additional merged scope: #3351 adds safe static session-release, active-client-stop and failed-startup-client-stop notes inGitHubCopilotTarget._cleanup_owned_resources_async, with regression coverage for phase labels and original errors. Complete: the identified dataset/provider, scenario-worker, backend service-close, runtime-shutdown and Copilot-cleanup context gaps are addressed. Current-code assessment found no further concrete, worthwhile propagation-context gap requiring another contribution for this item. Other examined groups already describe their operation or preserve ordered/chained causes; this is not a requirement to annotate every exception. Retain original exception identity/type/tracebacks, prior notes, cancellation, cleanup/retry ownership and single/grouped behavior; never add secrets or sensitive prompt content. - P311-09: Use scoped operation timeouts with
asyncio.timeout. Investigate multi-await operations that should share one timeout scope instead of restarting a timeout for every step. Existing analytics, objective-generation, observation, and A2A usage provides prior art. Preserve cancellation and cleanup behavior. Merged scope: #3205 gives each Copilot receive loop one overall budget across streamed messages, with separate handshake/response budgets and cancellation/cleanup coverage. Completed: #3239 givesWebSocketTarget._initialize_connection_asyncone separate receive budget across all discarded initial messages, with aggregate-budget and cancellation/connection-cleanup coverage. Initialization sends, restoration, and normal responses retain their separate scopes. Current timeout-site assessment found no further worthwhile shared-budget migration: remainingwait_forcalls bound individual operations, explicitly per-attempt probes, or already use a shared remaining deadline. Do not mechanically replace those calls. - P311-10: Propagate absolute deadlines with
asyncio.timeout_at. Investigate nested operations/retries that should share one monotonic-clock deadline. Distinguish operation deadlines from per-step timeouts and keep cleanup semantics explicit. Merged scope: #3610 bounds_run_invocation_asyncinpyrit/_code_execution/docker_client.pywithasyncio.timeout_at(deadline)during shielded Docker client startup and passes that same monotonic deadline into invocation handling. Capture and attached-client waiting already deducted elapsed time before this change; the new bound is startup. Startup-specific errors and cleanup ownership remain intact. Assessment: sufficiently addressed; no further worthwhile production migration was demonstrated in the inspected current timeout/retry surfaces. Existing A2A polling, WebSocket initialization, Copilot receive streaming and objective-generation execution already use shared scoped budgets, while QueryControl already computes one absolute deadline. Preserve independently bounded control commands and cancellation/container/workspace cleanup outside the expired invocation budget. The added test checks deadline wiring; behavioral expiry/cancellation regression coverage is an optional test follow-up, not an additional production migration or active reservation. - P311-11: Investigate structured concurrency with
asyncio.TaskGroup. Identify parent-scoped child tasks with appropriate fail-fast semantics. The message-send service already uses it. It is not a blanketgatherreplacement or a concurrency limiter: sibling cancellation and exception shapes differ. Preserve ordered results and the deliberate cancellation/cleanup behavior inpyrit/common/task_utils.py. Merged scope: #3959 covers_AnthropicModelWrittenEvalsDataset._fetch_dataset_async: ordered file fetches now use an awaited owned task containing a TaskGroup. First-error identity and caller cancellation are preserved, including the Python 3.11/3.12 internal-cancellation-count regression; failure and outer cancellation drain async siblings. File/cache writes already running in threads cannot be interrupted. Further merged scope: #3980 coversConverter._convert_token_regions_asyncusing the existinggather_with_cleanup_asynchelper, not a new TaskGroup migration. Failed or externally cancelled selected-region conversions drain async siblings while preserving result order, original errors and caller cancellation counts. File writes already running in threads remain non-interruptible. Víctor Fernández Ayllón (@vayllon301)'s reservation is released. No further focused case is identified here; propose a concrete remaining gap before claiming more work. The broad structured-concurrency opportunity remains unchecked pending maintainer acceptance. - P311-12: Investigate
ExceptionGroup/except*handling. Preserve multiple failures and, where useful, handle matching subsets without losing other errors. Groups already exist in scenario execution and service shutdown. Carefully assess public exception contracts and cancellation (BaseExceptionGroup). Focused contribution merged: #3983 preserves shutdown errors and disposal for mixed groups, propagates live-apply cancellation groups, retains secondary Anthropic fetch failures as notes, and includes nested child failures and explicit causes in scenario diagnostics. Ordinary-error output and the original Anthropic exception contract remain unchanged, with focused regression coverage. Muhammad Waqar Anwar (@WaqarAnwar)'s reservation is released. No further focused case is identified here; propose a concrete remaining gap before claiming more work. The broad opportunity remains unchecked pending maintainer acceptance. Claiming the investigation does not preapprove behavior changes or blanket migrations. - P311-13: Investigate explicit task context with
asyncio.create_task(..., context=...). Identify background operations that need selected tracing/execution ContextVars or a clean context. Default task creation already copies the current context; do not change it gratuitously. Investigation reported: @WaqarAnwar's findings report no current production bug and an execution-context carryover risk when a scheduled run fails in its own task rather than an attack worker. The report used a mock scenario and did not inspect every task creation call; this is not an independently verified production reproducer. A clean context for each scheduled run is proposed, not accepted. No change or completion is recorded; maintainer acceptance of the investigation is pending. - P311-14: Investigate
asyncio.Runnerfor synchronous entry points. Consider multiple top-level calls sharing one managed loop/context in a shell or lifecycle harness. This does not permit nested loops inside an already-running notebook event loop. Preserve resource and loop ownership. Current investigation (claimed by Slai.life (@huong8373tt-beep)): Report finds no justified Runner change in the one-shot scan/migration CLIs, persistent shell loop, synchronous embedding facade or platform-specific browser-authentication loop. This is the contributor's reported conclusion, not independently verified acceptance. The opportunity remains unchecked pending maintainer acceptance; no mechanical substitution is requested.
Standard-library simplifications
- P311-15: Parse project TOML with
tomllib. Investigate replacing the version-extraction regex inbuild_scripts/stamp_compatibility.pywith structured access toproject.version. Preserve validation/provenance behavior.tomllibreads TOML; it does not write it. Completed: #3208 replaces the version regex with structuredproject.versionparsing and covers table ordering, quoting, invalid metadata, and provenance errors. - P311-16: Simplify ISO timestamp parsing. Investigate removing trailing-
Zcompatibility rewrites, such as_RemoteDatasetLoader._parse_datetime, because 3.11 parses validZtimestamps directly. Preserve timezone and accepted-input contracts. Do not edit immutable historical Alembic revisions merely for cleanup. Completed: #3176 removes the active loader shim and preserves invalid/non-string fallback behavior. Post-merge search found no other actionable ISO-parsing compatibility shims inpyritorbuild_scripts; the remaining rewrite in historical migrationd7e9f1a3b5c6is intentionally immutable. - P311-17: Assess
enum.StrEnumfor appropriate enums. It is already used for harm categories and message-send models. Existingstr, Enummigrations can changestr(member)and formatting, and Ruff's automatic replacement is deliberately disabled. Focus on cases where string behavior is intended and compatibility is understood. Merged scope: #3850 migratesConversationRetryReasontoStrEnum.str()and formatting intentionally now returnjson_parsinginstead of the class-qualified name; members, equality/hash/repr, JSON, value lookup and theConversationRetrymodel remain compatible. Current focused case (claimed by Julio César Suástegui (@juliosuas)): investigateSeedOrigininpyrit/models/seeds/seed_origin.py. Check qualified string/format consumers, equality/hash/repr, schema/JSON and persisted-seed round trips before deciding whether to migrate. Do not assume every existing string enum is safe. - P311-18: Investigate atomic regex groups and possessive quantifiers. Consider
(?>...),*+, or++in a concrete scorer/parser pattern with unnecessary backtracking on large model outputs. Measure behavior and performance; these constructs change matching semantics and are not a universal regex-safety fix. Merged scope: #3702 uses atomic groups and whole-flag-token guards inShellCommandOutputScorer's destructivermbranch. Historical detection cases and malformed/repeated-input regressions preserve the tested heuristic behavior; measured long-input families scale approximately linearly. Assessment: sufficiently addressed for this concrete opportunity. The actual squash and assessed current code have identical scorer/test files. This is not a proof for every regex/input or a requirement to migrate every pattern; retain command-separator behavior,/tmpexclusion and whole flag tokens. The scorer evaluates text, not shell execution. Additional merged scope: #3764 adds multiline Markdown-image detection with possessive scanner patterns and a CommonMark block parser. Reference normalization, code/HTML context, custom-pattern opt-out and bounded malformed-input regressions are covered. The actual merged scorer/tests and dependency files are unchanged in assessed current main. Existing regex matching is retained, so this is not complete CommonMark equivalence or a globally linear scorer claim. It does not change the completed shell assessment or require a blanket migration. The earlier unclaimed item overlap remains in the historical row; no reservation was retroactively granted. Additional merged scope: #3687 deliberately uses possessive shell-word/option scans and atomic operand alternatives while changing command-substitution detection semantics. Twelve checked-in malformed-input shapes time regex search below 0.5 seconds and separately check the public false verdict. This is supplemental performance-bounded use, not a semantics-preserving migration or a second completion. The inherited destructive-shell atomic pattern is unchanged and remains counted under FIX: Make ShellCommandOutputScorer destructive-shell regex linear time #3702. The merged shell scorer and dedicated tests match assessed current main; the general regex tests later changed default-pattern identity assertions separately. Shell-oracle and growth results in the PR are author-reported, not independently rerun here, and do not establish global linearity or atomic-versus-nonatomic equivalence. No blanket migration or new reservation is needed. - P311-19: Investigate
hashlib.file_digestfor whole-file hashes. Consider classifier artifact verification or other complete-file checks. Keep blocking I/O off async paths. Do not replace bounded evidence-reading loops that enforce budgets/cancellation and retain content. Merged scope: #3392 streams the two classifiercompute_dataset_hashes()helpers through binaryhashlib.file_digesthandles, preserving SHA-256 values and dictionary order; multi-buffer regression tests cover current file contents. Assessment: sufficiently addressed. Keep_load_training_rows()hashing the same bytes it subsequently parses; keepLocalFileSurfaceSource._read_entrybounded/cancellable with retained evidence. Async serializer hashing supports storage backends that may be remote, memory media hashing reuses copied content, andFileDocumentStorage._compute_versionreceives bytes already in memory. No further worthwhile direct whole-file replacement identified; do not add synchronous local-file reads to async storage paths. - P311-20: Investigate
inspect.getmembers_staticfor safe introspection. Identify registry inspection that should avoid invoking descriptors/properties. It intentionally bypasses dynamic lookup and lazy exports, so verify the discovery contract before adoption. Merged scope: #3531 statically inspectsdiscover_in_directorymodules, resolves only explicitly declared missing__all__exports dynamically, and logs failed declared exports without losing other classes. Registration-outcome tests cover declared lazy exports with/without__dir__, unrelated dynamic names and failed exports; the earlier lazy-export regression is addressed. Additional merged scope: #3580 statically inspectsInitializerRegistry._module_defined_initializersfor single-file discovery and custom scripts, with targeted resolution of declared lazy exports. Own-module, base-class and abstract filters remain intact. Best-effort discovery logs failed declared exports; strict script loading and custom-script validation still reject them. Registration and strict/best-effort entry-point regressions cover these contracts. Assessment: the registry inspection opportunity is sufficiently addressed by fix: use static member discovery in registry inspection #3531 and FIX avoid dynamic attribute evaluation in single-file initializer discovery #3580; no further worthwhile blanket migration was found. KeepRegistry._discover's intentional lazy-export materialization dynamic.
SQLite-specific memory opportunities
These must stay in the SQLite-specific memory/driver layer and preserve portable SQL Server behavior. Check what SQLAlchemy/aiosqlite exposes before choosing an implementation.
- P311-21: Improve SQLite error handling using extended error codes/names. Analytics already checks
sqlite_errorcodefor plainSQLITE_BUSY. Investigate other real cases where structured codes are preferable to error-message matching, including distinctions such as stale snapshots. Completed scope: #3893 adds the driver'ssqlite_errorcode/sqlite_errornameas deduplicated exception notes on propagated SQLite analytics failures, making distinctions such asSQLITE_BUSY_SNAPSHOTvisible. Plain-BUSY retry policy, exception identity/messages and non-SQLite behavior are unchanged. The current bounded memory-handler survey found no worthwhile remaining message-based SQLite classification to replace; existing structured-code and domain-state handling stays intact. This completes the implemented diagnostics case, not a new retry policy or blanket database migration, and does not retroactively accept the earlier no-change proposal. - P311-22: Investigate SQLite connection
getlimit/setlimit. Identify a real need for connection-level runtime limits and how limits interact with pooling and workloads. Avoid arbitrary limits that break existing behavior. Current focused case (claimed by Pedro Holanda (@AoiUltramar)): investigate SQLite runtime bind-variable limits and the conservative batching constants inMemoryInterfaceandMemorySession. The proposal and preliminary benchmark are not accepted implementation evidence. Verify limits on the actual pooled connection rather than assuming a separate probe connection represents it, preserve Azure SQL behavior, and report reproducible pinned-environment measurements before proposing a capped SQLite-only change or no-change conclusion. - P311-23: Investigate SQLite custom window functions. Assess whether an actual analytics use case benefits from
create_window_function, with SQLite runtime support, driver integration, and portability considered. Prefer existing SQL when sufficient. - P311-24: Investigate incremental SQLite BLOB I/O. Assess whether any actual large-BLOB storage path benefits from
blobopen. Do not redesign media storage just to use this API; a no-change conclusion may be appropriate. - P311-25: Investigate SQLite database serialization/deserialization. Assess a concrete snapshot/test/export use case for
serialize/deserialize, including runtime availability, memory cost, concurrency, and driver boundaries.
Claim and PR tracking
The tracker is authoritative for per-item ownership; GitHub issue assignees are the aggregate set of active contributors. Available items have no active tracker row. Preserve released/expired rows as history when updating the tracker.
| Item | Contributor | Claim comment / source | Claimed on | PR due | PR | Status |
|---|---|---|---|---|---|---|
| P311-14 | Slai.life (@huong8373tt-beep) | Claim comment | 2026-11-06 | 2026-11-11 | None yet | Investigation reported no focused change; maintainer acceptance pending; GitHub assignment unavailable |
| P311-22 | Pedro Holanda (@AoiUltramar) | Claim comment | 2026-11-06 | 2026-11-11 | None yet | Claimed; focused SQLite runtime-limit and batch-size investigation; GitHub assignment unavailable |
| P311-01 | Slai.life (@huong8373tt-beep) | Claim comment | 2026-11-06 | 2026-11-11 | #3999 | ✅ PR merged 2026-11-06; focused SeedIdentifier factory typing complete; reservation released |
| P311-01 | Slai.life (@huong8373tt-beep) | Claim comment | 2026-11-05 | 2026-11-10 | #3992 | ✅ PR merged 2026-11-06; ToolExecutionMetadata factory typing complete; reservation released; SeedIdentifier factory case available |
| P311-03 | B Sainath Reddy (@Sainath-Reddy7) | Claim comment | 2026-11-05 | 2026-11-10 | #3998 | ✅ PR merged 2026-11-06; capability-behavior dispatch and typing regression complete; reservation released |
| P311-12 | Muhammad Waqar Anwar (@WaqarAnwar) | Claim comment | 2026-11-05 | 2026-11-10 | #3983 | ✅ PR merged 2026-11-06; focused exception-group investigation complete; reservation released |
| P311-11 | Víctor Fernández Ayllón (@vayllon301) | Claim comment | 2026-11-05 | 2026-11-10 | #3980 | ✅ PR merged 2026-11-06; selected-region sibling cleanup complete; reservation released |
| P311-11 | Muhammad Waqar Anwar (@WaqarAnwar) | Claim comment | 2026-11-04 | 2026-11-09 | #3959 | ✅ PR merged 2026-11-05; focused Anthropic file-fetch case complete; reservation released; selected-region converter case available |
| P311-03 | B Sainath Reddy (@Sainath-Reddy7) | Claim comment | 2026-11-04 | 2026-11-09 | #3924 | ✅ PR merged 2026-11-05; focused ScorerEvaluator dispatch and typing regression complete; reservation released; capability-behavior case available |
| P311-17 | Julio César Suástegui (@juliosuas) | Claim comment | 2026-11-04 | 2026-11-09 | None yet | Claimed; focused SeedOrigin investigation reserved; GitHub assignment unavailable |
| P311-01 | Slai.life (@huong8373tt-beep) | Claim comment | 2026-11-03 | 2026-11-08 | #3897 | ✅ PR merged 2026-11-04; focused RequestTraceContext factory complete; reservation released; ToolExecutionMetadata case available |
| P311-01 | Slai.life (@huong8373tt-beep) | Claim comment | 2026-11-03 | 2026-11-08 | #3884 | ✅ PR merged 2026-11-03; focused TokenUsage factory complete; reservation released; RequestTraceContext case available |
| P311-17 | Julio César Suástegui (@juliosuas) | Claim comment | 2026-11-02 | 2026-11-07 | #3850 | ✅ PR merged 2026-11-03; focused ConversationRetryReason case complete; reservation released; SeedOrigin investigation available |
| P311-03 | Muhammad Waqar Anwar (@WaqarAnwar) | Claim comment | 2026-11-02 | 2026-11-07 | #3830 | ✅ PR merged 2026-11-03; dataset-entry dispatch complete; reservation released; evaluator dispatch available |
| P311-01 | Slai.life (@huong8373tt-beep) | Claim comment | 2026-11-02 | 2026-11-07 | #3823 | ✅ PR merged 2026-11-03; metadata factory typing complete; reservation released; TokenUsage case available |
| P311-02 | XY (@XY-code1) | Claim comment | 2026-11-02 | 2026-11-07 | None yet | Claimed; focused internal payload investigation reserved; GitHub assignment unavailable |
| P311-21 | B Sainath Reddy (@Sainath-Reddy7) | Claim comment | 2026-11-01 | 2026-11-06 | #3893 | ✅ PR merged 2026-11-04; SQLite codes/names diagnostics assessed complete; reservation released; earlier survey retained as background |
| P311-18 | Jaafer Rahmani (@bIackr0se) | PR opened without prior claim | 2026-10-31 | N/A; not assigned | #3764 | ✅ PR merged 2026-11-02; Markdown-image scope complete; historical unclaimed overlap with #3702 retained; no reservation granted |
| P311-18 | Muhammad Waqar Anwar (@WaqarAnwar) | PR opened without prior claim | 2026-10-31 | N/A; PR already open | #3702 | ✅ PR merged 2026-11-01; destructive-shell scope complete; reservation released; prior GitHub assignment unavailable |
| P311-03 | Slai.life (@huong8373tt-beep) | Claim comment | 2026-10-31 | 2026-11-05 | #3797 | ✅ PR merged 2026-11-02; PromptShield dispatch complete; reservation released; dataset-entry dispatch available |
| P311-03 | Slai.life (@huong8373tt-beep) | Claim comment | 2026-10-29 | 2026-11-03 | #3654 | ✅ PR merged 2026-10-31; RepeatToken dispatch complete; reservation released; PromptShield parser available |
| P311-04 | Muhammad Waqar Anwar (@WaqarAnwar) | Request comment | 2026-11-01 | N/A; not assigned | #3770 | ✅ PR merged 2026-11-01; singleton checks complete; no active reservation; opened while #3702 active (historical one-item conflict) |
| P311-04 | mehmetl33t | Claim comment | 2026-10-30 | 2026-11-04 | #3686 | ✅ PR merged 2026-10-31; instance-registry checks complete; reservation released; singleton-factory checks available |
| P311-04 | Slai.life (@huong8373tt-beep) | Claim comment | 2026-10-29 | 2026-11-03 | #3625 | ✅ PR merged 2026-10-29; gather result-type check complete; reservation released; instance-registry type checks available |
| P311-01 | Julio César Suástegui (@juliosuas) | Claim comment | 2026-11-01 | 2026-11-06 | #3779 | ✅ PR merged 2026-11-02; adversarial-prompts factory complete; reservation released; attack-result metadata factory available |
| P311-01 | Julio César Suástegui (@juliosuas) | Request comment | 2026-10-28 | 2026-11-02 | #3626 | ✅ PR merged 2026-10-31; WebSearchConfig factory complete; reservation released; adversarial-prompts factory available |
| P311-01 | Slai.life (@huong8373tt-beep) | Claim, withdrawal | 2026-10-29 | N/A | None | Withdrawn 2026-10-29 before assignment; no reservation |
| P311-10 | Slai.life (@huong8373tt-beep) | Claim comment | 2026-10-28 | 2026-11-02 | #3610 | ✅ PR merged 2026-10-29; Docker sandbox invocation deadline complete; opportunity complete; reservation released |
| P311-20 | Slai.life (@huong8373tt-beep) | Claim comment | 2026-10-27 | 2026-11-01 | #3580 | ✅ PR merged 2026-10-28; single-file inspection complete; opportunity complete; reservation released |
| P311-03 | Julio César Suástegui (@juliosuas) | Claim comment | 2026-10-27 | 2026-11-01 | #3601 | ✅ PR merged 2026-10-28; CodeAttack encoding dispatch complete; reservation released; RepeatToken mode dispatch available |
| P311-01 | Slai.life (@huong8373tt-beep) | Claim comment | 2026-10-26 | 2026-10-31 | #3577 | ✅ PR merged 2026-10-27; JSON response factory complete; reservation released; web-search policy factory available |
| P311-20 | Slai.life (@huong8373tt-beep) | Claim comment | 2026-10-25 | 2026-10-30 | #3531 | ✅ PR merged 2026-10-26; directory discovery complete; reservation released; single-file initializer inspection available |
| P311-01 | Julio César Suástegui (@juliosuas) | Claim comment | 2026-10-25 | 2026-10-30 | #3529 | ✅ PR merged 2026-10-26; HarmDefinition factory complete; reservation released; JSON response factory available |
| P311-01 | Niladri Das (@bniladridas) | Claim comment | 2026-10-24 | 2026-10-29 | #3520 | ✅ PR merged 2026-10-25; run-size estimate factory complete; HarmDefinition factory available, reservation released |
| P311-03 | Slai.life (@huong8373tt-beep) | Claim comment | 2026-10-24 | 2026-10-29 | #3504 | Closed without merging 2026-10-25; reopened 2026-10-26; ✅ PR merged 2026-10-26; puzzle dispatch complete; reservation released; CodeAttack encoding dispatch available |
| P311-03 | Slai.life (@huong8373tt-beep) | Claim comment | 2026-10-23 | 2026-10-28 | #3472 | ✅ PR merged 2026-10-23; analytics option-kind dispatch complete; puzzle dispatch available, reservation released |
| P311-01 | Julio César Suástegui (@juliosuas) | Claim comment | 2026-10-22 | 2026-10-27 | #3470 | ✅ PR merged 2026-10-23; TagQuery factory family complete; run-size estimate factory available, reservation released |
| P311-03 | Slai.life (@huong8373tt-beep) | Claim comment | 2026-10-22 | 2026-10-27 | #3456 | ✅ PR merged 2026-10-23; CLI result-view dispatch complete; analytics option dispatch available, reservation released |
| P311-03 | Slai.life (@huong8373tt-beep) | Claim comment | 2026-10-21 | 2026-10-26 | #3441 | ✅ PR merged 2026-10-22; AttackOutcome logging complete; CLI view dispatch available, reservation released |
| P311-01 | Slai.life (@huong8373tt-beep) | Claim comment | 2026-10-21 | 2026-10-26 | #3411 | ✅ PR merged 2026-10-21; placeholder factory complete; further TagQuery family available, reservation released |
| P311-01 | Slai.life (@huong8373tt-beep) | Claim comment | 2026-10-20 | 2026-10-25 | #3383 | ✅ PR merged 2026-10-21; factory family complete; remaining MessagePiece case available, reservation released |
| P311-19 | ARJUN K (@arjun-k047) | Claim comment | 2026-10-20 | 2026-10-25 | #3392 | ✅ PR merged 2026-10-20; P311-19 complete; reservation released |
| P311-08 | Slai.life (@huong8373tt-beep) | Claim comment | 2026-10-19 | 2026-10-24 | #3351 | ✅ PR merged 2026-10-20; opportunity complete; reservation ended |
| P311-08 | Slai.life (@huong8373tt-beep) | Claim comment | 2026-10-19 | 2026-10-24 | #3333 | ✅ PR merged 2026-10-19; contribution complete; Copilot cleanup-phase investigation available, reservation released |
| P311-08 | Slai.life (@huong8373tt-beep) | Claim comment | 2026-10-17 | 2026-10-22 | #3281 | ✅ PR merged 2026-10-19; contribution complete; runtime-shutdown phase context remains available, reservation released |
| P311-08 | Slai.life (@huong8373tt-beep) | Claim comment | 2026-10-16 | 2026-10-21 | #3277 | ✅ PR merged 2026-10-17; contribution complete; service-shutdown context remains available, reservation released |
| P311-01 | 成功したナイロンカエル (@vito233) | Claim comment | 2026-10-16 | 2026-10-21 | #3266 | ✅ PR merged 2026-10-17; contribution complete; further model factory cases available, reservation released |
| P311-09 | Aizen__28 (@Yudeeswaran) | Claim comment | 2026-10-15 | 2026-10-20 | #3239 | ✅ PR merged 2026-10-16; opportunity complete; reservation ended |
| P311-15 | Shivam Maurya (@shivampwm2020) | Claim comment | 2026-10-14 | 2026-10-19 | #3208 | ✅ PR merged 2026-10-14; opportunity complete; reservation ended |
| P311-01 | Kushal Talati (@kushaltalati) | Claim comment | 2026-10-14 | 2026-10-19 | #3201 | ✅ PR merged 2026-10-14; contribution complete; further model factory cases available, reservation released |
| P311-09 | Aizen__28 (@Yudeeswaran) | Claim comment | 2026-10-14 | 2026-10-19 | #3205 | ✅ PR merged 2026-10-14; contribution complete; initialization timeout investigation available, reservation released |
| P311-01 | Kushal Talati (@kushaltalati) | Claim comment | 2026-10-13 | 2026-10-18 | #3194 | ✅ PR merged 2026-10-14; contribution complete; remaining cases available, reservation released |
| P311-08 | Aizen__28 (@Yudeeswaran) | Claim comment | 2026-10-13 | 2026-10-18 | #3175 | ✅ PR merged 2026-10-13; contribution complete; remaining cases available, reservation released |
| P311-16 | Kushal Talati (@kushaltalati) | Claim comment | 2026-10-13 | 2026-10-18 | #3176 | ✅ PR merged 2026-10-13; opportunity complete; reservation ended |
| P311-04 | Roman Lutz (@romanlutz) | Historical PR, opened before this issue; no claim | N/A | N/A | #3064 | ✅ PR merged; prior art covers part of the opportunity; remaining cases available |
Describe alternatives you've considered, if relevant
Keep existing implementations when they already express the required behavior clearly. Avoid a repo-wide modernization sweep: each contributor investigates and opens a focused PR for ONE opportunity at a time. Separate changes within an item should remain cohesive; coordinate here before splitting or expanding scope.
Additional context
- Python 3.11 release notes
- Article motivating this inventory
- Existing adoption includes
datetime.UTC,Self,NotRequired, genericTypedDict,StrEnum,TaskGroup, timeouts, exception groups, and SQLite extended error codes. - Better traceback locations and interpreter improvements arrive automatically on 3.11+; they do not need modernization PRs, and model-call performance is often network-bound.
typing.override,typing.TypeAliasType, thetype Alias = ...statement, andclass Foo[T]syntax are 3.12+, not 3.11. Keep neededtyping_extensionsimports while supporting 3.11.- Other 3.11 APIs such as
dataclass_transformcan be considered when there is a concrete fitting abstraction, but should not motivate new abstractions on their own.
Activity
romanlutz commentedon Oct 13, 2026
Roman Lutz (@romanlutz): P311-04 has related work in #3064, which opened on 2026-10-09 at 22:00:12 UTC, before this tracking issue existed, and merged on 2026-10-13 at 07:06:53 UTC. There was no prior claim here, so this is historical overlap, not a contributor bypassing the new claim rule.
That PR adds
assert_typechecks for retry helpers and async task return types, and brings the unit tests into themake tycheck. It partially addresses the type-regression opportunity. I am linking it as prior art but leaving P311-04 unchecked and available pending your assessment of the remaining scope. No duplicate claim exists and no new assignment is needed for the already-merged PR.Yudeeswaran commentedon Oct 13, 2026
Claiming P311-08 (
BaseException.add_note).I’ll investigate dataset-fetch failure propagation in
SeedDatasetProvider.fetch_datasets_async(pyrit/datasets/seed_datasets/seed_dataset_provider.py). Providers are fetched concurrently, and a failing provider’s exception may propagate without identifying the dataset (e.g., “SeedDataset cannot be empty”).I plan to add diagnostic context using
add_note(), including the dataset name and provider class, while preserving the original exception type and traceback.I’ll add a regression test to verify that the original exception is preserved and that the diagnostic note contains the expected dataset and provider information, without exposing prompt content or other sensitive data.
romanlutz commentedon Oct 13, 2026
Assigning P311-08 to you Aizen__28 (@Yudeeswaran).
kushaltalati commentedon Oct 13, 2026
Claiming P311-16 (Simplify ISO timestamp parsing).
I'll remove the .replace("Z", "+00:00") workaround in _parse_datetime since Python 3.11 handles it natively. Historical Alembic revisions will remain untouched. I'll preserve timezone and accepted-input contracts and update test coverage.
romanlutz commentedon Oct 13, 2026
Assigning P311-16 to you Kushal Talati (@kushaltalati). Linked #3176 and Aizen__28 (@Yudeeswaran)'s #3175 in the table.
romanlutz commentedon Oct 13, 2026
Post-merge assessment for P311-08: #3175 is merged and successfully covers dataset/provider context on concurrent dataset-fetch failures. Thank you Aizen__28 (@Yudeeswaran); your contribution is complete and your reservation is released, so you may claim another opportunity.
The broader exception-note opportunity remains unchecked and available. I assessed the merged diff and current main at
20b35221c514b7f0898caac12f14b71631889027and added two concrete follow-up cases to the checklist:Scenario._run_atomic_attacks_parallel_async: caught errors are stored without their atomic-attack association before aggregation. Safe scenario/atomic-group notes could identify which concurrent operation failed without logging seed text.close_services_async: exception notes could identify which close operation produced each grouped shutdown error.These are focused investigations, not instructions to annotate every exception. Preserve cancellation, exception identities/types, group contracts, and sensitive-data boundaries. Claim exactly one opportunity on this thread before starting.
kushaltalati commentedon Oct 13, 2026
PR #3176 is approved with all CI checks passing. Once merged, I would like to claim P311-01 (Extend typing.Self adoption).
I plan to investigate adopting typing.Self across classmethod factories and fluent methods (such as YamlLoadable.from_yaml_file and Seed.from_yaml_file) while preserving independent generic parameters.
romanlutz commentedon Oct 13, 2026
P311-16 is complete with #3176. No other active ISO-parsing workarounds remain; the historical migration stays unchanged. Assigning P311-01 to you Kushal Talati (@kushaltalati) now that #3176 is merged.
203 remaining items
Sainath-Reddy7 commentedon Nov 6, 2026
PR for the reserved P311-03
TargetConfiguration.ensure_can_handlecase: #3998 (opened within the claim window, due 2026-11-10).romanlutz commentedon Nov 6, 2026
#3992 completes the ToolExecutionMetadata case. Thanks Slai.life (@huong8373tt-beep); your reservation is released. P311-01 remains open for the SeedIdentifier factory case described above. Linked the timely P311-03 PR #3998, B Sainath Reddy (@Sainath-Reddy7).
huong8373tt-beep commentedon Nov 6, 2026
Claiming P311-01, the focused
SeedIdentifier.from_seedcase described in the tracker.I will investigate changing the factory return annotation to
Self, since it delegates tocls.of(...), which constructs through the subclass. I will add focused subclass runtime andassert_typecoverage while preserving behavioral parameter projection, content/evaluation hashes, promoted fields, serialization, and the existing runtime factory behavior. I will keep the scope limited to this factory and its directly coupled tests, and will open the PR within the claim window.Slai Agent Runtime performed the investigation, implementation, and test execution. The account owner authorized this contribution and is responsible for its content.
romanlutz commentedon Nov 6, 2026
#3983 is merged. Thanks Muhammad Waqar Anwar (@WaqarAnwar); the focused exception-group investigation is complete and your reservation is released. P311-12 remains unchecked pending maintainer acceptance of the broader opportunity; any further claim should identify a concrete remaining gap.
romanlutz commentedon Nov 6, 2026
#3980 is merged. Thanks Víctor Fernández Ayllón (@vayllon301); selected-region sibling cleanup is complete and your reservation is released. It reuses the existing cleanup helper rather than adding TaskGroup. P311-11 remains unchecked pending maintainer acceptance of the broader opportunity.
romanlutz commentedon Nov 6, 2026
Assigning P311-01 to you Slai.life (@huong8373tt-beep) and linking #3999. Please change its
Closes #3166reference toRelates to #3166; this PR completes one focused case, not the whole checklist.romanlutz commentedon Nov 6, 2026
#3998 is merged. Thanks B Sainath Reddy (@Sainath-Reddy7); the capability-behavior dispatch and its typing regression are complete, and your reservation is released. P311-03 remains unchecked pending maintainer acceptance of the broader opportunity.
WaqarAnwar commentedon Nov 6, 2026
Hello Roman Lutz (@romanlutz) ,
P311-13: Investigation result
Is there a bug? Not today, but there is one weak spot.
What I checked: a background task gets a copy of its creator's context, so a queued scenario run could inherit values from the failed run before it. I traced all 18 ContextVars in
pyrit/and the main long-lived background tasks, but didn't read everycreate_taskcall. I also tested failed runs followed by a queued run, using a mock scenario that copies the real structure.What I found:
Context(), so nothing is needed here.The weak spot: when the test failed in the run's own task instead of a worker, the value did reach the next queued run. It could then appear in that run's error details. No scenario in the repo fails this way today, but one that ran attacks directly in its own task could.
Recommendation: no change is needed for current behaviour. If that weak spot needs closing, starting each scheduled run with
create_task(..., context=Context())would do it.romanlutz commentedon Nov 6, 2026
Thanks Muhammad Waqar Anwar (@WaqarAnwar). I've linked your investigation under P311-13. Its no-change conclusion remains pending maintainer acceptance, so the item stays unchecked.
romanlutz commentedon Nov 6, 2026
#3999 is merged. Thanks Slai.life (@huong8373tt-beep); the SeedIdentifier factory case is complete and your reservation is released. P311-01 remains unchecked pending maintainer acceptance of the broader opportunity.
huong8373tt-beep commentedon Nov 6, 2026
Claiming P311-14 (
asyncio.Runnerfor synchronous entry points).\n\nI will investigate PyRIT synchronous entry points and determine whether multiple top-level calls currently create/dispose separate event loops where one managedasyncio.Runnerwould preserve lifecycle/context more safely. I will keep notebook/already-running-loop behavior unchanged, preserve resource and loop ownership, and only open a focused PR if a concrete improvement is justified, with regression coverage.\n\nSlai Agent Runtime performed the investigation, implementation, and test execution. The account owner authorized this contribution and is responsible for its content.huong8373tt-beep commentedon Nov 6, 2026
P311-14 investigation result — no focused code change justified\n\nI inspected the current synchronous entry points and their ownership/lifecycle contracts:\n\n-
pyrit_scan.mainandpyrit_migrate.maineach execute one top-level coroutine and then exit. Replacing their one-shotasyncio.run(...)withasyncio.Runnerwould not share a loop or context across calls and would not change resource ownership.\n-pyrit_shell.PyRITShellalready owns one persistent event loop on its background thread and schedules commands onto it withrun_coroutine_threadsafe; changing this to a Runner would not improve the existing lifecycle.\n-OpenAITextEmbedding.generate_text_embeddingis a synchronous facade over an async client. A reusable Runner would require a new long-lived runner/client shutdown contract, while a per-call Runner has the same one-shot semantics. It also would not make this facade safe to call from an already-running event loop, which this opportunity explicitly does not target.\n- The browser-authenticator thread deliberately creates a platform-specific loop (Proactor on Windows) for Playwright and runs one operation; a generic Runner replacement would risk changing that loop policy without providing a shared-call benefit.\n\nConclusion: the current production surfaces do not have a concrete multi-call synchronous lifecycle gap whereasyncio.Runnerimproves behavior without introducing a new ownership contract. I am not opening a code PR for a mechanical substitution.\n\nSlai Agent Runtime performed the investigation. The account owner authorized this contribution and is responsible for the report.AoiUltramar commentedon Nov 6, 2026
Claiming P311-22 (
getlimit/setlimit).Concrete case:
MemoryInterface._MAX_BIND_VARS = 500(and the duplicate inMemorySession) is documented as a conservative default based on SQLite's 999 limit, which was the default before SQLite 3.32; current builds default to 32766 and some ship higher (Debian/Kali: 250000).Preliminary findings from local runs: with the pinned SQLAlchemy/aiosqlite,
getlimitisn't exposed on the async adapter or aiosqlite's public connection; the clean option would be probing a short-livedsqlite3connection.MemorySessionis shared with Azure SQL and its batch size doesn't depend on dialect, so raising it would need a dialect check; I'd leave it at 500. Early benchmarks (unpinned environment) ofget_message_pieceswith 10k IDs show the SQL portion dropping by ~20–26 ms of ~700 ms per call (~3–4%); end-to-end timings were too noisy (±200 ms) to show a reliable difference.Plan: re-run the benchmarks in a
uv sync --frozenenvironment with more repetitions, then report either a capped SQLite-only change or a no-change conclusion here.romanlutz commentedon Nov 6, 2026
Assigning P311-14 to you Slai.life (@huong8373tt-beep) and P311-22 to you Pedro Holanda (@AoiUltramar). GitHub assignment is unavailable, so both reservations are recorded in the table. Huong's no-change report is linked; P311-14 stays unchecked pending maintainer acceptance.
AoiUltramar commentedon Nov 6, 2026
Update on P311-22: the data ended up pointing to a small, capped change rather than a no-change result.
What I measured (
uv sync --frozen, SQLite 3.46.1, one VM, interleaved runs, 40–80 samples per cell,get_message_pieces): bumping SQLite's_MAX_BIND_VARSfrom 500 to 2000 makes the async path ~11% faster at 10k IDs and ~17% faster at 2k IDs (the p25–p75 ranges don't overlap at 2k). Sync barely moves. Going above 2000 only helps calls with more than 2000 IDs (async at 10k IDs: 260 → 240 ms with a limit of 10000), and it would put SQLite well above Azure's 2000, so I'd stop there.What I'm planning:
SQLiteMemory._MAX_BIND_VARS = 2000whengetlimit(SQLITE_LIMIT_VARIABLE_NUMBER) >= 2000, and 500 otherwise, so older builds behave exactly as today. Neither the async adapter nor aiosqlite exposesgetlimitpublicly, so I'd read it from a short-livedsqlite3connection.MemorySession(500) and Azure's_MAX_BIND_VARS(2000) stay as they are, andtest_batching_scale.pywill size its inputs from the SQLite value so batching stays covered.One thing I'd like your call on: raising the limit broke
_execute_get_prompt_scores. Its JSONORbatches cost 2 bind params per term, because the JSON path repeats in every term under the qmark drivers (I confirmed this by compiling the real SQLite and Azure builders). At 1999 terms, SQLite rejects the query withExpression tree is too large (maximum depth 1000), so the PR needs to boundjson_batch_size.The same math says Azure is probably already affected: 1999 terms is ~3998 params, over SQL Server's 2100 limit, so
get_prompt_scoreswith more than ~1050 IDs likely fails there today. I haven't been able to verify that against a live Azure instance.Bounding it as
min(_MAX_BIND_VARS, 500) - 1(499 terms: 998 binds, depth ~499) keeps SQLite's current behavior. Locally, it fixes the SQLite failure: the failing test passes at 2000, and the batching suite still passes at 500. On paper it should fix Azure too (998 params). Would you like that included in the P311-22 PR, or should I file the Azure part as its own issue?