[Aditya Malkar (adimalkar)]

Description

Fixes #3718.

LLMGenericTextConverter._send_with_retries_async called self._converter_target.send_prompt_async directly, so nothing persisted the exchange. On main (f3a88daf), with in-memory SQLite and a local MockPromptTarget, a convert_async call leaves one piece in the converter's conversation: ['system']. That's the system prompt, which the target writes itself. The request, the response and every retry attempt are lost. With this change the same call persists ['system', 'user', 'assistant'].

What changed (in pyrit/converter/llm_generic_text_converter.py)

• Persistence. Each attempt goes through PromptNormalizer.send_prompt_async, which persists the request and the response. When the target raises, the normalizer persists the request and its error response. A PromptNormalizer is created per call, so converters can still be constructed before memory is initialized. _process_response and the retry policy stay in the converter.
• One conversation per attempt. Each attempt gets a new conversation, and the system prompt is set on it. Before this change retries were independent only because nothing was persisted. Reusing the conversation would make the retry replay the failed turn. Rolling the turn back, as send_json_with_retry_async does, would drop the retry exchanges the issue asks to keep. Retries send request.duplicate(), so persisted piece IDs stay unique.
• Error behaviour is unchanged. The normalizer re-raises a target error as Exception("Error sending prompt ...") from error. After the failed exchange is persisted, the converter raises the target's own exception, so retry_exceptions (e.g. TranslationConverter's, or a user's (ConnectionError,)) and callers see the same exception types as before. The normalizer turns an EmptyResponseException into an empty response with response_error="empty". The converter raises EmptyResponseException for that, instead of returning "".

The normalizer records the converter conversation under the current attack-result ID (get_current_attack_result_id()), as it does for scorer exchanges. The attack_result_scope docstring lists converters as one of the conversation creators meant to register that ownership.

Tests and Documentation

New tests in tests/unit/converter/test_generic_llm_converter.py use real in-memory SQLite and a local MockPromptTarget subclass. All five fail on main.

• test_convert_async_persists_the_target_exchange: checks the system, user and assistant pieces. The user piece keeps the raw prompt as original_value and the rendered template as converted_value.
• test_each_retry_attempt_is_persisted_in_its_own_conversation: two conversations, each with all three pieces. The target sees only [system, user] on both attempts.
• test_failed_target_call_is_persisted_and_raises_the_target_error: the original ValueError is raised, and the request and response_error="processing" piece are kept.
• test_retry_on_target_error_matches_the_unwrapped_error: retry_exceptions=(ConnectionError,) still retries, and both failed exchanges are kept.
• test_empty_target_response_raises_empty_response_exception: an empty reply still raises, and the response_error="empty" piece is kept.

Existing converter tests:

• Nine converter test modules that send through a mocked target now use patch_central_database (module-level pytestmark), since the exchange is persisted.
• Their mocks return response.duplicate() per call instead of one shared Message object, which would collide on primary keys once persisted. A real target builds new pieces for every call.

tests/unit has the same failures on main and on this branch (55 failed / 89 errors locally, all litellm-dependent or otherwise environmental), plus 5 new passing tests. ruff and ty check pyrit (lockfile versions) report nothing new.

[Copilot]

Copilot was unable to review this pull request because the user who requested the review has insufficient remaining quota. Try requesting a review at a lower effort level.

[Aditya Malkar (adimalkar)]

The one red job (main-job (macos-latest, 3.12, pyrit, dev)) failed tests/unit/prompt_target/target/test_websocket_copilot_target.py::TestConnectAndSend::test_connect_and_send_overall_timeout_during_stream. That is a timing assertion in the WebSocket Copilot target, which doesn't use converters or the normalizer. It passes locally on this branch, and the remaining Windows/macOS jobs were cancelled by fail-fast, so a re-run should clear it.