Skip to content

test(agents): shorten stream cache eviction test - #4171

Draft
dpiet-oai wants to merge 1 commit into
mainfrom
dp/project-fast-ci-02-python-stream-cache
Draft

dpiet-oai wants to merge 1 commit into
mainfrom
dp/project-fast-ci-02-python-stream-cache

Conversation

@dpiet-oai

Copy link
Copy Markdown
Contributor

The agent-session stream cache test delivers 5,000 distinct text deltas twice to exercise a 1,024-event cache. Use 1,026 deltas and explicitly verify that the original tool event is delivered after eviction while its handler still runs once. Both client modes retain duplicate-delta, cache-bound, input-count, and closure checks. This reduces sustained eviction churn from 3,978 to four evictions before the tool replay.

Validation: Python 3.10.16 / Pydantic 1.10.26, three serial local trials per revision (DEFER_PYDANTIC_BUILD=false, pytest -n 0): median call time 2.67s → 1.24s sync and 2.56s → 0.68s async. Python 3.10.16 / Pydantic 2.12.5: modified test file 90 passed, 4 skipped; Ruff lint and format checks passed. Whole-gate savings have not been measured.

@github-actions

Copy link
Copy Markdown
Contributor

Castiron custom code

Evaluated main: 43045fcaf91aec08571e54c57706597cefd32d45.

Trusted main policy: 43045fcaf91aec08571e54c57706597cefd32d45.

✅ No new custom-code files detected.

0 mixed files remain; 0 existing customizations changed.

Compared 43045fcaf91a → fc44b250a21b. Generated baselines verified.

A changed generated baseline means this report cannot reliably identify which handwritten lines changed.

Inspect the custom-code diff

Download the exact patch produced by this run (requires repository access):

gh run download 40029002619 --repo openai/openai-python \
  --name castiron-custom-code-40029002619-1 --dir /tmp/castiron-custom-code-40029002619-1
git apply --stat /tmp/castiron-custom-code-40029002619-1/custom-code.patch
cat /tmp/castiron-custom-code-40029002619-1/custom-code.patch

Or reproduce it from an SDK checkout containing the vendored reporter:

git fetch --no-tags origin 43045fcaf91aec08571e54c57706597cefd32d45 fc44b250a21b3c18fd86c7490c5d2872ab60f9d5
python3 scripts/castiron/custom_code_report.py report \
  --base 43045fcaf91aec08571e54c57706597cefd32d45 \
  --head fc44b250a21b3c18fd86c7490c5d2872ab60f9d5 --target-branch main --fetch --require-head-hash --public \
  --out /tmp/castiron-custom-code-fc44b250a21b
cat /tmp/castiron-custom-code-fc44b250a21b/custom-code.patch

This is the current full custom patch for mixed files, not an attribution of only the handwritten lines changed by this PR.

Full report and patch

@marcuswood-oai marcuswood-oai left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM 👍 No changes requested.

The focused cache-state probe passed. The draft CI run skipped test and lint jobs; I did not independently rerun the SDK tests.


Reviewed: fc44b250a21b · Base: 43045fcaf91a · sdk-pr-review

@markstuart-oai markstuart-oai left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed fc44b250a21b3c18fd86c7490c5d2872ab60f9d5. No blocking findings. The reduced event count still exceeds the actual cache capacity. Both client modes now explicitly verify tool-event eviction and replay while retaining duplicate suppression, one tool invocation, cache bounds, and stream cleanup.

Source-only review; I did not run candidate code or tests. Hosted checks: 13 skipped, 22 success.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants