Repository navigation
perf: rerun the CRM relationships check, and measure tools search on task phrases - #358
Merged
Merged
Conversation
…task phrases - performance/crm/relationships: rerun on b24 0.13.0 and memo 0.131.84 passes 13 of 13; check 11 holds since cli-memo #777. The runner chooses the CRM user b24 copies deals of, the fake portal answers user.admin, and the owner keeps one contact memo now hides as test data (cli-core #2862). - performance/agents/tools-search: 13 short and 13 task phrases over a real memo MCP server. Ranking by matched words (cli-core #2878) finds 13 of 13 task phrases instead of 6; short phrases unchanged. The after run used local builds. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Task
Rerun the CRM relationships scenario (#315) on released b24 and memo, and measure cli-core's new tools search
(cli-core #2878) before and after.
Why
search all). Its fix, cli-memo #777, shipped since.memo_tools_searchfinds only 6 of 13 commands when an agent words a task; #2878 changes the ranking.What
performance/crm/relationships: rerun on b24 0.13.0 and memo 0.131.84 passes 13 of 13; check 11 holds.Three runner fixes, none a product bug: choose the CRM user whose deals b24 copies, the fake portal answers
user.admin, and the owner keeps one contact that memo now hides as test data (cli-core #2862).performance/agents/tools-search: 13 short and 13 task phrases over a real stdiomemo mcp serve.Task phrases found 6 → 13 of 13, first place 6 → 9, top 3 6 → 10; short phrases unchanged (13, 9, 12).
The after run used local builds (memo branch on cli-core main 0.643.0, unpublished); the README says to rerun it
on the release.
performance/agents/README.md; CHANGELOG entries under Unreleased.Testing
pnpm lint,pnpm docs:check, rumdl on the changed pages.🤖 Generated with Claude Code