Repository navigation
Conversation
|
No actionable comments were generated in the recent review.
 |
| Layer / File(s) | Summary |
|---|---|
Initialize the router sender lightllm/server/multi_level_kv_cache/manager.py, unit_tests/server/multi_level_kv_cache/test_manager_startup.py |
The manager stores the ZeroMQ context and router address, creates a bounded queue, and waits for the sender thread to connect. Startup installs a fatal thread exception hook. Subprocess tests cover socket creation, connection, ready, and send stages. |
Queue and send request groups lightllm/server/multi_level_kv_cache/manager.py, unit_tests/server/multi_level_kv_cache/test_manager_socket_ownership.py |
Timeout and cache-match paths enqueue request groups. The sender thread sends queued groups. Tests cover socket ownership, concurrent sends, receiver progress, and queue backpressure. |
Priority:
Estimated code review effort: 3 (Moderate) | ~20 minutes
Change: Bug fix
Sequence Diagram(s)
sequenceDiagram
participant CacheMatchWorker
participant SendQueue
participant SendLoop
participant Router
CacheMatchWorker->>SendQueue: enqueue request group
SendQueue->>SendLoop: provide queued request group
SendLoop->>Router: send request group
Merge Risk: 
8780a
-
No actionable merge-blocking risk is established by the reviewed change.
Pre-merge checks: 
-
Check name Status Explanation Resolution Docstring Coverage Docstring coverage is 4.55% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Documentation-only edits and deletions within survi… Write docstrings for the functions missing them to satisfy the coverage threshold. -
Check name Status Explanation Description Check Check skipped - CodeRabbit’s high-level summary is enabled. Title check The title names the main change: cache-to-router notifications are now sent from a dedicated thread. The raw summary confirms this. A new queue-fed sender thread that owns its own ZeroMQ PUSH socket i… Linked Issues check Check skipped because no linked issues were found for this pull request. Out of Scope Changes check Check skipped because no linked issues were found for this pull request. Full details: Title check
-
Explanation
The title names the main change: cache-to-router notifications are now sent from a dedicated thread. The raw summary confirms this. A new queue-fed sender thread that owns its own ZeroMQ PUSH socket is the core of the changeset, and the title's 'fix(cache)' prefix matches the stated objective of avoiding the receive loop delaying notifications and router backpressure blocking reception. The title is short, single-sentence, and specific enough for a teammate scanning history.
-
Full details: Docstring Coverage
-
Explanation
Docstring coverage is 4.55% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Documentation-only edits and deletions within surviving functions are included; unchanged diff context and fully deleted functions are excluded. Analyzed 22 functions across 3 files.
Resolution
Write docstrings for the functions missing them to satisfy the coverage threshold.
-
Finishing Touches: 💡 1
-
📝 Generate docstrings: 💡
-
- Commit to this branch
- Create a new PR
🧪 Generate unit tests (beta)
-
- Commit to this branch
- Create a new PR
-
- Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.
Comment @coderabbitai help to get the list of available commands.
sufubao
changed the title
Cache worker 将完成请求写入有界队列，由独立发送线程创建并独占 ZMQ PUSH socket。发送线程通过阻塞式 Queue.get() 等待结果，避免接收循环的 10ms 休眠延迟完成通知，以及 Router 背压阻塞接收。接收循环原有的空闲休眠仍保留。
发送线程用 Future 将本地 socket 创建、connect() 的结果传回构造函数。初始化失败在主线程重抛，沿原 pipe 路径向 launcher 报错；成功后才报告 init ok，不等待 Router 上线。初始化后的发送异常仍触发 fatal hook。完成队列容量沿用 running_max_req_size，常规日志改为 debug。
验证：38 项多级缓存测试及 11 项启动管理测试通过，Black、Flake8 通过。新增 4 项真实 spawn / Pipe / launcher 回归，覆盖 socket 创建失败、connect 失败、Router 未启动时初始化成功，以及运行期发送异常退出。未修复版本已通过 EMFILE 故障注入复现启动挂起。GPU、CPU cache 和共享内存依赖在启动回归中做了隔离；未进行完整 GPU serving、真实 cache/abort 集成或 TTFT 实测。
Summary by CodeRabbit