Repository navigation
minimaxm31flash-fp4-mi355x-sglang-agentic-dspark: update configs / minimaxm31flash-fp4-mi355x-sglang-agentic-dspark：更新配置 - #4534
Open
kevin-mii wants to merge 5 commits into
Conversation
…and 64 Conc 48 and 64 move to the cookbook's High-Throughput cell: HiCache with 180 GB of host KV per rank, written back on eviction, in hipHostMalloc host pools (HSA_USERPTR_FOR_PAGED_MEM=0). The matrix declares 734 GB of host DRAM for the TP4 cell. The base env states the INT4 quick all-reduce SGLang's MiniMax-M3.1 gfx950 override already selects, and sets SGLANG_SIMULATE_ACC_GREEDY=0 so synthetic acceptance keeps the draft and accept sampling path on HIP. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…I#4534 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
kevin-mii
requested review from
1am9trash,
billishyahao,
chunfangamd,
seungrokj and
zijiecode
as code owners
October 26, 2026 09:03
![claude[bot]](https://avatars.githubusercontent.com/in/1236702?s=60&v=4){kind=small}
… quick-reduce env SGLang's MiniMax-M3.1 gfx950 override already raises the image's INT8 to INT4 at TP4. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…hetic-acceptance greedy env AgentX requests carry top_p 0.95, so SGLang's HIP greedy shortcut never applies; with the base env unchanged, only conc 48 and 64 rerun. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
# Conflicts: # inferencex-e2e/perf-changelog.yaml
zijiecode
added
AMD
non-canary-full-sweep-enabled
Contributor
|
Dispatched trusted sweep run for approved external revision 已为获批的外部提交 |
Collaborator
|
@claude review |
zijiecode
added
non-canary-full-sweep-enabled
Contributor
|
Dispatched trusted sweep run for approved external revision 已为获批的外部提交 |
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Update the MI355X SGLang AgentX configs on top of the day-zero recipe from #4445 and #4469 by @Oseltamivir: conc 48 and 64 move from GPU-resident TP4 to TP4 with DRAM KV offload (HiCache, 180 GB per rank, write-back, host pools from
hipHostMallocviaHSA_USERPTR_FOR_PAGED_MEM=0). On one MI350X node (3600 s, same harness), conc 48 goes from 61,038 to 70,268 tok/s/GPU (p90 interactivity 58.3 to 84.0) and conc 64 from 41,922 to 85,397 tok/s/GPU (p90 interactivity 14.1 to 54.2, p90 TTFT 13.7 s to 1.24 s).中文
在 @Oseltamivir 已合入的 #4445 和 #4469 day-zero 配置基础上更新 MI355X SGLang AgentX 配置：并发 48 和 64 从纯 GPU 显存的 TP4 改为启用 DRAM KV offload 的 TP4（HiCache，每个 rank 180 GB，write-back，主机内存池通过
HSA_USERPTR_FOR_PAGED_MEM=0由hipHostMalloc分配）。在单台 MI350X 节点上（3600 s，相同测试框架），并发 48 从 61,038 提升到 70,268 tok/s/GPU（p90 interactivity 从 58.3 提升到 84.0），并发 64 从 41,922 提升到 85,397 tok/s/GPU（p90 interactivity 从 14.1 提升到 54.2，p90 TTFT 从 13.7 s 降到 1.24 s）。AI model disclosure
Prepared with Claude Code using
claude-opus-5-5.🤖 Generated with Claude Code