[YukioZzz]

Run Kimi-K3 FP4 PD on MI355X with vllm/vllm-openai-rocm:nightly-c50aca14416ef29f6ed0ed256302f30646726dcd, TP8/DCP8 workers, FP8 KV cache, and MoRIIO RDMA transfer. The image is pinned by digest. The sweep covers one latency point and four throughput configurations:

Topology	Concurrency	DSpark draft tokens	P / per-D max sequences	SSM state	CPU offload
1P1D	1	7	1 / 1	FP32	None
1P1D	48	4	96 / 96	BF16	SimpleCPUOffload
1P2D	24	4	48 / 24	BF16	SimpleCPUOffload
1P2D	48	4	96 / 48	BF16	SimpleCPUOffload
1P3D	24	4	48 / 16	BF16	SimpleCPUOffload

The c1 decode worker uses latency_specialized with the V2 model runner and graph capture sizes [1, 8]. Throughput configurations use BF16 SSM state on both prefill and decode. DSpark uses /models/Inferact-Kimi-K3-DSpark with probabilistic drafting and block rejection; throughput runs use the selected golden acceptance curve, while evals use real verification.

The recipe uses vllm-router with consistent-hash routing and the standard eval selection. Recipe documentation: vllm-project/recipes#1102.

AI model disclosure

AI assistance prepared the recipe and validation, reviewed upstream dependencies, and rebased the PR. A separate agent previously checked documentation discoverability. Exact model identifiers could not be verified from the runtime.

PR checklist and change type

Type of Change

• Bug fix
• New feature
• Configuration change
• Documentation update
• Other

Checklist

• I have completed the AI model disclosure and kept it current
• I have tested my changes locally
• I have updated documentation if necessary
• If python -m infx.matrix.changed --base origin/main lists affected config keys, I have appended one new entry covering them to the physical end of inferencex-e2e/perf-changelog.yaml; if it lists none, I have added none. I have not edited historical entries
• If this PR appends an inferencex-e2e/perf-changelog.yaml entry, it carries exactly one primary sweep label (a maintainer applies it on fork PRs): full-sweep-fail-fast (recommended), full-sweep-enabled, or non-canary-full-sweep-enabled. Optional modifiers all-evals, evals-only, and agentx-fast require a primary label; the last two block reuse while applied.
• Before merging via reuse, an authorized maintainer (OWNER/MEMBER/COLLABORATOR) has commented /use <run_id> (or the legacy /reuse-sweep-run) on this PR. Do this only once there is a final full sweep that is all green with evals passing, since after this comment the primary sweep label will no longer automatically kick off new sweeps. Remove and re-add the primary sweep label to force a new sweep.

中文

在 MI355X 上使用官方 vllm/vllm-openai-rocm:nightly-c50aca14416ef29f6ed0ed256302f30646726dcd 运行 Kimi-K3 FP4 PD，P/D worker 均采用 TP8/DCP8、FP8 KV cache，通过 MoRIIO RDMA 传输 KV，镜像以 digest 固定。

五种配置见上表：1P1D c1 使用 FP32 SSM、K7 DSpark，不启用 CPU offload；decode 端使用 V2 model runner 的 latency_specialized 后端，graph capture sizes 为 [1, 8]。其余四个吞吐配置使用 BF16 SSM、K4 DSpark 和 SimpleCPUOffload。

DSpark draft 为 /models/Inferact-Kimi-K3-DSpark，使用 probabilistic drafting 和 block rejection。吞吐测试采用对应的 golden acceptance curve，eval 使用真实验证。路由采用 vllm-router consistent hash，评测沿用标准选择。

AI 协助配置、验证、依赖核对及 rebase；此前有独立 agent 检查文档可发现性。精确模型标识无法从运行环境核实。

[github-actions]

Thanks for the contribution!

• Review: If this PR changes files owned by someone other than a repository admin or @SemiAnalysisAI/core, ask one eligible CODEOWNER to complete the latest PR_REVIEW_CHECKLIST.md before contacting a core maintainer on Slack. Follow the template exactly, including As a PR reviewer and CODEOWNER, I have reviewed this and have, so sign-off verification triggers.
• PR verification: Sweeps run only when the PR appends an inferencex-e2e/perf-changelog.yaml entry and carries exactly one primary label: full-sweep-fail-fast (strongly recommended; canary plus per-matrix fail-fast), full-sweep-enabled (canary; matrix jobs continue after a failure), or non-canary-full-sweep-enabled (no canary or fail-fast). The modifiers all-evals, evals-only, and agentx-fast require a primary label. On fork PRs, a maintainer applies the label. See sweep labels and reuse.
• After merging: PR authors must ensure all GitHub Actions jobs pass. Transient failures often pass on rerun; see how to rerun failed jobs.

中文

感谢你的贡献！

• **审阅：**如果 PR 修改的文件归属于仓库管理员及 @SemiAnalysisAI/core 之外的 CODEOWNER，请先联系一位有资格的 CODEOWNER 填写最新的 PR_REVIEW_CHECKLIST.md，再通过 Slack 联系核心维护者。必须严格遵循模板，并保留 As a PR reviewer and CODEOWNER, I have reviewed this and have，才能触发签核验证。
• **PR 验证：**只有当 PR 在 inferencex-e2e/perf-changelog.yaml 末尾追加了条目，并且恰好带有一个主标签时，才会运行扫描：full-sweep-fail-fast（强烈推荐；canary 加逐矩阵 fail-fast）、full-sweep-enabled（有 canary；矩阵任务在失败后继续运行）或 non-canary-full-sweep-enabled（无 canary，也无 fail-fast）。修饰标签 all-evals、evals-only 和 agentx-fast 必须与主标签一起使用。fork PR 的标签由维护者添加。参见扫描标签与复用。
• **合并后：**PR 作者必须确保所有 GitHub Actions 任务通过。临时性失败通常可以通过重新运行恢复；参见重新运行失败任务的说明。

[github-actions]

View unofficial run (performance): https://inferencex.semianalysis.com/inference?unofficialRun=40109242325

View unofficial run (accuracy): https://inferencex.semianalysis.com/evaluation?unofficialRun=40109242325