|
30 | 30 |
|
31 | 31 | --- |
32 | 32 |
|
33 | | -## 📐 2. 逐篇论文深度机制解构、数学公式与本仓库落地指南 (Per-Paper Deep-Dive Cards) |
| 33 | +## 🔎 2. 来源核验、推导边界与复现补充规范 (Source Verification & Reproducibility Notes) |
34 | 34 |
|
35 | | -### 2.1 [2026-09-30] SlimWise & CascadeEP: Decoupling Expert Pruning Across Prefill/Decode & Asynchronous MoE Execution under Attention Imbalance (`arXiv:2609.34117` & `arXiv:2609.33252`) |
| 35 | +### 🔎 来源核验与研究补充(2026-09-30) |
| 36 | + |
| 37 | +本日实际为 6 个主题组、12 篇论文。本次核对标题与编号,不代表已核对全部公式、实验表或完成复现。 |
| 38 | + |
| 39 | +**引用纠正**:SCOPD 的正确编号为 [2609.34044](https://arxiv.org/abs/2609.34044)。原笔记中的 `2609.33918` 实际对应 *Green AI: Cost of LLM-Based Code Completion*,后文涉及 SCOPD 的该编号均以此更正为准。 |
| 40 | + |
| 41 | +**指标纠正**:SCOPD 摘要在 10% 视觉 Token 保留率、13 个基准下报告相对未剪枝模型的性能保留率:Vanilla 86.37%、SCOPD 90.49%、SCOPD+ 92.43%。后文“99.5% 恢复率”、5,000 条训练指令、1 Epoch、68% 延迟降低及 79% 缓存压缩未获本次核验支持,撤回这些具体数值。ACPruner 与 SCOPD 的组合应视为研究建议,不能当作论文已报告的联合实验。 |
| 42 | + |
| 43 | +| 主题组 | 原始论文来源 | |
| 44 | +| :--- | :--- | |
| 45 | +| 视觉剪枝与蒸馏 | [ACPruner](https://arxiv.org/abs/2609.34558)、[SCOPD](https://arxiv.org/abs/2609.34044) | |
| 46 | +| MoE 服务 | [SlimWise](https://arxiv.org/abs/2609.34117)、[CascadeEP](https://arxiv.org/abs/2609.33252) | |
| 47 | +| 静态图与动态剪枝 | [Dynamic Flow, Static Graph](https://arxiv.org/abs/2609.34727)、[DORA](https://arxiv.org/abs/2609.34325) | |
| 48 | +| 流匹配 | [CAT-Flow](https://arxiv.org/abs/2609.01746)、[MSFM](https://arxiv.org/abs/2609.35454) | |
| 49 | +| 具身与世界模型 | [VLaRL](https://arxiv.org/abs/2609.30868)、[Programmable World Model](https://arxiv.org/abs/2609.10540) | |
| 50 | +| 自我改进智能体 | [AutoDataBench](https://arxiv.org/abs/2609.35025)、[SelfOp](https://arxiv.org/abs/2609.22792) | |
| 51 | + |
| 52 | +**推导与实现边界**:后文 KL 公式的方向为教师到学生,不应称为学生到教师的反向 KL;隐状态对齐等组合设计仍需全文逐式核验。次模近似保证需核对非负、单调、归一化与基数约束;流形收缩结论需明确成立区域与扰动假设。跨仓映射表仅为候选适配位置,本次没有检查其他仓库路径或执行跨仓写入。 |
| 53 | + |
| 54 | +**建议复现顺序**:先分别复现 ACPruner、SCOPD,再测组合;随后验证 MoE 在长短混合请求下的质量与吞吐,最后测试固定 NFE 下的流匹配误差。记录论文版本、代码 commit、模型与数据版本、随机种子、硬件及预算;同时报告分任务性能、端到端延迟和峰值显存。智能体技能更新应使用独立保留任务,防止验证集泄漏。详细实验建议见[同日新闻](https://github.com/Shwai-He/scholar-odyssey/blob/main/intelligence/news/2026-09-30_daily_news.md)。 |
| 55 | + |
| 56 | + |
| 57 | +--- |
| 58 | + |
| 59 | +## 📐 3. 逐篇论文深度机制解构、数学公式与本仓库落地指南 (Per-Paper Deep-Dive Cards) |
| 60 | + |
| 61 | +### 3.1 [2026-09-30] SlimWise & CascadeEP: Decoupling Expert Pruning Across Prefill/Decode & Asynchronous MoE Execution under Attention Imbalance (`arXiv:2609.34117` & `arXiv:2609.33252`) |
36 | 62 | * **论文标题**: |
37 | 63 | 1. *SlimWise: Decoupling Expert Pruning Across Prefill and Decode for Efficient MoE Serving* (`arXiv:2609.34117`) |
38 | 64 | 2. *CascadeEP: Asynchronous Expert Execution for MoE Prefill under Attention Imbalance* (`arXiv:2609.33252`) |
|
94 | 120 |
|
95 | 121 | --- |
96 | 122 |
|
97 | | -### 2.2 [2026-09-29] 🧩 *CoMoE-Spec: Efficient Mixture-of-Experts with Speculative Decoding via Expert Coactivation* |
| 123 | +### 3.2 [2026-09-29] 🧩 *CoMoE-Spec: Efficient Mixture-of-Experts with Speculative Decoding via Expert Coactivation* |
98 | 124 | > 🏷️ **核心关键词**:Mixture-of-Experts (MoE) · Speculative Decoding · Expert Coactivation Routing · Memory-Bandwidth Bottleneck |
99 | 125 | > 🔗 **arXiv 链接**:[`arXiv:2609.22471`](https://arxiv.org/abs/2609.22471) |
100 | 126 |
|
|
140 | 166 |
|
141 | 167 | --- |
142 | 168 |
|
143 | | -### 2.3 [2026-09-28] 🧩 *PiKV: KV Cache Management System for Mixture of Experts* |
| 169 | +### 3.3 [2026-09-28] 🧩 *PiKV: KV Cache Management System for Mixture of Experts* |
144 | 170 | > 🏷️ **核心关键词**:Mixture-of-Experts (MoE) · Expert-Sharded KV Cache · Distributed Serving · Memory & Communication Co-Design |
145 | 171 | > 🔗 **arXiv 链接**:[`arXiv:2508.06526`](https://arxiv.org/abs/2508.06526) (2026 v3) |
146 | 172 |
|
|
186 | 212 |
|
187 | 213 | --- |
188 | 214 |
|
189 | | -### 2.4 [2026-09-27] SHAPE: Coalition-Aware Expert Pruning for Sparse Mixture-of-Experts LLMs |
| 215 | +### 3.4 [2026-09-27] SHAPE: Coalition-Aware Expert Pruning for Sparse Mixture-of-Experts LLMs |
190 | 216 |
|
191 | 217 | * **论文信息**:`arXiv:2606.09886` (2026-06, 开源仓库:`github.com/Alizen-1009/Shapley-Moe`) |
192 | 218 | * **核心关键词**:Sparse MoE、Cooperative Game Theory、Shapley Value Attribution、Coalition-Aware Expert Pruning、Quality-Coverage Bisection |
|
286 | 312 |
|
287 | 313 | --- |
288 | 314 |
|
289 | | -### 2.5 [2026-09-27] L2R: Low-Rank and Lipschitz-Controlled Routing for Mixture-of-Experts |
| 315 | +### 3.5 [2026-09-27] L2R: Low-Rank and Lipschitz-Controlled Routing for Mixture-of-Experts |
290 | 316 |
|
291 | 317 | * **论文信息**:Minghao Yang, Ren Togo, Guang Li, Takahiro Ogawa, Miki Haseyama (`arXiv:2601.21349`, 2026-01) |
292 | 318 | * **核心关键词**:MoE Routing Geometry、Low-Rank Latent Space、Lipschitz Continuity、Saturated Inner-Product Scoring (SIPS)、Multi-Anchor Routing |
|
379 | 405 |
|
380 | 406 | --- |
381 | 407 |
|
382 | | -### 2.6 [2026-09-26] 🔄 *LoopMoE: Unifying Iterative Computation with Mixture-of-Experts for Language Modeling* |
| 408 | +### 3.6 [2026-09-26] 🔄 *LoopMoE: Unifying Iterative Computation with Mixture-of-Experts for Language Modeling* |
383 | 409 | > **聚焦领域**:Looped Transformers · Mixture of Experts (MoE) · Iterative Depth Scaling · Weight Sharing |
384 | 410 | > **arXiv**:[`arXiv:2606.04438`](https://arxiv.org/abs/2606.04438) |
385 | 411 |
|
|
432 | 458 |
|
433 | 459 | --- |
434 | 460 |
|
435 | | -### 2.7 [2026-09-26] 🧩 *MoE-nD: Per-Layer Mixture-of-Experts Routing for Multi-Axis KV Cache Compression* |
| 461 | +### 3.7 [2026-09-26] 🧩 *MoE-nD: Per-Layer Mixture-of-Experts Routing for Multi-Axis KV Cache Compression* |
436 | 462 | > **聚焦领域**:Multi-Axis KV Cache Compression · Per-Layer Routing · Heterogeneous Quantization |
437 | 463 | > **arXiv**:[`arXiv:2604.17695`](https://arxiv.org/abs/2604.17695) |
438 | 464 |
|
|
449 | 475 |
|
450 | 476 | --- |
451 | 477 |
|
452 | | -### 2.8 [2026-09-24] DriveMoE: Mixture-of-Experts for Vision-Language-Action Model in End-to-End Autonomous Driving |
| 478 | +### 3.8 [2026-09-24] DriveMoE: Mixture-of-Experts for Vision-Language-Action Model in End-to-End Autonomous Driving |
453 | 479 |
|
454 | 480 | * **论文信息**:`arXiv:2505.16278` (2025/2026) |
455 | 481 | * **核心关键词**:End-to-End Autonomous Driving、Scene-Specialized Vision MoE、Skill-Specialized Action MoE、Flow-Matching Planner |
|
497 | 523 |
|
498 | 524 | --- |
499 | 525 |
|
500 | | -### 2.9 [2026-09-23] HiMoE-VLA: Hierarchical Mixture-of-Experts for Generalist Vision-Language-Action Policies |
| 526 | +### 3.9 [2026-09-23] HiMoE-VLA: Hierarchical Mixture-of-Experts for Generalist Vision-Language-Action Policies |
501 | 527 |
|
502 | 528 | * **论文信息**:`arXiv:2512.05693` (2025/2026) |
503 | 529 | * **核心关键词**:Hierarchical MoE、Generalist VLA Policy、Task-Skill Decoupled Routing、Gradient Conflict Mitigation |
|
545 | 571 |
|
546 | 572 | --- |
547 | 573 |
|
548 | | -### 2.10 [2026-09-22] SPIN: Unifying Sparse Attention with Hierarchical Memory for Scalable Long-Context LLM Serving |
| 574 | +### 3.10 [2026-09-22] SPIN: Unifying Sparse Attention with Hierarchical Memory for Scalable Long-Context LLM Serving |
549 | 575 |
|
550 | 576 | * **论文信息**:`arXiv:2604.26837` (2026-04) |
551 | 577 | * **核心关键词**:Sparse Attention Serving、Hierarchical GPU-CPU Memory、Asynchronous Layer-Ahead Prefetching |
|
595 | 621 |
|
596 | 622 | --- |
597 | 623 |
|
598 | | -### 2.11 [2026-09-21] MoE-FM: Towards Faster Language Model Inference Using Mixture-of-Experts Flow Matching |
| 624 | +### 3.11 [2026-09-21] MoE-FM: Towards Faster Language Model Inference Using Mixture-of-Experts Flow Matching |
599 | 625 |
|
600 | 626 | * **论文信息**:`arXiv:2604.15009` (2026-04) |
601 | 627 | * **核心关键词**:Mixture-of-Experts Flow Matching、Piecewise-Linear Vector Fields、Latent Flow Language Models |
|
646 | 672 |
|
647 | 673 | --- |
648 | 674 |
|
649 | | -### 2.12 [2026-09-20] CARE: Spend Experts Where You Are Unsure — Confidence-Adaptive Routing for MoE-LoRA |
| 675 | +### 3.12 [2026-09-20] CARE: Spend Experts Where You Are Unsure — Confidence-Adaptive Routing for MoE-LoRA |
650 | 676 |
|
651 | 677 | * **论文信息**:`arXiv:2607.26052` (2026-07) |
652 | 678 | * **核心关键词**:Confidence-Adaptive Routing、MoE-LoRA、Nucleus Expert Activation、Router Uncertainty Entropy |
|
705 | 731 |
|
706 | 732 | --- |
707 | 733 |
|
708 | | -### 2.13 [2026-09-19] REAP: Router-Weighted Expert Activation Pruning for Sparse MoE Models |
| 734 | +### 3.13 [2026-09-19] REAP: Router-Weighted Expert Activation Pruning for Sparse MoE Models |
709 | 735 |
|
710 | 736 | * **论文信息**:`arXiv:2510.13999` (2025/2026) |
711 | 737 | * **核心关键词**:MoE Expert Pruning、Router Gate Weighting、Expert Activation Norm、Generative Reasoning Preservation |
|
762 | 788 |
|
763 | 789 | --- |
764 | 790 |
|
765 | | -### 2.14 [2026-09-18] 🧩 *MoE-Tile: Warp-Aligned Tensor Slicing for Zero-Overhead Dynamic Sparse Routing on Modern Accelerators* |
| 791 | +### 3.14 [2026-09-18] 🧩 *MoE-Tile: Warp-Aligned Tensor Slicing for Zero-Overhead Dynamic Sparse Routing on Modern Accelerators* |
766 | 792 | > **聚焦领域**:Mixture of Experts (MoE) · GPU Kernel Optimization · Warp Divergence · Hardware-Aware Sparsity |
767 | 793 | > **arXiv**:[`arXiv:2609.09112`](https://arxiv.org/abs/2609.09112) |
768 | 794 |
|
|
0 commit comments