Skip to content

Commit 77cce07

Browse files
author
Shwai He
committed
docs(literature): route 2026-09 frontier paper notes and normalize GitHub MathJax/KaTeX formula rendering
1 parent 9cab2d1 commit 77cce07

1 file changed

Lines changed: 41 additions & 15 deletions

File tree

‎docs/frontier_literature_connections_2026_09.md‎

Lines changed: 41 additions & 15 deletions
Original file line numberDiff line numberDiff line change
@@ -30,9 +30,35 @@
3030

3131
---
3232

33-
## 📐 2. 逐篇论文深度机制解构、数学公式与本仓库落地指南 (Per-Paper Deep-Dive Cards)
33+
## 🔎 2. 来源核验、推导边界与复现补充规范 (Source Verification & Reproducibility Notes)
3434

35-
### 2.1 [2026-09-30] SlimWise & CascadeEP: Decoupling Expert Pruning Across Prefill/Decode & Asynchronous MoE Execution under Attention Imbalance (`arXiv:2609.34117` & `arXiv:2609.33252`)
35+
### 🔎 来源核验与研究补充(2026-09-30)
36+
37+
本日实际为 6 个主题组、12 篇论文。本次核对标题与编号,不代表已核对全部公式、实验表或完成复现。
38+
39+
**引用纠正**:SCOPD 的正确编号为 [2609.34044](https://arxiv.org/abs/2609.34044)。原笔记中的 `2609.33918` 实际对应 *Green AI: Cost of LLM-Based Code Completion*,后文涉及 SCOPD 的该编号均以此更正为准。
40+
41+
**指标纠正**:SCOPD 摘要在 10% 视觉 Token 保留率、13 个基准下报告相对未剪枝模型的性能保留率:Vanilla 86.37%、SCOPD 90.49%、SCOPD+ 92.43%。后文“99.5% 恢复率”、5,000 条训练指令、1 Epoch、68% 延迟降低及 79% 缓存压缩未获本次核验支持,撤回这些具体数值。ACPruner 与 SCOPD 的组合应视为研究建议,不能当作论文已报告的联合实验。
42+
43+
| 主题组 | 原始论文来源 |
44+
| :--- | :--- |
45+
| 视觉剪枝与蒸馏 | [ACPruner](https://arxiv.org/abs/2609.34558)、[SCOPD](https://arxiv.org/abs/2609.34044) |
46+
| MoE 服务 | [SlimWise](https://arxiv.org/abs/2609.34117)、[CascadeEP](https://arxiv.org/abs/2609.33252) |
47+
| 静态图与动态剪枝 | [Dynamic Flow, Static Graph](https://arxiv.org/abs/2609.34727)、[DORA](https://arxiv.org/abs/2609.34325) |
48+
| 流匹配 | [CAT-Flow](https://arxiv.org/abs/2609.01746)、[MSFM](https://arxiv.org/abs/2609.35454) |
49+
| 具身与世界模型 | [VLaRL](https://arxiv.org/abs/2609.30868)、[Programmable World Model](https://arxiv.org/abs/2609.10540) |
50+
| 自我改进智能体 | [AutoDataBench](https://arxiv.org/abs/2609.35025)、[SelfOp](https://arxiv.org/abs/2609.22792) |
51+
52+
**推导与实现边界**:后文 KL 公式的方向为教师到学生,不应称为学生到教师的反向 KL;隐状态对齐等组合设计仍需全文逐式核验。次模近似保证需核对非负、单调、归一化与基数约束;流形收缩结论需明确成立区域与扰动假设。跨仓映射表仅为候选适配位置,本次没有检查其他仓库路径或执行跨仓写入。
53+
54+
**建议复现顺序**:先分别复现 ACPruner、SCOPD,再测组合;随后验证 MoE 在长短混合请求下的质量与吞吐,最后测试固定 NFE 下的流匹配误差。记录论文版本、代码 commit、模型与数据版本、随机种子、硬件及预算;同时报告分任务性能、端到端延迟和峰值显存。智能体技能更新应使用独立保留任务,防止验证集泄漏。详细实验建议见[同日新闻](https://github.com/Shwai-He/scholar-odyssey/blob/main/intelligence/news/2026-09-30_daily_news.md)。
55+
56+
57+
---
58+
59+
## 📐 3. 逐篇论文深度机制解构、数学公式与本仓库落地指南 (Per-Paper Deep-Dive Cards)
60+
61+
### 3.1 [2026-09-30] SlimWise & CascadeEP: Decoupling Expert Pruning Across Prefill/Decode & Asynchronous MoE Execution under Attention Imbalance (`arXiv:2609.34117` & `arXiv:2609.33252`)
3662
* **论文标题**:
3763
1. *SlimWise: Decoupling Expert Pruning Across Prefill and Decode for Efficient MoE Serving* (`arXiv:2609.34117`)
3864
2. *CascadeEP: Asynchronous Expert Execution for MoE Prefill under Attention Imbalance* (`arXiv:2609.33252`)
@@ -94,7 +120,7 @@ $$
94120

95121
---
96122

97-
### 2.2 [2026-09-29] 🧩 *CoMoE-Spec: Efficient Mixture-of-Experts with Speculative Decoding via Expert Coactivation*
123+
### 3.2 [2026-09-29] 🧩 *CoMoE-Spec: Efficient Mixture-of-Experts with Speculative Decoding via Expert Coactivation*
98124
> 🏷️ **核心关键词**:Mixture-of-Experts (MoE) · Speculative Decoding · Expert Coactivation Routing · Memory-Bandwidth Bottleneck
99125
> 🔗 **arXiv 链接**:[`arXiv:2609.22471`](https://arxiv.org/abs/2609.22471)
100126
@@ -140,7 +166,7 @@ $$
140166

141167
---
142168

143-
### 2.3 [2026-09-28] 🧩 *PiKV: KV Cache Management System for Mixture of Experts*
169+
### 3.3 [2026-09-28] 🧩 *PiKV: KV Cache Management System for Mixture of Experts*
144170
> 🏷️ **核心关键词**:Mixture-of-Experts (MoE) · Expert-Sharded KV Cache · Distributed Serving · Memory & Communication Co-Design
145171
> 🔗 **arXiv 链接**:[`arXiv:2508.06526`](https://arxiv.org/abs/2508.06526) (2026 v3)
146172
@@ -186,7 +212,7 @@ $$
186212

187213
---
188214

189-
### 2.4 [2026-09-27] SHAPE: Coalition-Aware Expert Pruning for Sparse Mixture-of-Experts LLMs
215+
### 3.4 [2026-09-27] SHAPE: Coalition-Aware Expert Pruning for Sparse Mixture-of-Experts LLMs
190216

191217
* **论文信息**:`arXiv:2606.09886` (2026-06, 开源仓库:`github.com/Alizen-1009/Shapley-Moe`)
192218
* **核心关键词**:Sparse MoE、Cooperative Game Theory、Shapley Value Attribution、Coalition-Aware Expert Pruning、Quality-Coverage Bisection
@@ -286,7 +312,7 @@ $$
286312

287313
---
288314

289-
### 2.5 [2026-09-27] L2R: Low-Rank and Lipschitz-Controlled Routing for Mixture-of-Experts
315+
### 3.5 [2026-09-27] L2R: Low-Rank and Lipschitz-Controlled Routing for Mixture-of-Experts
290316

291317
* **论文信息**:Minghao Yang, Ren Togo, Guang Li, Takahiro Ogawa, Miki Haseyama (`arXiv:2601.21349`, 2026-01)
292318
* **核心关键词**:MoE Routing Geometry、Low-Rank Latent Space、Lipschitz Continuity、Saturated Inner-Product Scoring (SIPS)、Multi-Anchor Routing
@@ -379,7 +405,7 @@ $$
379405

380406
---
381407

382-
### 2.6 [2026-09-26] 🔄 *LoopMoE: Unifying Iterative Computation with Mixture-of-Experts for Language Modeling*
408+
### 3.6 [2026-09-26] 🔄 *LoopMoE: Unifying Iterative Computation with Mixture-of-Experts for Language Modeling*
383409
> **聚焦领域**:Looped Transformers · Mixture of Experts (MoE) · Iterative Depth Scaling · Weight Sharing
384410
> **arXiv**:[`arXiv:2606.04438`](https://arxiv.org/abs/2606.04438)
385411
@@ -432,7 +458,7 @@ $$
432458

433459
---
434460

435-
### 2.7 [2026-09-26] 🧩 *MoE-nD: Per-Layer Mixture-of-Experts Routing for Multi-Axis KV Cache Compression*
461+
### 3.7 [2026-09-26] 🧩 *MoE-nD: Per-Layer Mixture-of-Experts Routing for Multi-Axis KV Cache Compression*
436462
> **聚焦领域**:Multi-Axis KV Cache Compression · Per-Layer Routing · Heterogeneous Quantization
437463
> **arXiv**:[`arXiv:2604.17695`](https://arxiv.org/abs/2604.17695)
438464
@@ -449,7 +475,7 @@ $$
449475

450476
---
451477

452-
### 2.8 [2026-09-24] DriveMoE: Mixture-of-Experts for Vision-Language-Action Model in End-to-End Autonomous Driving
478+
### 3.8 [2026-09-24] DriveMoE: Mixture-of-Experts for Vision-Language-Action Model in End-to-End Autonomous Driving
453479

454480
* **论文信息**:`arXiv:2505.16278` (2025/2026)
455481
* **核心关键词**:End-to-End Autonomous Driving、Scene-Specialized Vision MoE、Skill-Specialized Action MoE、Flow-Matching Planner
@@ -497,7 +523,7 @@ $$
497523

498524
---
499525

500-
### 2.9 [2026-09-23] HiMoE-VLA: Hierarchical Mixture-of-Experts for Generalist Vision-Language-Action Policies
526+
### 3.9 [2026-09-23] HiMoE-VLA: Hierarchical Mixture-of-Experts for Generalist Vision-Language-Action Policies
501527

502528
* **论文信息**:`arXiv:2512.05693` (2025/2026)
503529
* **核心关键词**:Hierarchical MoE、Generalist VLA Policy、Task-Skill Decoupled Routing、Gradient Conflict Mitigation
@@ -545,7 +571,7 @@ $$
545571

546572
---
547573

548-
### 2.10 [2026-09-22] SPIN: Unifying Sparse Attention with Hierarchical Memory for Scalable Long-Context LLM Serving
574+
### 3.10 [2026-09-22] SPIN: Unifying Sparse Attention with Hierarchical Memory for Scalable Long-Context LLM Serving
549575

550576
* **论文信息**:`arXiv:2604.26837` (2026-04)
551577
* **核心关键词**:Sparse Attention Serving、Hierarchical GPU-CPU Memory、Asynchronous Layer-Ahead Prefetching
@@ -595,7 +621,7 @@ $$
595621

596622
---
597623

598-
### 2.11 [2026-09-21] MoE-FM: Towards Faster Language Model Inference Using Mixture-of-Experts Flow Matching
624+
### 3.11 [2026-09-21] MoE-FM: Towards Faster Language Model Inference Using Mixture-of-Experts Flow Matching
599625

600626
* **论文信息**:`arXiv:2604.15009` (2026-04)
601627
* **核心关键词**:Mixture-of-Experts Flow Matching、Piecewise-Linear Vector Fields、Latent Flow Language Models
@@ -646,7 +672,7 @@ $$
646672

647673
---
648674

649-
### 2.12 [2026-09-20] CARE: Spend Experts Where You Are Unsure — Confidence-Adaptive Routing for MoE-LoRA
675+
### 3.12 [2026-09-20] CARE: Spend Experts Where You Are Unsure — Confidence-Adaptive Routing for MoE-LoRA
650676

651677
* **论文信息**:`arXiv:2607.26052` (2026-07)
652678
* **核心关键词**:Confidence-Adaptive Routing、MoE-LoRA、Nucleus Expert Activation、Router Uncertainty Entropy
@@ -705,7 +731,7 @@ $$
705731

706732
---
707733

708-
### 2.13 [2026-09-19] REAP: Router-Weighted Expert Activation Pruning for Sparse MoE Models
734+
### 3.13 [2026-09-19] REAP: Router-Weighted Expert Activation Pruning for Sparse MoE Models
709735

710736
* **论文信息**:`arXiv:2510.13999` (2025/2026)
711737
* **核心关键词**:MoE Expert Pruning、Router Gate Weighting、Expert Activation Norm、Generative Reasoning Preservation
@@ -762,7 +788,7 @@ $$
762788

763789
---
764790

765-
### 2.14 [2026-09-18] 🧩 *MoE-Tile: Warp-Aligned Tensor Slicing for Zero-Overhead Dynamic Sparse Routing on Modern Accelerators*
791+
### 3.14 [2026-09-18] 🧩 *MoE-Tile: Warp-Aligned Tensor Slicing for Zero-Overhead Dynamic Sparse Routing on Modern Accelerators*
766792
> **聚焦领域**:Mixture of Experts (MoE) · GPU Kernel Optimization · Warp Divergence · Hardware-Aware Sparsity
767793
> **arXiv**:[`arXiv:2609.09112`](https://arxiv.org/abs/2609.09112)
768794

0 commit comments

Comments
 (0)