Skip to content

Commit 9213558

Browse files
author
liuzhiyi
committed
quant knowledge addded
Signed-off-by: liuzhiyi <liuzhiyi02@baidu.com>
1 parent 14cb6b1 commit 9213558

59 files changed

Lines changed: 7522 additions & 1658 deletions

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

‎docs/cuda/quantization cuda.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -33,7 +33,7 @@
3333
- 输出浮点激活值（Float Activation）
3434

3535
### 优化
36-
- 显存增加的原因是要把int8的结果累加到int32类型中，因此相比于BFloat1增加了额外的显存
36+
- 显存增加的原因是要把int8的结果累加到int32类型中，因此相比于BFloat16增加了额外的显存
3737
- 为了减少显存占用，作者提出了一个优化方案：将乘法和累加操作融合在一起，直接计算出rescaled的结果（XWrescaled），我们可以用 bf16 存储中间的值。
3838
- 动态量化的数学表达：
3939
- 原始公式：Y = X.W

‎docs/llm_inference/deepep_architecture_notes.md‎

Lines changed: 1077 additions & 0 deletions
Large diffs are not rendered by default.

‎docs/llm_inference/img/dspark-cycle-parallel.svg‎

Lines changed: 33 additions & 0 deletions
Loading

‎docs/llm_inference/img/dspark-cycle-prefill.svg‎

Lines changed: 31 additions & 0 deletions
Loading

‎docs/llm_inference/img/dspark-cycle-scheduler.svg‎

Lines changed: 38 additions & 0 deletions
Loading

‎docs/llm_inference/img/dspark-cycle-serial.svg‎

Lines changed: 34 additions & 0 deletions
Loading

‎docs/llm_inference/img/dspark-cycle-verify.svg‎

Lines changed: 41 additions & 0 deletions
Loading

‎docs/llm_inference/img/dspark-overview.svg‎

Lines changed: 51 additions & 0 deletions
Loading

‎docs/llm_inference/img/kv_cache_quant_layout.svg‎

Lines changed: 93 additions & 0 deletions
Loading

0 commit comments

Comments
 (0)