Skip to content

Commit c279245

Browse files
tom-jerrtom-jerr
committed
add AI summary
Signed-off-by: Zhi Yiliu <2584074296@qq.com>
1 parent 545842a commit c279245

10 files changed

Lines changed: 269 additions & 183 deletions

File tree

‎.github/workflows/ci.yml‎

Lines changed: 7 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -38,13 +38,17 @@ jobs:
3838
# pip install ./mkdocs-document-dates --no-build-isolation
3939
# rm -rf mkdocs-document-dates
4040
- run: pip cache purge
41+
# 先进行一次严格构建，提前暴露错误
42+
- name: Build (strict)
43+
env:
44+
DEEPSEEK_API_KEY: ${{ secrets.DEEPSEEK_API_KEY }}
45+
DASHSCOPE_API_KEY: ${{ secrets.DASHSCOPE_API_KEY }}
46+
run: mkdocs build --verbose
4147
# 使用 AI 摘要部署
4248
- name: 使用 AI 摘要部署
4349
env:
44-
AI_SUMMARY_CI_ENABLED: 'true'
45-
AI_SUMMARY_CACHE_ENABLED: 'true'
46-
AI_SUMMARY_CACHE_EXPIRE_DAYS: '300'
4750
DEEPSEEK_API_KEY: ${{ secrets.DEEPSEEK_API_KEY }}
51+
DASHSCOPE_API_KEY: ${{ secrets.DASHSCOPE_API_KEY }}
4852
run: mkdocs gh-deploy --force
4953
# 自动提交 AI 缓存文件
5054
# - name: 自动提交 AI 缓存

‎.gitignore‎

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -3,4 +3,6 @@ site
33
.obsidian/
44
TaskNotes/
55
.env
6-
.ai_cache
6+
.ai_cache
7+
docs/assets
8+
_ai_summary*.json

‎docs/about/index.md‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -50,3 +50,7 @@ icon: /fontawesome/solid/face-smile-wink
5050
---
5151

5252
In my spare time, I enjoy listening to music, watching anime, and reading. Occasionally, I write about these hobbies on my blog to share my thoughts✍.
53+
54+
=== "Acadamy Page 🎓"
55+
56+
欢迎访问我的[主页🎓](../academy.md)，了解我的教育背景、项目经验和技能专长。如果你对我的经历感兴趣，或者想进一步交流，欢迎随时联系我！

‎docs/notes/sglang/一条 Request 在 SGLang 的前世今生.md‎

Lines changed: 10 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -2,12 +2,20 @@
22
title: 一条 Request 在 SGLang 的前世今生
33
tags:
44
- LLMInference
5-
date: 2025/11/20
5+
created: 2025-11-25
6+
ai_summary_lang: zh
7+
include:
8+
- ai-summary
9+
ai-summary-config:
10+
api: "tongyi"
11+
model: "qwen-turbo"
12+
prompt: "帮我把这篇文章总结为500字以内的摘要："
613
---
714

815
# 一条 Request 在 SGLang 的前世今生
916

10-
> 这里我们设置一些参数更接近真实的推理场景，启用 `mixed chunked` 参数，prefill 开启 `chunked_prefill`，cache 使用 `page_size > 1`，scheduler 使用 `overlap scheduler`
17+
> [!NOTE]
18+
> 这里我们设置一些参数更接近真实的推理场景，启用 `mixed chunked` 参数，prefill 开启 `chunked_prefill`，cache 使用 `page_size > 1`，scheduler 使用 `overlap scheduler`
1119
1220
**Continuous Batching** 的重点在 Scheduler[^sglang]，**Page Attention** 重点在算子层以及对应的封装，所以我们主要关注这两部分。在开始前，我们提出几个问题，在我们梳理完整个代码流程后再来回答：
1321

‎docs/notes/sglang/从代码看 SGLang 的 KV Cache.md‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -7,6 +7,7 @@ tags:
77

88
# 从代码看 SGLang 的 KV Cache
99

10+
> [!INFO]
1011
> 建议阅读 [SGLang Scheduler 技术变迁](https://tom-jerr.github.io/blogs/posts/SGLang%20Scheduler%20%E6%8A%80%E6%9C%AF%E5%8F%98%E8%BF%81/)后进行阅读，效果更好
1112
1213
实际上 SGLang 的 Cache 管理由三个组件构成 `tree_cache` 、 `req_to_token_pool` 和 `token_to_kv_pool_allocator` 。

‎docs/notes/sglang/大模型推理服务中的 Batching.md‎

Lines changed: 178 additions & 159 deletions
Large diffs are not rendered by default.

‎docs/notes/template.md‎

Lines changed: 13 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,13 @@
1+
---
2+
title: Template Note
3+
tags:
4+
- LLMInference
5+
created: 2025-11-20
6+
updated: 2025-12-2
7+
include:
8+
- ai-summary
9+
ai-summary-config:
10+
api: "tongyi"
11+
model: "qwen-turbo"
12+
prompt: "帮我把下面的内容总结为200字以内的摘要："
13+
---

‎mkdocs.yml‎

Lines changed: 9 additions & 15 deletions
Original file line numberDiff line numberDiff line change
@@ -97,24 +97,16 @@ plugins:
9797
- academy.md
9898
- links.md
9999
- changelog.md
100-
date_format: '%Y-%m-%d' # Date format strings (e.g., %Y-%m-%d, %b %d, %Y)
101-
time_format: '%H:%M:%S' # Time format strings (valid only if type=datetime)
102-
show_created: true # Show creation date: true false, default: true
103-
show_updated: true # Show last updated date: true false, default: true
104-
show_author: true # Show author: true(avatar) text(text) false(hidden), default: true
105100

106-
- ai-summary:
107-
ai_service: "deepseek"
108-
# local_enabled: true
109-
enabled_folders:
110-
- docs # 处理 docs 文件夹
111-
exclude_patterns:
112-
- index.md # 排除 index.md 文件
113-
- academy.md
114-
- links.md
115-
- changelog.md
116101
- statistics:
117102
words_per_minute: 200
103+
- ai-summary:
104+
api: "tongyi"
105+
model: "qwen-turbo"
106+
ignore_code: true
107+
cache: true
108+
cache_dir: "./"
109+
prompt: "Please help me summarize the following content into an abstract within 800 words: "
118110

119111
markdown_extensions:
120112
- abbr
@@ -189,6 +181,7 @@ extra_css:
189181
- css/flink.css
190182
- css/tasklist.css
191183
- css/footer-stats.css
184+
- css/ai-summary.css
192185

193186
extra:
194187
social:
@@ -371,6 +364,7 @@ nav:
371364
- Parallelization in LLM Inference: notes/llm_inference/LLMparallelization.md
372365
- PageAttention: blogs/posts/PageAttention.md
373366
- FlashAttention: blogs/posts/FlashAttention 原理 v1-v2.md
367+
- 大模型推理服务中的 Batching: notes/sglang/大模型推理服务中的 Batching.md
374368
- CUDA:
375369
- GPU 内存系统演进: notes/cuda/GPU 内存系统演进：最大化带宽利用与延迟隐藏的技术路径.md
376370
- CUDA Optimization for LLM Inference: notes/cuda/cudaopt.md

‎overrides/css/ai-summary.css‎

Lines changed: 41 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,41 @@
1+
:root {
2+
--md-admonition-icon--tongyiai-summary: url('data:image/svg+xml;base64,PHN2ZyB2ZXJzaW9uPSIxLjEiIHZpZXdCb3g9IjAgMCAyMTAgMjEwIiB4bWxucz0iaHR0cDovL3d3dy53My5vcmcvMjAwMC9zdmciPjxwYXRoIGZpbGw9IiM2MTVDRUQiIGQ9Im0yMTIgMTI3LjEtMjQuMi00Mi0yLjktNSAxMi44LTIyLjFjMC40LTAuOCAwLjctMS42IDAuNy0yLjVzLTAuMi0xLjctMC43LTIuNWwtMTQuMy0yNC43Yy0wLjQtMC44LTEuMS0xLjQtMS44LTEuOHMtMS42LTAuNy0yLjUtMC43aC01NC4ybC0xMi44LTIyLjFjLTAuOS0xLjUtMi41LTIuNS00LjMtMi41aC0yOC40Yy0wLjkgMC0xLjcgMC4yLTIuNSAwLjctMC43IDAuNC0xLjQgMS0xLjggMS44bC0yNC4zIDQyLTIuOSA0LjloLTI1LjZjLTAuOSAwLTEuNyAwLjItMi41IDAuNy0wLjcgMC40LTEuNCAxLTEuOCAxLjhsLTE0LjMgMjQuN2MtMC40IDAuOC0wLjcgMS42LTAuNyAyLjVzMC4yIDEuNyAwLjcgMi41bDI3LjEgNDctMTIuOCAyMi4xYy0wLjQgMC44LTAuNyAxLjYtMC43IDIuNXMwLjIgMS43IDAuNyAyLjVsMTQuMyAyNC43YzAuNCAwLjggMS4xIDEuNCAxLjggMS44czEuNiAwLjcgMi41IDAuN2g1NC4ybDEyLjggMjIuMmMwLjkgMS41IDIuNSAyLjUgNC4zIDIuNWgyOC41YzAuOSAwIDEuNy0wLjIgMi41LTAuNyAwLjctMC40IDEuNC0xIDEuOC0xLjhsMjcuMS00N2gyNS43YzAuOSAwIDEuNy0wLjIgMi41LTAuNyAwLjctMC40IDEuNC0xIDEuOC0xLjhsMTQuMy0yNC43YzAuNC0wLjggMC43LTEuNiAwLjctMi41LTAuMi0wLjgtMC40LTEuNy0wLjgtMi41eiIvPjxwYXRoIGZpbGw9IiNGRkZGRkYiIGQ9Ik03OS40LDYuMWwxNC4yLDI0LjhMNzkuNCw1NS42aDExNGwtMTQuMiwyNC43SDcwLjhMNTMuNyw1MC42TDc5LjQsNi4xeiBNMjIuMyw1NS42aDI4LjVsNTQuMiw5My45TDg3LjksMTc5ICBIMzYuNmwxNC4yLTI0LjdoMjguNkwyMi4zLDU1LjZ6IE05My42LDE3OWwxNC4zLDI0LjhsNTcuMS05OC45bDE0LjIsMjQuN2gyOC42TDE4Miw4NS4yaC0zNC4yTDkzLjYsMTc5eiIvPjwvc3ZnPg==');
3+
--md-admonition-icon--chatgpt-summary: url('data:image/svg+xml;base64,PHN2ZyBpZD0ib3BlbmFpLXN5bWJvbCIgeG1sbnM9Imh0dHA6Ly93d3cudzMub3JnLzIwMDAvc3ZnIiB2aWV3Qm94PSIwIDAgMzIgMzIiPjxwYXRoIGQ9Ik0yOS43MSwxMy4wOUE4LjA5LDguMDksMCwwLDAsMjAuMzQsMi42OGE4LjA4LDguMDgsMCwwLDAtMTMuNywyLjlBOC4wOCw4LjA4LDAsMCwwLDIuMywxOC45LDgsOCwwLDAsMCwzLDI1LjQ1YTguMDgsOC4wOCwwLDAsMCw4LjY5LDMuODcsOCw4LDAsMCwwLDYsMi42OCw4LjA5LDguMDksMCwwLDAsNy43LTUuNjEsOCw4LDAsMCwwLDUuMzMtMy44NkE4LjA5LDguMDksMCwwLDAsMjkuNzEsMTMuMDlabS0xMiwxNi44MmE2LDYsMCwwLDEtMy44NC0xLjM5bC4xOS0uMTEsNi4zNy0zLjY4YTEsMSwwLDAsMCwuNTMtLjkxdi05bDIuNjksMS41NmEuMDguMDgsMCwwLDEsLjA1LjA3djcuNDRBNiw2LDAsMCwxLDE3LjY4LDI5LjkxWk00LjgsMjQuNDFhNiw2LDAsMCwxLS43MS00bC4xOS4xMSw2LjM3LDMuNjhhMSwxLDAsMCwwLDEsMGw3Ljc5LTQuNDlWMjIuOGEuMDkuMDksMCwwLDEsMCwuMDhMMTMsMjYuNkE2LDYsMCwwLDEsNC44LDI0LjQxWk0zLjEyLDEwLjUzQTYsNiwwLDAsMSw2LjI4LDcuOXY3LjU3YTEsMSwwLDAsMCwuNTEuOWw3Ljc1LDQuNDdMMTEuODUsMjIuNGEuMTQuMTQsMCwwLDEtLjA5LDBMNS4zMiwxOC42OGE2LDYsMCwwLDEtMi4yLTguMThabTIyLjEzLDUuMTQtNy43OC00LjUyTDIwLjE2LDkuNmEuMDguMDgsMCwwLDEsLjA5LDBsNi40NCwzLjcyYTYsNiwwLDAsMS0uOSwxMC44MVYxNi41NkExLjA2LDEuMDYsMCwwLDAsMjUuMjUsMTUuNjdabTIuNjgtNC0uMTktLjEyLTYuMzYtMy43YTEsMSwwLDAsMC0xLjA1LDBsLTcuNzgsNC40OVY5LjJhLjA5LjA5LDAsMCwxLDAtLjA5TDE5LDUuNGE2LDYsMCwwLDEsOC45MSw2LjIxWk0xMS4wOCwxNy4xNSw4LjM4LDE1LjZhLjE0LjE0LDAsMCwxLS4wNS0uMDhWOC4xYTYsNiwwLDAsMSw5Ljg0LTQuNjFMMTgsMy42LDExLjYxLDcuMjhhMSwxLDAsMCwwLS41My45MVpNMTIuNTQsMTQsMTYsMTJsMy40NywydjRMMTYsMjBsLTMuNDctMloiLz48L3N2Zz4=')
4+
}
5+
6+
.md-typeset .admonition.tongyiai-summary,
7+
.md-typeset details.tongyiai-summary {
8+
border-color: rgb(91, 97, 232);
9+
}
10+
11+
.md-typeset .tongyiai-summary>.admonition-title,
12+
.md-typeset .tongyiai-summary>summary {
13+
background-color: rgba(91, 97, 232, 0.1);
14+
}
15+
16+
.md-typeset .tongyiai-summary>.admonition-title::before,
17+
.md-typeset .tongyiai-summary>summary::before {
18+
background-color: rgb(91, 97, 232);
19+
-webkit-mask-image: var(--md-admonition-icon--tongyiai-summary);
20+
mask-image: var(--md-admonition-icon--tongyiai-summary);
21+
background-image: var(--md-admonition-icon--tongyiai-summary);
22+
}
23+
24+
25+
.md-typeset .admonition.chatgpt-summary,
26+
.md-typeset details.chatgpt-summary {
27+
border-color: rgb(33, 100, 95);
28+
}
29+
30+
.md-typeset .chatgpt-summary>.admonition-title,
31+
.md-typeset .chatgpt-summary>summary {
32+
background-color: rgba(33, 100, 95, .1);
33+
}
34+
35+
.md-typeset .chatgpt-summary>.admonition-title::before,
36+
.md-typeset .chatgpt-summary>summary::before {
37+
background-color: rgb(33, 100, 95);
38+
-webkit-mask-image: var(--md-admonition-icon--chatgpt-summary);
39+
mask-image: var(--md-admonition-icon--chatgpt-summary);
40+
background-image: var(--md-admonition-icon--chatgpt-summary);
41+
}

‎requirements.txt‎

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -5,8 +5,8 @@ mkdocs-minify-plugin
55
mkdocs-glightbox
66
mkdocs-video
77
mkdocs-rss-plugin
8-
mkdocs-document-dates
9-
mkdocs-ai-summary-wcowin
8+
mkdocs-document-dates>=3.4.9
109
mkdocs-statistics-plugin
11-
mkdocs-document-dates
10+
dashscope
11+
mkdocs-ai-summary
1212
#rich # for mkdocs-toc-plugin

0 commit comments

Comments
 (0)