You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Commit cc7b4ec
Browse filesBrowse the repository at this point in the historyBrowse files
feat: add enable_thinking option to template rendering and new MiniCPM5-2B model (#1571)
## Description
Adds option to switch-on/off thinking (for chat templates that respect
that setting) and new MiniCPM5-2B
https://huggingface.co/openbmb/MiniCPM5-2B as a substitute for Qwen 3.5
whose hybrid architecture doesn't work well with our runtime.
### Introduces a breaking change?
- [ ] Yes
- [x] No
### Type of change
- [ ] Bug fix (change which fixes an issue)
- [x] New feature (change which adds functionality)
- [ ] Documentation update (improves or adds clarity to existing
documentation)
- [ ] Other (chores, tests, code style improvements etc.)
### Tested on
- [x] iOS
- [ ] Android
### Testing instructions
- [ ] Run the NLP example app and test new LLM
- [ ] Check the HF repo
https://huggingface.co/software-mansion/react-native-executorch-minicpm-5
### Screenshots
<!-- Add screenshots here, if applicable -->
### Related issues
Closes#935
### Checklist
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [x] I have updated the documentation accordingly
- [x] My changes generate no new warnings
### Additional notes
<!-- Include any additional information, assumptions, or context that
reviewers might need to understand this PR. -->
---------
Co-authored-by: Mateusz Sluszniak <56299341+msluszniak@users.noreply.github.com>
Copy file name to clipboardExpand all lines: docs/docs/02-extensions/natural-language/02-llm-chat-and-generation.md
+16Lines changed: 16 additions & 0 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -151,6 +151,21 @@ type LLMChatTurnResult = {
151
151
152
152
Under the hood, [`createChatPreprocessor`](../../06-api-reference/react-native-executorch/namespaces/llm/functions/createChatPreprocessor.md) renders [`ChatMessage[]`](../../06-api-reference/react-native-executorch/namespaces/llm/type-aliases/ChatMessage.md) arrays using the model's official Jinja2 template from `tokenizer_config.json` (formatting special tokens, roles, and generation headers).
153
153
154
+
For models that support answering without thinking, set
155
+
`generationConfig: { enableThinking: false }` in `useLLMChatSession` or
156
+
`createLLMChatSession`. Override it per turn with
157
+
`session.sendMessage(message, onToken, { enableThinking: true })`. Direct
158
+
preprocessor users can pass `enableThinking` in the options to `render()` or
159
+
`process()`.
160
+
161
+
This exposes `enable_thinking` to Jinja. The template must honor this switch;
162
+
a template that always appends `<think>` needs to be corrected separately. Omit
163
+
the option to preserve the template's default. The raw runner accepts already
164
+
formatted prompts and ignores this field. When thinking is disabled, tool parsers
165
+
must accept answers without a generated `</think>` tag. Switching thinking modes
166
+
between turns assumes the template preserves previously cached messages; use
167
+
`resetOnTurn: true` if the switch also changes how earlier messages are rendered.
168
+
154
169
To keep multi-turn chat responsive without re-encoding past history on every message, the preprocessor uses incremental prompt diffing:
155
170
156
171
1. It verifies that the rendered prefix of previously committed turns is an exact substring match of the newly updated conversation.
@@ -381,6 +396,7 @@ The library provides ready-to-use models from the [Software Mansion HuggingFace
381
396
|**Microsoft Phi-4 Mini**|[`3.8B`](../../06-api-reference/variables/models.md#llmphi4_mini)| 2.62 GB – 7.15 GB | XNNPACK (CPU), MLX (Apple) | High-density reasoning model for STEM problem solving & coding. |
382
397
|**HuggingFace SmolLM2**|[`135M`](../../06-api-reference/variables/models.md#llmsmollm2_135m), [`360M`](../../06-api-reference/variables/models.md#llmsmollm2_360m), [`1.7B`](../../06-api-reference/variables/models.md#llmsmollm2_1_7b)| 158 MB – 1.83 GB | XNNPACK (CPU), MLX (Apple) | Smallest footprint family, for constrained devices and fast drafting. |
|**OpenBMB MiniCPM5**|[`2B`](../../06-api-reference/variables/models.md#llmminicpm5_2b)| 1.36 GB – 1.42 GB | XNNPACK (CPU), MLX (Apple) | Compact reasoning model with thinking and tool calling. |
384
400
385
401
:::tip Using Custom Models
386
402
To use your own fine-tuned LLM `.pte` model, pass an [`LLMModel`](../../06-api-reference/type-aliases/LLMModel.md) configuration object to [`useLLMChatSession`](../../06-api-reference/functions/useLLMChatSession.md) or [`createLLMChatSession`](../../06-api-reference/functions/createLLMChatSession.md):
0 commit comments