You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Commit d488d1b
Browse filesBrowse the repository at this point in the historyBrowse files
fix(privacy-filter)!: resolve labels from modelName, fix fromCustomModel identity collapse (#1198)
## Description
`PrivacyFilterModelSources` carried `labelNames` / `viterbiBiases` on it
but `usePrivacyFilter`'s deps only watched `modelName` / `modelSource` /
`tokenizerSource`, so two custom loads with the same sources but
different labels collapsed to the same identity and the runner kept the
old config silently. `fromCustomModel` made it worse by hardcoding
`modelName: 'custom'`.
Followed @chmjkb's suggestion on #1116: match the `ObjectDetection` /
`Segmentation` shape — drop `labelNames` / `viterbiBiases` from
`PrivacyFilterModelSources`, resolve them from `modelName` via an
internal lookup, and keep custom-label support only on `fromCustomModel`
(unreachable from the hook). The hook's existing deps now fully cover
the prop surface, so the collapse is gone.
### Introduces a breaking change?
- [x] Yes
- [ ] No
(`labelNames` / `viterbiBiases` are no longer accepted as fields on
`PrivacyFilterModelSources` or on the `PRIVACY_FILTER_*` constants.
Custom label spaces and Viterbi tuning move to
`PrivacyFilterModule.fromCustomModel`.)
### Type of change
- [x] Bug fix (change which fixes an issue)
- [ ] New feature (change which adds functionality)
- [ ] Documentation update (improves or adds clarity to existing
documentation)
- [ ] Other (chores, tests, code style improvements etc.)
### Tested on
- [ ] iOS
- [ ] Android
### Testing instructions
1. Run \`apps/llm\` and open the Privacy Filter screen.
2. Switch between the OpenAI and Nemotron presets — both should load and
produce the expected entity spans.
3. Re-trigger the hook with the same preset — should not reload.
### Related issues
Closes#1116
### Checklist
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [x] I have updated the documentation accordingly
- [x] My changes generate no new warnings
Copy file name to clipboardExpand all lines: docs/docs/03-hooks/01-natural-language-processing/usePrivacyFilter.md
+2-2Lines changed: 2 additions & 2 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -53,7 +53,7 @@ try {
53
53
54
54
`usePrivacyFilter` takes [`PrivacyFilterProps`](../../06-api-reference/interfaces/PrivacyFilterProps.md) that consists of:
55
55
56
-
-`model` of type [`PrivacyFilterModelSources`](../../06-api-reference/interfaces/PrivacyFilterModelSources.md)containing the model source, tokenizer source, and BIOES label list.
56
+
-`model` of type [`PrivacyFilterModelSources`](../../06-api-reference/type-aliases/PrivacyFilterModelSources.md)— a built-in preset (`modelName` + `modelSource` + `tokenizerSource`). The label list and Viterbi defaults are resolved from `modelName`; for custom fine-tunes use [`PrivacyFilterModule.fromCustomModel`](../../04-typescript-api/01-natural-language-processing/PrivacyFilterModule.md) directly.
57
57
- An optional flag [`preventLoad`](../../06-api-reference/interfaces/PrivacyFilterProps.md#preventload) which prevents auto-loading of the model.
58
58
59
59
You need more details? Check the following resources:
@@ -78,7 +78,7 @@ Token indices in returned entities are positions in the tokenizer's output (the
78
78
79
79
### Tuning precision and recall
80
80
81
-
Both built-in models ship with neutral, validity-only Viterbi decoding by default. If you want to shift the precision/recall tradeoff, pass an optional [`viterbiBiases`](../../06-api-reference/interfaces/PrivacyFilterModelSources.md#viterbibiases) object — six floats matching the operating-point schema in OpenAI's `viterbi_calibration.json`. Negative `backgroundToStart` makes the decoder enter spans more eagerly (higher recall); positive `backgroundStay` keeps it in the background label more often (higher precision).
81
+
Built-in presets run with neutral, validity-only Viterbi decoding. To shift the precision/recall tradeoff, load the model directly through [`PrivacyFilterModule.fromCustomModel`](../../04-typescript-api/01-natural-language-processing/PrivacyFilterModule.md) (reusing the preset's `modelSource` / `tokenizerSource` and label list) and pass a [`viterbiBiases`](../../06-api-reference/interfaces/ViterbiBiases.md) object — six floats matching the operating-point schema in OpenAI's `viterbi_calibration.json`. Negative `backgroundToStart` makes the decoder enter spans more eagerly (higher recall); positive `backgroundStay` keeps it in the background label more often (higher precision).
Copy file name to clipboardExpand all lines: docs/docs/04-typescript-api/01-natural-language-processing/PrivacyFilterModule.md
+3-5Lines changed: 3 additions & 5 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -27,20 +27,18 @@ All methods of `PrivacyFilterModule` are explained in details here: [`PrivacyFil
27
27
28
28
## Loading the model
29
29
30
-
To create a ready-to-use instance, call the static [`fromModelName`](../../06-api-reference/classes/PrivacyFilterModule.md#frommodelname) factory with the following parameters:
30
+
To create a ready-to-use instance for a built-in preset, call the static [`fromModelName`](../../06-api-reference/classes/PrivacyFilterModule.md#frommodelname) factory with the following parameters:
31
31
32
32
-`namedSources` — Object containing:
33
-
-`modelName` — Model name identifier.
33
+
-`modelName` — Built-in preset identifier (`'privacy-filter-openai'` or `'privacy-filter-nemotron'`). The runner resolves the matching BIOES label list from this value.
34
34
-`modelSource` — Location of the `.pte` model binary.
35
35
-`tokenizerSource` — Location of the `tokenizer.json` file.
36
-
-`labelNames` — BIOES label list. Index 0 must be `"O"`; the rest must follow the model's `id2label` mapping exactly.
37
-
-`viterbiBiases` (optional) — Six-field bias struct that shifts the decoder's precision/recall tradeoff. Defaults to neutral (validity-only Viterbi).
38
36
39
37
-`onDownloadProgress` — Optional callback to track download progress (value between 0 and 1).
40
38
41
39
The factory returns a promise that resolves to a loaded `PrivacyFilterModule` instance.
42
40
43
-
For custom-exported models, use [`fromCustomModel`](../../06-api-reference/classes/PrivacyFilterModule.md#fromcustommodel) instead — it takes the same fields as positional arguments and is convenient when you only have the raw resource locations.
41
+
For custom-exported models with a non-standard label space, use [`fromCustomModel`](../../06-api-reference/classes/PrivacyFilterModule.md#fromcustommodel) instead. It takes `modelSource`, `tokenizerSource`, and a `labelNames` array (BIOES; index 0 must be `"O"`, the rest must follow the model's `id2label` mapping exactly), plus an optional `options` object with `viterbiBiases` (six-field bias struct that shifts the decoder's precision/recall tradeoff; defaults to neutral validity-only Viterbi) and `onDownloadProgress`.
44
42
45
43
For more information on loading resources, take a look at [loading models](../../01-fundamentals/02-loading-models.md) page.
0 commit comments