Skip to content

Commit fb9429e

Browse files
authored
Update README with v1.0.0.rc.x integration progress (#2575)
Add banner with v1.0.0 integration progress
1 parent 7f5360e commit fb9429e

1 file changed

Lines changed: 7 additions & 1 deletion

File tree

‎README.md‎

Lines changed: 7 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -15,11 +15,17 @@ The SOTA tokenization library on all languages, all models, all hardwares.
1515

1616
Our goal with `tokenizers` is to develop and maintain the industry's standard tokenization engine, making it the defacto place for everyone to contribute to the whole ecosystem.
1717

18-
### Release candidate: v1.0.0.rc.0
18+
<picture>
19+
<source media="(prefers-color-scheme: dark)" srcset="https://huggingface.co/spaces/sbrandeis/tokenizers-v1-compat/resolve/main/banner-dark.png">
20+
<img alt="tokenizers v1.0.0 progress" src="https://huggingface.co/spaces/sbrandeis/tokenizers-v1-compat/resolve/main/banner-light.png">
21+
</picture>
22+
23+
### Release candidate: v1.0.0.rc.x
1924

2025
As we are switching from 0.23 to v1.0.0, the current library does not ship all features. If you are worried, check the [v1 features](#v1-features) section for more details of what we are bringing back.
2126
We will publish a blog about what breaking changes we introduced. We strived to keep them as small as possible.
2227

28+
2329
# Installation
2430

2531
```bash

0 commit comments

Comments
 (0)