Repository navigation

Commit 3f61a8a

Browse files

GregoryComer

authored and

jirioc

committed

Bump tokenizers to b007644 (pytorch#14638)

### Summary Update tokenizers to pick up meta-pytorch/tokenizers@b007644, which fixes an issue when referencing tokenizers via CMake find_package on Windows. ### Test plan CI

1 parent 6a74efd commit 3f61a8aCopy full SHA for 3f61a8a

1 file changed

extension/llm
- tokenizers

`‎extension/llm/tokenizers‎`

Submodule tokenizers updated 1 file

cmake/tokenizers-config.cmake.in
+5
-1

Comments

(0)

File tree

‎extension/llm/tokenizers‎

0 commit comments

`‎extension/llm/tokenizers‎`