Skip to content

Commit 54b3f56

Browse files
authored
Cover tokenizer build when running install_executorch.sh --clean (pytorch#16264)
As title, it was erroring out when building tokenizers when running `install_executorch.sh`. Remove the build folder and rebuild fix the issue
1 parent df626bd commit 54b3f56

2 files changed

Lines changed: 13 additions & 1 deletion

File tree

‎Makefile‎

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -194,4 +194,6 @@ gemma3-cpu:
194194
@echo " Binary: cmake-out/examples/models/gemma3/gemma3_e2e_runner"
195195

196196
clean:
197-
rm -rf cmake-out
197+
rm -rf cmake-out \
198+
extension/llm/tokenizers/build \
199+
extension/llm/tokenizers/pytorch_tokenizers.egg-info

‎install_executorch.py‎

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -54,6 +54,16 @@ def clean():
5454
print("Cleaning buck cached state and metadata ...")
5555
shutil.rmtree(os.path.expanduser("~/.buck/buckd"), ignore_errors=True)
5656

57+
# tokenizers build cleanup
58+
tokenizer_dirs = [
59+
"extension/llm/tokenizers/build",
60+
"extension/llm/tokenizers/pytorch_tokenizers.egg-info",
61+
]
62+
63+
for d in tokenizer_dirs:
64+
print(f"Cleaning {d}...")
65+
shutil.rmtree(d, ignore_errors=True)
66+
5767
# Clean ccache if available
5868
try:
5969
result = subprocess.run(["ccache", "--version"], capture_output=True, text=True)

0 commit comments

Comments
 (0)