Skip to content

Commit f7a8da9

Browse files
ronibhakta1ronibhakta1
committed
Remove obsolete test files and add comprehensive benchmarks for ZON format
- Deleted `testing.py` and `types_test.py` as they were no longer needed. - Introduced `BENCHMARKS_ALL.md` for unified benchmark results comparing JSON, ZON, and TOON formats. - Added detailed benchmark scripts and data files for the `hikes` dataset, including `hikes.json` and `hikes.toon`. - Implemented token analysis scripts to evaluate performance using the `tiktoken` library. - Enhanced the README and other documentation to reflect the new benchmark results and comparisons. - Added unit tests for core features of the ZON format to ensure data integrity and type safety.
1 parent ece6d21 commit f7a8da9

67 files changed

Lines changed: 1420 additions & 11620 deletions

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

‎.github/workflows/python-package.yml‎

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -52,3 +52,8 @@ jobs:
5252
- name: Test basic encode/decode
5353
run: |
5454
python -c "import sys; sys.path.insert(0, 'zon-format/src'); import zon; data = [{'id': 1, 'name': 'test'}]; encoded = zon.encode(data); decoded = zon.decode(encoded); assert decoded == data; print('✓ Basic encode/decode works')"
55+
56+
- name: Run Regression Benchmark
57+
run: |
58+
cd zon-format
59+
python test_comprehensive.py

‎ALL.md‎

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,8 @@
1+
# All Benchmarks
2+
3+
This repository now includes a consolidated benchmark file with the latest token analysis and a format comparison (JSON vs ZON vs TOON) for the `hikes` sample.
4+
5+
- Full analysis and raw output: `BENCHMARKS_ALL.md`
6+
- Quick summary: ZON is smaller than TOON for the `hikes` example (22 bytes smaller, 8 tokens fewer using `o200k_base`).
7+
8+
If you want this text embedded into `README.md` or `zon-format/README.md`, I can insert the short summary sections there as well.

‎BENCHMARKS_ALL.md‎

Lines changed: 153 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,153 @@
1+
# Unified Benchmark Results
2+
3+
This file contains the consolidated benchmark results (JSON vs ZON) and a focused comparison with TOON for the `hikes` sample used during analysis.
4+
5+
The token counts were computed using `tiktoken` with the `o200k_base` tokenizer (GPT-5 style) and `cl100k_base` for cross-checking TOON's published numbers.
6+
7+
Below is the raw token analysis output used for the documentation and comparison.
8+
9+
```
10+
====================================================================================================
11+
TOKEN ANALYSIS - Using GPT-5 Tokenizer (o200k_base)
12+
====================================================================================================
13+
14+
This matches TOON's official benchmark methodology
15+
Reference: https://github.com/toon-format/toon#benchmarks
16+
17+
────────────────────────────────────────────────────────────────────────────────────────────────────
18+
Format | Bytes | o200k_base | cl100k_base | p50k_base
19+
────────────────────────────────────────────────────────────────────────────────────────────────────
20+
JSON (formatted) | 680 | 229 | 235 | 266
21+
JSON (compact) | 451 | 139 | 142 | 148
22+
ZON | 264 | 96 | 98 | 111
23+
TOON | 286 | 104 | 106 | 115
24+
25+
====================================================================================================
26+
TOON'S CLAIMED NUMBERS vs ACTUAL
27+
====================================================================================================
28+
29+
JSON (formatted):
30+
TOON claims: 235 tokens
31+
o200k_base: 229 tokens
32+
cl100k_base: 235 tokens
33+
p50k_base: 266 tokens
34+
35+
TOON:
36+
TOON claims: 106 tokens
37+
o200k_base: 104 tokens
38+
cl100k_base: 106 tokens
39+
p50k_base: 115 tokens
40+
41+
ZON:
42+
o200k_base: 96 tokens
43+
cl100k_base: 98 tokens
44+
p50k_base: 111 tokens
45+
46+
====================================================================================================
47+
COMPRESSION ANALYSIS (using o200k_base)
48+
====================================================================================================
49+
50+
Format | Tokens | Reduction
51+
──────────────────────────────────────────────────
52+
JSON (formatted) | 229 | 0.0%
53+
JSON (compact) | 139 | 39.3%
54+
ZON | 96 | 58.1%
55+
TOON | 104 | 54.6%
56+
57+
====================================================================================================
58+
ZON vs TOON
59+
====================================================================================================
60+
61+
Token count:
62+
TOON: 104
63+
ZON: 96
64+
Difference: 8 tokens
65+
ZON is 7.7% fewer tokens
66+
67+
====================================================================================================
68+
FULL TEXT OUTPUT
69+
====================================================================================================
70+
71+
JSON (formatted) (229 tokens, 680 bytes):
72+
────────────────────────────────────────────────────────────────────────────────────────────────────
73+
{
74+
"context": {
75+
"task": "Our favorite hikes together",
76+
"location": "Boulder",
77+
"season": "spring_2025"
78+
},
79+
"friends": [
80+
"ana",
81+
"luis",
82+
"sam"
83+
],
84+
"hikes": [
85+
{
86+
"id": 1,
87+
"name": "Blue Lake Trail",
88+
"distanceKm": 7.5,
89+
"elevationGain": 320,
90+
"companion": "ana",
91+
"wasSunny": true
92+
},
93+
{
94+
"id": 2,
95+
"name": "Ridge Overlook",
96+
"distanceKm": 9.2,
97+
"elevationGain": 540,
98+
"companion": "luis",
99+
"wasSunny": false
100+
},
101+
{
102+
"id": 3,
103+
"name": "Wildflower Loop",
104+
"distanceKm": 5.1,
105+
"elevationGain": 180,
106+
"companion": "sam",
107+
"wasSunny": true
108+
}
109+
]
110+
}
111+
112+
JSON (compact) (139 tokens, 451 bytes):
113+
────────────────────────────────────────────────────────────────────────────────────────────────────
114+
{"context":{"task":"Our favorite hikes together","location":"Boulder","season":"spring_2025"},"friends":["ana","luis","sam"],"hikes":[{"id":1,"name":"Blue Lake Trail","distanceKm":7.5,"elevationGain":320,"companion":"ana","wasSunny":true},{"id":2,"name":"Ridge Overlook","distanceKm":9.2,"elevationGain":540,"companion":"luis","wasSunny":false},{"id":3,"name":"Wildflower Loop","distanceKm":5.1,"elevationGain":180,"companion":"sam","wasSunny":true}]}
115+
116+
ZON (96 tokens, 264 bytes):
117+
────────────────────────────────────────────────────────────────────────────────────────────────────
118+
context:"{task:Our favorite hikes together,location:Boulder,season:spring_2025}"
119+
friends:"[ana,luis,sam]"
120+
121+
@hikes(3):companion,distanceKm,elevationGain,id,name,wasSunny
122+
ana,7.5,320,1,Blue Lake Trail,T
123+
luis,9.2,540,2,Ridge Overlook,F
124+
sam,5.1,180,3,Wildflower Loop,T
125+
126+
TOON (104 tokens, 286 bytes):
127+
────────────────────────────────────────────────────────────────────────────────────────────────────
128+
context:
129+
task: Our favorite hikes together
130+
location: Boulder
131+
season: spring_2025
132+
friends[3]: ana,luis,sam
133+
hikes[3]{id,name,distanceKm,elevationGain,companion,wasSunny}:
134+
1,Blue Lake Trail,7.5,320,ana,true
135+
2,Ridge Overlook,9.2,540,luis,false
136+
3,Wildflower Loop,5.1,180,sam,true
137+
138+
====================================================================================================
139+
140+
```
141+
142+
Summary (hikes sample):
143+
144+
| Format | Bytes | o200k_base tokens | Reduction vs JSON (formatted) |
145+
|---|---:|---:|---:|
146+
| JSON (formatted) | 680 B | 229 | 0.0% |
147+
| JSON (compact) | 451 B | 139 | 39.3% |
148+
| ZON | 264 B | 96 | 58.1% |
149+
| TOON | 286 B | 104 | 54.6% |
150+
151+
Notes:
152+
- Token counts use `o200k_base` unless otherwise noted. TOON's published numbers align with `cl100k_base` in this example.
153+
- The ZON encoding for the hikes sample is smaller than TOON by 22 bytes and 8 tokens (7.7% fewer tokens).

‎CHANGELOG.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
# Changelog
22

3-
## [0.1.0] - 2023-10-27
3+
## [1.0.2] - 2025-11-25
44

55
### Added
66
- Initial release.

‎EDGE_CASES.md‎

Lines changed: 0 additions & 140 deletions
This file was deleted.

zon-format/LICENSE renamed to LICENSE

File renamed without changes.

0 commit comments

Comments
 (0)