Skip to content

Commit 48744db

Browse files
Anthony GordonAnthony Gordon
authored andcommitted
Initial release: DejaJson wire format v1
Binary envelope (DJ magic + version + mode) over a zlib stream, in dictionary (d) and plain (z) modes. Dio interceptor advertises X-Deja-Json with the dictionary id, fetches bytes transparently, and restores plain JSON — including error responses. Cross-implementation vectors shared with the nylo/dejajson Laravel package.
0 parents  commit 48744db

26 files changed

Lines changed: 1802 additions & 0 deletions

.github/workflows/publish.yml

Lines changed: 35 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,35 @@
1+
# .github/workflows/publish.yml
2+
name: Publish to pub.dev
3+
4+
on:
5+
push:
6+
tags:
7+
- 'v[0-9]+.[0-9]+.[0-9]+*' # tag-pattern on pub.dev: 'v'
8+
9+
# Publish using the reusable workflow from dart-lang.
10+
jobs:
11+
publish-package:
12+
runs-on: ubuntu-latest
13+
14+
steps:
15+
- name: Checkout code
16+
uses: actions/checkout@v3
17+
18+
- name: Set up Flutter
19+
uses: subosito/flutter-action@v2
20+
21+
- name: Get dependencies
22+
run: flutter pub get
23+
24+
- name: Format code
25+
run: dart format .
26+
27+
- name: Check publish warnings
28+
run: flutter pub publish --dry-run
29+
30+
- name: Publish package
31+
uses: k-paxian/dart-package-publisher@master
32+
with:
33+
credentialJson: ${{ secrets.CREDENTIAL_SECRET }}
34+
flutter: true
35+
skipTests: true

.github/workflows/tests.yml

Lines changed: 28 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,28 @@
1+
name: tests
2+
3+
on:
4+
push:
5+
pull_request:
6+
7+
jobs:
8+
dart:
9+
runs-on: ubuntu-latest
10+
11+
steps:
12+
- uses: actions/checkout@v4
13+
14+
- uses: dart-lang/setup-dart@v1
15+
with:
16+
sdk: stable
17+
18+
- name: Install dependencies
19+
run: dart pub get
20+
21+
- name: Analyze
22+
run: dart analyze --fatal-infos
23+
24+
- name: Check formatting
25+
run: dart format --output=none --set-exit-if-changed lib test tool example
26+
27+
- name: Run tests (VM + browser)
28+
run: dart test -p vm,chrome

.gitignore

Lines changed: 27 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,27 @@
1+
# Dart / Pub
2+
# https://dart.dev/guides/libraries/private-files
3+
.dart_tool/
4+
.packages
5+
build/
6+
doc/api/
7+
8+
# Library packages should not commit the lockfile
9+
# https://dart.dev/guides/libraries/private-files#pubspeclock
10+
pubspec.lock
11+
12+
# Test coverage
13+
coverage/
14+
.test_coverage.dart
15+
16+
# IDEs & editors
17+
.idea/
18+
*.iml
19+
*.ipr
20+
*.iws
21+
.vscode/
22+
*.swp
23+
*~
24+
25+
# OS files
26+
.DS_Store
27+
Thumbs.db

CHANGELOG.md

Lines changed: 20 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,20 @@
1+
# Changelog
2+
3+
## 0.1.0 — 2026-07-16
4+
5+
Initial release.
6+
7+
- DejaJson wire format v1: a 4-byte binary envelope over a zlib (RFC 1950)
8+
stream, in dictionary (`d`) and plain (`z`) modes.
9+
- `DejaJsonInterceptor` for Dio: advertises `X-Deja-Json` (with the
10+
dictionary id when one is loaded), fetches bytes transparently, and
11+
restores plain JSON responses — including error responses — exactly as Dio
12+
would have produced them.
13+
- `DejaJsonDictionary` for loading the dictionary trained by
14+
`php artisan dejajson:train` (shipped as an app asset).
15+
- `DejaJsonCodec` for manual encode/decode (websockets, cached blobs,
16+
isolates).
17+
- Web builds advertise nothing (zlib needs `dart:io`) and pass plain JSON
18+
through untouched.
19+
- Cross-implementation test vectors shared with the `nylo/dejajson` Laravel
20+
package.

LICENSE

Lines changed: 21 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,21 @@
1+
MIT License
2+
3+
Copyright (c) 2026 Anthony Gordon
4+
5+
Permission is hereby granted, free of charge, to any person obtaining a copy
6+
of this software and associated documentation files (the "Software"), to deal
7+
in the Software without restriction, including without limitation the rights
8+
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
9+
copies of the Software, and to permit persons to whom the Software is
10+
furnished to do so, subject to the following conditions:
11+
12+
The above copyright notice and this permission notice shall be included in all
13+
copies or substantial portions of the Software.
14+
15+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
16+
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
17+
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
18+
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
19+
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
20+
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
21+
SOFTWARE.

README.md

Lines changed: 161 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,161 @@
1+
# dejajson
2+
3+
Transparently decode DejaJson-encoded API responses with a drop-in
4+
[Dio](https://pub.dev/packages/dio) interceptor.
5+
6+
Pairs with the [`nylo/dejajson`](https://github.com/nylo-core/laravel-deja-json)
7+
Laravel package: the server compresses JSON responses against a **dictionary
8+
trained on your own API** (zlib preset dictionaries — the compressor has
9+
effectively *already seen* each response before it starts), and
10+
`DejaJsonInterceptor` restores the original JSON before your app sees it.
11+
Your models, decoders and error handling don't change at all.
12+
13+
Measured on realistic Laravel API payloads:
14+
15+
| Payload | plain JSON | transport gzip | **dejajson** |
16+
|---|---|---|---|
17+
| single resource | 693 B | 437 B (−37%) | **58 B (−92%)** |
18+
| list of 25 | 17,469 B | 1,581 B (−91%) | **887 B (−95%)** |
19+
| list of 200 | 137,919 B | 8,002 B (−94%) | **7,303 B (−95%)** |
20+
21+
- **Zero app-code changes**`response.data` is plain JSON data, exactly as
22+
if the server had never encoded it. Error responses (422 validation errors
23+
and friends) are decoded too, and plain-JSON servers pass through exactly
24+
as Dio would have produced them.
25+
- **Safe by negotiation** — the interceptor advertises support (and its
26+
dictionary id) with an `X-Deja-Json` request header; servers only encode
27+
for clients that ask, and only use the dictionary when both sides hold
28+
identical bytes. A stale dictionary degrades to plain zlib, never to
29+
garbage.
30+
- **Native platforms** — zlib comes from `dart:io`. Web builds advertise
31+
nothing and receive plain JSON, so code stays portable.
32+
33+
## Getting started
34+
35+
```bash
36+
dart pub add dejajson # or: flutter pub add dejajson
37+
```
38+
39+
Train a dictionary server side and ship it as an asset:
40+
41+
```bash
42+
# on the Laravel side
43+
php artisan dejajson:train storage/dejajson-samples
44+
cp storage/app/dejajson/dictionary.bin <your_app>/assets/dejajson/dictionary.bin
45+
```
46+
47+
```yaml
48+
# pubspec.yaml
49+
flutter:
50+
assets:
51+
- assets/dejajson/dictionary.bin
52+
```
53+
54+
```dart
55+
import 'package:dio/dio.dart';
56+
import 'package:dejajson/dejajson.dart';
57+
import 'package:flutter/services.dart' show rootBundle;
58+
59+
final bytes = await rootBundle.load('assets/dejajson/dictionary.bin');
60+
final dictionary = DejaJsonDictionary.fromBytes(bytes.buffer.asUint8List());
61+
62+
final dio = Dio(BaseOptions(baseUrl: 'https://api.example.com'))
63+
..interceptors.add(DejaJsonInterceptor(dictionary: dictionary));
64+
65+
final response = await dio.get('/users');
66+
print(response.data); // plain JSON — decoded transparently
67+
```
68+
69+
No dictionary yet? `DejaJsonInterceptor()` without one advertises mode `z`
70+
and still gets you ~gzip-level compression with no setup.
71+
72+
After retraining on the server, ship the new file with your next app release.
73+
In between, old clients keep working: the ids no longer match, so the server
74+
simply falls back to mode `z` for them.
75+
76+
## Using with Nylo
77+
78+
Register the interceptor on your API service:
79+
80+
```dart
81+
class ApiService extends NyApiService {
82+
ApiService({BuildContext? buildContext})
83+
: super(buildContext, decoders: modelDecoders);
84+
85+
@override
86+
String get baseUrl => getEnv('API_BASE_URL');
87+
88+
@override
89+
Map<Type, Interceptor> get interceptors => {
90+
...super.interceptors,
91+
DejaJsonInterceptor: DejaJsonInterceptor(dictionary: dictionary),
92+
};
93+
}
94+
```
95+
96+
Everything else — `network<User>()`, decoders, caching — keeps working
97+
unchanged, just with far smaller payloads on the wire.
98+
99+
## Options
100+
101+
```dart
102+
DejaJsonInterceptor(
103+
dictionary: dictionary, // the trained dictionary (null = mode "z" only)
104+
advertise: true, // add the request header (default true)
105+
modes: 'dz', // advertised modes; defaults to the platform's best
106+
rewriteContentType: true, // report application/json after decoding (default true)
107+
requestHeader: 'X-Deja-Json', // must match the server's dejajson.request_header
108+
codec: DejaJsonCodec(), // swap in a configured codec if needed
109+
)
110+
```
111+
112+
Notes:
113+
114+
- Envelopes are binary, so for JSON requests the interceptor fetches bytes
115+
and rebuilds the response itself — plain-JSON and DejaJson servers both
116+
come out as ordinary decoded JSON. Requests with `ResponseType.bytes`,
117+
`stream` or `plain` are never advertised for and never touched.
118+
- A corrupt envelope (or one for a dictionary you don't hold) surfaces as a
119+
`DioException` (`badResponse`) whose `error` is a
120+
`DejaJsonFormatException`, rather than silently handing your app garbage.
121+
- If you call the API from browsers as well, remember to whitelist
122+
`X-Deja-Json` in the server's CORS `allowed_headers`.
123+
124+
## Manual use
125+
126+
The codec works without Dio — websockets, cached blobs, isolates:
127+
128+
```dart
129+
const codec = DejaJsonCodec();
130+
131+
final data = codec.decode(envelopeBytes, dictionary: dictionary);
132+
final envelope = codec.encode(data, dictionary: dictionary); // symmetric
133+
```
134+
135+
## Wire format (v1)
136+
137+
```
138+
byte 0–1 magic "DJ"
139+
byte 2 format version (0x01)
140+
byte 3 mode — "d" (zlib + shared dictionary) or "z" (zlib)
141+
byte 4+ zlib stream (RFC 1950) of the minified UTF-8 JSON document
142+
```
143+
144+
Sent with content type `application/vnd.dejajson`. zlib framing is
145+
deliberate: the stream embeds an Adler-32 of the dictionary (so a wrong
146+
dictionary fails loudly) and of the content (so corruption does too).
147+
148+
One platform caveat inherited from JavaScript, not from DejaJson: on web
149+
builds, JSON numbers like `1.0` decode as the integer `1` — identical to how
150+
plain JSON behaves there.
151+
152+
## Testing
153+
154+
```bash
155+
dart test # VM suite
156+
dart test -p vm,chrome # additionally runs the web-platform tests in Chrome
157+
```
158+
159+
## License
160+
161+
MIT © [Anthony Gordon](https://nylo.dev)

analysis_options.yaml

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,6 @@
1+
include: package:lints/recommended.yaml
2+
3+
analyzer:
4+
language:
5+
strict-casts: true
6+
strict-inference: true

example/dejajson_example.dart

Lines changed: 29 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,29 @@
1+
import 'dart:convert';
2+
3+
import 'package:dejajson/dejajson.dart';
4+
import 'package:dio/dio.dart';
5+
6+
Future<void> main() async {
7+
// Ship the server's trained dictionary as an asset and load it once at
8+
// startup. In Flutter:
9+
//
10+
// final data = await rootBundle.load('assets/dejajson/dictionary.bin');
11+
// final dictionary = DejaJsonDictionary.fromBytes(data.buffer.asUint8List());
12+
final dictionary =
13+
DejaJsonDictionary.fromBytes(utf8.encode('…dictionary bytes…'));
14+
15+
final dio = Dio(BaseOptions(baseUrl: 'https://api.example.com'))
16+
..interceptors.add(DejaJsonInterceptor(dictionary: dictionary));
17+
18+
// The interceptor adds `X-Deja-Json: dz; dict=<id>` to every JSON request.
19+
// When the server answers with application/vnd.dejajson, the binary body
20+
// is decoded in place — response.data is ordinary JSON data, exactly as if
21+
// the server had sent plain JSON.
22+
final response = await dio.get<Object?>('/users');
23+
print(response.data);
24+
25+
// Manual decoding (websockets, cached blobs, …) works too:
26+
const codec = DejaJsonCodec();
27+
final envelope = codec.encode({'hello': 'world'}, dictionary: dictionary);
28+
print(codec.decode(envelope, dictionary: dictionary));
29+
}

lib/dejajson.dart

Lines changed: 13 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,13 @@
1+
/// Transparently decode DejaJson-encoded API responses with Dio.
2+
///
3+
/// Pairs with the `nylo/dejajson` Laravel package: the server compresses
4+
/// JSON responses against a dictionary trained on your own API (zlib preset
5+
/// dictionaries — the compressor has effectively already seen each response
6+
/// before it starts), and [DejaJsonInterceptor] restores the original JSON
7+
/// before your app sees it.
8+
library;
9+
10+
export 'src/codec.dart' show DejaJsonCodec;
11+
export 'src/dictionary.dart' show DejaJsonDictionary;
12+
export 'src/exceptions.dart';
13+
export 'src/interceptor.dart' show DejaJsonInterceptor;

0 commit comments

Comments
 (0)