Skip to content

Commit 14d4722

Browse files
committed
feat: Add zero-copy decoding examples and content-aware routing
- Introduced `zero_copy_usage.rs` example demonstrating zero-copy vs owned decoding. - Added `zerocopy_similarity.rs` example showcasing zero-copy embedding similarity computations. - Implemented `EmbeddingView` for efficient zero-copy access to embeddings with SIMD-optimized similarity functions. - Created `content_routing.rs` example for content-aware routing decisions using field inspection. - Developed `ContentAwarePolicy` and `ContentRule` for flexible routing based on record content. - Enhanced performance benchmarks for zero-copy routing and similarity computations.
1 parent d78946f commit 14d4722

42 files changed

Lines changed: 7518 additions & 393 deletions

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

CHANGELOG.md

Lines changed: 52 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -100,8 +100,59 @@ lnmp-envelope = "0.5.6"
100100

101101
## [Unreleased]
102102

103-
## [0.5.15] - 2025-12-17
103+
## [0.5.15] - 2025-12-19
104104

105+
### Added
106+
107+
- **NEW: `EmbeddingView` in `lnmp-embedding`** - Quasi-zero-copy embedding access
108+
- `EmbeddingView::from_bytes()` - Parse embedding from raw bytes
109+
- `EmbeddingView::as_f32_vec()` - Safe copy to Vec<f32> (recommended)
110+
- `EmbeddingView::as_f32_slice()` - Experimental zero-copy (alignment-dependent)
111+
- `cosine_similarity()`, `dot_product()`, `euclidean_distance()` - SIMD-optimized
112+
- AVX2 optimization for x86_64 with scalar fallback
113+
- Works seamlessly with `decode_view()` for high-performance embedding pipelines
114+
115+
- **NEW: Content-Aware Routing in `lnmp-net`** - Zero-copy field inspection
116+
- `ContentRouter` with rule-based routing decisions
117+
- `RoutingRule` enum: HeaderContains, FieldEquals, HasField, EmbeddingDim
118+
- `RoutingPriority` levels: Critical, High, Normal, Low, Background
119+
- Zero-copy field access from `LnmpRecordView` for routing decisions
120+
121+
- **Enhanced Examples**
122+
- `zerocopy_similarity.rs` - Complete embedding similarity pipeline demo
123+
- `content_routing.rs` - Content-based routing with benchmarks
124+
- `zero_copy_routing.rs` - Routing performance comparison
125+
126+
### Changed
127+
128+
- **`bytemuck` dependency** added to `lnmp-embedding` (optional, via `zerocopy` feature)
129+
- All similarity methods now use safe `as_f32_vec()` internally for reliability
130+
131+
### Technical Notes
132+
133+
- **Memory Alignment**: True zero-copy f32 slice access requires aligned memory pointers.
134+
Since arbitrary buffer slices may not be aligned, `as_f32_vec()` (safe copy) is recommended.
135+
The experimental `as_f32_slice()` may panic on unaligned buffers.
136+
137+
- **Performance**: For 256-dim embeddings, `as_f32_vec()` adds ~1.4μs overhead for allocation.
138+
This is negligible for most use cases and guarantees reliability.
139+
140+
### Examples
141+
142+
```rust
143+
use lnmp_codec::binary::BinaryDecoder;
144+
use lnmp_embedding::EmbeddingView;
145+
146+
let decoder = BinaryDecoder::new();
147+
let view = decoder.decode_view(&bytes)?;
148+
149+
if let Some(field) = view.get_field(512) {
150+
if let LnmpValueView::Embedding(emb_bytes) = &field.value {
151+
let emb_view = EmbeddingView::from_bytes(emb_bytes)?;
152+
let similarity = emb_view.cosine_similarity(&other_view)?;
153+
}
154+
}
155+
```
105156

106157
## [0.5.14] - 2025-12-17
107158

Cargo.lock

Lines changed: 10 additions & 0 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

README.md

Lines changed: 14 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -680,15 +680,24 @@ cd tests/compliance/cpp && make test
680680

681681
## Performance
682682

683-
- Zero-copy where possible
684-
- No regex dependencies
685-
- Hand-written lexer for optimal performance
683+
**Zero-Copy Decoding (v0.5.15):**
684+
685+
| Payload Size | Standard | Zero-Copy | Speedup |
686+
|--------------|----------|-----------|---------|
687+
| Small (3 fields) | 248 ns | 92 ns | **2.70x** |
688+
| Medium (7 fields) | 610 ns | 164 ns | **3.71x** |
689+
| Large (embeddings) | 18.3 μs | 2.1 μs | **8.91x** |
690+
691+
**Throughput:**
692+
- Large records: **1.04 GiB/s** (zero-copy) vs 0.12 GiB/s (standard)
693+
- Batch processing: **3x faster** (1000 records in 55 μs)
694+
695+
**Other Performance Metrics:**
696+
- No regex dependencies, hand-written lexer
686697
- Minimal allocations during parsing
687698
- SC32 checksum computation: <1μs per field
688699
- Nested parsing: <10μs for 3-level nesting
689700
- Token reduction: 7-12× vs JSON (ShortForm mode)
690-
- Binary nested encoding: <2μs per field (v0.5)
691-
- Streaming overhead: <5% vs non-streaming (v0.5)
692701
- Delta encoding savings: >50% for typical updates (v0.5)
693702

694703
## Roadmap

crates/lnmp-codec/Cargo.toml

Lines changed: 7 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -23,10 +23,16 @@ lnmp-sfe = { workspace = true }
2323
log = { version = "0.4", optional = true }
2424

2525
[dev-dependencies]
26-
criterion = { version = "0.5", features = ["html_reports"] }
26+
criterion = "0.5"
27+
tokio = { version = "1.0", features = ["full"] }
28+
bytemuck = "1.16"
2729
# property tests
2830
proptest = "1.2"
2931

32+
[[bench]]
33+
name = "zero_copy_bench"
34+
harness = false
35+
3036
[[bench]]
3137
name = "v05_performance"
3238
harness = false

crates/lnmp-codec/README.md

Lines changed: 182 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -468,6 +468,188 @@ let decoder = BinaryDecoder::with_config(decoder_config);
468468
- **Decoding Speed**: < 1μs per field for simple types
469469
- **Round-trip**: < 10μs for typical 10-field record
470470

471+
### Zero-Copy Decoding (v0.5)
472+
473+
The `decode_view()` API enables zero-copy parsing for **high-throughput** scenarios like routing, filtering, and logging. Instead of allocating owned values, it borrows directly from the input buffer.
474+
475+
#### When to Use
476+
477+
| Use Case | API | Reason |
478+
|----------|-----|--------|
479+
| **Routing/Filtering** | `decode_view()` | Decision based on field values without full parse |
480+
| **Logging/Monitoring** | `decode_view()` | Extract trace ID, timestamps without allocation |
481+
| **Proxying/Forwarding** | `decode_view()` | Inspect headers, forward payload unchanged |
482+
| **Processing/Storage** | `decode()` | Need to mutate, persist, or own the data |
483+
484+
#### Performance Comparison
485+
486+
Based on `cargo bench --bench zero_copy_bench` (v0.5.15):
487+
488+
| Payload Type | `decode()` | `decode_view()` | Speedup |
489+
|--------------|------------|-----------------|---------|
490+
| **Small** (3 fields, ~22 bytes) | ~248 ns | ~92 ns | **2.70x** |
491+
| **Medium** (strings + ints, ~100 bytes) | ~610 ns | ~164 ns | **3.71x** |
492+
| **Large** (10KB string + embedding) | ~18.3 μs | ~2.1 μs | **8.91x** |
493+
494+
**Throughput Comparison:**
495+
496+
| Scenario | Standard | Zero-Copy | Improvement |
497+
|----------|----------|-----------|-------------|
498+
| Small records | 0.09 GiB/s | 0.24 GiB/s | **+170%** |
499+
| Medium records | 0.16 GiB/s | 0.59 GiB/s | **+271%** |
500+
| Large records | 0.12 GiB/s | 1.04 GiB/s | **+791%** |
501+
502+
**Batch Processing (1000 records):**
503+
- Standard decode: 165 μs
504+
- Zero-copy view: 55 μs
505+
- **3x faster batch throughput**
506+
507+
#### Basic Usage
508+
509+
```rust
510+
use lnmp_codec::binary::BinaryDecoder;
511+
use lnmp_core::LnmpValueView;
512+
513+
let decoder = BinaryDecoder::new();
514+
let bytes = vec![...]; // From network/file
515+
516+
// Create zero-copy view (borrows from 'bytes')
517+
let view = decoder.decode_view(&bytes).unwrap();
518+
519+
// Access fields without allocation
520+
for field in view.fields() {
521+
match &field.value {
522+
LnmpValueView::String(s) => {
523+
println!("String: {}", s); // s is &str (borrowed!)
524+
}
525+
LnmpValueView::Embedding(raw) => {
526+
println!("Embedding: {} bytes", raw.len()); // raw is &[u8]
527+
}
528+
_ => {}
529+
}
530+
}
531+
```
532+
533+
#### Example: High-Throughput Router
534+
535+
```rust
536+
use lnmp_codec::binary::BinaryDecoder;
537+
use lnmp_core::LnmpValueView;
538+
539+
let decoder = BinaryDecoder::new();
540+
541+
// Process 1M+ messages/sec without allocation
542+
for payload in incoming_messages {
543+
let view = decoder.decode_view(&payload)?;
544+
545+
// Zero-copy field inspection
546+
if let Some(field) = view.get_field(50) { // F50: status
547+
match &field.value {
548+
LnmpValueView::String(status) if *status == "critical" => {
549+
route_to_llm(&payload)?;
550+
}
551+
_ => route_locally(&payload)?
552+
}
553+
}
554+
}
555+
```
556+
557+
#### Example: Trace ID Extraction
558+
559+
```rust
560+
use lnmp_codec::binary::BinaryDecoder;
561+
use lnmp_core::LnmpValueView;
562+
563+
fn extract_trace_id(bytes: &[u8]) -> Option<&str> {
564+
let decoder = BinaryDecoder::new();
565+
let view = decoder.decode_view(bytes).ok()?;
566+
567+
// Zero-copy trace ID access (F80)
568+
if let Some(field) = view.get_field(80) {
569+
if let LnmpValueView::String(trace_id) = &field.value {
570+
return Some(trace_id); // Returns &str (zero-copy!)
571+
}
572+
}
573+
None
574+
}
575+
576+
// Usage in HTTP middleware
577+
let trace_id = extract_trace_id(&request_body)?;
578+
println!("Trace-ID: {}", trace_id); // No allocation
579+
```
580+
581+
#### Zero-Copy Limitations
582+
583+
| Type | Zero-Copy | Notes |
584+
|------|-----------|-------|
585+
| **String** | ✅ Full | Returns `&str` borrowed from input |
586+
| **StringArray** | ✅ Full | Returns `Vec<&str>` (only refs allocated) |
587+
| **Embedding** | ✅ Lazy | Returns `&[u8]` raw bytes (parse on demand) |
588+
| **Int/Float/Bool** | ✅ Natural | Scalars copied (4-8 bytes, negligible) |
589+
| **IntArray** | ❌ Allocates | VarInt encoding requires parse → `Vec<i64>` |
590+
| **FloatArray** | ❌ Allocates | VarInt length + values → `Vec<f64>` |
591+
| **BoolArray** | ❌ Allocates | Byte-per-bool → `Vec<bool>` |
592+
| **Nested** | ⚠️ Partial | Currently allocates (future: zero-copy traversal) |
593+
594+
**Why IntArray allocates:**
595+
VarInt encoding stores integers as variable-length sequences. To access `[1, 2, 3]`, the decoder must parse each VarInt, which requires allocation. Future versions will support packed (fixed-width) arrays for zero-copy access.
596+
597+
#### Advanced: Content-Based Routing
598+
599+
See [`examples/zero_copy_routing.rs`](./examples/zero_copy_routing.rs) for a complete routing example with benchmarks.
600+
601+
```bash
602+
cargo run -p lnmp-codec --example zero_copy_routing
603+
```
604+
605+
**Expected Output:**
606+
```
607+
=== Zero-Copy Routing Demo ===
608+
Critical message → ROUTE_TO_LLM
609+
Normal message → ROUTE_LOCALLY
610+
611+
=== Performance (100k iterations) ===
612+
Standard decode: 210ms (2.10 μs/iter)
613+
Zero-copy view: 90ms (0.90 μs/iter)
614+
Speedup: 2.33x faster
615+
```
616+
617+
#### Migration Guide
618+
619+
**Before (Standard decode):**
620+
```rust
621+
let record = decoder.decode(&bytes)?;
622+
if let Some(field) = record.get_field(50) {
623+
match &field.value {
624+
LnmpValue::String(s) => process(s), // s is String (owned)
625+
_ => {}
626+
}
627+
}
628+
```
629+
630+
**After (Zero-copy view):**
631+
```rust
632+
let view = decoder.decode_view(&bytes)?;
633+
if let Some(field) = view.get_field(50) {
634+
match &field.value {
635+
LnmpValueView::String(s) => process(s), // s is &str (borrowed)
636+
_ => {}
637+
}
638+
}
639+
```
640+
641+
**Key Difference:**
642+
- `LnmpValue::String(String)``LnmpValueView::String(&str)`
643+
- Must convert to owned if needed: `s.to_string()`
644+
645+
#### Future Enhancements (v0.6+)
646+
647+
- **Sparse array encoding:** Compressed storage for sparse vectors
648+
- **Nested zero-copy:** Traverse nested structures without allocation
649+
- **SIMD similarity:** Direct cosine similarity on embedding views
650+
651+
**Use `decode_view()` for maximum throughput. Use `decode()` when you need to own/mutate the data.**
652+
471653
## v0.5.14 Features
472654

473655
### Dynamic FID Discovery Protocol

0 commit comments

Comments
 (0)