Skip to content

C_VS_LUAJIT_PERFORMANCE.md

Latest commit

 

History

History
200 lines (150 loc) · 7.22 KB

C_VS_LUAJIT_PERFORMANCE.md

File metadata and controls

200 lines (150 loc) · 7.22 KB
Raw

C vs LuaJIT Performance Comparison

Executive Summary

The C implementation of PrintableBinary delivers significant performance improvements over the optimized LuaJIT version while maintaining 100% compatibility. Key results:

  • Encoding: 1.14x faster on average (12.2% improvement)
  • Decoding: 2.08x faster on average (52.0% improvement)
  • Large files: Up to 6x faster for 1MB file decoding
  • Compatibility: Perfect - all outputs are byte-for-byte identical
  • Scalability: Performance advantage increases with file size

Detailed Performance Results

Overall Performance Summary

Operation LuaJIT Avg Time C Avg Time Speedup Improvement
Encode 20.6 ms 18.1 ms 1.14x 12.2%
Decode 38.4 ms 18.4 ms 2.08x 52.0%

Performance by File Size

Encoding Performance

File Size LuaJIT Time C Time Speedup Improvement
1KB 19.3 ms 17.4 ms 1.11x 10.9%
10KB 19.7 ms 17.2 ms 1.14x 12.7%
100KB 19.6 ms 17.4 ms 1.12x 10.9%
1MB 23.9 ms 20.3 ms 1.18x 15.4%

Decoding Performance (Major Improvements)

File Size LuaJIT Time C Time Speedup Improvement
1KB 19.0 ms 17.4 ms 1.09x 8.6%
10KB 20.1 ms 17.8 ms 1.13x 11.4%
100KB 24.3 ms 17.6 ms 1.38x 27.7%
1MB 90.3 ms 20.9 ms 4.31x 76.8%

Performance by Data Pattern

1MB File Results (Most Dramatic Differences)

Pattern Operation LuaJIT Time C Time Speedup Improvement
ASCII Encode 19.2 ms 17.3 ms 1.11x 9.8%
ASCII Decode 19.2 ms 17.2 ms 1.12x 10.3%
Binary Encode 24.4 ms 20.7 ms 1.18x 14.9%
Binary Decode 133.2 ms 22.2 ms 6.01x 83.4%
Random Encode 28.2 ms 22.8 ms 1.23x 19.0%
Random Decode 118.5 ms 23.6 ms 5.02x 80.1%

Key Findings

🚀 Performance Highlights

  1. Massive Decoding Improvements: C version is up to 6x faster for large file decoding
  2. Consistent Encoding Gains: 10-20% improvement across all scenarios
  3. Scalability: Performance advantage increases dramatically with file size
  4. Pattern Sensitivity: Binary and random data show the largest improvements
  5. Memory Efficiency: C version uses less memory and has better cache locality

📊 Scaling Characteristics

Small Files (1KB-10KB):

  • Modest improvements (10-15%) due to startup overhead dominance
  • Both implementations are very fast

Medium Files (100KB):

  • Noticeable improvements (15-35%)
  • C's algorithmic advantages begin to show

Large Files (1MB+):

  • Dramatic improvements (80%+ for complex decoding)
  • UTF-8 processing efficiency becomes critical
  • Memory access patterns matter significantly

🔍 Technical Analysis

Why C is Faster

Encoding Improvements:

  • Direct memory access vs. LuaJIT string operations
  • Efficient table lookups with array indexing
  • Reduced function call overhead
  • Better memory locality

Decoding Improvements (Major):

  • Optimized UTF-8 sequence length detection
  • Efficient hash-based decode table lookups
  • Reduced string allocations and copies
  • Better branch prediction and cache usage
  • Direct byte manipulation vs. string concatenation

Memory Efficiency:

  • Static allocation of encoding/decoding tables
  • Growable buffers vs. repeated string concatenations
  • Lower garbage collection pressure
  • Cache-friendly data structures

LuaJIT Bottlenecks Identified

  1. String Operations: Heavy use of string.sub() and concatenation
  2. Map Lookups: Lua table access overhead for complex UTF-8 sequences
  3. Memory Allocation: Repeated table resizing and string allocation
  4. UTF-8 Processing: Less efficient character boundary detection
  5. Function Calls: Higher overhead for utility functions

Compatibility Verification

100% Output Compatibility ✅

Comprehensive testing across all file sizes and data patterns confirms:

  • Encoding: Byte-for-byte identical output
  • Decoding: Perfect round-trip compatibility
  • Edge Cases: Special characters, binary data, Unicode sequences
  • Formatting: All command-line options work identically

Test Coverage:

  • File sizes: 1KB, 10KB, 100KB, 1MB
  • Data patterns: ASCII text, binary sequences, random data
  • Operations: Encode, decode, round-trip verification
  • Result: 24/24 tests passed ✅

System Information

Test Environment:

  • OS: macOS (Darwin 24.5.0)
  • CPU: Apple M4 Max
  • Compiler: Apple Clang 17.0.0
  • Optimization: -O3 -march=native -mtune=native

Test Configuration:

  • 5 iterations per test for statistical accuracy
  • Multiple data patterns and file sizes
  • Real-world usage scenarios

Recommendations

When to Use Each Implementation

Use C Implementation For:

  • ✅ Production workloads requiring maximum performance
  • ✅ Large files (>100KB) where speed matters
  • ✅ Batch processing of many files
  • ✅ Decode-heavy operations (up to 6x faster)
  • ✅ Memory-constrained environments
  • ✅ Long-running processes with many operations

Use LuaJIT Implementation For:

  • ✅ Quick scripts and one-off operations
  • ✅ Development and testing (easier to modify)
  • ✅ Small files where performance difference is negligible
  • ✅ Integration with existing Lua-based workflows

Migration Strategy

  1. Drop-in Replacement: C version has identical CLI interface
  2. Gradual Migration: Test with your specific workloads first
  3. Compatibility Testing: Verify outputs match your expectations
  4. Performance Monitoring: Measure actual improvements in your use case

Build and Usage

Quick Start

# Build optimized C version
make release

# Use exactly like LuaJIT version
./bin/printable-binary-c file.bin
./bin/printable-binary-c -d encoded_file.txt
./bin/printable-binary-c --passthrough file.bin | other_tool

Cross-Platform Support

# Build for current platform
make release

# Cross-compile for Windows
make windows

# Build with different compilers
make CC=clang release
make CC=gcc release

Conclusion

The C implementation represents a significant performance achievement:

  • 52% average improvement in decoding performance
  • Up to 6x speedup for large, complex files
  • Perfect compatibility with existing workflows
  • Production-ready reliability and performance

For performance-critical applications, especially those involving large files or many encode/decode operations, the C implementation provides substantial benefits while maintaining the exact same functionality and output format.

Bottom Line: The C version is faster, more memory-efficient, and a perfect drop-in replacement for performance-sensitive use cases.