Skip to content

ROADMAP.md

Latest commit

 

History

History
523 lines (418 loc) · 17.2 KB

ROADMAP.md

File metadata and controls

523 lines (418 loc) · 17.2 KB
Raw

Simple HTTP Daemon - Development Roadmap

Honesty note: Prefer project/PROGRESS_REPORT.md when phase checkmarks may be historical. Item-level tracking: project/ROADMAP_CHECKLIST.md. Overview: PROJECT_OVERVIEW.md. Operator manual: docs/README.md.

The daemon reads INI, YAML, or JSON. YAML blocks later in this file are a long-term sketch of nested enterprise knobs, not the parser target. Keys the binary actually accepts: docs/configuration.md.

Overview

The Simple HTTP Daemon (simple-httpd) is a lightweight, high-performance HTTP server implementation designed for modern systems. This roadmap outlines the development phases and milestones for creating a production-ready HTTP daemon.

Project Goals

  • Performance: High-throughput HTTP server with minimal resource usage
  • Security: Modern HTTP security features (HTTPS, HSTS, CSP)
  • Compatibility: Full HTTP/1.1 and HTTP/2 compliance
  • Simplicity: Easy configuration and deployment
  • Reliability: Robust error handling and logging

Development Phases

Phase 1: Foundation

Status: ✅ Completed (scaffold + 0.2.0 source tree) Timeline: Initial implementation

Core Infrastructure

  • Project structure and build system
  • CMake configuration with static linking support
  • Cross-platform build scripts (Linux, macOS, Windows)
  • CI/CD pipeline setup
  • Basic daemon framework
  • Configuration management system
  • Logging infrastructure
  • Signal handling and graceful shutdown

Development Tools

  • Standardized Makefile
  • Deployment configurations (systemd, launchd, Windows)
  • Docker containerization
  • Package generation (DEB, RPM, DMG, MSI)

Phase 2: Core HTTP Protocol Implementation

Status: ✅ Completed in 0.5.0 (HTTP/3 deferred) Timeline: 4-6 weeks

HTTP Protocol Stack

  • HTTP request parsing
  • HTTP response generation
  • HTTP/1.1 protocol support
  • HTTP/2 protocol support
  • HTTP/3 protocol support (future)
  • Protocol negotiation and version selection

Network Layer

  • TCP connection handling
  • Connection pooling and management
  • Request/response queuing
  • Timeout and retry mechanisms
  • Connection multiplexing
  • Keep-alive support

Security Features

  • HTTPS support (TLS/SSL)
  • Certificate management
  • HTTP Strict Transport Security (HSTS)
  • Content Security Policy (CSP)
  • HTTP authentication
  • Rate limiting

Phase 3: Web Server Features

Status: ✅ Completed in 0.6.0 Timeline: 6-8 weeks

Core Web Features

  • Static file serving
  • Directory browsing
  • MIME type handling
  • Content compression (gzip, deflate)
  • Range requests support
  • Conditional requests (ETag, Last-Modified)

Advanced Features

  • Virtual hosting
  • URL rewriting
  • Reverse proxy support (HTTP origins; HTTPS backends not implemented)
  • Load balancing (round-robin + connect failover)
  • Caching mechanisms (in-memory file-body cache)
  • WebSocket support (HTTP/1.1 Upgrade tunnel only)

Content Management

  • CGI support
  • FastCGI support (TCP client; no process manager)
  • Server-side includes (SSI)
  • Template processing (SSI; no separate engine)
  • Dynamic content generation (CGI / FastCGI / SSI)
  • API endpoint handling (/healthz, /metrics)

Phase 4: Performance & Monitoring

Status: ✅ Completed in 0.7.0 Timeline: 8-10 weeks

Performance Optimization

  • Connection pooling optimization (worker pool metrics; reject at capacity; TCP_NODELAY)
  • Memory management optimization (thread-local stream buffers; file-cache LRU)
  • I/O optimization (sendfile on Linux/macOS for cleartext; userspace fallback)
  • Request batching (HTTP/1.1 keep-alive reuses a worker connection)
  • Response caching (in-memory file-body cache with hit/miss metrics)
  • Load balancing (reverse-proxy round-robin from 0.6.0)

Monitoring & Management

  • Performance metrics collection
  • Health monitoring
  • Configuration hot-reloading (SIGHUP soft reload; listen/TLS/workers need restart)
  • Remote management interface (read-only JSON /status)
  • SNMP integration (deferred — Prometheus is the ops path)
  • Prometheus metrics export

High Availability

  • Clustering support (docs: multi-instance behind an external load balancer)
  • Failover mechanisms (proxy connect failover; process failover via supervisor / LB)
  • Data replication (N/A — daemon is stateless; sync the document root externally)
  • Backup and restore (operator procedures in docs/operations.md)
  • Disaster recovery (rebuild from config + TLS + document root backups)

Phase 5: Enterprise Features

Status: ✅ Completed in 0.8.0 (honest static-file scope) Timeline: shipped

Advanced Security

  • Advanced TLS features (TLS 1.3 preference / floor, ECDHE/PFS ciphers; OCSP stapling deferred)
  • Security headers
  • Request filtering
  • DDoS protection (request + connection rate limits, per-IP connection caps, IP allow/deny)
  • Security auditing (security_log)
  • Compliance reporting (operator checklist in docs/security.md)

Integration & APIs

  • REST API for management (GET {admin_path}/config, POST {admin_path}/reload, Basic auth)
  • GraphQL API for queries — out of scope for a static-file daemon
  • WebSocket support (HTTP/1.1 Upgrade tunnel via reverse proxy; shipped in 0.6.0)
  • Plugin architecture — out of scope
  • Third-party integrations — out of scope
  • Cloud storage backends — out of scope (serve from a mounted volume / CDN instead)

Scalability

  • Horizontal scaling (stateless multi-instance; docs)
  • Load balancing (proxy round-robin; external LB / k8s Service)
  • Distributed serving — N/A (no cluster protocol)
  • Cloud deployment (container + ops docs)
  • Container orchestration (Kubernetes sketch in docs/operations.md)
  • Microservices architecture — N/A (single-purpose static daemon)

Also in 0.8.0: Unix daemonize, user/group privilege drop after bind, pid_file.

Technical Specifications

Supported Protocols

  • HTTP/1.1: RFC 7230-7237
  • HTTP/2: RFC 7540
  • HTTP/3: RFC 9114 (future)
  • HTTPS: TLS 1.2, TLS 1.3
  • WebSocket: RFC 6455

Supported Features

  • Static File Serving: HTML, CSS, JS, images, etc.
  • Dynamic Content: CGI, FastCGI, SSI
  • Security: HTTPS, HSTS, CSP, authentication
  • Performance: Compression, caching, keep-alive
  • Management: Virtual hosts, URL rewriting, reverse proxy

Supported Platforms

  • Linux: Ubuntu, CentOS, RHEL, Debian, SUSE
  • macOS: 10.15+ (Catalina and later)
  • Windows: Windows 10/11, Windows Server 2016+

Performance Targets

  • Requests per Second: 100,000+ RPS
  • Concurrent Connections: 50,000+
  • Latency: <1ms for static content
  • Memory Usage: <50MB base + 1KB per connection
  • CPU Usage: <5% under normal load

Configuration

The running daemon reads INI, YAML, and JSON with flat keys (see docs/configuration.md). Reverse proxy, FastCGI, CGI, and SSI are real keys. The nested blocks below remain a wishlist of enterprise knobs the parser does not accept (unix sockets, per-location PHP, HTTPS origins, cache disk, …).

Basic Configuration (sketch)

# simple-httpd.conf
server:
  listen:
    - "0.0.0.0:80"
    - "[::]:80"
    - "0.0.0.0:443"
    - "[::]:443"
  
  document_root: "/var/www/html"
  index_files: ["index.html", "index.htm"]
  
  virtual_hosts:
    - name: "example.com"
      document_root: "/var/www/example.com"
      ssl_cert: "/etc/ssl/certs/example.com.crt"
      ssl_key: "/etc/ssl/private/example.com.key"
    
    - name: "www.example.com"
      document_root: "/var/www/example.com"
      ssl_cert: "/etc/ssl/certs/example.com.crt"
      ssl_key: "/etc/ssl/private/example.com.key"

  security:
    ssl_protocols: ["TLSv1.2", "TLSv1.3"]
    ssl_ciphers: "ECDHE+AESGCM:ECDHE+CHACHA20:DHE+AESGCM:DHE+CHACHA20:!aNULL:!MD5:!DSS"
    hsts: true
    hsts_max_age: 31536000
    
  performance:
    keep_alive: true
    keep_alive_timeout: 65
    max_connections: 1000
    compression: true
    compression_types: ["text/html", "text/css", "text/javascript", "application/javascript"]

Advanced Configuration (sketch)

# Advanced configuration
server:
  listen:
    - "0.0.0.0:80"
    - "[::]:80"
    - "0.0.0.0:443"
    - "[::]:443"
    - "0.0.0.0:8080"  # HTTP/2
    - "[::]:8080"
  
  document_root: "/var/www/html"
  index_files: ["index.html", "index.htm", "index.php"]
  
  virtual_hosts:
    - name: "example.com"
      document_root: "/var/www/example.com"
      ssl_cert: "/etc/ssl/certs/example.com.crt"
      ssl_key: "/etc/ssl/private/example.com.key"
      ssl_protocols: ["TLSv1.2", "TLSv1.3"]
      ssl_ciphers: "ECDHE+AESGCM:ECDHE+CHACHA20"
      hsts: true
      hsts_max_age: 31536000
      csp: "default-src 'self'; script-src 'self' 'unsafe-inline'"
      
      locations:
        - path: "/api"
          proxy_pass: "http://backend:8080"
          proxy_set_header: ["Host $host", "X-Real-IP $remote_addr"]
        
        - path: "/static"
          alias: "/var/www/static"
          expires: "1y"
          add_header: ["Cache-Control public"]
        
        - path: "/admin"
          auth_basic: "Admin Area"
          auth_basic_user_file: "/etc/httpd/htpasswd"
          allow: ["192.168.1.0/24"]
          deny: ["all"]
    
    - name: "api.example.com"
      document_root: "/var/www/api"
      ssl_cert: "/etc/ssl/certs/api.example.com.crt"
      ssl_key: "/etc/ssl/private/api.example.com.key"
      
      locations:
        - path: "/"
          try_files: ["$uri", "$uri/", "/index.php"]
          fastcgi_pass: "unix:/var/run/php-fpm.sock"
          fastcgi_index: "index.php"
          fastcgi_param: ["SCRIPT_FILENAME $document_root$fastcgi_script_name"]

  security:
    ssl_protocols: ["TLSv1.2", "TLSv1.3"]
    ssl_ciphers: "ECDHE+AESGCM:ECDHE+CHACHA20:DHE+AESGCM:DHE+CHACHA20:!aNULL:!MD5:!DSS"
    ssl_prefer_server_ciphers: true
    ssl_session_cache: "shared:SSL:10m"
    ssl_session_timeout: "10m"
    
    hsts: true
    hsts_max_age: 31536000
    hsts_include_subdomains: true
    
    security_headers:
      - "X-Frame-Options: DENY"
      - "X-Content-Type-Options: nosniff"
      - "X-XSS-Protection: 1; mode=block"
      - "Referrer-Policy: strict-origin-when-cross-origin"
    
    rate_limiting:
      enabled: true
      requests_per_minute: 1000
      burst_size: 100
      zone_size: "10m"
    
    access_control:
      - "192.168.0.0/16"
      - "10.0.0.0/8"
      - "172.16.0.0/12"
    
    deny_ips:
      - "192.168.1.100"
      - "10.0.0.50"
    
    allow_methods: ["GET", "POST", "PUT", "DELETE", "HEAD", "OPTIONS"]
    deny_methods: ["TRACE", "CONNECT"]
    
  performance:
    keep_alive: true
    keep_alive_timeout: 65
    keep_alive_requests: 1000
    
    max_connections: 10000
    max_requests_per_connection: 1000
    connection_timeout: 60
    request_timeout: 30
    
    compression: true
    compression_types: ["text/html", "text/css", "text/javascript", "application/javascript", "application/json", "text/xml", "application/xml"]
    compression_min_length: 1024
    compression_level: 6
    
    caching:
      enabled: true
      cache_size: "100m"
      cache_path: "/var/cache/httpd"
      cache_max_age: 3600
      cache_public: true
    
    sendfile: true
    tcp_nopush: true
    tcp_nodelay: true
    
    worker_processes: "auto"
    worker_connections: 1024
    multi_accept: true
    use: "epoll"  # Linux only
    
  logging:
    level: "info"
    access_log: "/var/log/httpd/access.log"
    error_log: "/var/log/httpd/error.log"
    log_format: "combined"
    log_rotation: true
    log_rotation_size: "100M"
    log_rotation_keep: 10
    
    custom_logs:
      - name: "security"
        file: "/var/log/httpd/security.log"
        format: "security"
        level: "warn"
      
      - name: "performance"
        file: "/var/log/httpd/performance.log"
        format: "performance"
        level: "info"
    
    metrics:
      enabled: true
      endpoint: "/metrics"
      format: "prometheus"
      interval: 60

Testing Strategy

Unit Testing

  • HTTP protocol implementation testing
  • Request/response handling testing
  • Security feature testing
  • Configuration parsing testing

Integration Testing

  • Cross-platform compatibility testing
  • Protocol compatibility testing
  • Performance benchmarking
  • Security testing

Load Testing

  • High request rate testing
  • Concurrent connection testing
  • Memory usage testing
  • Stress testing

Documentation

Canonical manuals: docs/README.md.

User Documentation

  • Installation guide
  • Configuration reference
  • Troubleshooting guide
  • Performance tuning guide
  • Security best practices

Developer Documentation

  • API documentation (Doxygen not published)
  • Architecture overview
  • Contributing guidelines
  • Code style guide (CONTRIBUTING.md + neighboring files)
  • Testing guidelines

Operations Documentation

  • Deployment guide
  • Monitoring setup (/healthz, /metrics)
  • Backup procedures
  • Disaster recovery
  • Maintenance procedures (restart to reload; logrotate)

Release Schedule

Version 0.1.0 (Scaffold)

  • Build, packaging, and documentation templates
  • Shipped: repository scaffold (no server binary)

Version 0.2.0 (Static files)

  • HTTP/1.1 GET / HEAD / OPTIONS
  • Static file serving from a document root
  • Shipped: August 2026

Version 0.3.0 (HTTPS and vhosts)

  • TLS 1.2+, virtual hosts, gzip, Cache-Control
  • Shipped: August 2026

Version 0.4.0 (Security and ops)

  • CSP, security headers, HTTP Basic, rate limits, URL rewrite, /healthz, Prometheus /metrics
  • Shipped: August 2026

Version 0.5.0 (HTTP/2)

  • HTTP/2 (nghttp2), TLS ALPN, h2c prior knowledge, worker pool, request bodies
  • Shipped: August 2026

Version 0.6.0 (Web server features)

  • Reverse proxy (HTTP), round-robin, file cache, WebSocket tunnel, CGI, FastCGI, SSI
  • Shipped: August 2026

Version 0.7.0 (Performance & monitoring)

  • Richer Prometheus metrics, /status JSON, SIGHUP soft reload, TCP_NODELAY, sendfile
  • Shipped: August 2026

Version 0.8.0 (Enterprise / ops hardening)

  • Advanced TLS (1.3 floor option, PFS ciphers), IP allow/deny, connection limits, security audit log
  • Auth-gated admin config/reload API, privilege drop, daemonize, pid file, k8s/HA docs
  • Shipped: August 2026

1.0.0 cut — production-usable static HTTP daemon

Milestones through 0.8.0 are released. 1.0.0 is a hygiene and contract cut on that series, not HTTP/3 and not an nginx replacement.

Contract (README / docs / CHANGELOG)

  • HTTP/1.1 and HTTP/2 static files; optional TLS; vhosts; Basic auth; rate / connection limits; CIDR allow/deny; /healthz / /metrics / /status; SIGHUP soft reload; privilege drop
  • Opt-in extras stay optional: reverse proxy, CGI, FastCGI, SSI, WebSocket Upgrade tunnel
  • One process, one host; run under systemd / launchd / a Windows service (--daemonize is optional; prefer the OS supervisor)
  • Known limits that stay: plaintext Basic htpasswd; no HTTP/3; no HTTPS origin proxy; FastCGI has no process manager; worker pool is not an event loop
  • Drop “early development” / “scaffold” packaging language; refresh SimpleDaemons overview blurbs

Packaging (must match production templates)

  • Create /var/lib/simple-httpd and /var/log/simple-httpd (or platform equivalents) with the service user
  • systemd / launchd / Windows unit: ExecStart path and --foreground --config must match the installed binary and /etc/simple-httpd/simple-httpd.conf

Optional polish (shipped with 1.0)

  • Production profile defaults for max_connections_per_ip, tls_min_version, and security_log
  • A CI workflow that builds and runs ctest

Do not pull into 1.0

Items under Later and Out of scope. A 1.1+ can add them without retconning 1.0.

Later (not scheduled)

Version 0.9.0-class / 1.1+ (optional protocols)

  • HTTP/3 support (optional)
  • Remaining wishlist items only where they fit a static-file daemon

Out of scope (1.0)

  • Replacing nginx / Apache / Caddy as a full application gateway
  • GraphQL, plugins, and S3-backed document roots
  • --daemonize fork as the primary lifecycle (use the OS supervisor)

Contributing

Getting Started

See CONTRIBUTING.md.

  1. Fork the repository
  2. Create a feature branch
  3. Make your changes
  4. Add tests and docs
  5. Submit a pull request

Development Setup

git clone https://github.com/SimpleDaemons/simple-httpd.git
cd simple-httpd
make build
make test

Code Style

  • Follow the existing code style
  • Use meaningful variable names
  • Add comments for complex logic
  • Write unit tests for new features

License

This project is licensed under the Apache License 2.0 - see the LICENSE file for details.

Contact