ip-enrichment

Enriches IP addresses with threat intelligence and geolocation data from multiple providers. Tails your access log for configurable status codes, caches results in SQLite, and serves a HTTP API for dashboards and automation.

I build this, as I needed some extra stats in my Grafana dashboard for my WAF, maybe you can use it for that or something else.

What it does

When a request matches one of the configured status codes (default: 403), this service automatically enriches the source IP across multiple providers:

Provider	Data	API key?	Free tier limit	Cache TTL
GeoIP (MaxMind GeoLite2)	Country, city, coordinates, ASN, ISP	No (auto-downloads)	Unlimited (offline)	Always fresh
AbuseIPDB	Abuse score, reports, tor flag, usage type, ISP, domain	Yes	1,000/day (free)	3-30d by score
Shodan InternetDB	Open ports, CVEs, hostnames, tags	No	Unlimited	7d
Reverse DNS (PTR)	PTR hostname	No	Unlimited	30d

Additional providers (disabled by default):

Provider	Data	API key?	Free tier limit	Status
ipapi.is	VPN/proxy/datacenter detection	Optional	1,000/day	Planned
VirusTotal	Malware associations, communicating files	Yes	500/day	Planned
AlienVault OTX	Community threat intel, pulses, reputation	Yes	Unlimited	Planned

Quick start

docker run -d \
  -v /var/log/nginx:/var/log/nginx:ro \
  -v ip-enrichment-data:/data \
  -e ABUSEIPDB_API_KEY=your-key-here \
  -p 8090:8090 \
  ghcr.io/saturate/ip-enrichment

Configuration

TOML config at /etc/ip-enrichment/config.toml. All settings have sane defaults; the service runs without a config file.

[server]
port = 8090
log_path = "/var/log/nginx/access.log"
db_path = "/data/enrichment.db"
status_codes = [403]              # HTTP status codes that trigger enrichment
auth_key_env = "IP_ENRICHMENT_API_KEY"  # env var for bearer token auth; unset = no auth
max_concurrent_enrichments = 20   # cap on parallel enrichment tasks

[providers.geoip]
enabled = true
auto_update = true
data_dir = "/data/geoip"
update_interval = "7d"

[providers.abuseipdb]
enabled = true
api_key_env = "ABUSEIPDB_API_KEY"
daily_limit = 900
ttl_high = "30d"    # score 75-100
ttl_medium = "7d"   # score 25-74
ttl_low = "3d"      # score 0-24

[providers.shodan]
enabled = true
ttl = "7d"

[providers.ptr]
enabled = true
ttl = "30d"

[providers.ipapi]
enabled = false

[providers.virustotal]
enabled = false
api_key_env = "VIRUSTOTAL_API_KEY"
daily_limit = 450

[providers.alienvault]
enabled = false
api_key_env = "OTX_API_KEY"

API keys are read from environment variables (referenced by api_key_env), not from the config file. This keeps secrets in your orchestrator's secret management.

Authentication

Set the IP_ENRICHMENT_API_KEY env var to require a bearer token on API endpoints:

curl -H "Authorization: Bearer your-token" http://localhost:8090/api/top

/healthz and /metrics are always accessible without auth (for k8s probes and Prometheus scraping).

API

GET /api/lookup?ip=1.2.3.4

Returns the merged enrichment profile for an IP, grouped into geo, network, threat, and services. The service reconciles data from multiple providers (for example, picking the most specific geolocation per the configured geo_strategy). If the IP is not cached, it is enriched on the fly. Returns 400 Bad Request if the IP is not valid.

Fields inside geo, network, and threat are omitted when no provider supplied a value; services arrays are always present and may be empty.

{
  "ip": "1.2.3.4",
  "geo": {
    "country": "China",
    "country_code": "CN",
    "city": "Beijing",
    "region": "Beijing",
    "latitude": 39.9042,
    "longitude": 116.4074,
    "timezone": "Asia/Shanghai"
  },
  "network": {
    "asn_org": "Tencent Cloud",
    "isp": "Tencent Cloud Computing",
    "domain": "tencent.com",
    "ptr": "server01.example.com",
    "usage_type": "Data Center/Web Hosting/Transit",
    "anycast": false
  },
  "threat": {
    "abuse_score": 85,
    "abuse_reports": 42,
    "is_tor": false,
    "is_vpn": false,
    "is_proxy": false,
    "is_datacenter": true,
    "is_crawler": false
  },
  "services": {
    "ports": [22, 80, 443, 8080],
    "vulns": ["CVE-2021-44228"],
    "hostnames": ["example.com"],
    "tags": ["cloud"]
  }
}

GET /api/lookup/raw?ip=1.2.3.4

Returns the same data without cross-provider reconciliation: each provider's raw response under its own key (geoip, abuseipdb, shodan, ptr, ipapi, ipinfo). Use this when you want to see what each source reported rather than the merged view. Same caching, on-the-fly enrichment, and 400 behavior as /api/lookup.

{
  "ip": "1.2.3.4",
  "geoip": {
    "country": "China",
    "city": "Beijing",
    "latitude": 39.9042,
    "longitude": 116.4074,
    "asn_org": "Tencent Cloud"
  },
  "abuseipdb": {
    "score": 85,
    "reports": 42,
    "country_code": "CN",
    "usage_type": "Data Center/Web Hosting/Transit",
    "isp": "Tencent Cloud Computing",
    "domain": "tencent.com",
    "is_tor": false
  },
  "shodan": {
    "ports": [22, 80, 443, 8080],
    "vulns": ["CVE-2021-44228"],
    "hostnames": ["example.com"],
    "tags": ["cloud"]
  },
  "ptr": {
    "hostname": "server01.example.com"
  },
  "ipapi": {
    "is_vpn": false,
    "is_proxy": false,
    "is_datacenter": true,
    "is_crawler": false,
    "is_tor": false
  },
  "ipinfo": {
    "hostname": "server01.example.com",
    "country": "CN",
    "city": "Beijing",
    "region": "Beijing",
    "latitude": 39.9042,
    "longitude": 116.4074,
    "org": "AS45090 Tencent",
    "timezone": "Asia/Shanghai",
    "anycast": false
  }
}

GET /api/top?limit=15

Returns the top IPs by abuse score from the cache, each in the same shape as /api/lookup. limit defaults to 15 and is capped at 1000.

GET /api/stats

api_calls_today is keyed by provider. Per-provider daily limits are exposed through /metrics, not here.

{
  "total_ips": 847,
  "enriched": 623,
  "api_calls_today": {
    "abuseipdb": 47,
    "ipapi": 12
  }
}

GET /healthz

Health check endpoint. Returns 200 with {"ok": true} when SQLite is accessible, 503 otherwise.

GET /metrics

Prometheus metrics endpoint. Per-provider series carry a provider label.

ip_enrichment_ips_total 847
ip_enrichment_ips_enriched 623
ip_enrichment_provider_enabled{provider="geoip"} 1
ip_enrichment_provider_enabled{provider="abuseipdb"} 1
ip_enrichment_provider_enabled{provider="shodan"} 0
ip_enrichment_api_daily_limit{provider="abuseipdb"} 900
ip_enrichment_api_calls_today{provider="abuseipdb"} 47
ip_enrichment_tasks_in_flight 3
ip_enrichment_tasks_max 20

Cache strategy

Results are cached in SQLite with tiered TTLs. Bad IPs stay cached longer since they rarely rehabilitate:

AbuseIPDB score	Cache TTL	Rationale
75-100	30 days	Known bad, not changing soon
25-74	7 days	Moderate, recheck weekly
0-24	3 days	Low/clean, might get reported

Shodan data caches for 7 days. PTR records are looked up once and cached for 30 days. GeoIP is always fresh (local database).

The cache persists across restarts. Over time it builds a local threat intelligence database of every IP that triggers your configured status codes.

Kubernetes deployment

Can run standalone or as a sidecar alongside a reverse proxy / WAF:

containers:
  - name: nginx
    image: nginx:latest
    volumeMounts:
      - name: logs
        mountPath: /var/log/nginx
  - name: enrichment
    image: ghcr.io/saturate/ip-enrichment
    ports:
      - containerPort: 8090
    env:
      - name: ABUSEIPDB_API_KEY
        valueFrom:
          secretKeyRef:
            name: abuseipdb
            key: API_KEY
      - name: IP_ENRICHMENT_API_KEY
        valueFrom:
          secretKeyRef:
            name: ip-enrichment
            key: AUTH_TOKEN
    volumeMounts:
      - name: logs
        mountPath: /var/log/nginx
        readOnly: true
      - name: enrichment-data
        mountPath: /data
      - name: enrichment-config
        mountPath: /etc/ip-enrichment

Grafana integration

Use the Infinity datasource to query the API, or scrape the /metrics endpoint with Prometheus.

Infinity datasource:

• URL: http://ip-enrichment:8090/api/top?limit=15
• Type: JSON
• Format: Table

Prometheus scrape config:

scrape_configs:
  - job_name: ip-enrichment
    static_configs:
      - targets: ['ip-enrichment:8090']

Building from source

cargo build --release

The release binary is ~5MB with LTO and stripping enabled.

License

MIT