11# ci4-request-analysis
22
33CodeIgniter 4 library that intercepts incoming HTTP requests, extracts structured
4- metadata (headers, body, files, tenant, source IP, etc.), and asynchronously sends a
5- JSON log to a dedicated Analysis Server — with local file-based queuing, retry,
6- sensitive-field redaction, body truncation, and IP whitelisting.
4+ metadata (headers, body, files, tenant, source IP, etc.), and writes one JSON
5+ Line per request directly to the CI4 ` writable/logs ` directory — with
6+ sensitive-field redaction, body truncation, IP whitelisting, and daily log
7+ rotation with gzip compression.
78
8- > Client side only. The Analysis Server (Docker: PHP-FPM + Nginx on Alpine, log
9- > rotation, Wazuh consumption) is delivered in the ` server/ ` directory of this
10- > repository.
9+ > Local logging only. No external server, no queue, no Guzzle.
1110
1211## Features
1312
1413- Attachable per-route (manual), not global.
15- - Non-blocking delivery: log is spooled locally, then sent by a background
16- ` analysis:send ` command (triggered by the filter and/or cron).
17- - Local file-based queue with max-size enforcement (oldest dropped) and
18- exponential retry; items are moved to ` failed/ ` after ` max_retries ` .
19- - Multi-tenant identification via subdomain ( ` HTTP_HOST ` ) .
14+ - Writes directly to ` writable/logs/analysis. log` as JSON Lines (JSONL).
15+ - Daily rotation: a file from a previous day is renamed and gzip-compressed
16+ automatically on the next write.
17+ - Retention pruning: compressed logs older than ` REQUEST_LOG_RETENTION_DAYS `
18+ (default 30) are deleted .
2019- Configurable sensitive-field redaction (default: ` password ` , ` nik ` , ` Api-Key ` , ` no_telp ` ).
2120- Raw body truncation at 3 MB (configurable) with ` ... [truncated] ` suffix.
2221- File upload metadata captured without binary content (name, size, MIME,
2322 extension, SHA-256 hash, double-extension detection).
2423- IP/CIDR whitelist to skip private/internal traffic.
25- - API-key authentication on the Analysis Server .
24+ - Multi-tenant identification via subdomain ( ` HTTP_HOST ` ) .
2625- PHP 7.4 and PHP 8.4+.
2726
2827## Requirements
2928
30- - PHP 7.4+ / 8.4+ (with ` exec ` enabled)
29+ - PHP 7.4+ / 8.4+
3130- CodeIgniter 4 (>= 4.3)
32- - Guzzle 7
33- - A writable local queue directory
31+ - A writable ` writable/logs ` directory
3432
3533## Installation
3634
@@ -44,34 +42,28 @@ Set the following environment variables in your CI4 application `.env`
4442(or system environment):
4543
4644``` ini
47- ANALYSIS_ENABLED = true
48- ANALYSIS_SERVER_URL = " http://host.docker.internal:8081/analyze"
49- ANALYSIS_API_KEY = " your-shared-api-key"
50- ANALYSIS_TIMEOUT = 2
51- ANALYSIS_MAX_QUEUE = 10000
52- ANALYSIS_QUEUE_DIR = " /tmp/analysis_queue"
53- ANALYSIS_REDACT_FIELDS = " password,nik,Api-Key,no_telp"
54- ANALYSIS_MAX_BODY_SIZE = 3145728
55- ANALYSIS_WHITELIST_IPS = " 10.0.0.0/8,172.16.0.0/12,192.168.0.0/16,127.0.0.1"
56- ANALYSIS_TRUNCATE_SUFFIX = " ... [truncated]"
57- ANALYSIS_MAX_RETRIES = 5
45+ REQUEST_LOG_ENABLED = true
46+ REQUEST_LOG_DIR = " writable/logs"
47+ REQUEST_LOG_FILE = " analysis.log"
48+ REQUEST_LOG_REDACT_FIELDS = " password,nik,Api-Key,no_telp"
49+ REQUEST_LOG_MAX_BODY_SIZE = 3145728
50+ REQUEST_LOG_WHITELIST_IPS = " 10.0.0.0/8,172.16.0.0/12,192.168.0.0/16,127.0.0.1"
51+ REQUEST_LOG_TRUNCATE_SUFFIX = " ... [truncated]"
52+ REQUEST_LOG_RETENTION_DAYS = 30
5853```
5954
6055### Config reference
6156
6257| Variable | Default | Description |
6358| ---| ---| ---|
64- | ` ANALYSIS_ENABLED ` | ` false ` | Master switch for the filter. |
65- | ` ANALYSIS_SERVER_URL ` | ` '' ` | Analysis Server endpoint (POST ` /analyze ` ). |
66- | ` ANALYSIS_API_KEY ` | ` '' ` | Shared static API key sent in the ` X-API-Key ` header. |
67- | ` ANALYSIS_TIMEOUT ` | ` 2 ` | HTTP timeout (seconds) when sending a log entry. |
68- | ` ANALYSIS_MAX_QUEUE ` | ` 10000 ` | Max queued items; oldest are dropped when exceeded. |
69- | ` ANALYSIS_QUEUE_DIR ` | ` sys_get_temp_dir()/analysis_queue ` | Directory for the local spool. |
70- | ` ANALYSIS_REDACT_FIELDS ` | ` password,nik,Api-Key,no_telp ` | Comma-separated sensitive fields (case-insensitive). |
71- | ` ANALYSIS_MAX_BODY_SIZE ` | ` 3145728 ` (3 MB) | ` raw_body ` truncation length (bytes). |
72- | ` ANALYSIS_WHITELIST_IPS ` | RFC1918 + localhost | CIDR ranges to skip. |
73- | ` ANALYSIS_TRUNCATE_SUFFIX ` | ` ... [truncated] ` | Appended when the body is truncated. |
74- | ` ANALYSIS_MAX_RETRIES ` | ` 5 ` | Send attempts before moving to ` failed/ ` . |
59+ | ` REQUEST_LOG_ENABLED ` | ` false ` | Master switch for the filter. |
60+ | ` REQUEST_LOG_DIR ` | ` '' ` (→ ` writable/logs ` ) | Directory for the log file. |
61+ | ` REQUEST_LOG_FILE ` | ` analysis.log ` | Log file name (single file, rotated daily). |
62+ | ` REQUEST_LOG_REDACT_FIELDS ` | ` password,nik,Api-Key,no_telp ` | Comma-separated sensitive fields (case-insensitive). |
63+ | ` REQUEST_LOG_MAX_BODY_SIZE ` | ` 3145728 ` (3 MB) | ` raw_body ` truncation length (bytes). |
64+ | ` REQUEST_LOG_WHITELIST_IPS ` | RFC1918 + localhost | CIDR ranges to skip. |
65+ | ` REQUEST_LOG_TRUNCATE_SUFFIX ` | ` ... [truncated] ` | Appended when the body is truncated. |
66+ | ` REQUEST_LOG_RETENTION_DAYS ` | ` 30 ` | Days of compressed logs kept before pruning. |
7567
7668## Usage
7769
@@ -82,112 +74,127 @@ In `app/Config/Filters.php`:
8274``` php
8375public $aliases = [
8476 // ...
85- 'analysis ' => \MrNaeem\Ci4RequestAnalysis\Filters\AnalysisFilter ::class,
77+ 'requestlog ' => \MrNaeem\Ci4RequestAnalysis\Filters\RequestLogFilter ::class,
8678];
8779```
8880
8981### 2. Attach to specific routes
9082
9183``` php
92- $routes->group('api', ['filter' => 'analysis '], function ($routes) {
84+ $routes->group('api', ['filter' => 'requestlog '], function ($routes) {
9385 $routes->post('profile/update', 'Profile::update');
9486 $routes->post('search', 'Search::index');
9587 $routes->post('upload/avatar', 'Upload::avatar');
9688});
9789```
9890
99- Only these routes will be analyzed — nothing is logged globally.
91+ Only these routes are logged — nothing is logged globally.
10092
101- ### 3. Ensure the queue directory is writable
93+ ### 3. Verify writable logs directory
10294
103- The directory must be writable by the web server. It is created automatically,
104- but on a persistent/containerized deployment mount it to a volume :
95+ The web server must be able to write to ` writable/logs ` . The directory is
96+ created automatically if missing :
10597
10698``` bash
107- mkdir -p /tmp/analysis_queue && chown -R www-data:www-data /tmp/analysis_queue
99+ mkdir -p writable/logs && chown -R www-data:www-data writable/logs
108100```
109101
110- ### 4. Run the sender
102+ ### 4. (Optional) Run rotation via cron
111103
112- The filter spawns ` analysis:send ` in the background on each analyzed request.
113- For retries of failed items , schedule it via cron (every minute) :
104+ Rotation happens automatically on write. For a guaranteed nightly pass and
105+ retention cleanup , schedule ` requestlog:rotate ` daily :
114106
115107``` cron
116- * * * * * cd /path/to/app && php spark analysis:send >/dev/null 2>&1
108+ 0 0 * * * cd /path/to/app && php spark requestlog:rotate >/dev/null 2>&1
117109```
118110
119111Or run it manually:
120112
121113``` bash
122- php spark analysis:send
114+ php spark requestlog:rotate
123115```
124116
125117## Log payload
126118
127- Each entry sent to the Analysis Server is a JSON object:
119+ Each line in ` analysis.log ` is a JSON object (envelope + ` log_data ` ) :
128120
129121``` json
130122{
131- "timestamp" : " 2026-08-31T02:15:04+00:00" ,
132- "domain" : " app.example.com" ,
133- "path" : " /api/profile/update" ,
134- "method" : " POST" ,
135- "srcip" : " 203.0.113.10" ,
136- "user_agent" : " Mozilla/5.0 ..." ,
137- "query_string" : " page=1" ,
138- "headers" : { "Content-Type" : " application/json" , ... },
139- "raw_body" : " {\" name\" :\" User\" ,\" email\" :\" user@example.com\" ,\" password\" :\" ***REDACTED***\" }" ,
140- "file_count" : 1 ,
141- "file_names" : [" shell.php.jpg" ],
142- "file_metadata" : [
143- {
144- "original_name" : " shell.php.jpg" ,
145- "size" : 20480 ,
146- "mime_type" : " image/jpeg" ,
147- "extension" : " jpg" ,
148- "hash" : " 3c98..." ,
149- "has_double_extension" : true
150- }
151- ]
123+ "log_data" : {
124+ "timestamp" : " 2026-08-31T02:15:04+00:00" ,
125+ "domain" : " app.example.com" ,
126+ "path" : " /api/profile/update" ,
127+ "method" : " POST" ,
128+ "srcip" : " 203.0.113.10" ,
129+ "user_agent" : " Mozilla/5.0 ..." ,
130+ "query_string" : " page=1" ,
131+ "headers" : { "Content-Type" : " application/json" , ... },
132+ "raw_body" : " {\" name\" :\" User\" ,\" email\" :\" user@example.com\" ,\" password\" :\" ***REDACTED***\" }" ,
133+ "file_count" : 1 ,
134+ "file_names" : [" shell.php.jpg" ],
135+ "file_metadata" : [
136+ {
137+ "original_name" : " shell.php.jpg" ,
138+ "size" : 20480 ,
139+ "mime_type" : " image/jpeg" ,
140+ "extension" : " jpg" ,
141+ "hash" : " 3c98..." ,
142+ "has_double_extension" : true
143+ }
144+ ]
145+ },
146+ "retry_count" : 0 ,
147+ "last_attempt" : null ,
148+ "created_at" : " 2026-08-31T02:15:04+00:00"
152149}
153150```
154151
152+ ## Log rotation
153+
154+ - Active file: ` writable/logs/analysis.log `
155+ - Rotated file: ` writable/logs/analysis-YYYY-MM-DD.log.gz `
156+
157+ When a write detects the active file belongs to a previous day, it is renamed
158+ to ` analysis-YYYY-MM-DD.log ` , compressed with gzip, and the original removed.
159+ Compressed files older than ` REQUEST_LOG_RETENTION_DAYS ` are pruned.
160+
155161## How it works
156162
157163```
158- Client (CI4)
159- Filter (before hook)
160- ├─ enabled? ──no──► done
161- ├─ IP whitelisted? ──yes──► done
162- ├─ collect: headers, body, files, tenant, srcip...
163- │ ├─ redact sensitive fields
164- │ ├─ truncate body at max size
165- │ └─ extract file metadata (no binary)
166- ├─ queue: write JSON to ANALYSIS_QUEUE_DIR
167- └─ triggerSend: background `php spark analysis:send`
168-
169- AnalysisSend (spark command)
170- └─ for each queue file:
171- ├─ send POST to ANALYSIS_SERVER_URL (X-API-Key header)
172- ├─ success → delete file
173- └─ failure → retry_count++, move to failed/ after max_retries
164+ Request → RequestLogFilter (before hook)
165+ ├─ enabled? ──no──► done
166+ ├─ IP whitelisted? ──yes──► done
167+ ├─ collect: headers, body, files, tenant, srcip...
168+ │ ├─ redact sensitive fields
169+ │ ├─ truncate body at max size
170+ │ └─ extract file metadata (no binary)
171+ ├─ rotate if the active log is from a previous day (rename + gzip + prune)
172+ └─ append one JSONL line to writable/logs/analysis.log
174173```
175174
175+ The write is a single append with ` LOCK_EX ` , so it does not block the request
176+ and is safe for concurrent PHP-FPM workers.
177+
176178## Security notes
177179
178- - Redaction happens client-side before the body is sent ; binary file content is
179- never transmitted .
180+ - Redaction happens before the body is written ; binary file content is never
181+ stored .
180182- Do not log ` Authorization ` /` Cookie ` values unless required — redact them by
181- adding their field names to ` ANALYSIS_REDACT_FIELDS ` .
182- - Keep ` ANALYSIS_API_KEY ` out of version control (use ` .env ` ).
183- - The ` exec ` PHP function must be enabled for background sending; otherwise use
184- cron only.
183+ adding their field names to ` REQUEST_LOG_REDACT_FIELDS ` .
184+ - ` writable/logs ` should not be publicly accessible (CI4 already blocks it by
185+ default); logs are git-ignored via ` /writable/ ` .
185186
186187## Multi-tenant
187188
188189The tenant is derived from the subdomain via ` HTTP_HOST ` and stored in the
189190` domain ` field of the payload. No extra configuration is required.
190191
192+ ## Wazuh integration
193+
194+ The library ships with a ready-to-use Wazuh setup: a JSON decoder for the
195+ ` analysis.log ` envelope, detection rules, and agent config that points to
196+ ` writable/logs/analysis.log ` . See [ docs/wazuh.md] ( docs/wazuh.md ) .
197+
191198## Testing
192199
193200``` bash
0 commit comments