Skip to content

v2: Separate request-body bucket handling from the reader - #3716

Open
vortexopenclaw wants to merge 2 commits into
owasp-modsecurity:v2/masterfrom
vortexopenclaw:refactor/v2-request-body-buckets
Open

vortexopenclaw wants to merge 2 commits into
owasp-modsecurity:v2/masterfrom
vortexopenclaw:refactor/v2-request-body-buckets

Conversation

@vortexopenclaw

@vortexopenclaw vortexopenclaw commented Oct 27, 2026 •

Copy link
Copy Markdown

Summary

Make ModSecurity v2's request-body handling easier to review and safer to maintain, without changing which requests it accepts or rejects.

When Apache receives a request body in chunks, ModSecurity must enforce size limits, store the data, and decide whether to reject the request or continue processing it. Today, those decisions are mixed together in a long reader function with repeated checks. That makes security-sensitive changes harder to review and increases the risk of accidentally changing how a configured limit is enforced.

This PR separates those responsibilities into small, focused helpers and adds 24 regression tests for requests just below, exactly at, and just above the configured limits. The benefit is clearer control flow and explicit test coverage of the existing policies—not a new blocking policy or a change to HTTP responses.

What this improves:

  • Makes request-size checks, body storage, and end-of-request handling easier to inspect independently.
  • Removes repeated whole-body limit error-message construction while preserving the existing decisions and messages.
  • Adds coverage for both overall-body and non-file-data limits, with the engine enabled or in DetectionOnly mode and with Reject or ProcessPartial configured.
  • Keeps this maintenance work separate from the JSON error-status fix, so that small functional correction can be reviewed on its own.

Scope and implementation

Split out of #3629 at the maintainer's request: #3629 (review). This PR targets v2/master independently of the JSON fix and changes only:

  • apache2/apache2_io.c: extract check_request_body_limit, store_request_body_chunk, and read_request_body_bucket from read_request_body().
  • tests/regression/config/10-request-buckets.t: add the 24 request-limit boundary cases described above.

There is no JSON return-code change or HTTP 400 mapping here. Completion failures retain the base's behavior. This PR does not resolve #2807 or choose between the overlapping status changes in #3629 and #3515.

Intended invariants

Preserve bucket-read failure handling, limit checks before storage, stream accounting, request-body length accounting, ProcessPartial stopping, storage-error policy, and end-of-stream flags/order. Preserve the existing distinction between whole-body and no-files limits rather than harmonizing their policies in a refactor.

The extraction removes duplicated whole-body error-message construction while retaining the same message and return decisions. It is submitted separately so maintainers can assess whether a maintenance-branch refactor is appropriate; the small JSON fix should not need to absorb this churn just to satisfy a complexity gate.

Verification

Fresh source export of this branch with the pinned libinjection submodule, Ubuntu 24.04 arm64, Apache 2.4.58 and YAJL. Container networking disabled; no published ports.

  • Build succeeds with --enable-pcre-study=yes --enable-pcre-jit CFLAGS=-Werror=format-security.
  • make test: 4,250/4,250 unit tests pass.
  • Full native Apache regressions: 336/336 pass, including the 24 bucket/limit cases.

The actual HTTP runner was USER=root LOGNAME=root perl run-regression-tests.pl -S . -p 8090 -v from tests/; use the actual test-process user for USER/LOGNAME.

These results cover the standard build, not certification of every optional build or every malformed transport path. Cross-platform CI and maintainer review are still required. No production deployment is included.

Separate bucket reads, nonempty chunk storage and whole-body limit handling. Collapse repeated limit diagnostics while preserving the distinct no-files limit policy, stream compile-time paths, byte accounting, EOS state and completion status mapping.

Add 24 native chunked boundary tests across both engines and limit actions, validated against the pre-refactor implementation. This addresses the SonarCloud cognitive-complexity finding without suppressing it.
@coderabbitai

coderabbitai Bot commented Oct 27, 2026 •

Copy link
Copy Markdown

Review in Change Stack →

Important

Review skipped

Auto reviews are disabled on base/target branches other than the default branch.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

Run configuration
  • Source mode: Auto
  • Repository: Unavailable
  • Branch: Unavailable
  • File: No configuration file from this repository applied
  • Commit: Unavailable
  • Layers: defaults
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: 11db4297-d42e-42e1-b733-b87d65875846

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review

No actionable comments were generated in the recent review.

Recent review info
Run configuration
  • Source mode: Auto
  • Repository: Unavailable
  • Branch: Unavailable
  • File: No configuration file from this repository applied
  • Commit: Unavailable
  • Layers: defaults
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: bfd88678-ab5c-4dbe-97bc-917a5108fe55
Commits

Reviewing files that changed from the base of the PR and between 42b02c6 and c4da907.

Files selected for processing: 2
  • apache2/apache2_io.c
  • tests/regression/config/10-request-buckets.t

Included review availability: This review used your included allowance. Your plan provides up to 8 included reviews per hour; 6 remain after this review.

Walkthrough

The request-body bucket loop now delegates reading, limit checks, storage, accounting, and EOS handling to helpers. New regression cases cover chunked requests across engine and action policies, body and no-files limits, and payload sizes around the configured limit.

Changes: Request-body bucket handling

Layer / File(s) Summary
Limit and storage policy handling
apache2/apache2_io.c
Helpers check whole-body limits and handle storage outcomes according to the engine mode and body-limit action.
Bucket reading and regression coverage
apache2/apache2_io.c, tests/regression/config/10-request-buckets.t
The bucket loop delegates reading, accounting, and EOS handling to a helper. Regression tests check expected response codes and phase-2 logging across the configured cases.

Priority: Low

Estimated code review effort: 3 (Moderate) | ~20 minutes

Change: Refactor

Suggested reviewers: airween

Merge Risk: Minimal · previous review c4da9

No concrete issue remains that would prevent merging after normal checks.

Pre-merge checks: Passed 2 · Failed 2

Failed Failed checks: 2 warnings

Check name Status Explanation Resolution
Linked Issues check Warning Issue #2807 requires premature EOF in a JSON request to fail with REQBODY_ERROR instead of producing HTTP 500. The whole-PR diff changes only apache2/apache2_io.c bucket handling and adds bucket-l… Implement the #2807 JSON premature-EOF behavior and add an automated regression test that verifies the required REQBODY_ERROR and HTTP status. Otherwise, remove #2807 as a direct link and link the issue that owns the bucket-handling refac…
Out of Scope Changes check Warning The linked issue concerns JSON premature EOF and its HTTP status. The added helpers and 24 tests concern chunked request-body limits, no-files limits, limit actions, and bucket accounting. The diff pr… Split the bucket refactor and bucket-limit tests into a PR with an appropriate linked issue, or make this PR implement #2807 and retain only changes that support that objective.
Passed Passed checks: 2 passed
Check name Status Explanation
Description Check Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check Passed The title "v2: Separate request-body bucket handling from the reader" matches the main change. The PR extracts request-body bucket reading, chunk storage, and whole-body limit handling out of read_req…
Full details: Title check

Explanation

The title "v2: Separate request-body bucket handling from the reader" matches the main change. The PR extracts request-body bucket reading, chunk storage, and whole-body limit handling out of read_request_body() in apache2/apache2_io.c, and the file summary confirms new helpers for these tasks. The title's "v2:" prefix is a version tag and adds no meaning, but the rest of the title is specific and accurate.

Full details: Linked Issues check

Explanation

Issue #2807 requires premature EOF in a JSON request to fail with REQBODY_ERROR instead of producing HTTP 500. The whole-PR diff changes only apache2/apache2_io.c bucket handling and adds bucket-limit tests in tests/regression/config/10-request-buckets.t. It contains no JSON status mapping or premature-EOF regression test. The PR description also states that it does not resolve #2807.

Resolution

Implement the #2807 JSON premature-EOF behavior and add an automated regression test that verifies the required REQBODY_ERROR and HTTP status. Otherwise, remove #2807 as a direct link and link the issue that owns the bucket-handling refactor.

Full details: Out of Scope Changes check

Explanation

The linked issue concerns JSON premature EOF and its HTTP status. The added helpers and 24 tests concern chunked request-body limits, no-files limits, limit actions, and bucket accounting. The diff provides no concrete connection between these changes and the #2807 behavior. The current PR description confirms that the changes are independent of the JSON fix.

Resolution

Split the bucket refactor and bucket-limit tests into a PR with an appropriate linked issue, or make this PR implement #2807 and retain only changes that support that objective.

Finishing Touches
🧪 Generate unit tests (beta)
  • Create a new PR
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@vortexopenclaw

Copy link
Copy Markdown
Author

@coderabbitai review

@coderabbitai

coderabbitai Bot commented Oct 27, 2026 •

Copy link
Copy Markdown
✅ Action performed

Review finished.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

@sonarqubecloud

Copy link
Copy Markdown

@airween

airween commented Oct 28, 2026

Copy link
Copy Markdown
Member

Thanks @vortexopenclaw for splitting this out of #3629!

I walked through the extracted helpers against the original loop, and they are equivalent: the whole-body limit decisions, the finished_reading handling, the -5 / 413 and On/Reject / 500 decisions, the stream and reqbody_length accounting and the EOS handling are all preserved, in the same order.

However, I have to put this PR on hold for now, sorry. As I explained in #3629, there is an older open PR by @hnakamur, #3483 (with its v3 counterpart #3476), which rewrites exactly this part of read_request_body() and also changes the behaviour this PR's tests assert:

  • Textual conflict: v2: Allow partial processing of XML, JSON, and multipart request body  #3483 restructures the same bucket loop (the ProcessPartial truncation, the -5 handling, and the rcbe handling after the loop), so the two PRs conflict in apache2/apache2_io.c.
  • Behavioural conflict: with v2: Allow partial processing of XML, JSON, and multipart request body  #3483, modsecurity_request_body_store() and modsecurity_request_body_end() return -5 for the no-files limit only when the engine is On and the action is Reject. With ProcessPartial the body is truncated and processed partially instead. So in 10-request-buckets.t, the 33-byte no-files cases for On/ProcessPartial, DetectionOnly/Reject and DetectionOnly/ProcessPartial would return 200 and run phase 2, instead of the 413 they assert now.

The current 413 under DetectionOnly is exactly the kind of known-wrong behaviour we don't want to lock in with a regression test: DetectionOnly is expected not to block, and v3 already handles the no-files limit by setting REQBODY_ERROR / INBOUND_DATA_ERROR and letting the rules decide.

Since #3483 has been waiting longer and keeps v2 and v3 consistent, I'd like to review and merge it first. After that, it would be great if you could rebase this refactor on top of it, so the helpers follow the new structure and the bucket tests assert the fixed behaviour. A refactor is much easier to adapt to a functional change than the other way around, so I think this order is the fairer one.

When you rebase, please also:

  • Squash and rebase onto v2/master instead of merging v2/master into the branch (127fb4e).
  • Reword the comment in store_request_body_chunk(). It currently says the -5 path continues for ProcessPartial "including when the engine is DetectionOnly", but modsecurity_request_body_store() never returns -5 in DetectionOnly. Please adjust it to whatever the logic is after v2: Allow partial processing of XML, JSON, and multipart request body  #3483.

Thanks for your patience, and for the boundary tests: they will be very useful for verifying #3483 as well.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants