Repository navigation
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
Audit's SQL Server and PostgreSQL persisters discard oversized and binary message bodies. Oversized text retains a searchable prefix, but the body API cannot return the complete body. Error's SQL persisters already support external storage for these bodies.
Change
Reuse Error's FileSystem, Azure Blob, and S3 providers for Audit by extracting them into
ServiceControl.Persistence.BodyStorage.RavenDB behavior is unchanged. Existing
TooLargeandNotTextrows retain their previous API responses; discarded bodies cannot be recovered. The new external-body state uses the existing integer column, so no schema migration is needed.Required configuration
External storage is required for Audit SQL Server and PostgreSQL instances. Existing instances must add
ServiceControl.Audit/MessageBody/StorageTypeand the selected provider's settings before restarting after an upgrade.MaxBodySizeToStorebecomes the inline threshold, not a limit on the complete stored body. Azure defaults to theaudit-bodiescontainer; S3 defaults to theaudit-bodies/key prefix.Each instance must use a separate directory, container, or key prefix. Sharing a location can let one instance's retention process delete another instance's bodies.
Design constraints
Sharing the providers avoids duplicating their storage formats, compression, authentication, and stream handling. Error retains its existing configuration and behavior.
Audit retention deletes by hourly prefixes on SQL Server and daily prefixes on PostgreSQL. This preserves PostgreSQL's partition-drop path without scanning message rows. SQL Server only visits hours containing rows, so bodies orphaned by failed inserts into otherwise empty hours can require manual cleanup.
The current design and configuration are documented in Message body storage.
Verification
git diff --checkpasses.Database-backed Audit, Azure/S3, and Windows packaging tests were not run because the required services or environment were unavailable. They remain release validation requirements.
The full Audit unit suite has an existing
PlatformSampleSettingssnapshot failure involving hostname/BOM differences. The testing-tool AppHost build is blocked by the existing MessagePack 2.5.302 vulnerability warning, GHSA-qhrr-8q5h-9q3h. Neither issue was changed or suppressed.