From b557315ffaf070c1eafe2426b4ab91bc7fef9753 Mon Sep 17 00:00:00 2001 From: Jakub Dyszkiewicz Date: Fri, 25 Sep 2026 14:51:47 +0200 Subject: [PATCH 1/2] docs(event-gateway): clarify mask fields failure modes and schema tag selection Move the failure mode recommendations into the failure mode tables, state that the parent Schema Validation policy must not use passthrough or mark, and document how to select fields from schema registry metadata tags. Co-Authored-By: Claude Opus 5 --- .../mask-fields-consume/index.md | 29 +++++++++++++++---- .../mask-fields-produce/index.md | 22 ++++++++++++-- 2 files changed, 43 insertions(+), 8 deletions(-) diff --git a/app/_event_gateway_policies/mask-fields-consume/index.md b/app/_event_gateway_policies/mask-fields-consume/index.md index a9783aeb3ea..bf289891190 100644 --- a/app/_event_gateway_policies/mask-fields-consume/index.md +++ b/app/_event_gateway_policies/mask-fields-consume/index.md @@ -152,9 +152,13 @@ columns: key: description rows: - failure: "`skip`" - description: Doesn't deliver the record to the client. + description: | + Doesn't deliver the record to the client. + Recommended for most situations. - failure: "`error`" - description: Doesn't deliver the batch to the client. + description: | + Doesn't deliver the batch to the client. + Use `error` sparingly, because an error on a batch makes clients stop at the problematic offset, and an operator must intervene to skip it. - failure: "`passthrough`" description: Delivers the record to the client without a mask. - failure: "`mark`" @@ -163,10 +167,23 @@ rows: The value of the header is the reason for the failure. {% endtable %} -For a redaction policy: -* In most situations, use `skip`. -* Use `error` sparingly, because an error on a batch makes clients stop at the problematic offset, and an operator must intervene to skip it. -* Don't use `passthrough` or `mark`, because both deliver the original value to the client. +The parent [Schema Validation Consume policy](/event-gateway/policies/schema-validation-consume/) must not use `passthrough` or `mark`. +With these failure modes, the parent delivers the record without parsing it. The Mask Fields Consume policy then never runs, and the client receives the fields unmasked. +Set the parent policy to `skip` or `error`. + +## Selecting fields from schema metadata + +Instead of listing paths one by one, the `paths` entry can take a single expression that reads the record's schema metadata and returns the tagged field paths. For example, with a Confluent Schema Registry, an expression can return every field path that carries a `PII` tag: + +``` +paths: 'record.value.schema.metadata.tags["PII"]' +``` + +This is the recommended way to mask fields. The mask follows the schema, so when the schema evolves and a new field gets tagged, it is masked without any policy change. + +In this case, the schema becomes the input that decides which fields are masked, so anyone who can edit the schema tags can change the masking. Keep schema registry write access restricted to trusted roles. + +See the full walkthrough in [Mask Kafka message fields selected by schema registry tags](/event-gateway/mask-kafka-message-fields-from-schema/). ## Policy order diff --git a/app/_event_gateway_policies/mask-fields-produce/index.md b/app/_event_gateway_policies/mask-fields-produce/index.md index e3955078d8e..e45c6f28959 100644 --- a/app/_event_gateway_policies/mask-fields-produce/index.md +++ b/app/_event_gateway_policies/mask-fields-produce/index.md @@ -146,7 +146,9 @@ columns: key: description rows: - failure: "`reject`" - description: Rejects the record batch that holds the record. + description: | + Rejects the record batch that holds the record. + Recommended for most situations. - failure: "`passthrough`" description: Sends the record to the backend cluster without a mask. - failure: "`mark`" @@ -155,7 +157,23 @@ rows: The value of the header is the reason for the failure. {% endtable %} -For a redaction policy, use `reject`. Don't use `passthrough` or `mark`, because both write the original value to the cluster. +The parent [Schema Validation Produce policy](/event-gateway/policies/schema-validation-produce/) must not use `passthrough` or `mark`. +With these failure modes, the parent sends the record without parsing it. The Mask Fields Produce policy then never runs, and the cluster stores the fields unmasked. +Set the parent policy to `reject`. + +## Selecting fields from schema metadata + +Instead of listing paths one by one, the `paths` entry can take a single expression that reads the record's schema metadata and returns the tagged field paths. For example, with a Confluent Schema Registry, an expression can return every field path that carries a `PII` tag: + +``` +paths: 'record.value.schema.metadata.tags["PII"]' +``` + +This is the recommended way to mask fields. The mask follows the schema, so when the schema evolves and a new field gets tagged, it is masked without any policy change. + +In this case, the schema becomes the input that decides which fields are masked, so anyone who can edit the schema tags can change the masking. Keep schema registry write access restricted to trusted roles. + +See the full walkthrough in [Mask Kafka message fields selected by schema registry tags](/event-gateway/mask-kafka-message-fields-from-schema/). ## Policy order From 90c2441b3e07895abe2134e54a6d357e864e4af8 Mon Sep 17 00:00:00 2001 From: lena-larionova <54370747+lena-larionova@users.noreply.github.com> Date: Tue, 20 Oct 2026 10:04:25 -0700 Subject: [PATCH 2/2] Apply copyedits Co-authored-by: lena-larionova <54370747+lena-larionova@users.noreply.github.com> --- app/_event_gateway_policies/mask-fields-consume/index.md | 5 +++-- app/_event_gateway_policies/mask-fields-produce/index.md | 5 +++-- 2 files changed, 6 insertions(+), 4 deletions(-) diff --git a/app/_event_gateway_policies/mask-fields-consume/index.md b/app/_event_gateway_policies/mask-fields-consume/index.md index bf289891190..f01138e3a9a 100644 --- a/app/_event_gateway_policies/mask-fields-consume/index.md +++ b/app/_event_gateway_policies/mask-fields-consume/index.md @@ -167,9 +167,10 @@ rows: The value of the header is the reason for the failure. {% endtable %} -The parent [Schema Validation Consume policy](/event-gateway/policies/schema-validation-consume/) must not use `passthrough` or `mark`. +Set the parent [Schema Validation Consume policy](/event-gateway/policies/schema-validation-consume/) to `skip` or `error` to use this policy. + +The parent policy must not use `passthrough` or `mark`. With these failure modes, the parent delivers the record without parsing it. The Mask Fields Consume policy then never runs, and the client receives the fields unmasked. -Set the parent policy to `skip` or `error`. ## Selecting fields from schema metadata diff --git a/app/_event_gateway_policies/mask-fields-produce/index.md b/app/_event_gateway_policies/mask-fields-produce/index.md index e45c6f28959..b0885ca053e 100644 --- a/app/_event_gateway_policies/mask-fields-produce/index.md +++ b/app/_event_gateway_policies/mask-fields-produce/index.md @@ -157,9 +157,10 @@ rows: The value of the header is the reason for the failure. {% endtable %} -The parent [Schema Validation Produce policy](/event-gateway/policies/schema-validation-produce/) must not use `passthrough` or `mark`. +Set the parent [Schema Validation Produce policy](/event-gateway/policies/schema-validation-produce/) to `reject` to use this policy. + +The parent policy must not use `passthrough` or `mark`. With these failure modes, the parent sends the record without parsing it. The Mask Fields Produce policy then never runs, and the cluster stores the fields unmasked. -Set the parent policy to `reject`. ## Selecting fields from schema metadata