Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
23 commits
Select commit Hold shift + click to select a range
cf601b6
Add Google SecOps (Chronicle) UDM parser for syslog output
seanthegeek Jun 4, 2026
7d9d693
Detect aggregate reports by "domain" instead of "adkim"
seanthegeek Jun 4, 2026
319986f
Fix JSON type handling and pre-json field init in SecOps parser
seanthegeek Jun 4, 2026
3fd3716
Detect aggregate reports by "xml_schema" instead of "domain"
seanthegeek Jun 4, 2026
16082c0
Expand README references with the sources used
seanthegeek Jun 4, 2026
091ba3d
Define CBN up front for new SecOps users
seanthegeek Jun 4, 2026
d8b8186
Cite the official Chronicle content-hub parser repo
seanthegeek Jun 4, 2026
53402c2
Store numbers as number_value; fix conditional guards to != ""
seanthegeek Jun 4, 2026
e52e6ab
Tag drop{} statements per content-hub convention
seanthegeek Jun 4, 2026
0e3f5f3
Include policy identity on SMTP TLS failure-detail rows
seanthegeek Jul 11, 2026
95e881b
Fix SecOps parser review findings
seanthegeek Jul 11, 2026
fba083f
Add Google SecOps (Chronicle) output via the v1 events.import API
seanthegeek Jul 11, 2026
c8e8cb5
Store CBN parser booleans as typed bool_value
seanthegeek Jul 11, 2026
4590c0b
Fix stale boolean-storage comments missed in the bool_value change
seanthegeek Jul 11, 2026
a131fa9
Explain the actual language constraint behind the 1b boolean conversion
seanthegeek Jul 11, 2026
571bbf7
Guarantee non-empty xml_schema: <feedback> identifies the report, not…
seanthegeek Jul 11, 2026
cd2785d
Frame aggregate detection accurately: <feedback> is the detector
seanthegeek Jul 11, 2026
8484841
Disambiguate "aggregate rows" in the xml_schema changelog entry
seanthegeek Jul 11, 2026
8935e73
Revert overanalyzed changes
seanthegeek Jul 11, 2026
7ce8b63
Ensure aggregate report output always has an xml_schema value
seanthegeek Jul 11, 2026
ae5a504
Cover the remaining gsecops.py branches with sample-driven tests
seanthegeek Jul 11, 2026
f15d4a3
Merge master into google-secops-parser
seanthegeek Aug 18, 2026
579626d
Don't log full event bodies when Google SecOps rejects an event
seanthegeek Aug 18, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
38 changes: 38 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,43 @@
# Changelog

## Unreleased

### Features

- **Google SecOps (Chronicle) output** (`[gsecops]` config section). Sends
reports to a Google SecOps instance as pre-normalized Unified Data Model
(UDM) events via the GA v1 Chronicle API `events.import` method — DMARC
aggregate and failure reports as `EMAIL_TRANSACTION` events, SMTP TLS
reports as `GENERIC_EVENT` events. Uses standard Google Cloud authentication
(a service account key file, or Application Default Credentials), batches
per the documented API best practices, and isolates invalid events by
bisecting rejected batches so one bad event cannot discard a whole run.
Because the events arrive already normalized, no parser needs to be
installed in the SecOps tenant.
- **Google SecOps (Chronicle) UDM parser** (`google_secops_parser/`). A
configuration-based normalizer (CBN) that maps the JSON events parsedmarc
emits through its `[syslog]` output to the same UDM shape as the `[gsecops]`
output, for deployments that prefer collector-based ingestion with raw-log
retention. Ships with real sample events for the SecOps parser-validation
tool; see `google_secops_parser/README.md` for installation, field mappings,
and caveats (not yet validated against a live tenant).

### Bug fixes

- **SMTP TLS failure-detail rows now carry `policy_domain` and `policy_type`**
in the flat row output (`parsed_smtp_tls_reports_to_csv_rows`). RFC 8460 §4.3
nests each failure detail inside a policy, but the serializer only attached
the policy identity to the per-policy summary row, so the `policy_domain` and
`policy_type` CSV columns were always empty on failure-detail rows and JSON
consumers (syslog, GELF) could not attribute a failure detail to its policy.
- **Per-policy fields no longer leak across policies** in the same serializer:
`policy_strings` / `mx_host_patterns` from an earlier policy were reused for
a later policy that did not define them, because the row template dict was
built once per report instead of once per policy.
- Ensure that the JSON or CSV output from a parsed DMARC aggregate report
always has an `xml_schema` value, so that the reports can be detected by
Google SecOps and other output parsers.

## 10.4.3

### Changes
Expand Down
29 changes: 29 additions & 0 deletions docs/source/usage.md
Original file line number Diff line number Diff line change
Expand Up @@ -748,6 +748,13 @@ The full set of configuration options are:
retry_delay = 5
```

:::{note}
To ingest the syslog output into Google SecOps (Chronicle), a custom UDM
parser is available in the
[google_secops_parser](https://github.com/domainaware/parsedmarc/tree/master/google_secops_parser)
directory.
:::

- `gmail_api`
- `credentials_file` - str: Path to file containing the
credentials, None to disable (Default: `None`)
Expand Down Expand Up @@ -803,6 +810,27 @@ The full set of configuration options are:
- `smtp_tls_url` - str: URL of the webhook which should receive the smtp_tls reports
- `timeout` - int: Interval in which the webhook call should timeout

- `gsecops` - Send reports to [Google SecOps](https://cloud.google.com/security/products/security-operations)
(Chronicle) as Unified Data Model (UDM) events via the v1 Chronicle API
[`events.import`](https://docs.cloud.google.com/chronicle/docs/reference/ingestion-methods)
method. Pre-normalized UDM events bypass SecOps's server-side parsing, so no
custom parser needs to be installed in the tenant (the alternative,
raw-log-preserving path is the syslog output plus the parser in the
[google_secops_parser](https://github.com/domainaware/parsedmarc/tree/master/google_secops_parser)
directory).
- `project_id` - str: The Google Cloud project ID linked to the SecOps
instance at onboarding. A correctly-permissioned account in any other
project will fail to authenticate.
- `instance_id` - str: The SecOps instance (customer) GUID, shown under
**SIEM Settings > Profile** in the SecOps console
- `region` - str: The SecOps instance region, e.g. `us` or `europe`
(Default: `us`)
- `credentials_file` - str: Path to a Google service account JSON key file.
When not set, [Application Default Credentials](https://cloud.google.com/docs/authentication/application-default-credentials)
are used. Either way, the account must hold the **Chronicle API Editor**
IAM role (the `chronicle.events.import` permission) in the linked
project.

:::{warning}
It is **strongly recommended** to **not** use the `nameservers`
setting. By default, `parsedmarc` uses
Expand Down Expand Up @@ -1076,6 +1104,7 @@ For sections with underscores in the name, the full section name is used:
| `log_analytics` | `PARSEDMARC_LOG_ANALYTICS_` |
| `gelf` | `PARSEDMARC_GELF_` |
| `webhook` | `PARSEDMARC_WEBHOOK_` |
| `gsecops` | `PARSEDMARC_GSECOPS_` |

## Using parsedmarc as a library

Expand Down
283 changes: 283 additions & 0 deletions google_secops_parser/README.md

Large diffs are not rendered by default.

Loading
Loading