You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Commit c17c8f6
Browse filesBrowse the repository at this point in the historyBrowse files
authored
Inventory packages on AWS Linux root volumes. (#113)
CVE enrichment can only match packages the graph already stores, and the API scan never read the disk. This adds an opt-in worker command that fills that list from the root volume.
Co-authored-by: O M <om@omLaptop.local>
Co-authored-by: Cursor <cursoragent@cursor.com>
Copy file name to clipboardExpand all lines: CONTRIBUTING.md
+1-1Lines changed: 1 addition & 1 deletion
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -24,7 +24,7 @@ On a PR you can comment `@coderabbitai review` (incremental) or `@coderabbitai f
24
24
25
25
## Development guidelines
26
26
27
-
- Keep collectors **read-only** toward cloud accounts by default. The Phase 4 workload scan is the opt-in exception: it may create and delete snapshots in the account, and it stores package identifiers only
27
+
- Keep collectors **read-only** toward cloud accounts by default. `om scan workload` is the opt-in exception: it may create and delete snapshots in the account, and it stores package identifiers only
28
28
- Document graph schema changes in `docs/`
29
29
- Prefer small, reviewable PRs
30
30
- Add tests for graph queries and normalization logic
Copy file name to clipboardExpand all lines: README.md
+5-2Lines changed: 5 additions & 2 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -21,7 +21,7 @@ OpenSourceOM Core is the platform behind [opensourceom.org](https://opensourceom
21
21
22
22
Traditional scanners flood you with CVEs and misconfigurations. OpenSourceOM connects the dots — showing which findings sit on paths from the internet to your sensitive data and privileged identities.
23
23
24
-
> **Status:** Phase 3 has shipped. Phase 4 (workload package inventory) and Phase 5 (secret and malware findings on that storage) are open. Further rule packs stay open for contributors ([#10](https://github.com/OpenSourceOM/core/issues/10)). See the [roadmap](./docs/ROADMAP.md).
24
+
> **Status:** Phase 3 has shipped. Phase 4 is open: `om scan workload` inventories AWS Linux root volumes. Phase 5 (secret and malware findings on that storage) is open. Further rule packs stay open for contributors ([#10](https://github.com/OpenSourceOM/core/issues/10)). See the [roadmap](./docs/ROADMAP.md).
25
25
26
26
## Why this exists
27
27
@@ -93,6 +93,8 @@ go build -o om ./cmd/om
93
93
# Or scan cloud / cluster inventory
94
94
export AWS_REGION=us-east-1
95
95
./om scan aws
96
+
# On an EC2 instance in that account, as root, after the API scan:
97
+
# sudo -E ./om scan workload
96
98
./om scan k8s # requires kubeconfig
97
99
98
100
# Or ingest a custom collector (stdout is a graph batch)
@@ -133,6 +135,7 @@ An API token is created only when an admin asks for one, from the console or wit
133
135
```bash
134
136
./om scan demo # sample environment, no credentials
135
137
./om scan aws
138
+
./om scan workload # root on an EC2 worker; Linux root-volume packages
136
139
./om scan azure # requires AZURE_SUBSCRIPTION_ID + az login
137
140
./om scan gcp # requires GCP_PROJECT_ID + ADC
138
141
./om scan k8s # requires kubeconfig or in-cluster credentials
@@ -162,7 +165,7 @@ Full documentation: [opensourceom.org](https://opensourceom.org) (docs at [opens
|**4**| Package inventory from persistent workload storage: root volumes, boot images, container layers, function bundles, shared filesystems. Open. |
168
+
|**4**| Package inventory from persistent workload storage. AWS Linux root volumes ship with `om scan workload`. The other clouds, boot images, container layers, function bundles, and shared filesystems are open. |
166
169
|**5**| Secret and malware findings on storage Phase 4 already opens. Kind and path only. Open. |
Copy file name to clipboardExpand all lines: collectors/README.md
+1-1Lines changed: 1 addition & 1 deletion
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -20,7 +20,7 @@ Cloud and platform ingestion plugins. Each collector normalizes provider APIs in
20
20
21
21
The **demo** collector loads a fixed environment (internet-exposed web tier, production database, public/private buckets, admin vs app identities, public Kubernetes service). Use it to exercise CSPM packs without cloud credentials.
22
22
23
-
The **AWS** collector emits properties the CIS pack matches on: `open_ingress`, `imdsv2`, `public_ip`, S3 `encryption` / `versioning` / `public_access_block`, and IAM user `mfa` / `unused_access_keys`. It also records RDS and Aurora instances as datastores. S3 bucket tags and the RDS `TagList` copy `sensitivity` or `data-class` onto the datastore. `sensitivity` wins when both are set. The same scan reads CloudTrail management events from the last 24 hours and stores the ones that name an identity and a resource on an exposed path. A failed lookup omits those events.
23
+
The **AWS** collector emits properties the CIS pack matches on: `open_ingress`, `imdsv2`, `public_ip`, S3 `encryption` / `versioning` / `public_access_block`, and IAM user `mfa` / `unused_access_keys`. It also records RDS and Aurora instances as datastores. S3 bucket tags and the RDS `TagList` copy `sensitivity` or `data-class` onto the datastore. `sensitivity` wins when both are set. The same scan reads CloudTrail management events from the last 24 hours and stores the ones that name an identity and a resource on an exposed path. A failed lookup omits those events.`om scan aws` does not read instance disks. `om scan workload` is the separate command that inventories packages on Linux root volumes.
24
24
25
25
The **Azure** collector records logical SQL servers. The same scan reads administrative Activity Log events from the last 24 hours and stores the ones that name an identity and a resource on an exposed path. A failed lookup omits those events. Entra ID sign-in logs are not part of this scan. The **GCP** collector records Cloud SQL instances. The same scan reads Admin Activity audit logs from the last 24 hours and stores the ones that name an identity and a resource on an exposed path. A failed lookup omits those events. Data Access logs are not part of this scan. A workload gets `CAN_ACCESS` only when a security group, firewall, or VPC path allows it. Azure copies the crown-jewel mark from storage-account and SQL-server tags. GCP copies it from a bucket label or a Cloud SQL user label. The keys are `sensitivity` and `data-class`. A plugin may set `sensitivity` on a datastore directly.
> **Status:** Phase 3 has shipped. The collector plugin SDK, Helm, the embedded rule pack, and cloud audit context from CloudTrail, Activity Log, and Admin Activity logs are in the tree, alongside Phase 2 CSPM rules, blast radius, Kubernetes ingest, and exports. Phase 4 (package inventory from persistent workload storage) and Phase 5 (secret and malware findings on that storage) are open. Further rule packs stay open for contributors.
8
+
> **Status:** Phase 3 has shipped. The collector plugin SDK, Helm, the embedded rule pack, and cloud audit context from CloudTrail, Activity Log, and Admin Activity logs are in the tree, alongside Phase 2 CSPM rules, blast radius, Kubernetes ingest, and exports. Phase 4 is open: `om scan workload` inventories AWS Linux root volumes. Phase 5 (secret and malware findings on that storage) is open. Further rule packs stay open for contributors.
Findings link to affected resources via `VIOLATES` edges. CSPM and CVE findings share the same `Finding` node type but use `finding_type` and `rule_id` / `cve_id` properties for provenance.
53
54
54
-
See [ADR 001](./adr/001-graph-schema-v0.md), [ADR 002](./adr/002-phase1-findings-ui.md), [ADR 003](./adr/003-phase2-cspm-rbac.md), [ADR 004](./adr/004-collector-plugin-sdk.md), and [ADR 005](./adr/005-console-admins.md).
55
+
See [ADR 001](./adr/001-graph-schema-v0.md), [ADR 002](./adr/002-phase1-findings-ui.md), [ADR 003](./adr/003-phase2-cspm-rbac.md), [ADR 004](./adr/004-collector-plugin-sdk.md), [ADR 005](./adr/005-console-admins.md), and [ADR 006](./adr/006-aws-root-volume-packages.md).
55
56
56
57
## Technology choices
57
58
@@ -84,7 +85,9 @@ Further community rule packs (PCI and additional CIS mappings) stay open for con
84
85
85
86
## Phase 4 and Phase 5
86
87
87
-
Planned, not in the tree. Phase 4 writes `packages` (CPE 2.3 names and package URLs) from persistent workload storage onto the workload node that enrichment already matches. The sources, in order, are AWS Linux root volumes, Azure VM and GCE root volumes, boot images, container image layers and function bundles, and shared filesystems (EFS, Azure Files, Filestore). The scan is opt-in. `om scan aws`, `om scan azure`, and `om scan gcp` stay read-only API collectors. The workload scan runs a worker in the account, and it stores package identifiers only.
88
+
Phase 4 writes `packages` (CPE 2.3 names and package URLs) from persistent workload storage onto the workload node that enrichment already matches. The sources, in order, are AWS Linux root volumes, Azure VM and GCE root volumes, boot images, container image layers and function bundles, and shared filesystems (EFS, Azure Files, Filestore). The scan is opt-in. `om scan aws`, `om scan azure`, and `om scan gcp` stay read-only API collectors.
89
+
90
+
`om scan workload` is the AWS root-volume scan. It runs as root on an EC2 instance in the account, snapshots the root volume, mounts the copy read-only, and stores package identifiers only. A later `om scan aws` keeps `packages`. Windows, LVM, and non-root disks are skipped. The other Phase 4 sources are still open. See [ADR 006](./adr/006-aws-root-volume-packages.md).
88
91
89
92
Phase 5 adds a finding for a secret pattern or a malware signature on storage that Phase 4 reader already opened. The finding stores the kind and the path. Object bodies and database rows are in neither phase.
Copy file name to clipboardExpand all lines: docs/ROADMAP.md
+3-1Lines changed: 3 additions & 1 deletion
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -68,12 +68,14 @@ The scan is opt-in and separate from `om scan aws`, `om scan azure`, and `om sca
68
68
69
69
The path, in order:
70
70
71
-
-[]**AWS Linux root volumes** — [#102](https://github.com/OpenSourceOM/core/issues/102)
71
+
-[x]**AWS Linux root volumes** — [#102](https://github.com/OpenSourceOM/core/issues/102)
72
72
-[ ]**Azure VM and GCE root volumes** — the same walker, those snapshot APIs — [#103](https://github.com/OpenSourceOM/core/issues/103)
73
73
-[ ]**Boot images** — an AMI and the Azure and GCE equivalents, scanned once and applied to instances launched from that image — [#104](https://github.com/OpenSourceOM/core/issues/104)
74
74
-[ ]**Container image layers and function bundles** — layers for images already stored on Kubernetes workloads, plus Lambda deployment packages and the Azure and GCP equivalents. No volume attach — [#105](https://github.com/OpenSourceOM/core/issues/105)
75
75
-[ ]**Shared filesystems** — EFS, Azure Files, and Filestore, when the application is installed there and a root-volume scan would miss it — [#106](https://github.com/OpenSourceOM/core/issues/106)
76
76
77
+
`om scan workload` reads Linux root volumes in the current AWS account. It runs as root on an EC2 instance in that account, snapshots the root volume, mounts the copy read-only, and writes CPE 2.3 names and package URLs onto `packages`. A later `om scan aws` leaves `packages` in place. Windows, LVM, and disks other than the root volume are skipped.
78
+
77
79
Object bodies and database rows stay out. They are customer data, not the installed software. A public bucket or database is already an exposure finding from the API scan.
# ADR 006: AWS Linux root-volume package inventory
7
+
8
+
**Status:** Accepted
9
+
**Date:** 2026-10-14
10
+
**Phase:** 4
11
+
12
+
## Context
13
+
14
+
`om enrich cve` writes a finding when `packages` on a workload matches a CPE 2.3 name or a versioned package URL. `om scan aws` records the instance from `DescribeInstances` and copies the AMI id into `image`. It does not read the root volume, so a package installed on the instance never reaches `packages`.
15
+
16
+
Reading a volume means creating a snapshot and attaching a copy. That is a write to the account. The API collectors stay read-only.
17
+
18
+
## Decision
19
+
20
+
`om scan workload` is a separate command. It runs as root on an EC2 instance in the account and region being scanned.
21
+
22
+
For each Linux instance with an EBS root volume, the command:
23
+
24
+
1. Creates a snapshot of the root volume and a volume from that snapshot in the worker's availability zone.
25
+
2. Attaches the volume to the worker and mounts it read-only.
26
+
3. Reads the dpkg, apk, or rpm database and writes CPE 2.3 names and versioned package URLs onto `packages` of the existing workload node.
27
+
4. Unmounts, detaches, and deletes the volume and the snapshot, including when a later step fails.
28
+
29
+
Windows instances, instance-store roots, LVM physical volumes, and non-root disks are skipped. The graph, logs, and exports store package identifiers. They do not store file contents.
30
+
31
+
An inventory upsert that omits `packages` keeps the list already stored. `om scan aws` can run again without erasing the workload scan. An upsert that sets `packages`, including an empty list, replaces it.
32
+
33
+
The IAM actions are `ec2:CreateSnapshot`, `ec2:AttachVolume`, `ec2:DeleteSnapshot`, `ec2:DeleteVolume`, the describe and tag calls around that lifecycle, and `kms:CreateGrant` plus `kms:Decrypt` (with `kms:DescribeKey`, `kms:GenerateDataKeyWithoutPlaintext`, and `kms:ReEncrypt*`) for an encrypted root volume. `om scan aws` does not gain these actions.
34
+
35
+
Azure, GCE, boot images, container layers, function bundles, and shared filesystems stay on later Phase 4 issues. Secret and malware findings stay on Phase 5, after this reader exists.
36
+
37
+
## Consequences
38
+
39
+
The command has to run on an instance in the account. A process killed with SIGKILL can leave a snapshot or volume; both are tagged `opensourceom:scan=workload`. The Helm chart does not schedule this command.
40
+
41
+
`om enrich cve` is unchanged. It attaches a CVE only when one of these identifiers matches.
0 commit comments