vmgrab QEMU dump/search reliability and misleading classification for SEV-SNP demos

[daniel-enclaive]

Summary

During the Buckypaper demo, I observed issues with vmgrab output and search behaviour. This appears to be a vmgrab tooling issue, not a Buckypaper or SEV-SNP failure.

Buckypaper was confirmed to launch with SEV-SNP enabled, and protected guest memory behaved as expected. However, vmgrab produced confusing or unreliable demo output in two areas:

1. vmgrab dump --backend qemu may return before the dump file is fully stable, causing immediate vmgrab search to scan only a partial dump.
2. vmgrab search may classify SEV-SNP QEMU dumps as READABLE DUMP, even when protected guest memory LOAD segments are zeroed/unreadable.

Environment

Host:

uname -a

Output:

Linux hz-runner-1 6.17.0-0.rc7.56.fc43.x86_64 #1 SMP PREEMPT_DYNAMIC Mon Sep 22 14:29:30 UTC 2025 x86_64 GNU/Linux

VMs tested:

Plain VM:
enclaive-demo-plain

Buckypaper VM:
enclaive-demo-bp-daniel-4vcpu

Buckypaper QEMU process included:

-machine q35,confidential-guest-support=sev0
-object sev-snp-guest,id=sev0

Issue 1: vmgrab dump --backend qemu can return before dump is fully stable

Command

vmgrab dump enclaive-demo-plain \
  --backend qemu \
  --output "$PLAIN_OUTPUT" \
  | tee "$RUN_DIR/plain-memory-dump.txt"

PLAIN_DUMP="$(find "$PLAIN_OUTPUT" -maxdepth 1 -type f -name '*.dump' \
  -printf '%T@ %p\n' | sort -nr | head -1 | cut -d' ' -f2-)"

vmgrab search "$PLAIN_DUMP" ENCLAIVE_MEMORY_MARKER_DANIEL_2026

Observed behaviour

vmgrab dump reported success quickly:

✅ Memory dump completed successfully!
Dump size: 298.6 MB
Backend: qemu

Immediate vmgrab search did not find the marker:

No matches found for pattern: ENCLAIVE_MEMORY_MARKER_DANIEL_2026
File size: 334.9 MB

However, grep found the marker later in the same dump at offsets around 2.2 GB:

2207751053:ENCLAIVE_MEMORY_MARKER_DANIEL_2026
2207751162:ENCLAIVE_MEMORY_MARKER_DANIEL_2026
2242711540:ENCLAIVE_MEMORY_MARKER_DANIEL_2026

A later vmgrab search on the same dump correctly found the marker:

⚠️  VULNERABLE - 10 match(es) found!
This indicates the memory is NOT encrypted and sensitive data is exposed!

Expected behaviour

vmgrab dump should only return after the dump file is fully written and stable, or vmgrab search should detect that the dump file is still growing and warn/refuse to scan.

Issue 2: Misleading READABLE DUMP classification on SEV-SNP QEMU dump

Command

vmgrab dump "$VALID_VM" --backend qemu -o "$VALID_VM_DIR"

vmgrab search "$BUCKY_DUMP" "$VHSM_WORKLOAD_ID"

Observed Buckypaper dump output

Target VM: enclaive-demo-bp-daniel-4vcpu
Security: SEV-SNP Protected
Backend: qemu

The QEMU dump was an ELF/core file:

ELF 64-bit LSB core file, x86-64, version 1 (SYSV), SVR4-style

The guest memory LOAD segments showed expected protected/zeroed output:

---- LOAD segment at file offset 0x001260 ----
00001260: 0000 0000 0000 0000 0000 0000 0000 0000
00001270: 0000 0000 0000 0000 0000 0000 0000 0000

---- LOAD segment at file offset 0x081260 ----
00081260: 0000 0000 0000 0000 0000 0000 0000 0000
00081270: 0000 0000 0000 0000 0000 0000 0000 0000

---- LOAD segment at file offset 0x101260 ----
00101260: 0000 0000 0000 0000 0000 0000 0000 0000
00101270: 0000 0000 0000 0000 0000 0000 0000 0000

Searches for guest strings and workload UUID returned no matches:

No matches found for pattern: Linux version
No matches found for pattern: root@localhost
No matches found for pattern: 4efe013e-3af8-42a9-b615-c9cd6a4b4707

But vmgrab search still printed:

📄 READABLE DUMP — process memory or unencrypted VM

Substantial printable content found; encryption is not active here.

Expected behaviour

For SEV-SNP QEMU dumps, vmgrab search should distinguish between:

• ELF/core metadata, which may be readable
• Actual guest memory LOAD segments, which may be zeroed/unreadable/protected

It should not report READABLE DUMP or encryption is not active here when the VM is detected as SEV-SNP Protected and protected guest memory segments are zeroed/unreadable.

Impact

This creates confusion during demos because the product behaviour is correct, but vmgrab output can suggest the opposite.

Validated outcome:

Plain VM:
- No encryption
- Marker recoverable from memory
- Expected vulnerable output confirmed

Buckypaper VM:
- SEV-SNP enabled
- QEMU backend used
- Guest memory LOAD segments zeroed/unreadable
- Guest strings and workload UUID not recoverable

Recommended fixes

1. Make vmgrab dump --backend qemu wait until the dump file is fully written/stable before returning.
2. Make vmgrab search detect if a dump file is still growing and warn or retry.
3. For ELF/core QEMU dumps, classify guest memory LOAD segments separately from ELF metadata.
4. For SEV-SNP dumps, avoid showing READABLE DUMP when only container metadata is readable.
5. Report SEV-SNP protected memory more clearly, for example:

SEV-SNP Protected dump detected.
ELF/core metadata is readable.
Guest memory LOAD segments are zeroed/unreadable.
No guest strings found.

Current workaround used in demo scripts

Plain VM:

Use qemu backend, wait for dump file to stabilise, then run search.

Buckypaper VM:

Use qemu backend.
Show SEV-SNP Protected status.
Show LOAD segment hexdump with zeroed memory.
Search for Linux version, root@localhost, and workload UUID.
Expected result: no matches.

Conclusion

This appears to be a vmgrab reliability/classification issue, not a Buckypaper or SEV-SNP failure. The tool needs clearer dump readiness handling and SEV-SNP-aware classification before being relied on in customer-facing demos.