Skip to content

Store phone-supplied images in a flash slot pool - #2049

Open
sjp4 wants to merge 3 commits into
coredevices:mainfrom
sjp4:steve/flash-images
Open

sjp4 wants to merge 3 commits into
coredevices:mainfrom
sjp4:steve/flash-images

Conversation

@sjp4

@sjp4 sjp4 commented Sep 9, 2026

Copy link
Copy Markdown
Member

Problem

Album art and notification images were reassembled into a ~34 KiB kernel-heap buffer. When that allocation failed (busy/fragmented heap), the image silently never appeared — the card stayed on its grey placeholder — and nothing was logged.

Change

emery/gabbro memory-map their NOR flash, so a bitmap can render directly from a flash address with no RAM copy. Images now stream into a two-slot pool carved from reserved flash and are rendered in place — no pixel-sized heap allocation. Slots are keyed by ImagingImageType (not wired to a specific consumer), so any two consumers can hold an image at once; a third falls back to the heap.

Also considered using the unused firmware slot for a larger image cache - that could be an alternative (advantage: cache a pool of images to avoid re-fetching from mobile, disadvantage: would be wiped/not usable if we enable background firmware updates in the future, so probably not worth it).

Testing

pbl test 333/333, with new cases for storage, the two-consumer pool, replace/supersede, no-image reclaim, heap fallback, and the erase-race (fails without the fix). Verified on obelix: both image types land in flash, no heap-fallback cascade.

sjp4 and others added 3 commits September 8, 2026 20:15
Expose flash_memory_mapped_address(), the CPU-addressable pointer for a
flash address on platforms whose external flash is permanently
memory-mapped (region addresses include the XIP base). Callers can then
read flash bytes directly instead of copying them out, which the imaging
service uses to render phone-supplied images straight from flash.

Also provide the accessor in the SPI-flash test fake.

Signed-off-by: Steve Penna <steve@penna.org.uk>
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Add IMAGING_0 and IMAGING_1, two 40 KiB slots taken from the previously
reserved RSVD1/RSVD2 areas on the gd25q256e (emery/gabbro) and QEMU
layouts. The imaging service stores decoded phone images here and renders
them through the memory-mapped flash window. Every other region keeps its
address, so the layout stays compatible with deployed devices.

Signed-off-by: Steve Penna <steve@penna.org.uk>
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Decoded album-art and notification images were reassembled into a
pixel-sized kernel-heap buffer (~34 KiB on the largest displays), which a
busy or fragmented heap could refuse, leaving the card on a grey
placeholder.

Stream each image instead into one of two fixed slots in the dedicated
IMAGING flash region and render it directly through the memory-mapped
window, so no pixel-sized heap allocation is needed. Slots are assigned to
whichever image types are displaying rather than wired to a specific
consumer, so any two consumers can hold an image at once; a third, or a
transfer whose erase loses the race to the response, falls back to the
heap. The reservation's erase overlaps the phone round trip, and a dropped
reservation is reclaimed rather than left to orphan its slot.

Delivered bitmaps now carry is_bitmap_heap_allocated so the consumers only
free pixels that actually live on the heap, and imaging_release() lets a
consumer hand its slot back when its window closes. Failure paths (no
image, malformed response) log and, where relevant, free the reserved
slot.

On boards without the region the whole pool compiles out and images use
the heap as before.

Signed-off-by: Steve Penna <steve@penna.org.uk>
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@sjp4
sjp4 requested review from gmarull and jplexer as code owners September 9, 2026 14:16
Comment on lines +128 to +130
if (s_music_ctx.album_art->info.is_bitmap_heap_allocated) {
kernel_free(s_music_ctx.album_art->addr);
}

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The stale-token branch in music_set_album_art() still calls kernel_free(bitmap->addr) unconditionally. If the track changes while a flash-backed image is transferring, the response takes that branch and passes a memory-mapped flash address to the heap allocator, potentially crashing the watch.

prv_rx_reset();
prv_deliver(hdr->token, type, NULL);
return;
}

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

At this point prv_slot_claim() may have moved a slot to SlotWriting, but prv_rx_reset() only frees heap buffers and clears s_rx; it never releases the uncommitted slot. The same leak occurs when a new first chunk supersedes an in-progress transfer or allocating the final GBitmap fails. These slots remain unavailable until disconnect, eventually forcing images back onto the heap.

//! Abandon the slot reserved for transfer `token` when the request resolves without filling it
//! (the phone had no image, the type is unsupported, or the response was malformed). Leaves any
//! already-delivered image (an Idle+live slot) untouched.
static void prv_slot_cancel(uint8_t token) {

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tokens are not globally unique: album art uses the track-generation counter, while notifications increment their own counter. If both consumers reserve token 1 and the notification returns NoImage, this loop cancels both reservations, forcing the unrelated album-art response onto the heap. prv_slot_claim() has the same ambiguity because it matches only the token. Can pending reservations be keyed by (image_type, token) throughout claim/cancel/write, with a test where both consumers use the same token and one returns NoImage?

@gmarull gmarull left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the thorough work here — the slot state machine is carefully done. My main concern is with the storage choice itself rather than the implementation, so raising it before the inline findings get addressed.

Flash wear

Every image load erases the whole 40 KiB slot (prv_slot_reserve()flash_erase_optimal_range() over the full slot). Neither slot is 64 KiB-aligned, so that's 10 × 4 KiB subsector erases per image, at fixed addresses, with no wear levelling. PFS spreads its erases over ~21 MiB; these two windows would be by far the hottest sectors on the chip.

Rough numbers against the GD25Q256E 100k-cycle rating: album art alone alternates between the two slots, so ~200 track changes/day → ~100 erases/slot/day → ~2.7 years. If a notification image is holding the other slot, album art is pinned to one slot → ~1.4 years. Heavy notification-image traffic makes it worse. That's inside the expected life of the device, and 100k is the spec minimum.

Non-wear costs of the same design:

  • ~0.5 s typical (seconds worst-case) of erase per image on the critical path. On a fast link the phone's first chunk can land before the erase completes, in which case we fall back to the heap anyway — so the benefit is only realised sometimes.
  • ~20 mA during the erase, and contention with PFS/resource reads (erase-suspend helps but doesn't remove it).

The unused-firmware-slot-as-ring alternative from the description would cut wear ~75×, but as you note it collides with background firmware updates.

Proposal: PSRAM instead

The sf32lb52 parts have PSRAM on MPI2 (0x60000000), and we've been holding it off (LDO18 disabled, PAD_SA* parked as analog in soc/sf32lb/sf32lb52x/init.c) mainly because nothing needed it yet. Enabling it is a better fit for this problem, and the imaging code gets simpler:

  • No slot pool, no erase state machine. PSRAM is plain memory-mapped, byte-writable RAM, so imaging.c goes back to roughly the pre-PR shape: allocate pixels, memcpy chunks in, hand the GBitmap over. The erase race, cancel_pending, leaked SlotWriting slots and token-collision issues all disappear because there's no erase to race and no fixed slot to leak.
  • Allocation comes from a second Heap instance over the PSRAM region (psram_malloc()/psram_free()); Heap already supports multiple instances. With MBs behind it, 34 KiB fragmentation stops being a concern, and a small LRU of recent images becomes cheap.
  • Consumers free through one imaging_free_bitmap() instead of picking kernel_free vs. not — which also closes the stale-token kernel_free(bitmap->addr) class of bug for good.
  • Rendering from PSRAM is fine: OPI PSRAM on MPI2 is faster than the QSPI flash XIP window this PR renders from, and it's CPU-cached the same way.

Costs, to be upfront:

  • Standby current is the real one. Keeping images across sleep needs the PSRAM in a data-retaining mode (HAL_MPI_PSRAM_SLEEP()), tens to ~100+ µA for a 64 Mb OPI part. The HAL has HAL_MPI_PSRAM_SET_PASR() (partial-array self-refresh), so we can refresh only a small fraction of the array and park images there; that needs measuring on hardware before we commit to "images survive standby". The fallback policy is HAL_MPI_PSRAM_DPD() in deep sleep and re-request on wake, at the cost of placeholder flicker after every wake.
  • Sleep integration in sleep.c (MPI2 sleep/wake, re-init if the level drops MPI2 power; bf0_hal_mpi_psram.c is already in ramfunc.ld), MPU/cache attributes for the region, and a few ms of boot.
Flash slots (this PR) PSRAM heap
Wear fixed hot sectors, ~1.5–3 yr at heavy use none
Per-image latency ~0.5 s+ erase, may lose the race to the first chunk none
Per-image energy ~20 mA × erase time negligible
Standby cost none PSRAM retention (PASR-mitigated)
Code slot state machine + erase race handling second heap, ~100 lines
Capacity 2 × 40 KiB MBs — image cache, bigger app/JS heaps later

Suggested sequencing

  1. PSRAM bring-up as its own PR: init + pin mux, MPU/cache attrs, sleep/wake with PASR, psram_heap, and a standby-current measurement before/after.
  2. Imaging on top: back to the simple heap path but allocating from psram_heap; keep the logging additions and the bitmap-ownership cleanup from this PR; drop IMAGING_0/1 from the flash region tables.

I'd rather not merge the flash-slot approach as a stopgap since the state machine is most of the diff and would be thrown away. The only thing flash gives that PSRAM doesn't is persistence across reboot, and nothing here uses that.

Separately, @jplexer's three inline findings (unconditional kernel_free on the stale-token path, prv_rx_reset() not releasing a claimed slot, non-unique tokens across image types) all look real to me if this direction is kept.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants