fix: retry Ring history after storage errors

Author: DevanMetz

.github/workflows/release.yml

@@ -35,6 +35,8 @@ jobs:
         run: python3 tools/test_notes_io.py --compiler cc
       - name: Test ride history storage
         run: python3 tools/test_ride_history.py --compiler cc
+      - name: Test Ring history storage
+        run: python3 tools/test_ring_history.py --compiler cc
       - name: Test capture reader
         run: |
           cc -std=c11 -Wall -Wextra -Werror -pedantic -I main main/capture_data.c tests/capture_data_test.c -o "$RUNNER_TEMP/capture_data_test"
@@ -147,6 +149,20 @@ jobs:
           if-no-files-found: warn
           retention-days: 14
 
+      - name: Retain Ring history test evidence
+        uses: actions/upload-artifact@v4
+        if: always()
+        with:
+          name: ring-history-logs
+          path: |
+            build/ring-history/compile.log
+            build/ring-history/result.log
+            build/ring-history/source.json
+            build/ring-history/ring_history.inc
+            build/ring-history/storage_source.inc
+          if-no-files-found: warn
+          retention-days: 14
+
   firmware:
     runs-on: ubuntu-latest
     container: espressif/idf:v5.4.2

ROADMAP.md

@@ -64,6 +64,7 @@ Phase 1 software progress is also underway:
 - [x] Notes and ride summaries use recoverable backup replacement; rides publish only after durable sync, and health logs repair incomplete final rows.
 - [x] Notes loads complete files up to 65,535 bytes, replacing a silent 2,047-byte prefix. Recovery/read/close errors, oversized files and NUL data disable Save; byte-length and SD checks run before storage changes. Its launcher leave hook clears editor ownership. Four preceding-source controls reproduce full-note truncation; 36 native file/fault cases and five real LVGL cases pass, including maximum ASCII/UTF-8 notes and 25 reopens without retained resources in the unchanged 96 KiB pool. Linux and Windows CI retain separate source-bound evidence. Physical SD/panel and power cuts remain open.
 - [x] Ride history reports recovery/open/read/close errors as unavailable instead of showing zero or partial totals. Summary appends check SD availability and preparation errors, reject failed read chunks before copying, and retain the first error through cleanup. All 50 native file/fault cases pass; the preceding callbacks fail 24 controls. Exact final/backup/temporary bytes and 25 reopens are checked with closed modeled handles. Linux CI retains source hashes and logs, and the unflashed firmware builds with unchanged configuration. Physical SD/panel and power cuts remain open.
+- [x] Ring logging commits its saved timestamp only after tail recovery and a complete, successfully closed read. Failed loads retain queued samples and defer history requests until a successful retry; known SD unavailability also blocks a cached checkpoint. Appends check directory errors and preserve the first seek/tell/header/row error through close and repair. All 59 native file/queue/fault cases pass; the preceding callbacks fail 32 controls. Checks verify exact bytes, retry/deduplication, timeout handling while blocked and 25 checkpoint reopens with closed modeled handles. Linux CI retains source hashes and logs. Physical SD, BLE, panel and power cuts remain open.
 - [x] Shared CSV tail recovery now scans backwards in 512-byte blocks. Eight previous-source controls exceed stdio seek budgets; a 64 KiB partial tail drops from 65,538 seeks to 130 with the same retained prefix. All 32 native byte/boundary/fault cases, original storage checks and five MQTT/HTTP logging cases pass with closed modeled handles. The unflashed firmware builds with unchanged configuration. Linux CI retains source hashes and compilation/result logs; physical SD timing and power cuts remain open.
 - [x] Built-in SD paths, timestamps, units, 8.3-safe collision names, `.TMP`, and `.BAK` behavior are documented.
 - [x] Persisted brightness choices, a dimmed two-minute screen, and configurable 5/10/30-minute or Never screen-off behavior are implemented.

docs/architecture.md

@@ -87,6 +87,8 @@ Notes uses a temporary PSRAM buffer to load an entire file of up to 65,535 bytes
 
 Ride history shows totals only after recovery, reading and closing the summary succeed; a missing index is valid empty history. Storage errors display an unavailable message and update the shared SD error. Appending checks known SD availability, directory creation and temporary-file removal before copying. A failed read chunk is rejected before writing, and the first read/write/open error survives source/destination close and temporary cleanup failures. The existing replacement helper still owns sync, backup, publication and rollback. Native tests verify exact file bytes and closed modeled handles; physical FAT/SD and panel behavior remain open.
 
+Ring's saved heart-rate timestamp becomes loaded only after incomplete-tail repair, seek/size checks, the expected bounded tail read and close succeed. Read or close errors leave the checkpoint unchanged and retryable. Its existing health timer retains queued samples and defers history requests until that load succeeds, then filters timestamps against the recovered checkpoint. Known SD unavailability blocks queue processing even when a checkpoint is cached; timeout state still advances before the storage gate. Appends preserve the first error through close and tail repair. Native files and controlled queue/timer/SD boundaries verify this behavior; physical SD, BLE and scheduling remain open.
+
 See [Storage and compatibility contract](compatibility.md) and [Data formats](data-formats.md).
 
 ## Network and radio

docs/data-formats.md

@@ -59,6 +59,8 @@ Heart-rate header:
 unix_time,local_time,bpm
 ```
 
+Ring loads its saved timestamp from the repaired final 512 bytes only after successful seek, size, complete read and close checks. A failed checkpoint load leaves queued samples pending and defers history requests until recovery; known SD unavailability also blocks a previously cached checkpoint. Directory and append failures preserve the first error through close and tail repair. An unconfirmed flush/sync/close can leave a complete row; a failed write or repair can retain an unfinished suffix. A retry can repeat a row whose persistence was unconfirmed, so consumers should use `unix_time` when handling duplicates. Native checks cover file bytes and controlled queue/error behavior; physical SD and BLE remain hardware checks.
+
 I2C watch header:
 
 ```text

docs/testing.md

@@ -13,6 +13,7 @@ Python 3 and Node.js are needed for the quick checks. The configuration checks a
 | Storage writes or incomplete CSV rows | [Core command reference](#core-command-reference), [CSV tail recovery](#csv-tail-recovery) and a firmware build. |
 | Notes loading, saving or lifecycle | [Notes storage and UI](#notes-storage-and-ui) and a firmware build. |
 | Ride history loading or summary appends | [Ride history storage](#ride-history-storage) and a firmware build. |
+| Ring heart-rate logging, saved timestamps or queue processing | [Ring history storage](#ring-history-storage) and a firmware build. |
 | Offline apps, SPI or I2C editor/result flows | [Offline and peripheral UI](#offline-and-peripheral-ui). |
 | Serial, saved serial logs or RTU Frames | [Serial and RTU UI](#serial-and-rtu-ui). |
 | Modbus TCP, NTP, Wake-on-LAN, UDP or HTTP fixtures | [Network fixtures and UI](#network-fixtures-and-ui). |
@@ -124,6 +125,18 @@ The preceding callbacks fail 24 controls, including misleading totals after reco
 
 Generated callbacks/storage source, full main/function/storage/header/test hashes and compile/result logs stay under `build/ride-history/`. Linux CI runs the fixture and retains these files as `ride-history-logs`, including diagnostics when compilation or execution fails. Its 50 cases are separate from the Windows native total.
 
+## Ring history storage
+
+```powershell
+python tools/test_ring_history.py --compiler 'C:/path/to/clang.exe'
+```
+
+This portable fixture compiles the three actual Ring append/checkpoint/health-timer callbacks extracted from `main/main.c` and the complete `storage_io.c`. Its 59 cases use native files to check exact headers/rows, missing/empty logs, bounded long-tail loads, incomplete-tail repair, seek/size/read/close failures, short reads without an error flag, full-count reads with an error flag, stored SD errors and directory/append/flush/sync/close/repair faults. Every case verifies exact file bytes with no remaining modeled file handles. Twenty-five checkpoint reopens retain the same bytes and close every read.
+
+The preceding callbacks fail 32 controls, including prematurely caching failed reads, consuming samples against an uncertain checkpoint, ignoring directory errors and losing seek/tell/header/row errors during close. Timer cases retain queued samples and stop history requests after failed loads, retry successfully without appending historical duplicates, clear recovered errors, retain a failed append for retry and update timeout state while storage is blocked. A cached checkpoint cannot bypass known SD unavailability. Missing I/O errno falls back to `EIO`, and an earlier successful seek cannot supply stale errno for a failed read. Sync/close errors can follow a complete append; repair failure can retain an unfinished suffix. These checks do not promise rollback, exactly-once logging or power-loss durability.
+
+Queue/timer and SD error APIs are controlled, time conversion uses UTC for independently expected rows, and no BLE traffic occurs. They do not run FreeRTOS/NimBLE or establish physical SD removal/full-card behavior, radio delivery, scheduling or panel rendering. Generated callbacks/storage source, full main/function/storage/header/test hashes and compile/result logs stay under `build/ring-history/`. Linux CI retains these files as `ring-history-logs`, including compiler/test failures; the 59 cases are separate from the Windows native total.
+
 ## OTA manifests
 
 ```powershell

main/main.c

@@ -1019,85 +1019,133 @@ static bool ring_hr_append_log(time_t sample_time, int heart_rate)
         errno = error ? error : ENODEV;
         return false;
     }
-    mkdir(HEALTH_PATH, 0775);
+    errno = 0;
+    if (mkdir(HEALTH_PATH, 0775) != 0 && errno != EEXIST) {
+        int error = errno ? errno : EIO;
+        sd_record_error(error);
+        errno = error;
+        return false;
+    }
     if (storage_repair_csv_tail(HEART_RATE_LOG) != 0) {
         int error = errno ? errno : EIO;
         sd_record_error(error);
         errno = error;
         return false;
     }
+    errno = 0;
     FILE *file = fopen(HEART_RATE_LOG, "ab+");
     if (!file) {
         int error = errno ? errno : EIO;
         sd_record_error(error);
         errno = error;
         return false;
     }
-    bool ok = fseek(file, 0, SEEK_END) == 0;
-    long size = ok ? ftell(file) : -1;
-    if (size < 0) ok = false;
-    if (ok && size == 0) ok = fputs("unix_time,local_time,bpm\n", file) >= 0;
+    int write_error = 0;
+    errno = 0;
+    if (fseek(file, 0, SEEK_END) != 0) write_error = errno ? errno : EIO;
+    long size = -1;
+    if (!write_error) {
+        errno = 0;
+        size = ftell(file);
+        if (size < 0) write_error = errno ? errno : EIO;
+    }
+    if (!write_error && size == 0) {
+        errno = 0;
+        if (fputs("unix_time,local_time,bpm\n", file) < 0) write_error = errno ? errno : EIO;
+    }
     struct tm local;
     char timestamp[24];
     localtime_r(&sample_time, &local);
     strftime(timestamp, sizeof(timestamp), "%Y-%m-%d %H:%M:%S", &local);
-    if (ok) ok = fprintf(file, "%lld,%s,%d\n", (long long)sample_time, timestamp, heart_rate) >= 0;
-    int write_error = 0;
-    if (ok && storage_sync_file(file) != 0) {
-        write_error = errno ? errno : EIO;
-        ok = false;
+    if (!write_error) {
+        errno = 0;
+        if (fprintf(file, "%lld,%s,%d\n", (long long)sample_time, timestamp, heart_rate) < 0)
+            write_error = errno ? errno : EIO;
     }
-    if (fclose(file) != 0) {
-        if (!write_error) write_error = errno ? errno : EIO;
-        ok = false;
+    if (!write_error) {
+        errno = 0;
+        if (storage_sync_file(file) != 0) write_error = errno ? errno : EIO;
     }
-    if (!ok) {
-        if (!write_error) write_error = errno ? errno : EIO;
+    errno = 0;
+    if (fclose(file) != 0 && !write_error) write_error = errno ? errno : EIO;
+    if (write_error) {
         storage_repair_csv_tail(HEART_RATE_LOG);
         sd_record_error(write_error);
         errno = write_error;
     }
-    return ok;
+    return write_error == 0;
 }
 
-static void ring_hr_load_last_saved(void)
+static bool ring_hr_load_last_saved(void)
 {
-    if (ring_hr_last_saved_loaded) return;
+    if (!sd_ready) {
+        int error = sd_error_snapshot();
+        errno = error ? error : ENODEV;
+        return false;
+    }
+    if (ring_hr_last_saved_loaded) return true;
     if (storage_repair_csv_tail(HEART_RATE_LOG) != 0) {
-        sd_record_error(errno ? errno : EIO);
-        return;
+        int error = errno ? errno : EIO;
+        sd_record_error(error);
+        errno = error;
+        return false;
     }
-    ring_hr_last_saved_loaded = true;
+    errno = 0;
     FILE *file = fopen(HEART_RATE_LOG, "rb");
-    if (!file) return;
+    if (!file) {
+        if (errno == ENOENT) {
+            ring_hr_last_saved_loaded = true;
+            return true;
+        }
+        int error = errno ? errno : EIO;
+        sd_record_error(error);
+        errno = error;
+        return false;
+    }
     char tail[513];
-    if (fseek(file, 0, SEEK_END) != 0) {
-        fclose(file);
-        return;
+    int error = 0;
+    errno = 0;
+    if (fseek(file, 0, SEEK_END) != 0) error = errno ? errno : EIO;
+    long end = 0, offset = 0;
+    if (!error) {
+        errno = 0;
+        end = ftell(file);
+        if (end < 0) error = errno ? errno : EIO;
+        else offset = end > (long)sizeof(tail) - 1 ? end - ((long)sizeof(tail) - 1) : 0;
     }
-    long end = ftell(file);
-    long offset = end > (long)sizeof(tail) - 1 ? end - ((long)sizeof(tail) - 1) : 0;
-    if (end < 0 || fseek(file, offset, SEEK_SET) != 0) {
-        fclose(file);
-        return;
+    if (!error) {
+        errno = 0;
+        if (fseek(file, offset, SEEK_SET) != 0) error = errno ? errno : EIO;
+    }
+    size_t length = 0;
+    if (!error) {
+        errno = 0;
+        length = fread(tail, 1, (size_t)(end - offset), file);
+        if (ferror(file) || length != (size_t)(end - offset)) error = errno ? errno : EIO;
+    }
+    errno = 0;
+    if (fclose(file) != 0 && !error) error = errno ? errno : EIO;
+    if (error) {
+        sd_record_error(error);
+        errno = error;
+        return false;
     }
-    size_t length = fread(tail, 1, sizeof(tail) - 1, file);
     tail[length] = '\0';
-    fclose(file);
     char *line = tail;
     if (offset) {
         line = strchr(line, '\n');
-        if (!line) return;
-        line++;
+        if (line) line++;
     }
     long long timestamp;
-    while (*line) {
+    while (line && *line) {
         if (sscanf(line, "%lld,", &timestamp) == 1 && timestamp > ring_hr_last_saved)
             ring_hr_last_saved = (time_t)timestamp;
         line = strchr(line, '\n');
         if (!line) break;
         line++;
     }
+    ring_hr_last_saved_loaded = true;
+    return true;
 }
 
 static int ring_request_history(void)
@@ -1161,7 +1209,12 @@ static void ring_health_tick(lv_timer_t *timer)
     }
     portEXIT_CRITICAL(&ring_lock);
 
-    ring_hr_load_last_saved();
+    if (!ring_hr_load_last_saved()) {
+        snprintf(ring_storage_error, sizeof(ring_storage_error),
+                 "Heart-rate log not saved: %s", strerror(errno));
+        return;
+    }
+    ring_storage_error[0] = '\0';
     ring_hr_sample_t sample;
     while (ring_hr_samples && xQueuePeek(ring_hr_samples, &sample, 0) == pdTRUE) {
         if (sample.timestamp > ring_hr_last_saved) {