fix: preserve ride history storage errors

Author: DevanMetz

.github/workflows/release.yml

@@ -33,6 +33,8 @@ jobs:
         run: python3 tools/test_storage_csv_tail.py --compiler cc
       - name: Test Notes storage
         run: python3 tools/test_notes_io.py --compiler cc
+      - name: Test ride history storage
+        run: python3 tools/test_ride_history.py --compiler cc
       - name: Test capture reader
         run: |
           cc -std=c11 -Wall -Wextra -Werror -pedantic -I main main/capture_data.c tests/capture_data_test.c -o "$RUNNER_TEMP/capture_data_test"
@@ -131,6 +133,20 @@ jobs:
           if-no-files-found: warn
           retention-days: 14
 
+      - name: Retain ride history test evidence
+        uses: actions/upload-artifact@v4
+        if: always()
+        with:
+          name: ride-history-logs
+          path: |
+            build/ride-history/compile.log
+            build/ride-history/result.log
+            build/ride-history/source.json
+            build/ride-history/ride_history.inc
+            build/ride-history/storage_source.inc
+          if-no-files-found: warn
+          retention-days: 14
+
   firmware:
     runs-on: ubuntu-latest
     container: espressif/idf:v5.4.2

ROADMAP.md

@@ -63,6 +63,7 @@ Phase 1 software progress is also underway:
 - [x] System reports the firmware/commit, reset reason, uptime, panel, heap, storage capacity, current OTA state, and persistent last OTA result.
 - [x] Notes and ride summaries use recoverable backup replacement; rides publish only after durable sync, and health logs repair incomplete final rows.
 - [x] Notes loads complete files up to 65,535 bytes, replacing a silent 2,047-byte prefix. Recovery/read/close errors, oversized files and NUL data disable Save; byte-length and SD checks run before storage changes. Its launcher leave hook clears editor ownership. Four preceding-source controls reproduce full-note truncation; 36 native file/fault cases and five real LVGL cases pass, including maximum ASCII/UTF-8 notes and 25 reopens without retained resources in the unchanged 96 KiB pool. Linux and Windows CI retain separate source-bound evidence. Physical SD/panel and power cuts remain open.
+- [x] Ride history reports recovery/open/read/close errors as unavailable instead of showing zero or partial totals. Summary appends check SD availability and preparation errors, reject failed read chunks before copying, and retain the first error through cleanup. All 50 native file/fault cases pass; the preceding callbacks fail 24 controls. Exact final/backup/temporary bytes and 25 reopens are checked with closed modeled handles. Linux CI retains source hashes and logs, and the unflashed firmware builds with unchanged configuration. Physical SD/panel and power cuts remain open.
 - [x] Shared CSV tail recovery now scans backwards in 512-byte blocks. Eight previous-source controls exceed stdio seek budgets; a 64 KiB partial tail drops from 65,538 seeks to 130 with the same retained prefix. All 32 native byte/boundary/fault cases, original storage checks and five MQTT/HTTP logging cases pass with closed modeled handles. The unflashed firmware builds with unchanged configuration. Linux CI retains source hashes and compilation/result logs; physical SD timing and power cuts remain open.
 - [x] Built-in SD paths, timestamps, units, 8.3-safe collision names, `.TMP`, and `.BAK` behavior are documented.
 - [x] Persisted brightness choices, a dimmed two-minute screen, and configurable 5/10/30-minute or Never screen-off behavior are implemented.

docs/architecture.md

@@ -85,6 +85,8 @@ CSV tail repair uses a 512-byte scratch buffer and scans backwards by blocks bef
 
 Notes uses a temporary PSRAM buffer to load an entire file of up to 65,535 bytes and frees that buffer after LVGL copies the text. It applies the editor's character limit after loading, using LVGL's bulk text path. Recovery/read/close failures, oversized files and NUL data disable editing and Save; the save callback rechecks ownership, SD availability and byte length before changing paths. The launcher leave hook clears its editor pointer and permission before content deletion. The existing replacement helper still syncs/closes `NOTE.TMP`, retains the previous complete generation in `NOTE.BAK`, and reports publication failures. Real LVGL checks use the unchanged 96 KiB pool; physical SD and panel behavior remain open.
 
+Ride history shows totals only after recovery, reading and closing the summary succeed; a missing index is valid empty history. Storage errors display an unavailable message and update the shared SD error. Appending checks known SD availability, directory creation and temporary-file removal before copying. A failed read chunk is rejected before writing, and the first read/write/open error survives source/destination close and temporary cleanup failures. The existing replacement helper still owns sync, backup, publication and rollback. Native tests verify exact file bytes and closed modeled handles; physical FAT/SD and panel behavior remain open.
+
 See [Storage and compatibility contract](compatibility.md) and [Data formats](data-formats.md).
 
 ## Network and radio

docs/data-formats.md

@@ -51,6 +51,8 @@ Ride summary header:
 start_unix,duration_s,distance_km,work_kj,avg_power_w,max_power_w,avg_hr,max_hr
 ```
 
+History totals require successful backup recovery, open, read and close. A missing index shows empty history; storage errors show `History unavailable` instead of zero or partial totals. Summary appends check SD availability and preparation errors, reject failed read chunks before copying, and preserve the first error through cleanup. Failed cleanup can leave an unpublished `SUMMARY.TMP`; publication or rollback errors can also retain the temporary or backup generation. The individual ride CSV remains authoritative. Native files and controlled errors verify these boundaries; physical SD and power-loss behavior remain hardware checks.
+
 Heart-rate header:
 
 ```text

docs/testing.md

@@ -12,6 +12,7 @@ Python 3 and Node.js are needed for the quick checks. The configuration checks a
 | Relay, parsers, calculations, clipboard or file formats | [Core command reference](#core-command-reference); choose the affected module and any independent reference test. |
 | Storage writes or incomplete CSV rows | [Core command reference](#core-command-reference), [CSV tail recovery](#csv-tail-recovery) and a firmware build. |
 | Notes loading, saving or lifecycle | [Notes storage and UI](#notes-storage-and-ui) and a firmware build. |
+| Ride history loading or summary appends | [Ride history storage](#ride-history-storage) and a firmware build. |
 | Offline apps, SPI or I2C editor/result flows | [Offline and peripheral UI](#offline-and-peripheral-ui). |
 | Serial, saved serial logs or RTU Frames | [Serial and RTU UI](#serial-and-rtu-ui). |
 | Modbus TCP, NTP, Wake-on-LAN, UDP or HTTP fixtures | [Network fixtures and UI](#network-fixtures-and-ui). |
@@ -111,6 +112,18 @@ The optional real LVGL mode runs five additional cases against the cached pinned
 
 Generated callbacks/config/storage source, full main/function/storage/header/test hashes and compile/result logs stay under the selected output directory. Linux CI retains the 36-case evidence as `notes-io-logs`. Windows CI runs the five real LVGL cases after the existing seven suites and retains separate `notes-ui-logs`; those five add to the prior 815 native checks. Both artifacts retain diagnostics when their test step fails.
 
+## Ride history storage
+
+```powershell
+python tools/test_ride_history.py --compiler 'C:/path/to/clang.exe'
+```
+
+This portable fixture compiles the two actual ride-history callbacks extracted from `main/main.c` and the complete `storage_io.c`. Its 50 cases check missing/empty indexes, exact aggregate totals and appended rows, backup recovery, unavailable SD, preparation/open/read/write/flush/sync/close/publication/rollback errors and 25 history reopens. Every case independently checks final, backup and temporary bytes with no remaining modeled file handles. UI and I/O error boundaries are controlled; the files use the native filesystem.
+
+The preceding callbacks fail 24 controls, including misleading totals after recovery/read/close errors, ignored SD/preparation errors and lost first errors during cleanup. Compounded read/copy/open failures retain their original error when later closes or removal fail. Missing read/write errno falls back to `EIO`; a successful earlier history read cannot supply a stale errno for a later failure. Multi-block and full-count read-error cases reject the failed chunk before copying. Cleanup failure can retain an unpublished partial temporary file, and publication/rollback failures can retain recovery generations. These checks do not validate every CSV field or prove physical FAT/SD latency, removal, power cuts or panel rendering.
+
+Generated callbacks/storage source, full main/function/storage/header/test hashes and compile/result logs stay under `build/ride-history/`. Linux CI runs the fixture and retains these files as `ride-history-logs`, including diagnostics when compilation or execution fails. Its 50 cases are separate from the Windows native total.
+
 ## OTA manifests
 
 ```powershell

main/main.c

@@ -6509,8 +6509,15 @@ static void govee_clicked(lv_event_t *event)
 static void ride_load_history(void)
 {
     if (!ride_history) return;
-    storage_recover_replace(SD_PATH "/RIDES/SUMMARY.CSV", SD_PATH "/RIDES/SUMMARY.BAK");
-    FILE *file = fopen(SD_PATH "/RIDES/SUMMARY.CSV", "rb");
+    int error = sd_ready ? 0 : sd_error_snapshot();
+    if (!sd_ready && !error) error = ENODEV;
+    if (!error && storage_recover_replace(SD_PATH "/RIDES/SUMMARY.CSV", SD_PATH "/RIDES/SUMMARY.BAK") != 0)
+        error = errno ? errno : EIO;
+    FILE *file = NULL;
+    if (!error) {
+        file = fopen(SD_PATH "/RIDES/SUMMARY.CSV", "rb");
+        if (!file && errno != ENOENT) error = errno ? errno : EIO;
+    }
     int rides = 0, best_power = 0;
     long long start;
     unsigned duration;
@@ -6520,15 +6527,23 @@ static void ride_load_history(void)
     float total_distance = 0;
     char line[160];
     if (file) {
-        while (fgets(line, sizeof(line), file)) {
+        for (;;) {
+            errno = 0;
+            if (!fgets(line, sizeof(line), file)) break;
             if (sscanf(line, "%lld,%u,%f,%f,%d,%d,%d,%d", &start, &duration, &distance,
                        &work, &average_power, &maximum_power, &average_hr, &maximum_hr) != 8) continue;
             rides++;
             total_seconds += duration;
             total_distance += distance;
             if (maximum_power > best_power) best_power = maximum_power;
         }
-        fclose(file);
+        if (ferror(file)) error = errno ? errno : EIO;
+        if (fclose(file) != 0 && !error) error = errno ? errno : EIO;
+    }
+    if (error) {
+        sd_record_error(error);
+        lv_label_set_text_fmt(ride_history, "History unavailable: %s", strerror(error));
+        return;
     }
     char history_text[160];
     snprintf(history_text, sizeof(history_text), "History: %d rides  |  %.1f mi  |  %.1f hr  |  best %d W",
@@ -6538,47 +6553,59 @@ static void ride_load_history(void)
 
 static bool ride_append_summary(unsigned duration)
 {
-    mkdir(SD_PATH "/RIDES", 0775);
+    if (!sd_ready) {
+        int error = sd_error_snapshot();
+        errno = error ? error : ENODEV;
+        return false;
+    }
+    if (mkdir(SD_PATH "/RIDES", 0775) != 0 && errno != EEXIST) return false;
     const char *temporary_path = SD_PATH "/RIDES/SUMMARY.TMP";
     const char *final_path = SD_PATH "/RIDES/SUMMARY.CSV";
     const char *backup_path = SD_PATH "/RIDES/SUMMARY.BAK";
     if (storage_recover_replace(final_path, backup_path) != 0) return false;
-    remove(temporary_path);
+    if (remove(temporary_path) != 0 && errno != ENOENT) return false;
     FILE *source = fopen(final_path, "rb");
     if (!source && errno != ENOENT) return false;
     FILE *file = fopen(temporary_path, "wb");
     if (!file) {
+        int error = errno ? errno : EIO;
         if (source) fclose(source);
+        errno = error;
         return false;
     }
-    bool ok = true;
+    int error = 0;
     bool empty = true;
     int last = '\n';
     char buffer[512];
-    while (source && ok) {
+    while (source && !error) {
+        errno = 0;
         size_t length = fread(buffer, 1, sizeof(buffer), source);
+        if (ferror(source)) {
+            error = errno ? errno : EIO;
+            break;
+        }
         if (length) {
             empty = false;
             last = (unsigned char)buffer[length - 1];
-            ok = fwrite(buffer, 1, length, file) == length;
+            if (fwrite(buffer, 1, length, file) != length) error = errno ? errno : EIO;
         }
-        if (length < sizeof(buffer)) {
-            if (ferror(source)) ok = false;
-            break;
-        }
-    }
-    if (source && fclose(source) != 0) ok = false;
-    if (ok && !empty && last != '\n') ok = fputc('\n', file) != EOF;
-    if (ok && empty)
-        ok = fputs("start_unix,duration_s,distance_km,work_kj,avg_power_w,max_power_w,avg_hr,max_hr\n", file) >= 0;
-    if (ok)
-        ok = fprintf(file, "%lld,%u,%.3f,%.1f,%d,%d,%d,%d\n", (long long)ride_started_at, duration,
+        if (length < sizeof(buffer)) break;
+    }
+    if (source && fclose(source) != 0 && !error) error = errno ? errno : EIO;
+    if (!error && !empty && last != '\n' && fputc('\n', file) == EOF) error = errno ? errno : EIO;
+    if (!error && empty &&
+        fputs("start_unix,duration_s,distance_km,work_kj,avg_power_w,max_power_w,avg_hr,max_hr\n", file) < 0)
+        error = errno ? errno : EIO;
+    if (!error &&
+        fprintf(file, "%lld,%u,%.3f,%.1f,%d,%d,%d,%d\n", (long long)ride_started_at, duration,
                      ride_distance_km, ride_work_kj,
                      ride_power_samples ? (int)(ride_power_sum / ride_power_samples) : 0, ride_max_power,
-                     ride_hr_samples_count ? (int)(ride_hr_sum / ride_hr_samples_count) : 0, ride_max_hr) >= 0;
-    if (!ok) {
+                     ride_hr_samples_count ? (int)(ride_hr_sum / ride_hr_samples_count) : 0, ride_max_hr) < 0)
+        error = errno ? errno : EIO;
+    if (error) {
         fclose(file);
         remove(temporary_path);
+        errno = error;
         return false;
     }
     return storage_commit_replace_file(&file, temporary_path, final_path, backup_path) == 0;