fix: report incomplete Files directory listings

Author: DevanMetz

.github/workflows/release.yml

@@ -41,6 +41,8 @@ jobs:
         run: python3 tools/test_ebook_pages.py --compiler cc
       - name: Test Files text previews
         run: python3 tools/test_file_preview.py --compiler cc
+      - name: Test Files directories
+        run: python3 tools/test_files_directory.py --compiler cc
       - name: Test capture reader
         run: |
           cc -std=c11 -Wall -Wextra -Werror -pedantic -I main main/capture_data.c tests/capture_data_test.c -o "$RUNNER_TEMP/capture_data_test"
@@ -196,6 +198,20 @@ jobs:
           if-no-files-found: warn
           retention-days: 14
 
+      - name: Retain Files directory evidence
+        uses: actions/upload-artifact@v4
+        if: always()
+        with:
+          name: files-directory-logs
+          path: |
+            build/files-directory/compile.log
+            build/files-directory/result.log
+            build/files-directory/source.json
+            build/files-directory/files_directory.inc
+            build/files-directory/files_directory_config.inc
+          if-no-files-found: warn
+          retention-days: 14
+
   firmware:
     runs-on: ubuntu-latest
     container: espressif/idf:v5.4.2

ROADMAP.md

@@ -68,6 +68,7 @@ Phase 1 software progress is also underway:
 - [x] Ebook page reads publish their text and byte positions only after read/close success. Failed navigation retains the previous page for retry, including zero-byte and full-count read errors. Initial navigation is disabled until a page loads; callbacks reject disabled/offscreen actions and invalid positions. All 44 native file/state/fault checks pass, including 25 navigation cycles with one reused buffer and closed handles. Thirty-one preceding-callback controls fail; three new unsafe-boundary guards are tested only with the new callbacks. Linux CI retains source hashes, initialization source and logs. Physical SD/panel and power behavior remain open.
 - [x] Recognized Scope/I2C and saved serial CSV loads require a successful close before showing a graph or row-copy controls. Failed or empty capture loads release their row buffer immediately. All 36 real LVGL/native-file checks pass; the preceding viewers fail 15 controls. Checks cover both row limits, I2C gaps, read/seek/close and allocation faults, unchanged file/clipboard bytes and 25 success/error/reopen cycles per viewer with stable memory in the unchanged 96 KiB pool. Existing serial-copy/inspector checks pass. Windows CI retains complete viewer/parser sources and logs; physical SD/panel behavior remains open.
 - [x] Files text previews require a successful read and close before showing their existing 4,095-byte prefix. Failed operations show the first error, using EIO when errno is absent; SD-path failures reach the existing error callback. All 36 native file/fault cases pass with unchanged bytes and closed handles, including read/close errors, retry, routing and 25 cycles. The preceding callback fails 29 controls: 23 reproduce ignored read/close failures or retry cycles, and six concern open-error details/reporting. UI, CSV probes and logical mount paths are controlled; Linux CI retains source hashes and logs. Physical storage/panel checks remain open.
+- [x] Files directory browsing reports open/read/close errors, marks interrupted listings incomplete, and discloses its entry cap and skipped long paths. Fresh errno checks distinguish normal EOF from read errors and preserve the first failure through close. All 31 native directory/file cases pass with unchanged bytes, valid callback path ownership and closed handles, including 25 success/error/reopen cycles. The previous callback fails 21 controls: 14 concern silent read/close errors or recovery, three disclose limits and four add open-error details/reporting. UI, mount paths and faults are controlled; Linux CI retains source hashes and logs. SDK mounts, physical SD and panel checks remain open.
 - [x] Shared CSV tail recovery now scans backwards in 512-byte blocks. Eight previous-source controls exceed stdio seek budgets; a 64 KiB partial tail drops from 65,538 seeks to 130 with the same retained prefix. All 32 native byte/boundary/fault cases, original storage checks and five MQTT/HTTP logging cases pass with closed modeled handles. The unflashed firmware builds with unchanged configuration. Linux CI retains source hashes and compilation/result logs; physical SD timing and power cuts remain open.
 - [x] Built-in SD paths, timestamps, units, 8.3-safe collision names, `.TMP`, and `.BAK` behavior are documented.
 - [x] Persisted brightness choices, a dimmed two-minute screen, and configurable 5/10/30-minute or Never screen-off behavior are implemented.

docs/architecture.md

@@ -95,6 +95,8 @@ The Scope/I2C capture and saved serial CSV viewers require a successful close be
 
 Files dispatches supported CSVs to those viewers before its text fallback. The fallback reads at most 4,095 bytes into its existing static buffer and publishes them only after checking the stream and closing successfully. Failed open/read/close operations display the first error; missing errno becomes EIO. Error reporting to the shared SD callback is scoped to the exact `/sdcard/` prefix. Native files and controlled UI/probe/path boundaries check this callback; physical storage and panel behavior remain open.
 
+Files directory browsing resets errno before each directory read so per-entry `stat` errors cannot masquerade as an enumeration error at EOF. A failed read or close keeps the visited entries visible with an explicit incomplete-list warning, preserving the first error. Entry capacity and skipped long paths are disclosed. The existing 64 path slots include a parent-navigation slot when needed, and each path remains bounded to 256 bytes. SD root and descendant errors reach the shared callback. Native directory/file checks verify these transitions and callback path ownership; UI, mount paths and error injection are controlled.
+
 See [Storage and compatibility contract](compatibility.md) and [Data formats](data-formats.md).
 
 ## Network and radio

docs/data-formats.md

@@ -33,6 +33,8 @@ After a supported capture or serial header is recognized, seek/read/close failur
 
 The fallback text preview displays up to the first 4,095 source bytes after a successful read and close. A failed open/read/close displays the first error instead of an unconfirmed prefix, including zero-byte or full-count reads with the stream error flag set. An error-free empty read remains an empty preview. Paths beginning `/sdcard/` forward failures to the existing SD error callback. The preview keeps its existing text/NUL behavior and never writes the file. Native checks use controlled UI/probes and logical mount paths; physical storage and panel checks remain open.
 
+Directory browsing uses 64 stored path slots of 256 bytes each, including a parent entry when needed. It shows an entry-limit warning when those slots fill and counts paths skipped for length. Read/close failures mark the visited listing incomplete; reopening can retry. Open/read/close errors retain their first cause and are forwarded to the SD callback for `/sdcard` and its descendants. A null directory read with zero errno is treated as normal EOF. Native checks verify file bytes and directory handles under controlled faults; SDK mounts and physical SD/panel behavior remain open.
+
 Choose a record from the current page and tap **COPY ROW** to put its original 1-128 bytes on the shared RAM clipboard. The selector and timeline use the same record numbers: positions among loaded valid rows, not physical CSV line numbers. Page/filter changes select the first matching record on the new page; changing Hex/ASCII preserves the selection. Rows of 129-256 bytes remain viewable but cannot be partially copied. Empty filters and failed copies preserve the clipboard. Record bytes can then be pasted into Byte Lab or RTU Frames; a logged RX/TX chunk may split or combine protocol frames, so a record is not a guarantee of a complete message. Copy changes neither the CSV nor the displayed byte values.
 
 ## CSV conventions

docs/testing.md

@@ -17,6 +17,7 @@ Python 3 and Node.js are needed for the quick checks. The configuration checks a
 | Ebook page loading or navigation | [Ebook pages](#ebook-pages) and a firmware build. |
 | Scope/I2C capture or saved serial CSV viewers | [CSV viewer storage and UI](#csv-viewer-storage-and-ui), existing serial-copy checks and a firmware build. |
 | Files fallback text previews | [Files text previews](#files-text-previews) and a firmware build. |
+| Files directory browsing | [Files directories](#files-directories) and a firmware build. |
 | Offline apps, SPI or I2C editor/result flows | [Offline and peripheral UI](#offline-and-peripheral-ui). |
 | Serial, saved serial logs or RTU Frames | [Serial and RTU UI](#serial-and-rtu-ui). |
 | Modbus TCP, NTP, Wake-on-LAN, UDP or HTTP fixtures | [Network fixtures and UI](#network-fixtures-and-ui). |
@@ -176,6 +177,18 @@ UI, CSV viewer probes and the SD error callback are controlled. Logical internal
 
 Generated callback/config source, full main/function/test hashes and compile/runtime logs stay under `build/file-preview/`. Linux CI retains `file-preview-logs`, including available compiler/test failure diagnostics. These 36 cases are separate from the Windows native total.
 
+## Files directories
+
+```powershell
+python tools/test_files_directory.py --compiler 'C:/path/to/clang.exe'
+```
+
+The portable runner compiles the actual `show_files` callback. Its 31 cases exercise native directory enumeration and `stat`, file/directory classification, explicit dot entries, empty directories, mount selection, entry/path limits, open/read/close faults, first-error preservation, stale errno, callback path ownership, reopen recovery and 25 success/error/success cycles. Windows Find APIs and POSIX directory APIs provide the host entries; native dot entries are normalized before separate dot-injection checks. Every case verifies unchanged file bytes and closed directory handles. The previous callback fails 21 controls: 14 silent read/close/recovery cases, three limit disclosures and four open-error details/reporting cases.
+
+UI and SD callbacks, logical mount paths and directory/stat failures are controlled. A null read with zero errno remains EOF; the fixture does not claim to detect an unreported backend error. The SD callback's filtering and SDK mount implementation are not executed. These checks do not establish LVGL allocation, panel rendering, physical SD, latency or power-loss behavior. The existing 64 path slots include parent navigation; paths stay bounded to 256 bytes.
+
+Generated callback/config source, full main/function/test hashes and compile/result logs stay under `build/files-directory/`. Linux CI retains `files-directory-logs`, including available failure diagnostics. These 31 checks are separate from the Windows native total.
+
 ## OTA manifests
 
 ```powershell

main/main.c

@@ -4311,19 +4311,33 @@ static void show_files(const char *path)
         lv_obj_add_event_cb(up, home_clicked, LV_EVENT_CLICKED, NULL);
     }
 
+    bool sd_directory = !strcmp(path, SD_PATH) || !strncmp(path, SD_PATH "/", sizeof(SD_PATH));
+    errno = 0;
     DIR *dir = opendir(path);
     if (!dir) {
-        lv_list_add_text(list, "Could not open directory");
+        int error = errno ? errno : EIO;
+        if (sd_directory) sd_record_error(error);
+        char message[96];
+        snprintf(message, sizeof(message), "Could not open directory: %s", strerror(error));
+        lv_list_add_text(list, message);
         return;
     }
-    struct dirent *entry;
-    while (file_path_count < 64 && (entry = readdir(dir))) {
+    int error = 0;
+    size_t skipped = 0;
+    while (file_path_count < sizeof(file_paths) / sizeof(file_paths[0])) {
+        errno = 0;
+        struct dirent *entry = readdir(dir);
+        if (!entry) {
+            error = errno;
+            break;
+        }
         if (!strcmp(entry->d_name, ".") || !strcmp(entry->d_name, "..")) continue;
         char *full = file_paths[file_path_count++];
         size_t path_len = strlen(path);
         size_t name_len = strlen(entry->d_name);
         if (path_len + name_len + 2 > sizeof(file_paths[0])) {
             file_path_count--;
+            skipped++;
             continue;
         }
         memcpy(full, path, path_len);
@@ -4334,7 +4348,21 @@ static void show_files(const char *path)
         lv_obj_t *item = lv_list_add_button(list, is_dir ? LV_SYMBOL_DIRECTORY : LV_SYMBOL_FILE, entry->d_name);
         lv_obj_add_event_cb(item, file_clicked, LV_EVENT_CLICKED, full);
     }
-    closedir(dir);
+    errno = 0;
+    if (closedir(dir) != 0 && !error) error = errno ? errno : EIO;
+    if (error) {
+        if (sd_directory) sd_record_error(error);
+        char message[96];
+        snprintf(message, sizeof(message), "Directory listing incomplete: %s", strerror(error));
+        lv_list_add_text(list, message);
+    }
+    if (file_path_count == sizeof(file_paths) / sizeof(file_paths[0]))
+        lv_list_add_text(list, "Directory entry limit reached");
+    if (skipped) {
+        char message[96];
+        snprintf(message, sizeof(message), "Skipped %u paths that exceed the path limit", (unsigned)skipped);
+        lv_list_add_text(list, message);
+    }
 }
 
 static void files_clicked(lv_event_t *event)