fix: PWD_USER consistency, CQ error detection, staging disk-space guards

- PWD_USER now derives as SUDO_UID:SUDO_GID (invoking user's uid and
  primary gid) — the exact same uid:gid convention setup.sh writes — so a
  restore never silently changes how an install is owned. Drops the
  docker-GID pairing, which was self-consistent but diverged from the
  rest of the project. Root-shell (UID 0) guard retained.
- CQ replay now detects failures: the influx 1.x CLI prints statement
  errors to stdout and exits 0 regardless, so the previous exit-code
  check reported success even if every CREATE failed. New replay_cqs
  helper captures output, filters harmless 'already exists' responses,
  greps for ERR, and surfaces the first errors to the user.
- Staging disk-space checks: restore.sh warns (with sizes and a TMPDIR
  override hint) if the staging filesystem can't hold ~6x the compressed
  archive; backup.sh aborts before copying a snapshot that won't fit
  (protects RAM-backed /tmp from multi-GB datasets, cron-safe: no prompt).
- Docs: README large-dataset/TMPDIR note, updated restore step list,
  RELEASE.md v5.3.0 entry updated to match final behavior (also folds in
  the previously-unreleased jasonacox#840 datasource UID note post-rebase).

Author: jasonacox

RELEASE.md

@@ -9,13 +9,14 @@
 ### New Features
 
 * **Restore script** — new `backups/restore.sh.sample`, an automated companion to `backup.sh.sample`, based on `restore_v16a.sh` by **@JonMurphy** ([#836](https://github.com/jasonacox/Powerwall-Dashboard/issues/836)). One command restores a backup archive on the same machine or migrates to a new one:
-  - Auto-detects the Powerwall-Dashboard directory by locating `compose-dash.sh`
+  - Auto-detects the Powerwall-Dashboard directory from the script's own location
+  - Checks staging disk space before extracting (warns and offers `TMPDIR` override — protects RAM-backed `/tmp` from multi-GB archives)
   - Non-destructive — moves existing InfluxDB `data`/`meta`/`wal` and `grafana.db` aside as `.pre-restore.<timestamp>` copies (rollback path) instead of deleting
-  - Restores InfluxDB from the `influxd backup -portable` snapshot into a clean instance, then re-creates continuous queries from `influxdb.sql` (`influxd restore -portable` does not reliably restore CQs — a known InfluxDB 1.x limitation)
-  - Restores Grafana database and provisioning files with correct ownership, deriving `PWD_USER` from live filesystem state (`compose-dash.sh` owner + docker group GID)
+  - Restores InfluxDB from the `influxd backup -portable` snapshot into a clean instance, then re-creates continuous queries from the archive's `continuous_queries.txt` (live CQ state, including customizations) with `influxdb.sql` as fallback — `influxd restore -portable` does not reliably restore CQs (known InfluxDB 1.x limitation). CQ replay output is checked for errors (the influx CLI exits 0 even on failed statements).
+  - Restores Grafana database and provisioning files with correct ownership, deriving `PWD_USER` as the invoking user's `uid:gid` (`SUDO_UID:SUDO_GID`) — the same convention `setup.sh` uses — with a guard against running from a root shell
   - Restores user configuration files (`*.env`, `telegraf.local`, etc.) while **skipping git-managed project files** (`powerwall.yml`, `telegraf.conf`, `influxdb.conf`, `VERSION`) so an older backup can never downgrade the stack or break future `git pull`/upgrades
   - Finishes with `compose-dash.sh up -d` so containers are **recreated** and restored settings (including the rewritten `PWD_USER`) actually take effect
-* **Backup script** — `backup.sh.sample` now auto-detects the dashboard directory (no more editing `DASHBOARD=` by hand) and exports `SHOW CONTINUOUS QUERIES` output into the archive as a safety net.
+* **Backup script** — `backup.sh.sample` now auto-detects the dashboard directory (no more editing `DASHBOARD=` by hand), exports live continuous queries into the archive as a safety net, and aborts with a clear message if the staging area can't hold the snapshot (with `TMPDIR` override).
 
 ### Documentation
 

backups/README.md

@@ -45,6 +45,8 @@ The following shows an example of how to set up automated backups (see backup.sh
 2. Make the script executable with `chmod +x backup.sh` (the script auto-detects the dashboard location by finding `compose-dash.sh`)
 3. Add to crontab for daily backups: `0 2 * * * /home/user/Powerwall-Dashboard/backups/backup.sh`
 
+> **Large datasets:** both `backup.sh` and `restore.sh` stage data in a temporary directory (`mktemp -d`, usually under `/tmp`). If your InfluxDB history is large (multi-GB) and `/tmp` is a RAM-backed tmpfs, staging there can exhaust memory. Both scripts check available space first — backup aborts and restore warns — and you can point staging at a disk with more room: `sudo TMPDIR=/path/with/space ./backup.sh`
+
 ## Backup Script Example
 
 ```bash
@@ -55,8 +57,8 @@ if [ "$EUID" -ne 0 ]
   exit
 fi
 
-# Dashboard location is auto-detected (script-relative, with a fallback
-# search for compose-dash.sh) - see backup.sh.sample
+# Dashboard location is auto-detected from the script's own location
+# (must live in Powerwall-Dashboard/backups/) - see backup.sh.sample
 BACKUP_FOLDER="${DASHBOARD}/backups"          # Destination folder for backups
 KEEP="5"                                      # Days to keep backup
 
@@ -89,11 +91,12 @@ A companion `restore.sh.sample` is provided to automate the restore process. It
 
 The restore script will:
 1. **Auto-detect** the Powerwall-Dashboard directory by locating `compose-dash.sh`
-2. **Stop all containers** before touching data
-3. **Restore InfluxDB** from the `influxd backup -portable` snapshot, moving existing data aside first (not deleted — you get a rollback path)
-4. **Restore Grafana** database and provisioning files with correct ownership
-5. **Restore configuration files**, rewriting `PWD_USER` in `compose.env` to match this host's actual user and docker group. Project files managed by git (`powerwall.yml`, `telegraf.conf`, `influxdb.conf`, `VERSION`) are kept in the archive for reference but are NOT restored over the current checkout — this prevents an older backup from downgrading the stack or breaking future upgrades.
-6. **Recreate the stack** (`compose-dash.sh up -d`, so restored settings take effect) and print a list of pre-restore backup paths to clean up once confirmed
+2. **Check staging disk space** and warn before extracting a large archive into a location that can't hold it (use `sudo TMPDIR=/path/with/space ./restore.sh` to relocate staging)
+3. **Stop all containers** before touching data
+4. **Restore InfluxDB** from the `influxd backup -portable` snapshot, moving existing data aside first (not deleted — you get a rollback path), then re-create continuous queries from the archive (with `influxdb.sql` as fallback)
+5. **Restore Grafana** database and provisioning files with correct ownership
+6. **Restore configuration files**, rewriting `PWD_USER` in `compose.env` to match this host's actual user and primary group (same `uid:gid` convention as `setup.sh`). Project files managed by git (`powerwall.yml`, `telegraf.conf`, `influxdb.conf`, `VERSION`) are kept in the archive for reference but are NOT restored over the current checkout — this prevents an older backup from downgrading the stack or breaking future upgrades.
+7. **Recreate the stack** (`compose-dash.sh up -d`, so restored settings take effect) and print a list of pre-restore backup paths to clean up once confirmed
 
 ### Manual restore from a backup script archive
 

backups/backup.sh.sample

@@ -62,6 +62,26 @@ mkdir -p influxdb/backups
 chmod g+w influxdb/backups
 docker exec influxdb influxd backup -portable -database powerwall /var/lib/influxdb/backups
 
+# ── Check staging disk space ──────────────────────────────────────────
+# The snapshot is copied into ${STAGING} (usually under /tmp) before being
+# archived. On systems where /tmp is a RAM-backed tmpfs, a multi-GB
+# snapshot can exhaust memory. Abort (rather than risk OOM) if the staging
+# filesystem can't hold the snapshot plus 10% headroom. Override the
+# staging location with: sudo TMPDIR=/path/with/space ./backup.sh
+SNAPSHOT_KB=$(du -sk "${DASHBOARD}/influxdb/backups" | cut -f1)
+NEEDED_KB=$((SNAPSHOT_KB + SNAPSHOT_KB / 10))
+AVAIL_KB=$(df -Pk "${STAGING}" | awk 'NR==2 {print $4}')
+if [ -n "${AVAIL_KB}" ] && [ "${AVAIL_KB}" -lt "${NEEDED_KB}" ]; then
+  echo "ERROR: Staging area ${STAGING} does not have enough free space."
+  echo "  Snapshot size: $((SNAPSHOT_KB / 1024)) MB"
+  echo "  Available:     $((AVAIL_KB / 1024)) MB"
+  echo "If /tmp is a RAM-backed tmpfs, staging there can exhaust memory."
+  echo "Point staging at a disk with more space and re-run:"
+  echo "  sudo TMPDIR=/path/with/space $0"
+  rm -rf "${DASHBOARD}/influxdb/backups"/*
+  exit 1
+fi
+
 # Copy the snapshot into staging (the backup files are the consistent copy)
 mkdir -p "${STAGING}/influxdb"
 cp -a "${DASHBOARD}/influxdb/backups/." "${STAGING}/influxdb/"

backups/restore.sh.sample

@@ -37,10 +37,10 @@ if [ ! -d "${BACKUP_FOLDER}" ]; then
 fi
 
 # ── Determine this host's correct owner:group for container files ─────
-# Under sudo, SUDO_UID gives the invoking user (not root). Pair that with
-# the docker GID so container files are owned by the real user + docker group.
-# Falls back to id -u/id -g when not under sudo.
-DOCKER_GID=$(getent group docker 2>/dev/null | cut -d: -f3)
+# Under sudo, SUDO_UID/SUDO_GID give the invoking user (not root).
+# This matches setup.sh, which derives PWD_USER as "$(id -u):$(id -g)" of
+# the installing user — keeping the same uid:primary-gid convention means a
+# restore never changes how the install is owned.
 HOST_UID="${SUDO_UID:-$(id -u)}"
 HOST_GID="${SUDO_GID:-$(id -g)}"
 
@@ -54,7 +54,7 @@ if [ "${HOST_UID}" -eq 0 ] 2>/dev/null; then
   exit 1
 fi
 
-PWD_USER="${HOST_UID}:${DOCKER_GID:-${HOST_GID}}"
+PWD_USER="${HOST_UID}:${HOST_GID}"
 echo "Detected owner:group for this host: ${PWD_USER}"
 
 # ── Determine which archive to restore ────────────────────────────────
@@ -82,6 +82,31 @@ cleanup() {
 }
 trap cleanup EXIT
 
+# ── Check staging disk space ─────────────────────────────────────────────
+# The full archive is extracted into ${STAGING} (usually under /tmp). On
+# systems where /tmp is a RAM-backed tmpfs, a multi-GB archive can exhaust
+# memory. xz compresses time-series data heavily, so require 6x the archive
+# size as a conservative uncompressed estimate. Override the staging
+# location with: sudo TMPDIR=/path/with/space ./restore.sh
+ARCHIVE_KB=$(du -k "${ARCHIVE}" | cut -f1)
+NEEDED_KB=$((ARCHIVE_KB * 6))
+AVAIL_KB=$(df -Pk "${STAGING}" | awk 'NR==2 {print $4}')
+if [ -n "${AVAIL_KB}" ] && [ "${AVAIL_KB}" -lt "${NEEDED_KB}" ]; then
+  echo "WARNING: Staging area ${STAGING} may not have enough free space."
+  echo "  Archive size:     $((ARCHIVE_KB / 1024)) MB (compressed)"
+  echo "  Estimated needed: $((NEEDED_KB / 1024)) MB (uncompressed estimate)"
+  echo "  Available:        $((AVAIL_KB / 1024)) MB"
+  echo "If /tmp is a RAM-backed tmpfs, extraction can exhaust memory."
+  echo "Point staging at a disk with more space and re-run:"
+  echo "  sudo TMPDIR=/path/with/space $0 ${1:-}"
+  echo ""
+  read -r -p "Continue anyway? [y/N] " SPACE_CONFIRM
+  if [[ ! "${SPACE_CONFIRM}" =~ ^([yY][eE][sS]|[yY])$ ]]; then
+    echo "Restore cancelled."
+    exit 1
+  fi
+fi
+
 echo "========================================="
 echo " Powerwall-Dashboard Restore - ${STAMP}"
 echo "========================================="
@@ -169,22 +194,41 @@ if [ -d "${STAGING}/influxdb" ] && [ -n "$(ls -A "${STAGING}/influxdb" 2>/dev/nu
   # SHOW CONTINUOUS QUERIES table output (older versions). Handle both.
   # Each statement must be terminated with ';' so the influx CLI can parse
   # multiple CREATE CONTINUOUS QUERY lines piped via stdin.
+  #
+  # NOTE: the influx 1.x CLI prints statement errors to STDOUT and still
+  # exits 0, so exit-code checks alone would report success even when every
+  # statement failed. replay_cqs captures the output and greps it for ERR
+  # instead. "already exists" responses are harmless and ignored.
+  replay_cqs() {
+    # stdin: CREATE CONTINUOUS QUERY statements, ';'-terminated
+    local OUT
+    OUT=$(docker exec -i influxdb influx -database powerwall 2>&1)
+    if echo "${OUT}" | grep -vi 'already exists' | grep -qiE '(^|\s)ERR'; then
+      echo "${OUT}" | grep -iE '(^|\s)ERR' | head -5 | sed 's/^/    /'
+      return 1
+    fi
+    return 0
+  }
+
   CQ_RESTORED=0
   if [ -f "${STAGING}/influxdb/continuous_queries.txt" ] && grep -q 'CREATE CONTINUOUS QUERY' "${STAGING}/influxdb/continuous_queries.txt"; then
     # Extract just the CREATE statement from each line (handles both
     # clean format and raw table output with query_id prefix), append ';'.
-    sed -n 's/.*\(CREATE CONTINUOUS QUERY.*\)/\1/p' "${STAGING}/influxdb/continuous_queries.txt" \
-      | sed 's/$/;/' \
-      | docker exec -i influxdb influx -database powerwall > /dev/null \
-      && { echo "  ✓ Continuous queries re-created from backup (continuous_queries.txt)"; CQ_RESTORED=1; } \
-      || echo "  ⚠ Failed to re-create continuous queries from continuous_queries.txt"
+    if sed -n 's/.*\(CREATE CONTINUOUS QUERY.*\)/\1/p' "${STAGING}/influxdb/continuous_queries.txt" \
+      | sed 's/$/;/' | replay_cqs; then
+      echo "  ✓ Continuous queries re-created from backup (continuous_queries.txt)"
+      CQ_RESTORED=1
+    else
+      echo "  ⚠ Errors re-creating continuous queries from continuous_queries.txt (see above)"
+    fi
   fi
   if [ "${CQ_RESTORED}" -eq 0 ] && [ -f "${DASHBOARD}/influxdb/influxdb.sql" ]; then
-    grep '^CREATE CONTINUOUS QUERY' "${DASHBOARD}/influxdb/influxdb.sql" \
-      | sed 's/$/;/' \
-      | docker exec -i influxdb influx -database powerwall > /dev/null \
-      && echo "  ✓ Continuous queries re-created from influxdb.sql (repo defaults)" \
-      || echo "  ⚠ Failed to re-create continuous queries — see influxdb/influxdb.sql"
+    if grep '^CREATE CONTINUOUS QUERY' "${DASHBOARD}/influxdb/influxdb.sql" \
+      | sed 's/$/;/' | replay_cqs; then
+      echo "  ✓ Continuous queries re-created from influxdb.sql (repo defaults)"
+    else
+      echo "  ⚠ Errors re-creating continuous queries — see influxdb/influxdb.sql and output above"
+    fi
   fi
 
   rm -rf influxdb/backups/*