Skip to content

Commit 99dbb6f

Browse files
atulmguptaCopilot
andauthored
Optimization (#102)
* fix(mqtt): bound telemetry database pressure Coalesce ordered per-vehicle telemetry into bounded multi-row writes while preserving source timestamps, durable signal history, and broker replay semantics. Separate liveness from dependency readiness, expose pool-pressure metrics, and tune API, worker, PostgreSQL, Helm, and Compose defaults for resource-constrained home servers. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: c4f5fe6d-5e00-4d43-a379-96a3491480e3 * feat(fsd): add drive attribution insights Add synchronized counter collection, confidence-aware per-drive attribution, dashboard comparisons, and approximate route evidence across the FSD and drives experiences. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: ecf38089-132f-4123-b33f-23d9133f2db9 * feat(fsd): add firmware spotlight and weekly digest FSD section Backend: add BuildFirmwareSpotlight to compare same-route FSD share before/after the latest firmware pair (high-confidence drives only, sorted by |share change|). Frontend: - Add FirmwareSpotlightPanel to FSD drive analytics showing before/after route comparisons - Add FsdSection + useFsdWeeklyDigestNotification to the weekly digest page, wiring in useFsdInsightsRange for the selected Monday-Sunday week (once-per-week browser notification, no permission prompting) - Add CSV/JSON export for contributing drives (SI meters, drops unmeasured FSD distance) - Add downloadJSON helper to csvExport.ts All changes preserve SI units end-to-end and null-safety for unmeasured data. * feat(web,api): add FSD observatory reset-safe journal Introduce an Observatory view stitching drive and counter-reset events into a single honesty-first timeline. Resets never contribute travelled distance, and unknown/ambiguous FSD kilometres stay null instead of being coerced to zero. Adds commute stories that split repeated routes into firmware-version chapters. Backend: buildObservatory in drive_aggregate.go computes stitched/high/estimated/ambiguous FSD totals, a capped timeline (max 200 events), and per-route commute chapters. New Observatory/ObservatoryEvent/ObservatoryTotals/ObservatoryCommuteStory types added to DriveAnalytics response. Frontend: new FsdObservatoryPanel component renders the journal, KPIs, and commute stories; wired into FSDInsightsPage. Types, fixtures, and mock API updated to match the new response shape. * fix(api): treat only Disconnected as charge end Tesla's Complete/Stopped/NoPower states mean the vehicle is still plugged in — only Disconnected means unplugged. This was causing charge sessions to end prematurely. Changes: - enums.IsChargeEnded now only returns true for Disconnected - FSM and telemetry session tracker updated to keep charging sessions alive through Complete/Stopped/NoPower - Gear=Park now debounces before ending a drive (Neutral is rolling, not parked, and no longer treated as park) - FSM confirmed debounced transitions now commit immediately instead of waiting for the next batch - MQTT: delay broker Connect until after PipelineSubscriber/DefaultPublishHandler are wired, so queued QoS 1 messages delivered right after CONNACK aren't dropped; split NewProductionPipelineMQTT into construct + ConnectProductionPipelineMQTT Updated tests across datarepair, telemetry sessions, fsm, enums, and mqtt packages to match. * feat(physics): add Tesla physics views Add Tesla-honest physics APIs and UI for charge, park, vampire drain, FSD trip meters, gear events, session certificates, and outage replay. Extend FSD analytics with commute comparisons and firmware-tagged counter resets. * Add TeslaSync-only physics views Adds a backend `/physics/exclusive` report that derives TeslaSync-only physics signals, including clocks, life tape, contradictions, meter genealogy, unknown budgets, firmware epochs, mode laws, and range disagreement without inventing missing values. Adds frontend routes, navigation, i18n, API hooks, types, and tests for the new Tesla-only page. * fix(mqtt): handle startup reconnect race Serialize reconnect epoch changes with subscription setup and retry stale initial SUBACKs. Add coverage for overlapping OnConnect and connection-loss startup races. * perf(api): stream and cap physics timeline reads Physics endpoints buffered every signal_log event for the window before folding, which made the 14-day exclusive and 7-day outage lookbacks unbounded in memory and latency. - Replace the buffer-then-fold-then-collapse pipeline in internal/signal with a streaming timelineFolder that folds, collapses, and caps rows as events arrive; add TimelineOptions.MaxRows and log truncation. - Apply CollapseBy and MaxRows to charge, theater, silent, cockpit, heartbeat, park-truth, vampire, exclusive, and outage timeline reads. - Split exclusiveFields into a collapsed history set plus a high-resolution 90s black-box query, and drop battery_level/driving_distance_m from collapse keys where they defeated collapsing. - Shrink vampire lookback to 7d and outage lookback to 2h. - Move charge, outage, and exclusive query policies from operational/live to historical. - gofmt alignment across teslaphysics/types.go. * Rename Tesla-only hub to Tesla Physics Updates navigation, route labels, page copy, i18n strings, catalog description, and tests to use the Tesla Physics naming. * Revamp Tesla physics pages Redesign the Tesla-only physics hub with feature cards, provenance, empty states, and richer detail views backed by data tables. Update related i18n labels and tests to preserve unknown/null semantics for clocks, MQTT, FSD meters, range, modes, and life tape states. * fix(web): remove inner frame from PWA app icons Android and iOS apply their own squircle/circle masks to launcher icons, so the pre-drawn inner frame and accent stroke were being clipped into visible border arcs on the splash screen. - Drop the inset surface rect + accent stroke from all icon modes in `appIcon.ts`, the static SVG sources, the favicon, and the index.html splash mark - Make launcher/splash icons full-bleed; the PWA `any` purpose now uses the apple (unrounded) canvas - Add missing `icon-maskable-192.svg` / `icon-maskable-512.svg` sources and regenerate all PNGs via sharp - Keep accent validation in `buildAppIconSvg` for API stability even though it is no longer drawn - Update tests and icons README with the regeneration command and the no-frame rule * fix(web): correct TS types and JSX in TeslaOnlyPage DataTable column definitions were typed as a single `Column<T>` instead of `Column<T>[]`, breaking type checking across all 13 panels. Also wrap the empty-state branch in a fragment so the JSX comment is no longer parsed as an object literal. * feat(settings): add guided Fleet Setup Add a routed, searchable Fleet Setup workspace for Tesla connection, token status, telemetry subscription, domain readiness, and stream health. Introduce typed hooks and coverage for Fleet API setup flows. --------- Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: c4f5fe6d-5e00-4d43-a379-96a3491480e3 Copilot-Session: ecf38089-132f-4123-b33f-23d9133f2db9
1 parent 4a3425e commit 99dbb6f

241 files changed

Lines changed: 21258 additions & 841 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

.env.example

Lines changed: 26 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -21,6 +21,19 @@ POSTGRES_USER=teslasync
2121
POSTGRES_PASSWORD=changeme
2222
POSTGRES_DB=teslasync
2323
POSTGRES_PORT=5432
24+
POSTGRES_MAX_CONNECTIONS=50
25+
# Home-lab-safe connection budget. Each worker has its own smaller pool so the
26+
# steady-state stack can use at most 22 connections (24 while migrations run),
27+
# leaving headroom below POSTGRES_MAX_CONNECTIONS for TimescaleDB maintenance.
28+
# Raise the server ceiling before raising any individual pool.
29+
DATABASE_MAX_CONNS=12
30+
DATABASE_MIN_CONNS=2
31+
NOTIFICATION_WORKER_DB_MAX_CONNS=2
32+
NOTIFICATION_WORKER_DB_MIN_CONNS=1
33+
EXPORT_WORKER_DB_MAX_CONNS=2
34+
EXPORT_WORKER_DB_MIN_CONNS=1
35+
AUTOMATION_WORKER_DB_MAX_CONNS=2
36+
AUTOMATION_WORKER_DB_MIN_CONNS=1
2437

2538
# Application
2639
TESLASYNC_PORT=8080
@@ -83,8 +96,21 @@ FLEET_TELEMETRY_PORT=4443
8396
FLEET_TELEMETRY_ENABLED=false
8497
FLEET_TELEMETRY_HOST=
8598
FLEET_TELEMETRY_TOPIC_BASE=telemetry
99+
# Coalesce per-field MQTT messages while bounding database concurrency below
100+
# DATABASE_MAX_CONNS so API requests retain connection headroom.
101+
FLEET_TELEMETRY_BATCH_MS=100
102+
FLEET_TELEMETRY_BATCH_MAX_MESSAGES=256
103+
# Two workers are sufficient for a typical 1-2 vehicle home deployment and
104+
# reserve ten API-pool connections for interactive traffic. Larger fleets can
105+
# raise this conservatively, keeping it below DATABASE_MAX_CONNS.
106+
FLEET_TELEMETRY_PERSISTENCE_CONCURRENCY=2
107+
FLEET_TELEMETRY_PERSISTENCE_QUEUE_CAPACITY=64
108+
FLEET_TELEMETRY_PERSISTENCE_TIMEOUT=30s
109+
FLEET_TELEMETRY_SNAPSHOT_WRITE_INTERVAL=10s
86110
FLEET_TELEMETRY_STALE_TIMEOUT=15m
87111
FLEET_TELEMETRY_FALLBACK_POLL_INTERVAL=5m
112+
FLEET_TELEMETRY_CLEANUP_INTERVAL=2m
113+
FLEET_TELEMETRY_STALE_SESSION_TIMEOUT=5m
88114
# FLEET_TELEMETRY_TLS_CERT=./certs/server.crt
89115
# FLEET_TELEMETRY_TLS_KEY=./certs/server.key
90116

cmd/automation-worker/main.go

Lines changed: 3 additions & 32 deletions
Original file line numberDiff line numberDiff line change
@@ -32,6 +32,7 @@ import (
3232
automationmodel "github.com/ev-dev-labs/teslasync/internal/models/automation"
3333
tsmqtt "github.com/ev-dev-labs/teslasync/internal/mqtt"
3434
"github.com/ev-dev-labs/teslasync/internal/notification"
35+
healthprobe "github.com/ev-dev-labs/teslasync/internal/health"
3536
"github.com/ev-dev-labs/teslasync/internal/resilience"
3637
"github.com/ev-dev-labs/teslasync/internal/tesla"
3738
"github.com/ev-dev-labs/teslasync/internal/tracing"
@@ -302,7 +303,8 @@ func main() {
302303
// ── Health Endpoint ───────────────────────────────────────────────
303304
port := healthPort()
304305
healthMux := http.NewServeMux()
305-
healthMux.HandleFunc("/healthz", healthHandler(db))
306+
healthMux.Handle("/healthz", healthprobe.LivenessHandler())
307+
healthMux.Handle("/readyz", healthprobe.ReadinessHandler(db))
306308
healthMux.Handle("/metrics", promhttp.Handler())
307309
go func() {
308310
log.Info().Str("port", port).Msg("health endpoint listening")
@@ -356,37 +358,6 @@ func healthPort() string {
356358
return port
357359
}
358360

359-
// healthChecker is the minimal database surface the health endpoint needs.
360-
// Narrowing to this port keeps the handler unit-testable with a fake.
361-
type healthChecker interface {
362-
Health(ctx context.Context) error
363-
}
364-
365-
// healthHandler returns the /healthz handler. It responds 200 with
366-
// {"status":"ok"} when the checker is healthy and 503 with a JSON-encoded
367-
// {"status":"unhealthy","error":...} otherwise. The error message is
368-
// marshalled rather than string-interpolated so a checker error containing
369-
// quotes or newlines still produces valid, non-injectable JSON, and the
370-
// JSON Content-Type is set on both the success and failure paths.
371-
func healthHandler(checker healthChecker) http.HandlerFunc {
372-
return func(w http.ResponseWriter, r *http.Request) {
373-
w.Header().Set("Content-Type", "application/json")
374-
if err := checker.Health(r.Context()); err != nil {
375-
w.WriteHeader(http.StatusServiceUnavailable)
376-
body, mErr := json.Marshal(struct {
377-
Status string `json:"status"`
378-
Error string `json:"error"`
379-
}{Status: "unhealthy", Error: err.Error()})
380-
if mErr != nil {
381-
body = []byte(`{"status":"unhealthy"}`)
382-
}
383-
_, _ = w.Write(body)
384-
return
385-
}
386-
_, _ = w.Write([]byte(`{"status":"ok"}`))
387-
}
388-
}
389-
390361
func safePrefix(token string) string {
391362
if len(token) <= 8 {
392363
return token[:len(token)/2] + "***"

cmd/automation-worker/main_test.go

Lines changed: 3 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -15,6 +15,7 @@ import (
1515
"github.com/ev-dev-labs/teslasync/internal/database"
1616
dbauto "github.com/ev-dev-labs/teslasync/internal/database/automation"
1717
automationmodel "github.com/ev-dev-labs/teslasync/internal/models/automation"
18+
healthprobe "github.com/ev-dev-labs/teslasync/internal/health"
1819
)
1920

2021
// ── compile-time contracts ────────────────────────────────────────────────
@@ -24,7 +25,7 @@ import (
2425
var (
2526
_ action.VariableRepo = (*variableRepoAdapter)(nil)
2627
_ variableStore = (*dbauto.AutomationVariableRepo)(nil)
27-
_ healthChecker = (*database.DB)(nil)
28+
_ healthprobe.Checker = (*database.DB)(nil)
2829
)
2930

3031
// ── healthPort ─────────────────────────────────────────────────────────────
@@ -171,7 +172,7 @@ func TestHealthHandler(t *testing.T) {
171172
for _, tc := range tests {
172173
t.Run(tc.name, func(t *testing.T) {
173174
fake := &fakeHealthChecker{err: tc.healthErr}
174-
handler := healthHandler(fake)
175+
handler := healthprobe.ReadinessHandler(fake)
175176

176177
req := httptest.NewRequest(http.MethodGet, "/healthz", nil)
177178
rec := httptest.NewRecorder()

cmd/export-worker/main.go

Lines changed: 3 additions & 30 deletions
Original file line numberDiff line numberDiff line change
@@ -2,7 +2,6 @@ package main
22

33
import (
44
"context"
5-
"encoding/json"
65
"fmt"
76
"net/http"
87
"os"
@@ -26,6 +25,7 @@ import (
2625
dbbackup "github.com/ev-dev-labs/teslasync/internal/database/backup"
2726
exportdb "github.com/ev-dev-labs/teslasync/internal/database/export"
2827
"github.com/ev-dev-labs/teslasync/internal/export"
28+
healthprobe "github.com/ev-dev-labs/teslasync/internal/health"
2929
"github.com/ev-dev-labs/teslasync/internal/resilience"
3030
"github.com/ev-dev-labs/teslasync/internal/tracing"
3131

@@ -211,7 +211,8 @@ func main() {
211211
// Health endpoint for Kubernetes probes.
212212
healthPort := resolveHealthPort()
213213
healthMux := http.NewServeMux()
214-
healthMux.HandleFunc("/healthz", newHealthHandler(db))
214+
healthMux.Handle("/healthz", healthprobe.LivenessHandler())
215+
healthMux.Handle("/readyz", healthprobe.ReadinessHandler(db))
215216
healthMux.Handle("/metrics", promhttp.Handler())
216217
go func() {
217218
log.Info().Str("port", healthPort).Msg("health endpoint listening")
@@ -292,34 +293,6 @@ func healthcheckExitCode(ctx context.Context, client *http.Client, url string) i
292293
return 0
293294
}
294295

295-
// healthChecker is the minimal surface newHealthHandler needs from the database
296-
// pool, letting the handler be exercised without a live connection.
297-
type healthChecker interface {
298-
Health(ctx context.Context) error
299-
}
300-
301-
// newHealthHandler returns the /healthz handler. It responds 200 with
302-
// {"status":"ok"} when the dependency is reachable and 503 with a JSON error
303-
// body otherwise. The error string is JSON-encoded (not string-interpolated) so
304-
// a driver message containing quotes cannot produce a malformed body, and
305-
// Content-Type is set on both paths.
306-
func newHealthHandler(hc healthChecker) http.HandlerFunc {
307-
return func(w http.ResponseWriter, r *http.Request) {
308-
w.Header().Set("Content-Type", "application/json")
309-
if err := hc.Health(r.Context()); err != nil {
310-
body, _ := json.Marshal(map[string]string{
311-
"status": "unhealthy",
312-
"error": err.Error(),
313-
})
314-
w.WriteHeader(http.StatusServiceUnavailable)
315-
_, _ = w.Write(body)
316-
return
317-
}
318-
body, _ := json.Marshal(map[string]string{"status": "ok"})
319-
_, _ = w.Write(body)
320-
}
321-
}
322-
323296
// newScheduledBackupRun builds the queued BackupRun the scheduler persists for a
324297
// due config. ConfigID points at the config's ID so the run is attributable to
325298
// its schedule, and the metadata records that a scheduled tick (not a manual

cmd/export-worker/main_test.go

Lines changed: 3 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -11,6 +11,7 @@ import (
1111

1212
backupmodel "github.com/ev-dev-labs/teslasync/internal/models/backup"
1313

14+
healthprobe "github.com/ev-dev-labs/teslasync/internal/health"
1415
"github.com/rs/zerolog"
1516
)
1617

@@ -177,7 +178,7 @@ func TestNewHealthHandler(t *testing.T) {
177178
}
178179
for _, tt := range tests {
179180
t.Run(tt.name, func(t *testing.T) {
180-
handler := newHealthHandler(&fakeHealthChecker{err: tt.healthErr})
181+
handler := healthprobe.ReadinessHandler(&fakeHealthChecker{err: tt.healthErr})
181182
rec := httptest.NewRecorder()
182183
req := httptest.NewRequest(http.MethodGet, "/healthz", nil)
183184

@@ -213,7 +214,7 @@ func TestNewHealthHandler_PropagatesRequestContext(t *testing.T) {
213214
const marker ctxKey = "marker"
214215

215216
fake := &fakeHealthChecker{}
216-
handler := newHealthHandler(fake)
217+
handler := healthprobe.ReadinessHandler(fake)
217218

218219
rec := httptest.NewRecorder()
219220
req := httptest.NewRequest(http.MethodGet, "/healthz", nil)

0 commit comments

Comments
 (0)