Skip to content

Refactor data layer to support PostgreSQL as an alternative to SQLite - #8

Merged
joeldickson merged 2 commits into
mainfrom
feature/postgresql-support
Mar 22, 2026
Merged

Refactor data layer to support PostgreSQL as an alternative to SQLite#8
joeldickson merged 2 commits into
mainfrom
feature/postgresql-support

Conversation

@joeldickson

Copy link
Copy Markdown
Contributor

Summary

Closes #5

  • Introduced ITelemetryRepository abstraction with EfTelemetryRepository base class containing all shared EF Core query logic, and two provider-specific implementations: SqliteTelemetryRepository (PRAGMA optimize + VACUUM) and PostgresTelemetryRepository (VACUUM ANALYZE)
  • Refactored all four services (IngestService, DashboardService, FilterService, DataCleanupService) to depend on ITelemetryRepository instead of TelemetryDbContext directly — no service class touches the DbContext anymore
  • Conditional provider registration in Program.cs: presence of POSTGRES_CONNECTION_STRING env var switches to PostgreSQL (with EnsureCreated()); absence keeps SQLite with existing migrations
  • All 16 integration test classes now run against both SQLite and PostgreSQL using [TestFixture(DatabaseProvider.Sqlite)] / [TestFixture(DatabaseProvider.PostgreSql)] parameterization, with PostgreSQL tests powered by Testcontainers — total test cases doubled from 62 to 124

New files

File Purpose
Core/Data/ITelemetryRepository.cs Data access interface
Core/Data/EfTelemetryRepository.cs Shared EF Core implementation
Core/Data/SqliteTelemetryRepository.cs SQLite maintenance (PRAGMA/VACUUM)
Core/Data/PostgresTelemetryRepository.cs PostgreSQL maintenance (VACUUM ANALYZE)
IntegrationTests/DatabaseProvider.cs Enum for test parameterization
IntegrationTests/PostgresTestServer.cs Shared Testcontainers PostgreSQL instance

Modified files

File Change
Core/Services/IngestService.cs Uses ITelemetryRepository
Core/Services/DashboardService.cs Delegates to ITelemetryRepository
Core/Services/FilterService.cs Delegates to ITelemetryRepository
Core/Services/DataCleanupService.cs Uses ITelemetryRepository
WebApi/Program.cs Conditional SQLite/PostgreSQL registration
Core/*.csproj Added Npgsql.EntityFrameworkCore.PostgreSQL
IntegrationTests/*.csproj Added Testcontainers + Npgsql
All 16 test classes Dual [TestFixture] parameterization

Test plan

  • All 38 unit tests pass
  • All 62 SQLite integration tests pass (identical behavior to before)
  • All 62 PostgreSQL integration tests pass (via Testcontainers)
  • Solution builds with no errors
  • Verify docker-compose up works with the PostgreSQL configuration
  • Verify app starts with no env vars set (SQLite default)
  • Verify app starts with POSTGRES_CONNECTION_STRING set

Introduce ITelemetryRepository abstraction with EfTelemetryRepository base
class and provider-specific implementations (SqliteTelemetryRepository,
PostgresTelemetryRepository). All services now depend on the repository
interface instead of TelemetryDbContext directly.

Provider selection is driven by the POSTGRES_CONNECTION_STRING env var:
when set, the app uses PostgreSQL with EnsureCreated(); otherwise SQLite
with migrations (existing default behavior).

All 16 integration test classes now run against both SQLite (in-memory)
and PostgreSQL (via Testcontainers), doubling coverage from 62 to 124
database-backed test cases.

Made-with: Cursor

@joeldickson joeldickson left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The refactoring to introduce the ITelemetryRepository abstraction looks very clean and effectively decouples the core domain logic from EF Core. Parameterizing the integration tests to run against both providers using Testcontainers is also an excellent strategy for verifying parity!

I left a couple of inline comments regarding long-term maintenance and test resource cleanup:

  1. Migrations vs. EnsureCreated: EnsureCreated() prevents future schema migrations for the Postgres provider. Depending on your long-term plans, you may want to generate Postgres-specific migrations.
  2. Test Database Leakage: The integration test setup creates a new Postgres database per test method but never drops them, which could lead to bloat inside the container as the test suite grows.

Otherwise, this is a really solid PR!

Comment thread src/Agoda.DevExTelemetry.WebApi/Program.cs
…onale

- Add DropDatabaseAsync to PostgresTestServer that terminates active
  connections then drops the database
- Call DropDatabaseAsync in CustomWebApplicationFactory.Dispose so test
  databases are cleaned up after each test, avoiding container bloat
- Add comment explaining why EnsureCreated() is used for PostgreSQL
  (MVP approach) and the migration path for long-term support

Made-with: Cursor
@joeldickson
joeldickson merged commit 64eed6d into main Mar 22, 2026
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Refactor data layer to support PostgreSQL as an alternative to SQLite

2 participants