Skip to content

Commit 3c37671

Browse files
committed
fix(cache): implement proper caching strategies per industry best practices
Audit and fix caching implementation against cache-aside, write-through, thundering herd prevention, and stale-while-revalidate patterns. Thundering herd prevention: - findById() and findByKey() now use Caffeine's get(key, loader) which coalesces concurrent cache misses into a single DB query. Previously used getIfPresent() + manual put, meaning N concurrent misses for the same key caused N parallel database queries. TTL jitter (mass expiry prevention): - Cache expiration times now include configurable random jitter (default 10% of TTL). A 30-minute TTL becomes 27-33 minutes. Prevents stampede when preload() writes thousands of entries at the same instant and they all expire together. Stale-while-revalidate: - New refreshAfterWrite option in CacheConfig. When set, expired entries serve stale data immediately while Caffeine triggers an async background reload. Users never block on cache revalidation. Leverages Caffeine's native refresh mechanism. Bounded preload: - Added preload(int limit) and preloadAsync(int limit) for large tables. Previous preload() loaded the entire table into memory with no safety bound. Cache observability: - Added logCacheStats() method that logs hit rate, hit/miss counts, eviction count, and cache size per layer at INFO level. Stats were recorded via recordStats() but never surfaced. Cache contract documentation: - Documented staleness window, invalidation strategy, cold start behavior, and consistency model in class javadoc. Cross-population: - findById() now cross-populates the key cache on hit, and findByKey() cross-populates the ID cache. Previously each layer was only populated through its own lookup path. CacheConfig additions: - refreshAfterWrite (Duration) — stale-while-revalidate trigger - jitterPercent (int, 0-50) — TTL jitter percentage
1 parent aec212c commit 3c37671

2 files changed

Lines changed: 205 additions & 97 deletions

File tree

README.md

Lines changed: 47 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -505,7 +505,18 @@ Supports: `equal`, `notEqual`, `like`, `in`, `isNull`, `isNotNull`, `greaterThan
505505

506506
## Caching
507507

508-
Extend `AbstractCachedRepository` for dual-layer Caffeine caching (by ID and by custom key):
508+
Extend `AbstractCachedRepository` for dual-layer Caffeine caching (by ID and by custom key). The cache uses industry-standard patterns: cache-aside reads, write-through mutations, thundering herd protection, and optional stale-while-revalidate.
509+
510+
### How It Works
511+
512+
| Pattern | What Happens |
513+
|---|---|
514+
| **Cache-Aside** (reads) | Check cache first. On miss, load from DB and populate cache. Concurrent misses for the same key are coalesced into a single DB query (thundering herd protection). |
515+
| **Write-Through** (mutations) | Every create/update/save writes to DB first, then updates the cache. Deletes evict before the DB write. |
516+
| **TTL Jitter** | Expiration times include random jitter (default 10% of TTL) to prevent mass expiry stampedes after bulk preload. |
517+
| **Stale-While-Revalidate** (optional) | When `refreshAfterWrite` is set, expired entries serve stale data immediately while reloading in the background. Users never block on revalidation. |
518+
519+
### Basic Setup
509520

510521
```java
511522
public class PlayerRepository extends AbstractCachedRepository<PlayerData, UUID, String> {
@@ -521,11 +532,26 @@ public class PlayerRepository extends AbstractCachedRepository<PlayerData, UUID,
521532
}
522533
```
523534

535+
### With Stale-While-Revalidate
536+
537+
For high-traffic lookups where slight staleness is acceptable (player profiles, leaderboards):
538+
539+
```java
540+
CacheConfig.builder()
541+
.expiration(Duration.ofMinutes(30))
542+
.refreshAfterWrite(Duration.ofMinutes(25)) // after 25min, serve stale + reload async
543+
.maxSize(5000)
544+
.jitterPercent(10) // TTL varies by +/-10% to prevent mass expiry
545+
.build()
546+
```
547+
548+
### Cache Operations
549+
524550
```java
525-
// Cache lookups
526-
repo.findByKey("alice"); // memory only
527-
repo.findByKey("username", "alice"); // DB fallback
528-
repo.getOrCreate("username", "alice", k -> new PlayerData(uuid, k));// get or create
551+
// Lookups (thundering herd safe)
552+
repo.findByKey("alice"); // memory only
553+
repo.findByKey("username", "alice"); // DB fallback
554+
repo.getOrCreate("username", "alice", k -> new PlayerData(uuid, k)); // get or create
529555

530556
// Eviction
531557
repo.evict(player);
@@ -534,13 +560,26 @@ repo.evictByKey("alice");
534560
repo.evictAll();
535561

536562
// Preloading (warm cache on startup)
537-
repo.preloadAsync();
563+
repo.preloadAsync(); // load all (small tables)
564+
repo.preloadAsync(1000); // load first 1000 (large tables)
538565

539-
// Stats
566+
// Monitoring
567+
repo.logCacheStats(); // logs hit rate, misses, evictions, size
540568
CacheStats stats = repo.getKeyCacheStats();
541569
long size = repo.getCacheSize();
542570
```
543571

572+
### Cache Contract
573+
574+
Before using caching, consider these questions for your use case:
575+
576+
| Question | Default Behavior |
577+
|---|---|
578+
| **Staleness window?** | Configurable TTL (default 30 min). Data may be stale up to this duration. |
579+
| **Who invalidates?** | Automatic on all mutation paths (create, update, save, delete, batch variants). |
580+
| **Cold start?** | Call `preload()` or `preload(limit)` in `onEnable()`. Cache populates on first access otherwise. |
581+
| **Wrong data consequence?** | For balances/permissions, use short TTL or skip cache. For display names/stats, longer TTL is fine. |
582+
544583
All mutations (create, update, save, delete) automatically maintain cache consistency.
545584

546585
---
@@ -859,4 +898,4 @@ de.jexcellence.jehibernate
859898

860899
[Apache License 2.0](https://www.apache.org/licenses/LICENSE-2.0)
861900

862-
Copyright 2024 JExcellence
901+
Copyright 2026 JExcellence

0 commit comments

Comments
 (0)