auto projection overlay / update docs

Author: kai-init

CHANGELOG.md

@@ -7,13 +7,13 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 
 ---
 
-## [4.0.0] — 2026-07-29
+## [4.0.0] — 2026-07-20
 
 ### Added
 
 - **Query Builder caching:** supported `DB::table()` reads now participate in NormCache automatically, alongside Eloquent reads. Writes through those cache-aware builders invalidate affected tables automatically.
 - **Unified dependencies:** `dependsOn()` now accepts Eloquent model classes and raw table names in one declaration.
-- **Result overlays:** `useResultCache()` adds a complete-result payload on top of canonical row storage, letting warm reads avoid row-by-row assembly while retaining canonical fallback and repair.
+- **Automatic Result Overlays:** canonical queries returning up to `auto_overlay_max_rows` (default 50) automatically store a complete result payload in Redis, letting warm reads execute in a single Redis `GET` (~80–120 µs) while retaining canonical fallback and self-healing.
 - **Global tags:** use `tag('name')` to group cached query payloads and `NormCache::flushTag('name')` to invalidate that group.
 - **Unified manual invalidation:** `NormCache::invalidate()` accepts a model instance, model class, table name, or an array of those targets. A model target uses its model connection; table targets use Laravel's default connection unless `connection:` is supplied.
 - **Runtime cache switch:** `normcache:disable` and `normcache:enable`, plus `disableCache()`, `enableCache()`, and `cacheDisabled()`, pause caching across application nodes. Re-enabling advances the global epoch before serving cached data again.

README.md

@@ -23,12 +23,6 @@ Publish the configuration:
 php artisan vendor:publish --tag=normcache-config
 ```
 
-### Optional igbinary serialization
-
-When the `ext-igbinary` PHP extension is available, NormCache detects it automatically and uses it for cached payloads. Otherwise it falls back to PHP's native serialization; no configuration is required.
-
-Every application node and worker sharing the same Redis cache must use the same serializer. After installing or removing igbinary, run `php artisan normcache:flush` before serving traffic so payloads written with the previous format are not reused.
-
 Add `Cacheable` to Eloquent models whose writes and reads NormCache should observe:
 
 ```php
@@ -58,10 +52,23 @@ Cache controls are available on Eloquent and Query Builder:
 Post::query()->withoutCache()->get();
 Post::query()->where('published', true)->ttl(600)->get();
 Post::query()->where('published', true)->tag('homepage')->get();
-Post::query()->orderBy('id')->useResultCache()->get();
 ```
 
-For queries that would normally use canonical storage, `useResultCache()` also stores the complete result as one payload. Warm reads can use that payload directly, while canonical storage remains available as a fallback.
+## Canonical & Normalized Row Caching
+
+At the core of NormCache is **normalized row storage**. Unlike traditional query caching—which stores duplicate, static copies of entire result sets for every unique SQL query—NormCache normalizes data in Redis:
+
+- **Single Storage for Model Rows**: Individual database rows are stored once under canonical primary key IDs (`table:r:<id>`).
+- **Lightweight Query Memberships**: Queries cache only a list of primary key IDs (`table:m:<query_hash>`), not full duplicate model attributes.
+- **$O(1)$ Invalidation Without Redis SCAN**: When a model is updated or deleted, NormCache invalidates only that specific row key (`table:r:<id>`) and advances the table version counter (`table:v`). There are no expensive `KEYS` or `SCAN` commands in Redis.
+- **Global Row Freshness**: Every query reading Post #42 automatically receives the updated row data on its next fetch, ensuring instant consistency across all application queries without clearing individual query keys.
+
+## Automatic Result & Projection Overlay
+
+NormCache automatically optimizes warm query performance by storing single-step result overlays in Redis for eligible canonical queries:
+
+- **Automatic Promotion**: Canonical queries returning up to `auto_overlay_max_rows` (default `50`) automatically store a serialized result payload in Redis (`table:e:v1:...`).
+- **Instant Synchronization & Self-Healing**: Updates to underlying models or dependency tables instantly invalidate the overlay alongside canonical storage. If an overlay key expires or misses, NormCache seamlessly falls back to canonical row assembly and repromotes automatically.
 
 ## Tags and selective flushing
 
@@ -84,8 +91,6 @@ NormCache::flushTag('homepage');
 
 `flushTag()` advances a Redis version counter; it does not scan for or delete matching keys. The affected queries miss and rebuild on their next read, while old payloads expire naturally. Tags are an additional manual invalidation boundary and do not replace automatic dependency invalidation when an underlying table changes.
 
-Tags must be non-empty valid UTF-8 strings of at most 128 bytes.
-
 ## Dependencies
 
 NormCache infers identifiable tables from ordinary joins, unions, subqueries, and relationship queries. If a query contains an opaque expression or source, declare every table it reads:
@@ -106,12 +111,7 @@ Writes through cache-aware Eloquent or Query Builder paths invalidate automatica
 Use the facade after writes performed elsewhere:
 
 ```php
-use App\Models\Comment;
-use App\Models\Post;
-use NormCache\Facades\NormCache;
-
-NormCache::invalidate('posts', connection: 'mysql');
-NormCache::invalidate([Post::class, Comment::class]);
+NormCache::invalidate([Post::class, Comment::class], connection: 'mysql');
 NormCache::invalidate(['posts', 'comments'], connection: 'mysql');
 NormCache::flushTag('homepage');
 NormCache::flushAll();
@@ -138,18 +138,6 @@ While disabled, reads bypass NormCache and go directly to the database, and writ
 
 `normcache:enable` atomically advances the global epoch before clearing the disabled flag. This prevents payloads cached before the pause from being served after writes occurred while invalidation was disabled.
 
-The same controls are available programmatically:
-
-```php
-use NormCache\Facades\NormCache;
-
-NormCache::disableCache();
-$disabled = NormCache::cacheDisabled();
-$newEpoch = NormCache::enableCache();
-```
-
-This runtime switch is separate from `NORMCACHE_ENABLED=false`. A cache disabled in configuration cannot be enabled with `normcache:enable`; update the configuration first.
-
 ## Configuration
 
 ```php
@@ -160,6 +148,7 @@ return [
 
     'row_ttl' => 604800,
     'query_ttl' => 3600,
+    'auto_overlay_max_rows' => 50,
 
     'max_precise_invalidation_keys' => 1000,
     'building_lock_ttl' => 5,
@@ -201,12 +190,7 @@ NormCache bypasses reads when correctness cannot be established, including:
 
 Canonical storage requires a supported single-column integer or string primary key. Queries can still use `result` storage when canonical routing is unavailable.
 
-Writes performed through raw SQL or a connection not installed by NormCache are invisible until `invalidate()` or `flushAll()` is called. After changing a connection's database, schema, or database objects at runtime, call `NormCache::clearSchemaMetadata()` for that connection.
-
-### Consistency & Failure Modes
-
-- **Fail-Open Invalidation**: Database availability is prioritized over cache state. If Redis is unreachable during a write operation, NormCache fails open on the writing node (bypassing cache for subsequent reads on that node) and logs a warning. Note that other application nodes connected to Redis may continue serving cached queries until TTL expiration or subsequent invalidation.
-- **Triggers & Database Cascades**: Foreign key `ON DELETE CASCADE` rules, database triggers, and stored procedures operating within the database engine are not intercepted at the application layer. When executing operations that trigger database-side side-effect updates, call `NormCache::invalidate([...])` explicitly for affected secondary tables.
+Direct database writes executed outside of Eloquent (such as raw SQL, triggers, or external services) bypass automatic cache interception. Use `NormCache::invalidate(...)` or `NormCache::flushAll()` to manually invalidate affected models or tables. If connection schemas or table definitions are modified at runtime, call `NormCache::clearSchemaMetadata($connection)` to reset cached schema metadata.
 
 ## Redis Cluster
 
@@ -216,6 +200,12 @@ All keys for one physical table share a Redis hash slot. Query-group entries use
 
 When `events` is enabled, NormCache dispatches cache hit, miss, bypass, repair, and invalidation events. When `fruitcake/laravel-debugbar` is installed and `debugbar` is enabled, cache activity appears in Laravel Debugbar.
 
+## Optional igbinary serialization
+
+When the `ext-igbinary` PHP extension is available, NormCache detects it automatically and uses it for cached payloads. Otherwise it falls back to PHP's native serialization; no configuration is required.
+
+Every application node and worker sharing the same Redis cache must use the same serializer. After installing or removing igbinary, run `php artisan normcache:flush` before serving traffic so payloads written with the previous format are not reused.
+
 ## License
 
 MIT

config/normcache.php

@@ -7,6 +7,7 @@
 
     'row_ttl' => (int) env('NORMCACHE_ROW_TTL', 604800),
     'query_ttl' => (int) env('NORMCACHE_QUERY_TTL', 3600),
+    'auto_overlay_max_rows' => (int) env('NORMCACHE_AUTO_OVERLAY_MAX_ROWS', 50),
 
     // Each group requires connection, database, and table metadata. Add schema to
     // restrict a PostgreSQL or SQL Server match. Types are integer or string.

src/Cache/Engine.php

@@ -1187,6 +1187,10 @@ private function promoteResultPayload(
         bool $wakeWaiters = true,
     ): bool {
         try {
+            if (count($rows) > $this->config->maxAutoOverlayRows) {
+                return false;
+            }
+
             $ttl = $query->configuredTtl() ?? $this->config->queryTtl;
             $encoded = $this->codec->encode(
                 $rows,

src/Database/QueryBuilder.php

@@ -35,8 +35,6 @@ final class QueryBuilder extends Builder
 
     private ?string $tag = null;
 
-    private bool $useResultCache = false;
-
     /** @var array<string, DependencyDeclaration> */
     private array $dependencies = [];
 
@@ -139,18 +137,6 @@ public function configuredTag(): ?string
         return $this->tag;
     }
 
-    public function useResultCache(): static
-    {
-        $this->useResultCache = true;
-
-        return $this;
-    }
-
-    public function usesResultCache(): bool
-    {
-        return $this->useResultCache;
-    }
-
     /** @param array<mixed> $dependencies */
     public function dependsOn(array $dependencies): static
     {

src/Planning/QueryPlanner.php

@@ -91,7 +91,7 @@ public function plan(
                 $root,
                 $dependencies,
                 $primaryKey,
-                materializeResult: $query->usesResultCache(),
+                materializeResult: $query->limit !== null,
             );
         }
 

src/Values/CacheConfig.php

@@ -11,6 +11,7 @@ public function __construct(
         public string $keyPrefix,
         public int $rowTtl,
         public int $queryTtl,
+        public int $maxAutoOverlayRows,
         public array $primaryKeys,
         public int $maxPreciseInvalidationKeys,
         public int $buildingLockTtl,
@@ -32,6 +33,11 @@ public static function fromArray(array $values): self
 
         $rowTtl = self::positive($values, 'row_ttl', 604_800);
         $queryTtl = self::positive($values, 'query_ttl', 3_600);
+        $maxAutoOverlayRows = self::positive(
+            $values,
+            'auto_overlay_max_rows',
+            50,
+        );
         $maxPreciseInvalidationKeys = self::bounded(
             $values,
             'max_precise_invalidation_keys',
@@ -46,6 +52,7 @@ public static function fromArray(array $values): self
             keyPrefix: $keyPrefix,
             rowTtl: $rowTtl,
             queryTtl: $queryTtl,
+            maxAutoOverlayRows: $maxAutoOverlayRows,
             primaryKeys: self::primaryKeys($values['primary_keys'] ?? []),
             maxPreciseInvalidationKeys: $maxPreciseInvalidationKeys,
             buildingLockTtl: $buildingLockTtl,

tests/Integration/ResultCacheStrategyTest.php

@@ -34,7 +34,7 @@ protected function setUp(): void
         }
     }
 
-    public function test_use_result_cache_materializes_result_over_canonical_storage(): void
+    public function test_small_canonical_result_automatically_materializes_an_overlay(): void
     {
         $query = fn() => DB::table('posts')
             ->where('published', true)
@@ -48,7 +48,6 @@ public function test_use_result_cache_materializes_result_over_canonical_storage
             ->orderByDesc('views')
             ->orderBy('id')
             ->limit(4)
-            ->useResultCache()
             ->get();
 
         $cold = $query();
@@ -71,7 +70,6 @@ public function test_eloquent_forwards_use_result_cache_to_the_query_builder():
             ->where('published', true)
             ->orderByDesc('views')
             ->limit(3)
-            ->useResultCache()
             ->get();
 
         $cold = $query();
@@ -85,22 +83,34 @@ public function test_eloquent_forwards_use_result_cache_to_the_query_builder():
         $this->assertSame([], DB::getQueryLog());
     }
 
-    public function test_large_result_is_published_without_an_admission_limit(): void
+    public function test_result_larger_than_the_row_limit_is_not_promoted(): void
     {
-        $title = str_repeat('x', 4_194_304 + 1_024);
-        $id = DB::table('posts')->insertGetId([
-            'title' => $title,
-            'views' => 0,
-            'published' => true,
-            'author_id' => $this->authorId,
-            'created_at' => now(),
-            'updated_at' => now(),
-        ]);
-
-        $row = DB::table('posts')->where('id', $id)->select('title')->first();
-
-        $this->assertSame(strlen($title), strlen((string) $row?->title));
-        $this->assertCount(1, $this->cacheKeysMatching(':e:v'));
+        foreach (range(1, 50) as $index) {
+            DB::table('posts')->insert([
+                'title' => "Extra {$index}",
+                'views' => $index,
+                'published' => true,
+                'author_id' => $this->authorId,
+                'created_at' => now(),
+                'updated_at' => now(),
+            ]);
+        }
+
+        $query = fn() => DB::table('posts')
+            ->where('published', true)
+            ->orderBy('id')
+            ->limit(55)
+            ->get();
+
+        $this->assertCount(55, $query());
+        $this->assertSame([], $this->cacheKeysMatching(':e:v'));
+
+        DB::flushQueryLog();
+        DB::enableQueryLog();
+        $this->assertCount(55, $query());
+        DB::disableQueryLog();
+
+        $this->assertSame([], DB::getQueryLog());
     }
 
     public function test_missing_result_overlay_falls_back_to_canonical_and_repromotes(): void
@@ -109,7 +119,6 @@ public function test_missing_result_overlay_falls_back_to_canonical_and_repromot
             ->where('published', true)
             ->orderBy('id')
             ->limit(4)
-            ->useResultCache()
             ->get();
 
         $expected = $query()->pluck('id')->all();
@@ -132,7 +141,6 @@ public function test_corrupt_result_overlay_falls_back_to_canonical_and_self_hea
             ->where('published', true)
             ->orderBy('id')
             ->limit(4)
-            ->useResultCache()
             ->get();
 
         $expected = $query()->pluck('id')->all();
@@ -162,7 +170,6 @@ public function test_write_invalidates_the_materialized_overlay(): void
             ->where('published', true)
             ->orderBy('id')
             ->limit(4)
-            ->useResultCache()
             ->get();
 
         $before = $query();
@@ -185,7 +192,6 @@ public function test_tag_flush_invalidates_the_materialized_overlay(): void
             ->orderBy('id')
             ->limit(4)
             ->tag('homepage')
-            ->useResultCache()
             ->get();
 
         $query();
@@ -207,7 +213,6 @@ public function test_dependency_version_invalidates_the_materialized_overlay():
             ->where('published', true)
             ->orderBy('id')
             ->limit(4)
-            ->useResultCache()
             ->get();
 
         $query();
@@ -229,7 +234,6 @@ public function test_query_ttl_applies_to_membership_and_result_overlay(): void
             ->orderBy('id')
             ->limit(4)
             ->ttl(30)
-            ->useResultCache()
             ->get();
 
         $connection = Redis::connection('normcache-test');

tests/Unit/CacheConfigTest.php

@@ -9,31 +9,22 @@
 
 final class CacheConfigTest extends UnitTestCase
 {
-    public function test_builds_configuration_contract(): void
+    public function test_populates_default_configuration_values(): void
     {
-        $config = CacheConfig::fromArray([
-            'connection' => 'normcache-test',
-            'key_prefix' => 'app:',
-            'row_ttl' => 600,
-            'query_ttl' => 60,
-            'primary_keys' => [],
-            'events' => true,
-        ]);
+        $config = CacheConfig::fromArray([]);
 
-        $this->assertSame('normcache-test', $config->connection);
-        $this->assertSame('app:', $config->keyPrefix);
-        $this->assertSame(600, $config->rowTtl);
-        $this->assertSame(60, $config->queryTtl);
+        $this->assertSame('cache', $config->connection);
+        $this->assertSame('', $config->keyPrefix);
+        $this->assertSame(604_800, $config->rowTtl);
+        $this->assertSame(3_600, $config->queryTtl);
+        $this->assertSame(50, $config->maxAutoOverlayRows);
         $this->assertSame(1000, $config->maxPreciseInvalidationKeys);
-        $this->assertFalse(property_exists($config, 'maxMembershipRows'));
-        $this->assertFalse(property_exists($config, 'maxMembershipBytes'));
-        $this->assertFalse(property_exists($config, 'maxCanonicalBytes'));
-        $this->assertFalse(property_exists($config, 'maxResultBytes'));
-        $this->assertTrue($config->dispatchEvents);
-        $this->assertFalse(property_exists($config, 'cooldown'));
-        $this->assertFalse(property_exists($config, 'deploymentIds'));
-        $this->assertFalse(property_exists($config, 'publicationGuardMarginSeconds'));
-        $this->assertFalse(property_exists($config, 'fallbackEnabled'));
+        $this->assertSame(5, $config->buildingLockTtl);
+        $this->assertSame(200, $config->stampedeWaitMs);
+        $this->assertSame(64, $config->stampedeWakeTokens);
+        $this->assertTrue($config->enabled);
+        $this->assertFalse($config->dispatchEvents);
+        $this->assertFalse($config->debugbar);
     }
 
     public function test_rejects_hash_tag_characters_in_key_prefix(): void
@@ -60,6 +51,7 @@ public static function invalidSafetyValues(): array
             ['building_lock_ttl', 0],
             ['row_ttl', 0],
             ['query_ttl', 0],
+            ['auto_overlay_max_rows', 0],
         ];
     }