stale-while-revalidate feature

[mharkins-cosm]

Would you support a stale-while-revalidate feature in this crate? If so, I'd be happy to start work on it.

EDIT: To clarify, I'd expect the mechanism to synchronize write access to the cache key as with the sync_writes feature, but rather than blocking, it would authorize failed attempts to return a stale value while the lock holder rebuilds the cache value.

[jaemk]

Hi @mharkins-cosm, the semantics you described in the edit compose today: cache_peek_with_expiry_status returns an expired entry as (Some(value), true) without removing it, {fn}_prime_cache recomputes and (since 3.0) runs the function body outside the cache write lock, and sync_writes = "by_key" still covers the cold path. One rule per state:

match value {
    Some(v) if !expired => v,
    // Expired: nobody waits. One caller claims the refresh, all of them get the stale value.
    Some(stale) => {
        if claim_refresh(id) {
            let owned = id.to_string();
            tokio::spawn(async move { single_flight_prime_cache(&owned).await; release_refresh(&owned); });
        }
        stale
    }
    // Cold: nothing to serve, so `by_key` deduplicates these and they wait together.
    None => single_flight(id).await,
}

See examples/stale_while_revalidate.rs (section 5) for the runnable version. It measures both states: 8 cold callers produce 1 compute and all agree, 8 stale callers are served in 0ms and produce 1 refresh.

Note that the claim set is separate from the by_key bucket lock on purpose. That lock covers the store write, not the recompute, which is what keeps a refresh from blocking readers, so it will not collapse concurrent refreshes on its own.

Leaving this open: the above is a recipe, not a feature, and an attribute that wraps it would still need a caller-supplied spawn since the crate carries no runtime dependency. If you still want to build it, that design question is the one to settle first.