Commit 1b0e40a
authored
Add task-level cache max age support (#1501)
## Summary
- add `Cache.max_age` as a task-level cache policy, accepting a
nonnegative `timedelta` or integer number of seconds
- serialize the setting to `TaskMetadata.cache_max_age`, preserving the
distinction between unset and explicit zero
- round-trip and override the setting through remote task details
- validate positive, zero, unset, negative, and invalid-type behavior
`None` uses the platform default, zero disables age-based expiration,
and a positive value limits the age of cached results that the task will
reuse.
## IDL dependency
This depends on `TaskMetadata.cache_max_age` from flyteorg/flyte#7938
([commit](flyteorg/flyte@6390805)).
The dependency pin is intentionally not changed in this PR, so CI is
expected to fail against `flyteidl2==2.0.44` until a new IDL version is
published and the SDK pin is bumped.
## Testing
Using a locally built `flyteidl2==2.0.45.dev0` wheel:
- `pytest -q tests/user_api/test_cache.py
tests/flyte/internal/runtime/test_task_serde.py
tests/flyte/remote/test_task.py tests/flyte/test_image.py` — 177 passed
- Ruff passed for all changed source and test files
- `git diff --check` passed
### Dogfood validation
Validated against `dogfood/flytesnacks/development` and the leaseworker
implementation in unionai/cloud#18043. The expiring task template
contained `cacheMaxAge: 60s`; the normal-cache task omitted the field.
| Scenario | Run | Cache status | Execution marker |
|---|---|---|---|
| Initial 60s cache |
[expiring-3](https://dogfood.cloud-staging.union.ai/v2/domain/development/project/flytesnacks/runs/pvditt-cache-age-expiring-3)
| `CACHE_MISS` | `04:46:48.396163Z` |
| Immediate repeat |
[expiring-4](https://dogfood.cloud-staging.union.ai/v2/domain/development/project/flytesnacks/runs/pvditt-cache-age-expiring-4)
| `CACHE_HIT` | Same marker |
| Repeat after 163s |
[expiring-5](https://dogfood.cloud-staging.union.ai/v2/domain/development/project/flytesnacks/runs/pvditt-cache-age-expiring-5)
| `CACHE_MISS` | `04:49:31.529448Z` |
| Immediate repeat of refreshed result |
[expiring-6](https://dogfood.cloud-staging.union.ai/v2/domain/development/project/flytesnacks/runs/pvditt-cache-age-expiring-6)
| `CACHE_HIT` | Same refreshed marker |
| Normal cache initial |
[normal-1](https://dogfood.cloud-staging.union.ai/v2/domain/development/project/flytesnacks/runs/pvditt-cache-age-normal-1)
| `CACHE_MISS` | `04:47:39.246597Z` |
| Normal cache immediate repeat |
[normal-2](https://dogfood.cloud-staging.union.ai/v2/domain/development/project/flytesnacks/runs/pvditt-cache-age-normal-2)
| `CACHE_HIT` | Same marker |
| Normal cache after the wait |
[normal-3](https://dogfood.cloud-staging.union.ai/v2/domain/development/project/flytesnacks/runs/pvditt-cache-age-normal-3)
| `CACHE_HIT` | Still the original marker |
Dogfood workers were still using an older SDK without the task-side
constructor argument, so the probe used this SDK build for registration
and an import-time compatibility guard for the older task runtime. Cache
lookup and expiration occur in leaseworker before task execution.
---------
Signed-off-by: Paul Dittamo <pvdittamo@gmail.com>1 parent c872581 commit 1b0e40a
36 files changed
Lines changed: 205 additions & 89 deletions
File tree
- plugins
- agento11y
- bigquery
- codegen
- dask
- databricks
- echo
- hitl
- huggingface
- hydra
- jsonl
- lance
- nsight
- omegaconf
- otel
- pandera
- polars
- pytorch
- ray
- redis
- sglang
- snowflake
- spark
- trackio
- vllm
- wandb
- rs_controller
- src/flyte
- _cache
- _internal/runtime
- remote
- tests
- flyte
- internal/runtime
- remote
- user_api
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.
0 commit comments