Skip to content

ADAPTER_SPEC.md

Latest commit

 

History

History
329 lines (230 loc) · 9.88 KB

ADAPTER_SPEC.md

File metadata and controls

329 lines (230 loc) · 9.88 KB
Raw

pluca Adapter Specification

This document defines the contract you must follow to create a new cache adapter for pluca.

If you follow this specification, your adapter should work with pluca.Cache.

Example names

This document uses pluca_xyz as an example adapter module name.

Compatibility targets

Your adapter should:

  • support Python 3.11+
  • depend on pluca
  • preserve pluca.Cache public behavior

What you implement

Implement an adapter class that satisfies pluca.adapter.CacheAdapter.

Example export:

# pluca_xyz/__init__.py
from .adapter import XyzAdapter

Adapter = XyzAdapter

Required adapter methods

Your adapter must provide these public methods:

  • put_mapped(mkey, value, max_age=None) -> None
  • get_mapped(mkey) -> Any
  • remove_mapped(mkey) -> None
  • flush() -> None
  • gc() -> None
  • shutdown() -> None

Misses and expired entries must raise:

  • KeyError(mkey)

For arguments with invalid types, raise:

  • TypeError

For invalid adapter values or configuration, raise:

  • ValueError

For backend runtime failures (for example disk full, DB engine errors, network/client failures), raise:

  • pluca.CacheBackendError

When a backend exception is available, adapters must always re-raise with exception chaining:

  • raise pluca.CacheBackendError(...) from ex

Use from ex for every CacheBackendError re-raise where ex exists.

Utility methods and native optimizations

pluca.Cache offers utility APIs such as has, put_many, get_many, and remove_many, and set_max_age.

If your backend infrastructure supports these operations natively, you should implement the corresponding adapter methods for better performance:

  • has_mapped(mkey) -> bool
  • put_many_mapped(data, max_age=None) -> None
  • get_many_mapped(keys, default=...) -> list[tuple[Any, Any]]
  • remove_many_mapped(keys) -> None
  • set_max_age_mapped(mkey, max_age=None) -> None

If native support is unavailable, keep these methods present and raise NotImplementedError. pluca.Cache will automatically fallback to single-item logic.

Example: if your backend has a native multi-set command, implement put_many_mapped using that command instead of per-item writes.

Example: if your backend supports native TTL refresh/update commands, implement set_max_age_mapped using that command instead of a read + rewrite fallback.

If native support is unavailable, keep set_max_age_mapped present and raise NotImplementedError. pluca.Cache will fallback to get_mapped + put_mapped with the new max_age.

set_max_age_mapped must raise KeyError(mkey) when the key is missing or expired.

Dependency companion keys

pluca.Cache may store dependency metadata for get_put(..., dependency=...) under internal companion keys. These keys are regular cache entries from the adapter point of view.

Adapter rules:

  • do not special-case or filter internal companion entries
  • apply the same expiration and persistence rules as normal entries
  • allow set_max_age_mapped to operate on companion keys normally

Treat all mapped keys as opaque and backend-agnostic.

Concurrency behavior

pluca.Cache does not require adapters to be thread-safe for shared cache instances.

Adapter authors should assume that a cache instance may be created and used within a single thread, but concurrent use of the same cache instance across threads is not part of the required contract unless the adapter explicitly documents stronger guarantees.

Adapter rules:

  • do not claim thread-safety unless concurrent shared-instance access is actually supported by the backend and adapter implementation
  • document any thread-affinity requirements imposed by the backend client (for example, connections that must only be used from the creating thread)
  • document any locking or coordination options that affect concurrent access
  • if your adapter is intended for concurrent shared-instance use, make that guarantee explicit in adapter documentation

Keep in mind that pluca.Cache.get_put(..., dependency=...) performs multiple cache operations. Adapters must preserve normal cache semantics for those operations, but they are not required to provide single-flight behavior, atomic dependency checks, or at-most-once producer execution.

Expiration behavior

Public behavior must be consistent:

  • max_age=None: no explicit expiration
  • max_age < 0: invalid (ValueError from public API)
  • max_age == 0: valid, behaves as immediately expired
  • expired entries behave like missing entries

How expiration is enforced depends on your backend:

  • If your backend supports native expiration (for example, memcache TTL), use native expiration.
  • If it does not, enforce expiration in Python in adapter logic.

Both approaches are valid as long as API behavior stays consistent.

Constructor guidance

Keep constructor arguments configuration-friendly:

  • use simple scalar types when possible (str, int, float, bool, None)
  • validate arguments early
  • raise ValueError for invalid values

Adapter class documentation

Document the adapter class with a class-level docstring so users can understand what the adapter does and how to configure and operate it without reading implementation code.

Adapter class docstrings should include relevant information such as:

  • what backend the adapter targets and any backend-specific constraints
  • constructor arguments and expected values
  • exceptions raised for invalid configuration and runtime backend failures
  • thread/concurrency guarantees (or explicit lack of guarantees)
  • any important lifecycle behavior (for example, what shutdown() does)

Keep this documentation high-level and user-facing.

Do include:

  • behavior and guarantees that matter to users configuring or operating adapter instances
  • externally visible backend constraints (for example, TTL limits imposed by the backend)

Do not include:

  • internal implementation details (for example, internal serialization formats, in-memory structures, private helper methods, or specific client classes)
  • code-level mechanics that users do not need to configure or operate the adapter

Use Google-style format for the docstring (recommended).

Add a brief explanation in your README about what pluca is, with a link to the package page.

Minimal skeleton

import pickle
import time
from collections.abc import Iterable, Mapping
from typing import Any

import pluca


class XyzAdapter:
    def __init__(self, endpoint: str, namespace: str | None = None) -> None:
        if not endpoint:
            raise ValueError('endpoint must be non-empty')
        if namespace == '':
            raise ValueError('namespace must not be empty')
        self.endpoint = endpoint
        self.namespace = namespace
        self._storage: dict[str, tuple[bytes, float | None]] = {}

    def _k(self, mkey: Any) -> str:
        base = str(mkey)
        if self.namespace is None:
            return base
        return f'{self.namespace}:{base}'

    def put_mapped(self, mkey: Any, value: Any,
                   max_age: float | None = None) -> None:
        expires = None if max_age is None else time.time() + max_age
        try:
            self._storage[self._k(mkey)] = (pickle.dumps(value), expires)
        except OSError as ex:
            raise pluca.CacheBackendError('xyz put failed') from ex

    def get_mapped(self, mkey: Any) -> Any:
        key = self._k(mkey)
        try:
            payload, expires = self._storage[key]
        except KeyError as ex:
            raise KeyError(mkey) from ex

        if expires is not None and expires <= time.time():
            del self._storage[key]
            raise KeyError(mkey)

        return pickle.loads(payload)

    def remove_mapped(self, mkey: Any) -> None:
        key = self._k(mkey)
        try:
            del self._storage[key]
        except KeyError as ex:
            raise KeyError(mkey) from ex

    def set_max_age_mapped(self, mkey: Any,
                           max_age: float | None = None) -> None:
        key = self._k(mkey)
        try:
            payload, _expires = self._storage[key]
        except KeyError as ex:
            raise KeyError(mkey) from ex
        expires = None if max_age is None else time.time() + max_age
        self._storage[key] = (payload, expires)

    def flush(self) -> None:
        self._storage.clear()

    def has_mapped(self, mkey: Any) -> bool:
        raise NotImplementedError

    def put_many_mapped(self,
                        data: Mapping[Any, Any] | Iterable[tuple[Any, Any]],
                        max_age: float | None = None) -> None:
        raise NotImplementedError

    def get_many_mapped(self, keys: Iterable[Any],
                        default: Any = ...) -> list[tuple[Any, Any]]:
        raise NotImplementedError

    def remove_many_mapped(self, keys: Iterable[Any]) -> None:
        raise NotImplementedError

    def gc(self) -> None:
        now = time.time()
        expired = [
            key for key, (_, exp) in self._storage.items()
            if exp is not None and exp <= now
        ]
        for key in expired:
            del self._storage[key]

    def shutdown(self) -> None:
        return None


Adapter = XyzAdapter

HOWTO: basic adapter test with AdapterTester

Use pluca.test.AdapterTester to validate core behavior.

import unittest

import pluca_xyz
from pluca.test import AdapterTester


class TestXyz(AdapterTester, unittest.TestCase):
    def get_adapter(self) -> pluca_xyz.Adapter:
        return pluca_xyz.Adapter(endpoint='127.0.0.1:11211')

This checks behavior through pluca.Cache wrapping and validates standard cache semantics.

Then add backend-specific tests for features such as:

  • connection/auth failures
  • native TTL handling
  • native bulk operation behavior
  • resource lifecycle in shutdown()

Final rule

If an optimization conflicts with standard pluca.Cache behavior, preserve standard behavior.