soev for developers
WarrenReferencewarren.storage

warren.storage.cache

Cache: the cache interface, and the in-memory and Redis caches.

Classes

MemoryCache

classMemoryCache[T](*, default_ttl_seconds=None, clock=time.monotonic, name=None)

Cache held in a dict, with TTLs honoured lazily on access.

Each instance is its own namespace, so there is no base_key: two caches never share keys. Expiry is checked when a key is read, not by a timer, which keeps the class free of background tasks.

paramdefault_ttl_secondsint | None
= None

TTL applied when set gets none. None means entries never expire.

paramclockCallable[[], float]
= time.monotonic

Monotonic clock, injectable so tests can advance time.

paramnamestr | None
= None

Optional name for logging.

methodgetasync (key) -> T | None
paramkeystr
methodsetasync (key, value, ttl_seconds=None) -> None
paramkeystr
paramvalueT
paramttl_secondsint | None
= None
methoddeleteasync (key) -> bool
paramkeystr
methodexistsasync (key) -> bool
paramkeystr
methodget_by_key_prefixasync (key_prefix) -> dict[str, T]
paramkey_prefixstr
methodset_manyasync (items, ttl_seconds=None) -> None
paramitemsdict[str, T]
paramttl_secondsint | None
= None
methoddelete_by_key_prefixasync (key_prefix) -> int
paramkey_prefixstr
methodclearasync () -> None

RedisBinaryCache

classRedisBinaryCache(client, *, base_key, default_ttl_seconds=3600, key_separator=':', name=None, scoping_enabled=False, scope_required=True)

Async Redis cache for binary data (pass-through, no serialization).

Suitable for binary values.

paramclientRedis
parambase_keystr
paramdefault_ttl_secondsint | None
= 3600
paramkey_separatorstr
= ':'
paramnamestr | None
= None
paramscoping_enabledbool
= False
paramscope_requiredbool
= True

RedisCacheBase

classRedisCacheBase[T](client, *, base_key, default_ttl_seconds=3600, key_separator=':', name=None, scoping_enabled=False, scope_required=True)

Abstract base class for async Redis-backed cache implementations.

Uses redis.asyncio for non-blocking I/O. Handles Redis connection, key namespacing, TTL management, and error handling. Subclasses must implement _serialize and _deserialize for their specific types.

Initialize the Redis cache.

paramclientRedis

Async Redis client instance (injected, not created internally).

parambase_keystr

Namespace prefix for all keys in this cache.

paramdefault_ttl_secondsint | None
= 3600

Default expiry in seconds. None uses 3600. Must be positive, or ValueError.

paramkey_separatorstr
= ':'

Separator between base_key and key. Defaults to ":" (Redis convention). Colon is preferred over dot because dots commonly appear in data (filenames, versions).

paramnamestr | None
= None

optional name for logging purposes

paramscoping_enabledbool
= False

Prefix each key with the scope of the current handler: s:{scope}:{base_key}:{key}.

paramscope_requiredbool
= True

With scoping on, raise HardFailureException (not CacheOperationError) when the handler has no scope. False: use the key without a prefix.

methodgetasync (key) -> T | None

Retrieve value from cache.

paramkeystr

Cache key.

Returns

T | None

Cached value, or None if not found.

Raises:

  • CacheOperationError: If Redis operation fails.
methodsetasync (key, value, ttl_seconds=None) -> None

Store value in cache.

paramkeystr

Cache key.

paramvalueT

Value to cache.

paramttl_secondsint | None
= None

Time-to-live in seconds. None uses default TTL. Must be positive, or CacheOperationError.

Raises:

  • CacheOperationError: If Redis operation fails.
methoddeleteasync (key) -> bool

Delete key from cache.

paramkeystr

Cache key.

Returns

bool

True if key existed and was deleted, False otherwise.

Raises:

  • CacheOperationError: If Redis operation fails.
methodexistsasync (key) -> bool

Check if key exists in cache.

paramkeystr

Cache key.

Returns

bool

True if key exists.

Raises:

  • CacheOperationError: If Redis operation fails.
methodget_by_key_prefixasync (key_prefix) -> dict[str, T]

Get all items where key starts with key_prefix.

IMPORTANT: The prefix is matched literally with a wildcard appended. To match only sub-keys of a hierarchical key, include the hierarchy separator in the prefix. Example with keys "doc123:0", "doc123:1", "doc1234:0":

  • key_prefix="doc123" matches: doc123:0, doc123:1, doc1234:0 (too broad!)
  • key_prefix="doc123:" matches: doc123:0, doc123:1 only (sub-keys only)
paramkey_prefixstr

Key prefix to match.

Returns

dict[str, T]

Dictionary mapping matching keys to their values.

Raises:

  • CacheOperationError: If Redis operation fails.
methodset_manyasync (items, ttl_seconds=None) -> None

Store multiple key-value pairs.

paramitemsdict[str, T]

Dictionary of key-value pairs to cache.

paramttl_secondsint | None
= None

Time-to-live in seconds. None uses default TTL. Must be positive, or CacheOperationError.

Raises:

  • CacheOperationError: If Redis operation fails.
methoddelete_by_key_prefixasync (key_prefix) -> int

Delete all items where key starts with key_prefix.

IMPORTANT: The prefix is matched literally with a wildcard appended. To delete only sub-keys of a hierarchical key, include the hierarchy separator in the prefix. Example with keys "doc123:0", "doc123:1", "doc1234:0":

  • key_prefix="doc123" deletes: doc123:0, doc123:1, doc1234:0 (too broad!)
  • key_prefix="doc123:" deletes: doc123:0, doc123:1 only (sub-keys only)
paramkey_prefixstr

Key prefix to match.

Returns

int

Number of keys deleted.

Raises:

  • CacheOperationError: If Redis operation fails.
methodclearasync () -> None

Clear all entries from this cache namespace.

Raises:

  • CacheOperationError: If Redis operation fails.

RedisDictCache

classRedisDictCache(client, *, base_key, default_ttl_seconds=3600, key_separator=':', name=None, scoping_enabled=False, scope_required=True)

Async Redis cache for Dict values using JSON serialization.

Suitable for caching document metadata, extracted text, processing results, and other JSON-serializable data. Datetime values are stored as ISO 8601 text and read back as strings; models that declare a datetime field (such as ResultDoc.created_at) parse them back on validation.

paramclientRedis
parambase_keystr
paramdefault_ttl_secondsint | None
= 3600
paramkey_separatorstr
= ':'
paramnamestr | None
= None
paramscoping_enabledbool
= False
paramscope_requiredbool
= True

Protocols

CacheInterface

protocolCacheInterface[T]

Generic async cache interface for storing typed values.

All methods are async — implementations must use non-blocking I/O (e.g., redis.asyncio) to avoid blocking the event loop.

Implementations should be specialized for specific value types (e.g., dict values or binary values).

A cache is one namespace: the Redis caches prefix each key with a base_key. Each MemoryCache instance is its own namespace.

Key hierarchy convention: keys can be hierarchical, with ":" as the separator (e.g., "doc123:0" and "doc123:1" for the parts of document doc123). For the prefix operations (get_by_key_prefix, delete_by_key_prefix), include the trailing separator to match only sub-keys:

  • "doc123" matches: doc123, doc123:0, doc1234:0 (can be too broad)
  • "doc123:" matches: doc123:0, doc123:1 only (sub-keys only)
methodgetasync (key) -> T | None

Retrieve value from cache.

paramkeystr

Cache key.

Returns

T | None

Cached value, or None if not found.

methodsetasync (key, value, ttl_seconds=None) -> None

Store value in cache.

paramkeystr

Cache key.

paramvalueT

Value to cache.

paramttl_secondsint | None
= None

Time-to-live in seconds. None uses default TTL.

methoddeleteasync (key) -> bool

Delete key from cache.

paramkeystr

Cache key.

Returns

bool

True if key existed and was deleted, False otherwise.

methodexistsasync (key) -> bool

Check if key exists in cache.

paramkeystr

Cache key.

Returns

bool

True if key exists.

methodget_by_key_prefixasync (key_prefix) -> dict[str, T]

Get all items where key starts with key_prefix.

IMPORTANT: The prefix is matched literally with a wildcard appended. To match only sub-keys, include the hierarchy separator (":") at the end. See class docstring for key hierarchy convention.

paramkey_prefixstr

Key prefix to match.

Returns

dict[str, T]

Dictionary mapping matching keys to their values.

methodset_manyasync (items, ttl_seconds=None) -> None

Store multiple key-value pairs.

paramitemsdict[str, T]

Dictionary of key-value pairs to cache.

paramttl_secondsint | None
= None

Time-to-live in seconds. None uses default TTL.

methoddelete_by_key_prefixasync (key_prefix) -> int

Delete all items where key starts with key_prefix.

IMPORTANT: The prefix is matched literally with a wildcard appended. To delete only sub-keys, include the hierarchy separator (":") at the end. See class docstring for key hierarchy convention.

paramkey_prefixstr

Key prefix to match.

Returns

int

Number of keys deleted.

methodclearasync () -> None

Clear all entries from this cache namespace.

Functions

get_or_set

functionget_or_setasync [T](cache, key, factory, ttl_seconds=None) -> T

Return cached value, or compute via async factory, cache it, and return.

This implements the cache-aside pattern: if the key exists in cache, return the cached value. Otherwise, call factory() to compute the value, store it in cache, and return it.

Example with functools.partial:

import asyncio
from functools import partial
from warren.storage.cache import MemoryCache, get_or_set

async def load(key: str) -> dict:
    return {"key": key}

async def main() -> None:
    cached = partial(get_or_set, MemoryCache[dict]())
    print(await cached("doc-1", lambda: load("doc-1")))

asyncio.run(main())
paramcacheCacheInterface[T]

Cache instance to use.

paramkeystr

Cache key.

paramfactoryCallable[[], Awaitable[T]]

Async callable that produces the value if not cached.

paramttl_secondsint | None
= None

Time-to-live in seconds. None uses cache default.

Returns

T

Cached or newly computed value.

Errors

CacheOperationError

errorCacheOperationError(message=None, operation=None)

Raised when a cache operation fails.

parammessagestr | None
= None

Optional error message with details.

paramoperationstr | None
= None

Optional operation name (e.g., "get", "set", "delete") to make the error message more specific.

warren.storage

Storage: the store errors, request scoping and the helpers that the stores share.

warren.storage.document_store

Document store: the document store interface and its MongoDB and in-memory implementations.

On this page

ClassesMemoryCacheRedisBinaryCacheRedisCacheBaseRedisDictCacheProtocolsCacheInterfaceFunctionsget_or_setErrorsCacheOperationError