Skip to main content
Hyperterse includes a built-in in-memory cache. The cache stores tool results keyed by tool name and substituted statement, reducing redundant connector execution for identical requests. Caching applies to DB-backed tools only. Script-backed tools (handler-only) bypass the cache.

​
Cache key derivation

Each entry is identified by: 
cache_key = hash(tool_name + substituted_statement)
The same tool with different input values produces different cache entries. Identical inputs for the same tool produce a cache hit.

​
Configuration

Caching is controlled at two levels: global defaults in the root config and per-tool overrides.

​
Global defaults

Set default caching in .hyperterse: 
tools:
  cache:
    enabled: true
    ttl: 60
FieldTypeDefaultDescription
enabledbooleanfalseWhether caching is active for all DB-backed tools.
ttlinteger120Time-to-live in seconds.

​
Per-tool override

cache:
  enabled: true
  ttl: 30

cache:
  enabled: false

​
Precedence

  1. Tool-level config (highest priority).
  2. Global root config (tools.cache).
  3. Runtime defaults (enabled: false, ttl: 120).

​
Behavior

The cache operates as a read-through layer in front of connectors and handler scripts.

​
On execution

  1. Before connector execution, the executor checks the cache using the derived key.
  2. Hit: cached result returned immediately; connector is not called.
  3. Miss: connector executes; result stored with configured TTL before being returned.

​
Eviction

  • TTL-based. Entries expire after their configured TTL.
  • Capacity-based. The cache enforces memory bounds (128 MiB default). When approaching capacity, least-recently-used entries are evicted.

​
No explicit invalidation

There is no API for manual cache invalidation. Entries are evicted only by TTL or capacity pressure. For immediate invalidation, disable caching on affected tools and manage cache externally (e.g., through a Redis adapter with handler logic).

​
Characteristics

PropertyValue
ScopeProcess-local. Not shared between instances.
StorageIn-memory. No persistence across restarts.
Thread safetyConcurrent-safe. Reads and writes do not block execution.
SerializationResults are cloned on store and retrieval to prevent mutation.
DistributedNone. Each instance maintains its own cache.

​
When to enable caching

Enable for:
  • Read-heavy tools with stable data (reference tables, configuration lookups).
  • Expensive queries where staleness within the TTL window is acceptable.
  • High-frequency tools where connector load reduction matters.
Disable for:
  • Write operations or tools that must return real-time data.
  • Highly variable inputs where cache fill exceeds hit rate.
  • Tools where result freshness is critical for correctness.

​
Monitoring

Cache hit/miss status is included in OpenTelemetry trace spans when observability is configured. Monitor hit ratio to evaluate whether TTL values are effective for your workload.