Caching¶

Reduce costs and latency with intelligent HTTP-level response caching. PHP LLM caches LLM responses automatically, making repeated requests nearly instant and free.

This page covers two independent features:

HTTP response caching (below) — identical requests are served from a local cache and never reach the API.
Provider-side prompt caching — the API bills the repeated part of a prompt at a reduced rate. Relevant for multi-turn tool loops.

Overview¶

All PHP LLM clients support caching at the HTTP request level. When enabled: - Identical requests return cached responses instantly - No API calls are made for cached requests - Original response time is preserved in metadata - Costs are eliminated for cached responses - Streaming and non-streaming requests share the same cache entries

File Cache¶

The built-in FileCache stores responses on the filesystem. The directory must already exist — the constructor throws a RuntimeException otherwise:

<?php
use Soukicz\Llm\Cache\FileCache;
use Soukicz\Llm\Client\Anthropic\AnthropicClient;

$cache = new FileCache(sys_get_temp_dir());
$client = new AnthropicClient('sk-xxxxx', $cache);

Characteristics: - ✅ Simple to set up - ✅ No additional dependencies - ✅ Works across requests - ⚠️ Limited to single server - ⚠️ Manual cleanup required

Custom Cache Directory¶

<?php
$cache = new FileCache('/var/cache/llm');
$client = new AnthropicClient('sk-xxxxx', $cache);

DynamoDB Cache¶

For distributed systems, use the DynamoDB cache extension:

composer require soukicz/llm-cache-dynamodb

<?php
use Soukicz\Llm\Cache\DynamoDB\DynamoDBCache;
use Aws\DynamoDb\DynamoDbClient;

$dynamodb = new DynamoDbClient([
    'region' => 'us-east-1',
    'version' => 'latest',
]);

$cache = new DynamoDBCache($dynamodb, 'llm-cache-table');
$client = new AnthropicClient('sk-xxxxx', $cache);

Characteristics: - ✅ Distributed across servers - ✅ Automatic TTL expiration - ✅ Scalable - ❌ Requires AWS setup - ❌ Additional costs

Custom Cache Implementation¶

The cache operates on PSR-7 HTTP messages. The CacheInterface has three methods:

<?php
use Psr\Http\Message\RequestInterface;
use Psr\Http\Message\ResponseInterface;

interface CacheInterface {
    public function fetch(RequestInterface $request): ?ResponseInterface;

    public function store(RequestInterface $request, ResponseInterface $response): void;

    public function invalidate(RequestInterface $request): void;
}

For custom backends, extend AbstractCache — it provides getCacheKey(RequestInterface): string (a SHA-512 hash of URL, method and body) plus responseToJson()/responseFromJson() helpers for serializing PSR-7 responses:

<?php
use Psr\Http\Message\RequestInterface;
use Psr\Http\Message\ResponseInterface;
use Soukicz\Llm\Cache\AbstractCache;

class RedisCache extends AbstractCache {
    public function __construct(
        private readonly Redis $redis,
        private readonly int $ttl = 3600
    ) {}

    public function fetch(RequestInterface $request): ?ResponseInterface {
        $json = $this->redis->get($this->getCacheKey($request));

        return $json !== false ? $this->responseFromJson($json) : null;
    }

    public function store(RequestInterface $request, ResponseInterface $response): void {
        $this->redis->setex($this->getCacheKey($request), $this->ttl, $this->responseToJson($response));
    }

    public function invalidate(RequestInterface $request): void {
        $this->redis->del($this->getCacheKey($request));
    }
}

<?php
$cache = new RedisCache($redisClient);
$client = new AnthropicClient('sk-xxxxx', $cache);

Cache Keys¶

Cache keys are a SHA-512 hash of the HTTP request: - Request URL (API endpoint, including the model for Gemini) - HTTP method - Request body (model, temperature, maxTokens, conversation messages, tool definitions, ...)

Any change to the request body produces a new cache key.

Important: Always use exact model versions to prevent stale cached responses.

Security caveat: The cache key does not include request headers, so API keys are not part of the key. Identical requests share cache entries regardless of which credentials were used. The cache is intended for development, testing and request deduplication — do not rely on it for multi-tenant isolation.

Best Practices¶

Use Exact Model Versions¶

❌ Bad - Generic naming

<?php
// Vague version could cache responses from old models
$model = new AnthropicClaude45Sonnet('latest');

✅ Good - Explicit version

<?php
// Specific version ensures cache correctness
$model = new AnthropicClaude45Sonnet(AnthropicClaude45Sonnet::VERSION_20250929);

Development vs Production¶

Development:

<?php
// Aggressive caching to save costs during development
$cache = new FileCache('/tmp/llm-cache');
$client = new AnthropicClient('sk-xxxxx', $cache);

Production:

<?php
// Distributed cache for multi-server setup
$cache = new DynamoDBCache($dynamodb, 'prod-llm-cache');
$client = new AnthropicClient('sk-xxxxx', $cache);

Cache Warming¶

Pre-cache common requests:

<?php
// Warm cache with common queries
$commonQueries = [
    'What is PHP?',
    'How do I install composer?',
    'What are PHP traits?',
];

foreach ($commonQueries as $query) {
    $response = $agentClient->run(
        client: $client,
        request: new LLMRequest(
            model: $model,
            conversation: new LLMConversation([
                LLMMessage::createFromUserString($query)
            ])
        )
    );
}

Disabling Cache¶

To bypass cache for specific requests, create a client without cache:

<?php
// No cache
$client = new AnthropicClient('sk-xxxxx', null);

Cache Behavior¶

What Gets Cached¶

✅ Successful responses ✅ Complete conversations ✅ Tool call results ✅ Multimodal requests

What Doesn't Get Cached¶

❌ Failed requests (errors) ❌ Incomplete responses ❌ Async requests in progress

Monitoring Cache Performance¶

Track cache hit rates by decorating another cache:

<?php
use Psr\Http\Message\RequestInterface;
use Psr\Http\Message\ResponseInterface;
use Soukicz\Llm\Cache\CacheInterface;

class CacheMonitor implements CacheInterface {
    private int $hits = 0;
    private int $misses = 0;

    public function __construct(
        private readonly CacheInterface $cache
    ) {}

    public function fetch(RequestInterface $request): ?ResponseInterface {
        $response = $this->cache->fetch($request);
        if ($response !== null) {
            $this->hits++;
        } else {
            $this->misses++;
        }

        return $response;
    }

    public function store(RequestInterface $request, ResponseInterface $response): void {
        $this->cache->store($request, $response);
    }

    public function invalidate(RequestInterface $request): void {
        $this->cache->invalidate($request);
    }

    public function getHitRate(): float {
        $total = $this->hits + $this->misses;
        return $total > 0 ? $this->hits / $total : 0;
    }
}

<?php
$cache = new CacheMonitor(new FileCache('/tmp/cache'));
$client = new AnthropicClient('sk-xxxxx', $cache);

// After some requests...
echo "Cache hit rate: " . ($cache->getHitRate() * 100) . "%\n";

Cache Expiration¶

Manual Cleanup¶

invalidate() removes the entry for a specific PSR-7 HTTP request. Since you usually don't have the underlying HTTP request at hand, the simplest cleanup for FileCache is to delete the cache files:

<?php
// Clear all cache (FileCache stores one .json file per entry)
array_map('unlink', glob('/tmp/llm-cache/*.json'));

Automatic Expiration¶

Implement TTL in a custom cache by extending AbstractCache:

<?php
use Psr\Http\Message\RequestInterface;
use Psr\Http\Message\ResponseInterface;
use Soukicz\Llm\Cache\AbstractCache;

class TTLFileCache extends AbstractCache {
    public function __construct(
        private readonly string $directory,
        private readonly int $ttl = 3600
    ) {}

    private function getPath(RequestInterface $request): string {
        return $this->directory . '/' . md5($this->getCacheKey($request)) . '.json';
    }

    public function fetch(RequestInterface $request): ?ResponseInterface {
        $file = $this->getPath($request);

        if (!file_exists($file)) {
            return null;
        }

        // Check if expired
        if (time() - filemtime($file) > $this->ttl) {
            unlink($file);

            return null;
        }

        return $this->responseFromJson(file_get_contents($file));
    }

    public function store(RequestInterface $request, ResponseInterface $response): void {
        file_put_contents($this->getPath($request), $this->responseToJson($response), LOCK_EX);
    }

    public function invalidate(RequestInterface $request): void {
        @unlink($this->getPath($request));
    }
}

Cost Savings¶

Example cost calculation:

<?php
$request = new LLMRequest(/*...*/);

// First request - hits the API
$response1 = $agentClient->run($client, $request);
echo "Cost: $" . ($response1->getInputPriceUsd() + $response1->getOutputPriceUsd()) . "\n";

// Identical request - served from the cache, no API call is made
$response2 = $agentClient->run($client, $request);

// 100% savings on repeated requests!

Note that the reported price is calculated from the token counts in the response, so a cached response still reports the original cost — but no API call is made and nothing is billed.

Provider-Side Prompt Caching¶

Independent of the HTTP cache above, providers can cache the repeated prefix of a prompt server-side and bill it at a fraction of the input price (Anthropic: ~10% for cache reads).

Static content¶

Mark stable content (instructions, documents) with the cached constructor flag — the Anthropic encoder emits a cache_control breakpoint for it:

<?php
use Soukicz\Llm\Message\LLMMessageText;

new LLMMessageText('Long extraction instructions and document...', cached: true);

Put the flag on user-message content. A cached flag on the system message itself is ignored — the Anthropic encoder sends the system prompt as a plain string without cache support.

Multi-turn tool loops¶

In an agent loop the conversation grows every turn (tool calls + tool results), and without further breakpoints this growing tail is re-billed at full input price on every turn — quadratically with the number of turns. Opt in to conversation caching to place a moving breakpoint at the end of the conversation on every request:

<?php
use Soukicz\Llm\Config\ConversationCacheConfig;

$request = new LLMRequest(
    model: $model,
    conversation: $conversation,
    tools: $tools,
    conversationCacheConfig: new ConversationCacheConfig(),
);

Each turn then reads all previous turns from cache and pays full price only for the newly added messages.

Behavior details:

Breakpoints are added at encode time and are never persisted into the conversation, so they move with the conversation instead of accumulating.
The encoder marks the last two user messages and always respects Anthropic's limit of 4 breakpoints per request; explicit cached flags set by you take priority.
The default cache lifetime is 5 minutes; pass new ConversationCacheConfig(ttl: CacheTtl::ONE_HOUR) for slow loops (1-hour cache writes cost more). A configured TTL applies to every breakpoint in the request, including content you marked with cached — Anthropic requires longer-lived cache entries to come before shorter-lived ones, so TTLs cannot be mixed in that order.
OpenAI and Gemini cache prompts automatically, so the option is a no-op there.
Prompts shorter than the provider's minimum (e.g. 1024 tokens for Claude Sonnet) are silently not cached.