Prompt caching

When to enable caching

• System prompt > ~5KB called repeatedly (chatbots, agent loops)
• RAG with a fixed retrieval prefix + varying questions
• Code assistant pinning a whole file as context
• Long-doc Q&A: many questions over same doc

Minimum token thresholds

Minimum prefix sizes and TTLs vary by model. Short prompts usually do not benefit much; verify cache hits through usage.cached_tokens before relying on it.

Example

import os
from openai import OpenAI

client = OpenAI(
    base_url="https://api.sealink.io/v1",
    api_key=os.environ["SEALINK_API_KEY"],
)

# Long, reusable system prompt — cached on the first call.
SYSTEM = open("knowledge_base.md").read()  # imagine 50KB

resp = client.chat.completions.create(
    model="claude-opus-5-5",
    messages=[
        {
            "role": "system",
            "content": SYSTEM,
            # Use only with models that support cache_control.
            "cache_control": {"type": "ephemeral"},
        },
        {"role": "user", "content": "Question 1"},
    ],
)
# Subsequent calls may report cached_tokens when the cache is used.

print(resp.usage.prompt_tokens_details)
# {"cached_tokens": 12500}

Models that support caching

• Check the model detail page for caching support, TTL, and billing behavior.
• When a response includes cached_tokens, SeaLink records it in usage data.

Seeing cache hits in your console

Check cached_tokens in usage records. If it stays at 0, the selected model, prompt shape, or call pattern is not producing effective cache hits.