Every CAELUM client keeps local metrics automatically. No setup, no account, nothing sent anywhere: metrics live in your process memory.
from caelum import OpenAI
client = OpenAI(cache=True)
# ... your application makes requests ...
client.metrics.print_summary()
CAELUM Local Metrics
Requests: 1 240
Cache hits: 743 (60%)
Cost saved: $ 24.30
Cost spent: $ 16.10
Avg latency: 180ms
p95 latency: 410ms
With the rich extra installed (pip install "caelum-ai[rich]") the summary is rendered as
a table; otherwise it is printed as plain text.
Programmatic access
summary = client.metrics.summary()
summary["cache_hit_rate"] # 0.6
summary["cost_saved_usd"] # 24.3
summary["latency_p95_ms"] # 410.0
client.metrics.export_json("metrics.json")
client.metrics.reset()
| Key | Meaning |
|---|---|
requests |
Requests handled by the client (including cache hits and failures). |
cache_hits, cache_hits_exact, cache_hits_semantic |
Requests answered from a cache. |
cache_misses |
Cacheable requests answered by the API (primary provider). Fallback answers and failed requests are counted in fallbacks and failed_requests instead. |
api_calls |
Requests answered by the provider. |
failed_requests |
Requests that raised an error. |
retries, fallbacks |
Retry attempts and provider switches. |
cost_usd |
Estimated spend (API calls + semantic-cache embeddings). |
cost_saved_usd |
Estimated cost of the answers served from cache. |
unpriced_requests |
Requests whose cost is unknown (model not in the price table, streams). |
latency_avg_ms, latency_p50_ms, latency_p95_ms |
End-to-end latency as seen by your code. |
requests_by_model |
Request count per requested model. |
Metrics are shared between a client and its copy() / with_options() derivatives, and
are thread-safe.
Cost estimates
Costs are estimates computed from the token usage returned by the provider and a
price table maintained by hand in caelum.pricing (last updated: see
caelum.pricing.PRICING_LAST_UPDATED). Prices change: always rely on your provider's
invoice for billing.
Add or override a model price (USD per 1M tokens):
from caelum import pricing
pricing.register_price("my-fine-tuned-model", input_per_1m=3.0, output_per_1m=12.0)
Dated snapshots resolve to their base model: gpt-4o-2024-08-06 uses the gpt-4o price.
Streaming requests are counted, but their cost is not estimated in this version.