Skip to main content
OpenTracy ghost mascot peacefully watching the traffic — traces are quiet, always-on observation A trace is a row that records one LLM call: what was asked, what came back, which model answered, how much it cost, and how long it took. Traces are the single source of truth for everything downstream: auto-routing, evaluation, dataset curation, and distillation. If you don’t have traces, you don’t have a product.

​
What’s in a trace

Every completion that flows through the OpenTracy engine is persisted with this shape:
FieldTypeMeaning
trace_iduuidPrimary key for this single request.
tenant_idstringWhich project / workspace this trace belongs to.
timestampdatetimeUTC start of the request.
modelstringConcrete model the request was answered by (e.g. gpt-4o-mini).
providerstringopenai, anthropic, groq, … or opentracy if the student.
input_textstringThe user message(s) — full content.
output_textstringThe assistant response — full content.
input_tokensintPrompt tokens.
output_tokensintCompletion tokens.
total_cost_usdfloatUSD cost for this request (uses live pricing tables).
latency_msfloatEnd-to-end time in milliseconds.
statusstringsuccess, error, timeout.
routing_aliasstring?The alias the app asked for, e.g. smart. Null if direct.
cluster_idint?Which semantic cluster the prompt landed in (after embedding).
metadatajsonAnything you attach — user_id, session_id, A/B flag, custom tags.
Where they live: a ClickHouse column-store, designed so that billion-row aggregations (cost per model per day, latency p99 per alias) stay fast. How they’re created: you don’t call a separate log_trace() API. Any request that hits /v1/chat/completions on the engine — whether from the Python SDK, the OpenAI SDK pointed at the engine URL, or a raw curl — is traced automatically.

​
How traces are created (three paths)

​
Path A — Python SDK

import opentracy as ot

resp = ot.completion(
    model="openai/gpt-4o-mini",
    messages=[{"role": "user", "content": "..."}],
)
# A trace for this request is already in ClickHouse. The response also
# carries the usage summary as attached metadata:
print(resp._cost, resp._latency_ms, resp._routing)

​
Path B — OpenAI SDK (drop-in)

from openai import OpenAI

client = OpenAI(
    base_url="http://localhost:8080/v1",   # OpenTracy engine
    api_key="any",                         # engine handles provider auth
)

resp = client.chat.completions.create(
    model="openai/gpt-4o-mini",
    messages=[{"role": "user", "content": "..."}],
)
# Traced. No library changes in your app.

​
Path C — Raw HTTP

curl http://localhost:8080/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{"model": "openai/gpt-4o-mini", "messages": [{"role": "user", "content": "..."}]}'
Any OpenAI-compatible client works, because the engine speaks that protocol.

​
Inspecting traces

The REST API exposes a search interface: 
curl "http://localhost:8000/v1/traces?model=gpt-4o-mini&limit=10"
Returns a list of trace rows matching the filter. Filters supported include model, provider, routing_alias, status, tenant_id, since, until, and full-text search on input_text / output_text. In the UI, traces live under Traces → Browse with filters, a cost/latency chart on top, and a detail drawer that shows the full prompt + response and lets you “add to dataset”.

​
Attaching metadata

You control metadata — use it to link traces back to your app’s world: 
resp = ot.completion(
    model="openai/gpt-4o-mini",
    messages=[{"role": "user", "content": "..."}],
    metadata={
        "user_id": "u_42",
        "session_id": "s_7af",
        "feature": "ticket_classifier",
        "ab_variant": "B",
    },
)
When you go to build a dataset later, these fields are how you say “give me all traces from the ticket_classifier feature where the user gave positive feedback”.

​
Privacy and PII

Traces store full prompt and response text by default — that’s what makes distillation possible. Two knobs for environments that can’t do that:
  • Set OPENTRACY_TRACE_REDACT=true to strip matched patterns (emails, phone numbers, credit cards) before persist.
  • Set OPENTRACY_TRACE_CONTENT=false to store only metadata + token counts + cost, dropping the text entirely. You lose distillation ability but keep cost analytics.
Per-trace override is possible via the trace header: 
resp = ot.completion(
    ...,
    extra_headers={"X-Opentracy-Trace": "metadata-only"},
)

​
From trace to dataset

One trace is a row. A dataset is a curated bundle of many rows, grouped by intent. The transformation is covered in Datasets — that’s where the pipeline starts producing training-ready artifacts.