RI2 — Agent Observability
Agent Trace Viewer
Every LLM call made by the ops agents, structured as a trace. Model, token counts, latency, estimated cost, and what the agent decided. No observability platform. Just DynamoDB and HTML.
How traces are recorded
Each agent writes a
Each agent writes a
TYPE#llm_trace record after every LLM call.
The record includes the model, token usage, latency, a SHA-256 hash of the
prompt prefix (not the raw prompt), and the agent's decision summary.
Loading LLM trace records...
What this demonstrates
Structured trace records
Each record has a stable schema: agent, model, input/output tokens, latency_ms, cost_usd, prompt_hash, decision_summary, run_id. The schema is enforced in
Each record has a stable schema: agent, model, input/output tokens, latency_ms, cost_usd, prompt_hash, decision_summary, run_id. The schema is enforced in
write_llm_trace() in agent_base.py.
No platform required
Langfuse, Honeycomb, and Datadog all require running infrastructure. This works with one DynamoDB table and a Lambda function that already exists. The trace viewer is 150 lines of HTML.
Langfuse, Honeycomb, and Datadog all require running infrastructure. This works with one DynamoDB table and a Lambda function that already exists. The trace viewer is 150 lines of HTML.
Privacy by default
The raw prompt is never stored. Only a SHA-256 hash of the first 200 characters. This lets you detect repeated prompts (e.g. recurring daily runs) without storing potentially sensitive input data.
The raw prompt is never stored. Only a SHA-256 hash of the first 200 characters. This lets you detect repeated prompts (e.g. recurring daily runs) without storing potentially sensitive input data.
Cost tracking
Each trace estimates cost from token counts using published Haiku pricing. Aggregating across traces gives a running total with no external billing API.
Each trace estimates cost from token counts using published Haiku pricing. Aggregating across traces gives a running total with no external billing API.
Read the companion article: Agent observability without the observability platform