Kalmia finds errors hiding in your agent traces

We triage your traces and spawn specialized investigator agents to catch and contextualize your hidden agent errors.

Get started

Kalmia

Failed2

7c31be09

Failed

[{"role": "user", "content": "Compile the weekly pipeline digest and email it to the founders list."}]

41.0s18.3k6Aug 30, 2:02 PM

d3a92f41

Failed

[{"role": "user", "content": "Compile the weekly pipeline digest and email it to the founders list."}]

72.0s86.9k8Aug 30, 2:02 PM

Healthy2

b81d4c26

Healthy

[{"role": "user", "content": "Compile the weekly pipeline digest and email it to the founders list."}]

41.0s18.3k6Aug 30, 2:02 PM

09e6f3a2

Healthy

[{"role": "user", "content": "Compile the weekly pipeline digest and email it to the founders list."}]

41.0s18.3k6Aug 30, 2:02 PM

Unreviewed7

a409fc51

Completed

[{"role": "user", "content": "Compile the weekly pipeline digest and email it to the founders list."}]

41.0s18.3k6Aug 30, 2:02 PM

543c854e

Completed

[{"role": "user", "content": "Compile the weekly pipeline digest and email it to the founders list."}]

41.0s18.3k6Aug 30, 2:02 PM

ddb0447b

Completed

[{"role": "user", "content": "Compile the weekly pipeline digest and email it to the founders list."}]

41.0s18.3k6Aug 30, 2:02 PM

a6242e13

Completed

[{"role": "user", "content": "Compile the weekly pipeline digest and email it to the founders list."}]

41.0s18.3k6Aug 30, 2:02 PM

0b169dc0

Completed

[{"role": "user", "content": "Compile the weekly pipeline digest and email it to the founders list."}]

41.0s18.3k6Aug 30, 2:02 PM

1e8ede22

Completed

[{"role": "user", "content": "Compile the weekly pipeline digest and email it to the founders list."}]

41.0s18.3k6Aug 30, 2:02 PM

d4606b25

Completed

[{"role": "user", "content": "Compile the weekly pipeline digest and email it to the founders list."}]

41.0s18.3k6Aug 30, 2:02 PM

Healthy3

f8b12892

Healthy

[{"role": "user", "content": "Compile the weekly pipeline digest and email it to the founders list."}]

41.0s18.3k6Aug 30, 2:02 PM

6a77d408

Healthy

[{"role": "user", "content": "Compile the weekly pipeline digest and email it to the founders list."}]

41.0s18.3k6Aug 30, 2:02 PM

631a1a2c

Healthy

[{"role": "user", "content": "Compile the weekly pipeline digest and email it to the founders list."}]

41.0s18.3k6Aug 30, 2:02 PM

Unreviewed7

247abd59

Completed

[{"role": "user", "content": "Compile the weekly pipeline digest and email it to the founders list."}]

41.0s18.3k6Aug 30, 2:02 PM

05036ffa

Completed

[{"role": "user", "content": "Compile the weekly pipeline digest and email it to the founders list."}]

41.0s18.3k6Aug 30, 2:02 PM

93d95770

Completed

[{"role": "user", "content": "Compile the weekly pipeline digest and email it to the founders list."}]

41.0s18.3k6Aug 30, 2:02 PM

9f159139

Completed

[{"role": "user", "content": "Compile the weekly pipeline digest and email it to the founders list."}]

41.0s18.3k6Aug 30, 2:02 PM

c637821c

Completed

[{"role": "user", "content": "Compile the weekly pipeline digest and email it to the founders list."}]

41.0s18.3k6Aug 30, 2:02 PM

dd24e4a9

Completed

[{"role": "user", "content": "Compile the weekly pipeline digest and email it to the founders list."}]

41.0s18.3k6Aug 30, 2:02 PM

7978c3ba

Completed

[{"role": "user", "content": "Compile the weekly pipeline digest and email it to the founders list."}]

41.0s18.3k6Aug 30, 2:02 PM

source=kalmia-agent-eventsoutreach-digest-agent

d3a92f41-7c1e-4b8a-9f03-52ee71c0a8b4

17 spans · 1.2m · 86.9k tok

Trace summary

The agent was asked to compile the weekly pipeline digest and email it to the founders list. It paged open deals out of the CRM, hit a 429 rate limit on page 3, and re-exported the full 214-row set in 50-row batches. It then computed week-over-week deltas, rendered the digest, and delivered it to 8 recipients. 8 tool calls across 4 tools, 1 retry.

Errors

Retry merged instead of replaced

After the CRM export was rate-limited on page 3, the agent re-exported the complete 214-row set in batches, then appended it to the 160 partial rows it already held.

90b17f3eanthropic chat

Pipeline totals double-counted

The merged set reports 374 deals and $6.3M of open pipeline. 63 of those deals appear twice; the CRM holds 214.

e51f70d2compute_deltas

Inflated digest sent without a check

The digest claimed +38% week-over-week growth against a true +9%, and went out to all 8 founders before any reconciliation ran.

a9e07c44send_email

“For auditing agent behavior, Kalmia is now the best tool we have, and it isn't close.”

“Using Kalmia, we've been able to cut total API spend in half.”

Aayush · Founder & CEO

How Kalmia catches agent errors

Detector agents triage your traces

Detector agents triage your traces and cluster what looks wrong.

Investigator agents dig in

Investigator agents dig through the trace and find errors other tools miss.

Kalmia learns your system over time

Findings become knowledge that improves future investigations.

Connects to the
tools you already use

Keep the observability stack you already run: whatever emits your traces today can send them to Kalmia too.

Agent failures and needs-attention findings reach you as alerts and digests in Slack, Microsoft Teams, or email, with the evidence attached.

Export from your existing observability stack

OpenTelemetryBraintrustLangfuseLangSmithVercel AI SDK

Agent failures and errors as alerts and digests

SlackMicrosoft TeamsEmail

Native SDKs for your agent stack

PythonTypeScriptLangGraphOpenAIAnthropic

Repo-grounded verdicts and fix PRs

GitHubGitLab

Kalmia is MCP-native.

Claude Code, Cursor, or any MCP client can pull Kalmia's insights as context, so your coding agent fixes errors and hill-climbs your agent.

outreach-agent - claude - 120×30

Claude Code v2.1.237
Opus 5 (1M context) with xhigh effort · Claude Max
~/Documents/outreach-agent

❯ Why did last night's outreach eval regress?

⏺ kalmia - list_recent_runs (MCP)(limit: 5)

⎿ 5 runs · newest run_0819_night: 41 pass · 3 fail · 2 needs_attention

⏺ kalmia - compare_runs (MCP)(baseRunId: "run_0818_night", candidateRunId: "run_0819_night")

⎿ pass rate 95% → 89% · 3 regressed: net-new-gate-07, cohort-verdicts-12, enrichment-outage-03 · 1 fixed

⏺ kalmia - get_case_diagnosis (MCP)(runId: "run_0819_night", caseExternalId: "net-new-gate-07")

⎿ root cause #1102: the collision check compares approved requests only; pending requests are ignored

● One change explains the regression: the collision gate stopped treating pending requests as collisions, so three "net-new" candidates with pending requests were submitted. Fix: match on status in (pending, approved) at #1102, then add a workspace rule so Kalmia fails any run that submits a pending lead.

❯

⏵⏵ accept edits on (shift+tab to cycle) · ⇐ for agents/rc

Ask, don't dig. The same read tools our agents use, in your editor.

traces.sh

curl https://kalmia.dev/api/traces?verdict=failed \
  -H "Authorization: Bearer $KALMIA_API_KEY"

{
  "traces": [{
    "id": "22c32b3c",
    "verdict": "failed",
    "root_cause": { "span": "#1102", "title": "collision check ignored pending requests" },
    "citations": ["#920", "#1014", "#1094"]
  }]
}

200 OK · 41 ms

Everything is an endpoint. Anything the UI can do, an API key can do too.

More tools for agent debugging

Live run watching Built for long-horizon agents: watch behaviors and errors as the run goes, not after it ends.

Prompt audit Find where your system prompt and your traces disagree.

Behaviors and rules Describe a rule in plain language. Runs that break it fail.

Cost and latency Cost and latency for every run, and totals per agent version.

Variant comparison Run the same workload across prompts and models, and compare.

See what's hiding in your traces

Instrument in two minutes with our SDK or your existing observability stack.

Book a demo

Kalmia finds errors hiding in your agent traces

How Kalmia catches agent errors

Detector agents triage your traces

Investigator agents dig in

Kalmia learns your system over time

Connects to thetools you already use

Kalmia is MCP-native.

More tools for agent debugging

See what's hiding in your traces

Connects to the
tools you already use