Skip to main content
FailproofAI Observability is a self-hosted platform for observing, evaluating, and improving your AI agents in production. It records everything your agents do (every tool call, model request, hook, and error), scores the quality of each run, and surfaces the failures you didn’t know to look for, all in a dashboard you run inside your own infrastructure. If you ship AI agents and you’re tired of guessing why a run went wrong, this is the page to start on. It explains what FailproofAI Observability gives you and how the pieces fit together, before you install anything.
FailproofAI Observability is an enterprise product from FailproofAI. Want to see it in action? Request a demo: email nikita@befailproof.ai.
A FailproofAI Observability session drawn as a git-style execution graph beside its event timeline, with a per-run breakdown of tools, models, and hooks in the right rail Every agent run is drawn as a git-style execution graph (left) beside its event timeline. Parallel sub-agents each get their own lane; the right rail breaks down the tools, models, hooks, and token spend for the run.

​
See it in action

Two short videos show the two things teams reach for first: tracing a run, and finding failures automatically.
Agent tracing: follow a single run step by step, from goal to tools to final answer.
Failproof Audit: let FailproofAI Observability mine your logs across sessions and tell you what to fix.

​
Why teams use it

  • See what your agent actually did. Every run becomes a readable, git-style execution graph: which tools ran in parallel, which sub-agents branched off, where it stalled, and what it spent.
  • Catch quality regressions automatically. Connect a small scoring service and FailproofAI Observability scores every finished run, so a drop in helpfulness or a spike in hallucinations shows up on its own.
  • Find failures you didn’t write a rule for. Recurring audits mine your logs across sessions for error clusters, latency outliers, low scores, and stuck runs, then hand you ranked, evidence-backed findings.
  • Get paged when it matters. Threshold rules fire on error rate, latency, cost, or evaluator scores and open incidents you can acknowledge, assign, and resolve.
  • Ask questions in plain English. An in-dashboard AI assistant answers “how is quality trending in prod this week?” over your own data. Any change it makes is approval-gated.
  • Keep your data. FailproofAI Observability is self-hosted: events, prompts, and analytics stay in infrastructure you control.

​
What you get

FailproofAI Observability is organized around three ideas (observe, analyze, and admin), mirrored in the dashboard’s left sidebar. Observe (the raw truth of what happened):
  • Event stream: the live, per-step trail of every run (tool calls, model calls, hooks, errors).
  • Sessions: those events rolled up into one row per run, each ready to be scored, with a git-style execution graph.
  • Performance metrics: per-surface latency heat-maps and p50/p95/p99 vitals for models, tools, and hooks, so a tail spike stands out from the median.
  • Error tracking: one triage surface for everything that went wrong, one click from a firing alert.
The Tools observe page: a latency heat-map, a percentile band, and a tool-distribution bar over 24 time bins Each observe surface pairs a sparkline and p50/p95/p99 vitals with a latency heat-map and a percentile band. Shown here: Tools. Analyze (turn activity into answers):
  • Queries and dashboards: saved SQL over your events and evaluations, charted into shared, org-scoped dashboards.
  • Evaluations: quality scores produced by your own evaluator service, with per-score reasoning.
  • Audits: recurring investigations that surface failure patterns across sessions.
  • Alerts and incidents: threshold rules that page you, plus an incident workflow to triage them.
Interfaces (reach your data your way):
  • CLI: drive your whole deployment from the terminal or a script, and let a coding agent do it for you in plain English.
  • AI assistant: ask questions about your agents in plain English, right inside the dashboard.
Admin (run it for your team):
  • API keys: scoped tokens for the collector, the dashboard, and the assistant.
  • Users: passwordless, email-based sign-in with an allowlist.
  • Settings: per-org configuration, including model context-window overrides.

​
How the pieces fit

Data flows in one direction, from your agent code to the dashboard: your agent (via the Python SDK) emits events to the agenteye-collector, which ships them to the server, which serves the dashboard. Two optional services round it out — a scoring service (evaluations) and an AI assistant service (the in-dashboard chat).
  • Python SDK: you add a few agenteye.event.* calls to your agent; events are buffered locally.
  • agenteye-collector: a lightweight daemon on each agent machine that batches events and ships them to the server.
  • Server: ingests your events and keeps operational state in your own databases.
  • Dashboard: where you explore everything.
  • Optional services: a scoring service (evaluations), and an AI assistant service (the in-dashboard chat).
For the vocabulary used throughout the docs (event, session, evaluation, audit, finding, incident), see Concepts.

​
Getting FailproofAI Observability

FailproofAI Observability is an enterprise product from FailproofAI, and it works alongside FailproofAI Enforcement — the policy and guardrail product — under the FailproofAI brand. It runs entirely in your own environment. If you don’t have access to the packages yet, request a demo and we’ll get you set up: email nikita@befailproof.ai.

​
Next steps

  • Concepts: the FailproofAI Observability vocabulary in one place.
  • Observability: follow what your agents do, run by run.
  • Security: how FailproofAI Observability keeps your data isolated and in your control.