Set up an audit - Failproof AI

The quality of an audit starts with its scope. A broad request such as “find problems” produces less useful results than a concrete failure question.

Configure the audit

Dashboard
CLI

Go to Analyze → Audits → New audit and enter the name and description.
Set the cadence, time window, agent/environment scope, ignored errors, sensitivity, and maximum findings.
In agents, add or review agent context, then add the operator brief and any public HTTPS reference URLs.
Choose notification channels and select create audit. The first run queues immediately.

fp audits create checkout-reliability \
  --description "Find checkout failures that agents do not recover from" \
  --scope '{"environments":["production"],"agent_ids":["checkout-agent"]}' \
  --schedule-interval-secs 86400 \
  --window-mode since_last \
  --sensitivity medium \
  --text-file ./checkout-audit-brief.txt \
  --url https://runbooks.example.com/checkout

The first run queues immediately. Include the brief and reference URLs during creation so that run receives them.See the complete fp audits create reference.

Start with a known failed session and several normal sessions. This gives the audit both a positive example and a comparison set.

Audit context is stored as its own resource so editing an audit does not accidentally remove the reference material.

​Configure the audit

Configure the audit