# Failproof AI: English Integrations and reference > Trace agent behavior, find failures with audits, and prevent repeat failures with deployed policies. ## Integrations and reference ### Connect your agents - [Integrations and reference](https://docs.befailproof.ai/reference/overview.md): Connect supported agent harnesses, SDKs, CLIs, and the HTTP API. - [Agent harnesses](https://docs.befailproof.ai/reference/harnesses.md): Capture sessions and enforce policies across all 12 supported agent harnesses. - [Custom agents](https://docs.befailproof.ai/reference/custom-agents.md): Configuration, the event catalog, correlation rules, and delivery for failproofai-sdk. - [Custom agents (TypeScript)](https://docs.befailproof.ai/reference/custom-agents-typescript.md): Configuration, the event catalog, the scopes and the framework adapters for @failproofai/sdk. - [Evaluator SDK](https://docs.befailproof.ai/reference/evaluator-sdk.md): Run your own evaluation worker, for LLM judges and anything else hosted Python cannot do. - [Custom policies](https://docs.befailproof.ai/reference/policy-sdk.md): Author, test, and deploy JavaScript or TypeScript policies for failures specific to your agents. - [Self-host Failproof AI Cloud](https://docs.befailproof.ai/reference/self-hosting.md): Deploy the Failproof AI control plane on a customer-managed Kubernetes cluster. ### Jev reference - [Jev integration reference](https://docs.befailproof.ai/reference/jev.md): Configuration, providers, keys, request data, and failure behavior for Jev. - [Jev evaluation reference](https://docs.befailproof.ai/reference/jev-evaluations.md): Question types, calibrated scores, limits, and backfill for Jev session evaluations. - [Jev providers and own-key setup](https://docs.befailproof.ai/reference/jev-providers.md): Provider endpoints, model IDs, configuration, and failure behavior for live Jev policy review with your own key. - [Jev through FailproofAI Cloud](https://docs.befailproof.ai/reference/jev-cloud.md): Cloud machine keys, connection state, limits, and failure behavior for live Jev policy review. ### Reference - [Failproof AI CLI](https://docs.befailproof.ai/reference/failproof-cli.md): Install hooks, manage local policies, connect Cloud, and operate the local daemon. - [OpenTelemetry CLI](https://docs.befailproof.ai/reference/otel.md): Opt in to OTEL for coding agents, inspect delivery, and restore your previous settings. - [Local dashboard](https://docs.befailproof.ai/reference/local-dashboard.md): Review local projects, sessions, policy activity, configuration, audits, and scheduled scans. - [Local configuration](https://docs.befailproof.ai/policies/local-configuration.md): Control policy scope, parameters, custom files, and machine-level Failproof AI settings. - [Failproof Cloud CLI](https://docs.befailproof.ai/reference/cloud-cli.md): Complete reference for querying and administering Failproof AI Cloud with fp. - [HTTP API](https://docs.befailproof.ai/reference/http-api.md): Authenticate to the public Failproof AI Cloud `/v1` API and use the generated endpoint reference. - [Events and configuration](https://docs.befailproof.ai/reference/events-and-configuration.md): Reference the event model, environments, local storage, and configuration boundaries. - [Troubleshooting](https://docs.befailproof.ai/reference/troubleshooting.md): Diagnose missing sessions, missing policies, failed delivery, and blocked agent actions. ### HTTP API #### Users - [List who can grant you a permission.](https://docs.befailproof.ai/api-reference/users/list-who-can-grant-you-a-permission.md): ANY valid key may call this — it is the one authenticated endpoint that requires no particular permission, deliberately, because the caller who needs it is by definition the one who lacks access. Only active members of the calling organization holding `users:update` are listed, at most 50 of them, a… - [List this organization's members.](https://docs.befailproof.ai/api-reference/users/list-this-organizations-members.md): The `permissions` on each row are that member's grants in THIS organization, alongside the `permission_set` / `permission_added` / `permission_removed` they were resolved from. `disabled_at` is global rather than per-org: a member showing as disabled cannot sign in to any organization. Responses are… - [Add a member to this organization.](https://docs.befailproof.ai/api-reference/users/add-a-member-to-this-organization.md): Send `permission_set` plus optional `permission_added` / `permission_removed` overrides; the server resolves them and returns the effective grants in `permissions`. The older flat `permissions` array still works and is read as `permission_added` against no set. The address is trimmed and lowercased. - [The permissions a new member starts with.](https://docs.befailproof.ai/api-reference/users/the-permissions-a-new-member-starts-with.md): `GET /users/defaults` — returns the dashboard-configured starting permission set used to seed the new-user invite form. Gated on `users:create` so an invite-capable operator who lacks `settings:read` can still load the form without going through the locked /settings surface. If the configured set ha… - [Fetch one member of this organization.](https://docs.befailproof.ai/api-reference/users/fetch-one-member-of-this-organization.md): Permissions are the member's grants in THIS organization. Someone who has an account but no membership here reads as 404 — the same as an id that does not exist, so one organization can't probe another's membership. Cached for 60 seconds. - [Replace a member's permissions in this organization.](https://docs.befailproof.ai/api-reference/users/replace-a-members-permissions-in-this-organization.md): The set/added/removed triple replaces what was stored — send the whole picture, not a delta. Two guards apply: a protected member (the bootstrap admin, or an org admin designated with `agenteye-orgctl`) can never be edited here, and a disabled member is read-only until re-enabled, which is a 409 rat… - [Disable a member.](https://docs.befailproof.ai/api-reference/users/disable-a-member.md): Despite the verb, nothing is deleted: the account is stamped `disabled_at`, the membership and all history stay, and the change is reversible with `POST /users/{id}/enable`. The flag is GLOBAL — a user disabled here can no longer sign in to ANY organization they belong to, not only this one — so tre… - [Re-enable a disabled member.](https://docs.befailproof.ai/api-reference/users/re-enable-a-disabled-member.md): Requires `users:delete`, NOT `users:create`. Enabling and disabling are two halves of the same lifecycle action and share one permission, so a key that can invite members cannot bring a disabled one back — that takes the same grant as offboarding. Clearing `disabled_at` restores sign-in across every… #### Alerts - [List this organization's alert rules.](https://docs.befailproof.ai/api-reference/alerts/list-this-organizations-alert-rules.md): Newest first. Each entry carries the rule itself plus two derived fields: `open_incidents`, the number of its issues that are not yet resolved, and `last_attempted_at`, when the dispatcher last evaluated it (`null` if it never has). The list is cached for 60 seconds per organization, so a rule you j… - [Create an alert rule.](https://docs.befailproof.ai/api-reference/alerts/create-an-alert-rule.md): `trigger_kind` picks how the rule is evaluated — `metric_threshold`, `custom_sql`, `evaluation_score`, `eval_compound` or `per_event` — and `trigger_spec` is the free-form JSON that kind expects. - [List the member emails an alert can notify.](https://docs.befailproof.ai/api-reference/alerts/list-the-member-emails-an-alert-can-notify.md): Returns `[{ "email": … }]` for your organization — active members only, alphabetised. Gated on `alerts:read` OR `alerts:write` rather than `users:read`, so whoever views or edits alerts can fill in the recipient picker without being able to read user records; only emails come back, never the permiss… - [Fetch one alert rule.](https://docs.befailproof.ai/api-reference/alerts/fetch-one-alert-rule.md): Same shape as an entry in `GET /alerts`, `open_incidents` and `last_attempted_at` included, and cached for 60 seconds in the same way. An alert belonging to another organization reads as 404, not 403. - [Replace an alert rule.](https://docs.befailproof.ai/api-reference/alerts/replace-an-alert-rule.md): A whole-object replace, not a patch: any field you leave out reverts to its default rather than keeping its current value — `enabled` to `true`, `severity` to `warning`, `channels` to empty, which silently stops the rule notifying anyone. Read the alert first and send the full object back. Validatio… - [Delete an alert rule.](https://docs.befailproof.ai/api-reference/alerts/delete-an-alert-rule.md): This also deletes every issue the rule ever opened — the issue rows cascade off the definition, taking their comments, subscribers and activity history with them. If you only want the rule to stop firing, `PUT` it back with `enabled: false` instead. Returns 204 with no body. - [Send a test notification through an alert's channels.](https://docs.befailproof.ai/api-reference/alerts/send-a-test-notification-through-an-alerts-channels.md): Fires one synthetic notification so you can check a channel is actually wired up. It touches no state: nothing is opened or resolved, no issue is stored, and the `synthetic_incident_id` in the response belongs to no row — do not try to fetch it. #### Issues - [List one alert's issues.](https://docs.befailproof.ai/api-reference/issues/list-one-alerts-issues.md): The same rows as `GET /issues`, restricted to this alert. The default `state` differs: this returns the alert's full history — `firing,acknowledged,resolved` — because it backs the per-alert page rather than the board. An `alert_id` in the query string is ignored; the path wins. - [Open an issue against an existing alert, by hand.](https://docs.befailproof.ai/api-reference/issues/open-an-issue-against-an-existing-alert-by-hand.md): `summary` is required and capped at 2000 characters; `title` is optional and defaults to the alert's own name. - [List issues.](https://docs.befailproof.ai/api-reference/issues/list-issues.md): The whole issues board in one feed: issues opened by an alert rule, issues graduated from an audit finding, and issues opened by hand. Newest first. - [Open an issue by hand, with no parent alert.](https://docs.befailproof.ai/api-reference/issues/open-an-issue-by-hand-with-no-parent-alert.md): For a failure that doesn't map to any configured rule. `summary` is required and capped at 2000 characters; `title` is optional and falls back to the summary's first 200 characters; `severity` is `info`, `warning` (the default) or `critical`. - [Count issues without fetching them.](https://docs.befailproof.ai/api-reference/issues/count-issues-without-fetching-them.md): Returns `{ "count": n }` for the same `state` filter `GET /issues` takes (default `firing,acknowledged`). This is what the sidebar badge polls, so a dashboard does not pull 200 rows to render one number. Cached for 15 seconds, so it can trail a state change by that long. - [Fetch one issue, with its whole thread.](https://docs.befailproof.ai/api-reference/issues/fetch-one-issue-with-its-whole-thread.md): Beyond the issue's own fields this returns `notifications` (the last 100 delivery attempts, newest first, each with its channel and status), `subscribers`, `comments` (oldest first — deleted ones appear as tombstones with `body: null`) and `activity`, the append-only log of who did what. An issue th… - [Acknowledge an issue — "I'm looking at this".](https://docs.befailproof.ai/api-reference/issues/acknowledge-an-issue-—-"im-looking-at-this".md): Needs only `issues:read`, deliberately: signalling that you have picked something up must not require write authority. It does not change who is accountable — `POST /issues/{iid}/assign` does that. - [Replace an issue's assignees.](https://docs.befailproof.ai/api-reference/issues/replace-an-issues-assignees.md): A whole-list replace, not an add: `{"assignees": []}` clears everyone. (The older single `assigned_to` string is still accepted — a string becomes a one-element list, `null` clears — and is ignored when `assignees` is present.) Every address must belong to an active member of your organization; a ty… - [List an issue's comments.](https://docs.befailproof.ai/api-reference/issues/list-an-issues-comments.md): Oldest first. Deleted comments are still returned, as tombstones with `body: null`, so the thread keeps its shape and reply order. - [Comment on an issue.](https://docs.befailproof.ai/api-reference/issues/comment-on-an-issue.md): `body` is required and capped at 8000 characters after trimming. Needs only `issues:read`: a responder who can see the issue can talk on its thread — putting discussion behind a write grant would make read-only reviewers silent observers. Commenting subscribes you to the thread and emails the other… - [Delete a comment.](https://docs.befailproof.ai/api-reference/issues/delete-a-comment.md): Authorship, not permission tier, is the gate: you may delete your own comment, and holders of `issues:close` may moderate anyone's — someone has to be able to remove a departed colleague's. A bare API-key call carries no human identity and so cannot prove authorship; it needs `issues:close`. - [Resolve an issue.](https://docs.befailproof.ai/api-reference/issues/resolve-an-issue.md): The one triage action with its own permission — `issues:close`, not the `issues:read` the rest of the thread runs on — because closing is what silences the thread and takes the issue off the board. - [Subscribe to an issue's activity emails.](https://docs.befailproof.ai/api-reference/issues/subscribe-to-an-issues-activity-emails.md): With no body you subscribe yourself, using the email your dashboard session forwards. Passing `{"email": …}` for SOMEONE ELSE is a create-tier action: it needs `issues:create`, and that address must belong to an active member of your organization. Subscribing yourself works at `issues:read`, so a re… - [List who gets an issue's activity emails.](https://docs.befailproof.ai/api-reference/issues/list-who-gets-an-issues-activity-emails.md): Active subscribers only — anyone who unsubscribed is omitted. Note that people are subscribed automatically when they ack, comment, are assigned, or open the issue, so this is not just the list of manual subscribes. - [Stop an issue's activity emails.](https://docs.befailproof.ai/api-reference/issues/stop-an-issues-activity-emails.md): The mirror of subscribe. Removing YOURSELF — the default with no body — works at `issues:read`, so nobody can be trapped on a thread; passing another person's `email` takes them off it and needs `issues:create`. #### Audits - [List this organization's audits, newest first.](https://docs.befailproof.ai/api-reference/audits/list-this-organizations-audits-newest-first.md): Each entry is the full definition plus its schedule state (`next_attempt_at`, `last_attempted_at`, `last_error`), its open finding count, and the status of its most recent run. Edits made through this API show up immediately, but the response is cached for 60 seconds, so schedule state advanced by t… - [Create a recurring audit.](https://docs.befailproof.ai/api-reference/audits/create-a-recurring-audit.md): An audit sweeps a window of your events on a schedule and files findings for the failure patterns it sees. `schedule_interval_secs` must be between 3600 (1h) and 604800 (7d) and defaults to daily; `lookback_window_secs` must be between 3600 (1h) and 7776000 (90d) and defaults to 7d. `schedule_anchor… - [Get auditscontracts](https://docs.befailproof.ai/api-reference/audits/get-auditscontracts.md): List this organization's agent contracts. A contract states what one agent is for — what it must produce, how often it should run, what it must never do — and audits grade that agent against it. Contracts are organization-wide: there is one per agent, shared by every audit that covers it. Contract t… - [Post auditscontracts](https://docs.befailproof.ai/api-reference/audits/post-auditscontracts.md): Write a contract for one agent. `agent_id` must match the agent id in your telemetry exactly — take it from the agent list rather than typing it, because a name that matches nothing is not rejected and the contract simply never applies. Case and surrounding whitespace are ignored when matching. One… - [Get auditscontracts 1](https://docs.befailproof.ai/api-reference/audits/get-auditscontracts-1.md): Read one agent contract, including its full text. `sections` lists the parts of the document that were recognised, so a heading that was not understood is visible rather than silently having no effect. - [Put auditscontracts](https://docs.befailproof.ai/api-reference/audits/put-auditscontracts.md): Replace a contract. This is a full replacement, not a merge: the agent id and body are set to exactly what you send. Editing a contract changes what future audits expect; findings already raised keep referring to the text that was in force when they were raised. - [Delete auditscontracts](https://docs.befailproof.ai/api-reference/audits/delete-auditscontracts.md): Delete a contract. The agent is audited from then on as though one had never been written. Audits that already ran keep their record of what they graded against. - [List findings across every audit in the organization, highest priority first.](https://docs.befailproof.ai/api-reference/audits/list-findings-across-every-audit-in-the-organization-highest-priorityfirst.md): This is the triage queue. Without a `status` filter it shows only the live set — `open` and `recurring` — so triaged findings disappear from it; pass `status` explicitly to see them. Narrow to one audit with `audit_id`, or to the findings one run produced or touched with `run_id`. Each finding's `is… - [Fetch one finding in full.](https://docs.befailproof.ai/api-reference/audits/fetch-one-finding-in-full.md): The same shape the list returns — evidence, priority factors, recommendation, suggested alert, and the issue it graduated into. - [Triage a finding.](https://docs.befailproof.ai/api-reference/audits/triage-a-finding.md): `action` is one of `ack`, `mute`, `dismiss`, `resolve`, `reopen` or `assign`; `assign` additionally requires `assigned_to`, and any action may carry a free-text `reason`. - [Fetch one audit definition.](https://docs.befailproof.ai/api-reference/audits/fetch-one-audit-definition.md): Same shape as an entry in the list, read straight from the database rather than the 60-second list cache — schedule state and the last-run summary are current as of the request. - [Update an audit definition.](https://docs.befailproof.ai/api-reference/audits/update-an-audit-definition.md): `name` is required and always overwritten; every other field is optional and an omitted one keeps its current value — there is no way to clear a field back to NULL here. The same ranges as create apply. A changed `schedule_interval_secs` or `schedule_anchor` takes effect at the next reschedule, not… - [Delete an audit.](https://docs.befailproof.ai/api-reference/audits/delete-an-audit.md): Cascades: the audit's queue row, its run history and all of its findings go with it, and this cannot be undone. Muting feedback survives — it is keyed on the fingerprint, not the audit. Requires `audits:write`; there is no separate delete permission. - [Get audits context](https://docs.befailproof.ai/api-reference/audits/get-audits-context.md): Read an audit's extra context: the operator's written brief, plus the reference URLs it draws on and the state of each one's stored snapshot. Snapshot text is returned as a bounded preview rather than in full. - [Put audits context](https://docs.befailproof.ai/api-reference/audits/put-audits-context.md): Replace an audit's extra context. This is a full replacement, not a merge: the brief and the reference URL list are both set to exactly what you send, so send an empty value to clear one. Context is edited here rather than through the audit itself, so an ordinary unrelated edit to the audit can neve… - [Post audits contextrefresh](https://docs.befailproof.ai/api-reference/audits/post-audits-contextrefresh.md): Queue a re-fetch of this audit's reference URLs. Returns as soon as the work is queued, not when it finishes. Sources whose URL was refused are left alone, since nothing about them can change until the URL is edited, and a source that already has usable text keeps serving it while the re-fetch runs. - [Run an audit now.](https://docs.befailproof.ai/api-reference/audits/run-an-audit-now.md): Makes the audit's queue row due immediately; the dispatcher claims it on its next tick, so the 202 means "queued", not "finished" — poll `GET /audits/{id}/runs` for the result. Three cases all answer 409 and are distinguished by the `error` field: the audit is unknown or belongs to another organizat… - [List an audit's runs, newest first.](https://docs.befailproof.ai/api-reference/audits/list-an-audits-runs-newest-first.md): Each run carries its status, the window it covered, its finding counts and the LLM report when there is one. An unknown audit id — or one in another organization — returns an empty list rather than 404. #### Auth - [Get authintrospect](https://docs.befailproof.ai/api-reference/auth/get-authintrospect.md): Describe the API key making this request: the organization it belongs to and the permissions it holds. Requires no permission of its own — any valid key may describe itself, and only itself. Intended to be called once when provisioning a machine, so a bad key, a missing permission and a wrong organi… #### Dashboards - [List the organization's dashboards, newest first.](https://docs.befailproof.ai/api-reference/dashboards/list-the-organizations-dashboards-newest-first.md): Dashboards belong to the organization, not to whoever created them — everyone with `dashboards:read` sees the same list. Each entry carries a `tile_count` but not the tiles themselves; fetch a single dashboard for those. The list is cached for up to 30 seconds, so a dashboard created a moment ago ma… - [Create a dashboard.](https://docs.befailproof.ai/api-reference/dashboards/create-a-dashboard.md): `definition` is an opaque JSON object owned by the dashboard UI (filters and display settings) — it is stored and returned verbatim and the server never looks inside it. A new dashboard has no tiles; add them with `POST /dashboards/{id}/tiles`. - [Fetch one dashboard together with its tiles.](https://docs.befailproof.ai/api-reference/dashboards/fetch-one-dashboard-together-with-its-tiles.md): The response is the dashboard record plus a `tiles` array, ordered by grid position (top to bottom, then left to right) — the same order the UI draws them in. - [Replace a dashboard's name, description and definition.](https://docs.befailproof.ai/api-reference/dashboards/replace-a-dashboards-name-description-and-definition.md): A full replacement, not a patch: send every field you want to keep, because an omitted `description` is written back as empty. Tiles and their layout are not touched — change those through the tile endpoints. - [Delete a dashboard and every tile on it.](https://docs.befailproof.ai/api-reference/dashboards/delete-a-dashboard-and-every-tile-on-it.md): This is the one dashboards route that needs `dashboards:delete`; editing a dashboard — including adding and removing its tiles — needs only `dashboards:write`, so a key that can author dashboards cannot necessarily destroy one. The tiles go with the dashboard; the saved queries they render are left… - [List a dashboard's tiles, ordered by grid position.](https://docs.befailproof.ai/api-reference/dashboards/list-a-dashboards-tiles-ordered-by-grid-position.md): A dashboard id that does not exist in this organization returns an empty `tiles` array rather than a 404 — check the dashboard itself if you need to tell "no tiles" from "no dashboard". - [Add a tile to a dashboard.](https://docs.befailproof.ai/api-reference/dashboards/add-a-tile-to-a-dashboard.md): A tile renders exactly one saved query: `query_id` names it, `chart_type` is one of `line`, `area`, `bar`, `stacked_bar`, `pie`, `kpi` or `table`, and `pos_x` / `pos_y` / `pos_w` / `pos_h` place it on the grid (origin top-left, width and height in grid cells). `chart_config` and `param_bindings` are… - [Reposition a dashboard's tiles in one request.](https://docs.befailproof.ai/api-reference/dashboards/reposition-a-dashboards-tiles-in-one-request.md): Send `{"tiles": [{id, pos_x, pos_y, pos_w, pos_h}, …]}`; every entry is applied in a single transaction, so a whole drag-and-resize rearrangement is one round-trip. Only geometry changes — the query, title and config on each tile are left as they are. - [Replace a tile.](https://docs.befailproof.ai/api-reference/dashboards/replace-a-tile.md): A full replacement, not a patch — every field is written, so send the ones you want to keep. The tile must live on the dashboard named in the path; otherwise it reads as not found. - [Remove a tile from a dashboard.](https://docs.befailproof.ai/api-reference/dashboards/remove-a-tile-from-a-dashboard.md): Needs `dashboards:write`, NOT `dashboards:delete` — taking a chart off a dashboard is an edit; only deleting the whole dashboard needs the delete grant. The saved query the tile rendered is untouched. #### Evaluations - [List the evaluations that are queued or in flight, newest enqueue first.](https://docs.befailproof.ai/api-reference/evaluations/list-the-evaluations-that-are-queued-or-in-flight-newest-enqueue-first.md): This is the queue, not the results — a job leaves this list once it finishes, at which point its scores appear on `/evaluations`. Each row carries the attempt count, the next retry time, and the last error, so a session stuck behind a failing evaluator is visible here rather than merely absent from… - [List completed evaluations, newest first.](https://docs.befailproof.ai/api-reference/evaluations/list-completed-evaluations-newest-first.md): Cursor-paginated: read a page, then pass its `next_cursor` back as `cursor`. `next_cursor` is null on the final page. Every filter below AND-composes. - [Rolled-up evaluation health for a filtered slice.](https://docs.befailproof.ai/api-reference/evaluations/rolled-up-evaluation-health-for-a-filtered-slice.md): Takes the same filters as `/evaluations` and returns totals, a status breakdown, per-score-key stats, and a time-bucketed timeline — computed over the whole matching set, not a sample of it. - [The distinct environment names seen in this organization's evaluations.](https://docs.befailproof.ai/api-reference/evaluations/the-distinct-environment-names-seen-in-this-organizations-evaluations.md): Returns a bare JSON array of strings, suitable for an environment filter. Read from the evaluations dataset rather than the event store, so a key with `evaluations:read` but not `events:read` can still populate the filter. Cached per organization for 60 seconds, so a brand-new environment can take u… - [The magnitude-metric names under `scores.metrics`, each with its unit.](https://docs.befailproof.ai/api-reference/evaluations/the-magnitude-metric-names-under-`scoresmetrics`-each-with-its-unit.md): Returns a JSON array of `{name, unit, count}` objects. Kept separate from `score_keys` because the two are on different scales: score keys are 0-1 rates that share one axis, metrics carry units (ms, USD, tokens) and must not be plotted against each other. `unit` is an empty string for metrics record… - [The numeric score keys present across this organization's evaluations.](https://docs.befailproof.ai/api-reference/evaluations/the-numeric-score-keys-present-across-this-organizations-evaluations.md): Returns a bare JSON array of strings. Score keys are emergent — whatever your evaluators emit — so this is the complete all-time catalog rather than a sample of recent rows. Only NUMERIC keys are listed, which is exactly the set `score_filters` can range-filter on; string and boolean score values ar… - [Enqueue a fresh evaluation job for a session. Used by the dashboard's re-evaluate button when a session was missed by the automatic pipeline (e.g. no agent_end ever arrived) or when the user wants fresh scores after deploying a new evaluator.](https://docs.befailproof.ai/api-reference/evaluations/enqueue-a-fresh-evaluation-job-for-a-session-used-by-the-dashboardsre-evaluate-button-when-a-session-was-missed-by-the-automatic-pipelineeg-no-agent_end-ever-arrived-or-when-the-user-wants-fresh-scoresafter-deploying-a-new-evaluator.md): The `eval_enqueued` guard on `agent_sessions` only gates ingest-driven enqueue (so duplicate `agent_end` events can't re-evaluate). Manual re-eval bypasses it: we insert directly into `evaluation_jobs`. The dispatcher's terminal-result writer already upserts on `session_id`, so the new run naturally… #### Events - [List raw events, including each event's full payload.](https://docs.befailproof.ai/api-reference/events/list-raw-events-including-each-events-full-payload.md): Cursor-paginated: read a page, then pass its `next_cursor` back as `cursor`. `next_cursor` is null on the final page. For the dashboard's lighter feed without payloads, use `/events/summary`. - [Ingest a batch of events.](https://docs.befailproof.ai/api-reference/events/ingest-a-batch-of-events.md): The body is newline-delimited JSON — one event object per line, not a JSON array. Malformed lines are SKIPPED, not rejected: the response reports `accepted` and `skipped` counts and the status is 200 even when every line was bad. A 5xx means "retry the whole batch"; duplicate events are collapsed se… - [List the agent ids this organization has sent events for, most-used first.](https://docs.befailproof.ai/api-reference/events/list-the-agent-ids-this-organization-has-sent-events-for-most-used-first.md): A JSON array of strings — the values accepted by the `agent_id` filter on `/events`, `/events/summary` and `/sessions`. Capped at the 500 busiest agent ids and cached for up to a minute. - [List the environment names this organization has actually sent events from.](https://docs.befailproof.ai/api-reference/events/list-the-environment-names-this-organization-has-actually-sent-events-from.md): A JSON array of strings, alphabetically sorted — the values accepted by the `environment` filter on `/events`, `/events/summary` and `/sessions`. Derived from stored events, so an environment appears only once something has been ingested under it, and results are cached for up to a minute: a brand-n… - [Summarise the errors matching a filter, across the whole window.](https://docs.befailproof.ai/api-reference/events/summarise-the-errors-matching-a-filter-across-the-whole-window.md): Returns `total`, the number of distinct `sessions` and `agents` involved, `last_ts` (null when nothing matched), a `bins` array of exactly 24 counts, and the `bounds` in epoch milliseconds that those bins span. - [List the error types seen in this organization's events, most-used first.](https://docs.befailproof.ai/api-reference/events/list-the-error-types-seen-in-this-organizations-events-most-used-first.md): A JSON array of strings — the values accepted by the `error_type` filter. Only events that carry a classified error type are counted, which is a narrower set than `errored=true`: a failed tool result that reports an error message without a type contributes to that filter but not to this list. Capped… - [List the event types this organization has actually sent, alphabetically.](https://docs.befailproof.ai/api-reference/events/list-the-event-types-this-organization-has-actually-sent-alphabetically.md): A JSON array of strings — the values accepted by the `event_type` filter. This is the observed set, not the full catalogue the SDKs can emit: a type you have never ingested is absent. Cached for up to five minutes, the longest of the facet endpoints. - [List the hook names seen in this organization's events, most-used first.](https://docs.befailproof.ai/api-reference/events/list-the-hook-names-seen-in-this-organizations-events-most-used-first.md): A JSON array of strings — the values accepted by the `hook_name` filter. Counted from `hook_triggered` events only (the start leg of a hook), capped at 500 names, cached for a couple of minutes. - [Roll up the latency between two event types into a chart-ready summary.](https://docs.befailproof.ai/api-reference/events/roll-up-the-latency-between-two-event-types-into-a-chart-ready-summary.md): Pairs each `start_event_type` event with its matching `end_event_type` event and reports the duration between them: `bins` (a time series over the window), `heatmap` (time against latency band), `summary` (pair count with p50/p95/p99 and max), `events` (window totals), and, when `distribution_key_pa… - [List the model names seen in this organization's events, most-used first.](https://docs.befailproof.ai/api-reference/events/list-the-model-names-seen-in-this-organizations-events-most-used-first.md): A JSON array of strings — the values accepted by the `model` filter. Capped at the 500 most-used models and cached for up to a minute. Every event that names a model contributes, not just model responses, so both legs of a model call count towards the ordering. - [List events WITHOUT their payloads — the light feed the dashboard tails.](https://docs.befailproof.ai/api-reference/events/list-events-without-their-payloads-—-the-light-feed-the-dashboard-tails.md): Same filters and cursor semantics as `/events`; each row carries a derived one-line summary instead of the full payload, so a page is far smaller. - [List the tool names seen in this organization's events, most-used first.](https://docs.befailproof.ai/api-reference/events/list-the-tool-names-seen-in-this-organizations-events-most-used-first.md): A JSON array of strings — the values accepted by the `tool_name` filter. Counted from `tool_use` and `tool_result` events only, capped at 500 names, cached for a couple of minutes. - [List the trigger events that fired hooks recently, most-used first.](https://docs.befailproof.ai/api-reference/events/list-the-trigger-events-that-fired-hooks-recently-most-used-first.md): A JSON array of strings — the values accepted by the `trigger_event` filter. Only the last 90 days are scanned, so a trigger that has not fired in that window is absent even though older events still carry it. Capped at 500 values and cached for up to a minute. #### Health - [Liveness probe.](https://docs.befailproof.ai/api-reference/health/liveness-probe.md): The only endpoint on `/v1` that needs no credential — it reports that the process is answering, nothing about your data. #### Keys - [List this organization's API keys.](https://docs.befailproof.ai/api-reference/keys/list-this-organizations-api-keys.md): Secrets are never returned — only metadata. A key's secret is shown once, at creation, and cannot be recovered afterwards. - [Mint a scoped API key.](https://docs.befailproof.ai/api-reference/keys/mint-a-scoped-api-key.md): You supply the secret in `key`; the server stores only its hash and never returns it again. Generate it with a cryptographically secure random source. `permissions` defaults to `["events:add"]` — an ingest-only key. - [Replace a key's permissions.](https://docs.befailproof.ai/api-reference/keys/replace-a-keys-permissions.md): No API key can call this. `keys:update` is deliberately not grantable to any key — editing keys is a human, dashboard-only action — so a bearer token always gets 403 here, permanently and by design. Use the dashboard's key editor instead. The list you send replaces the key's grants outright rather t… - [Revoke an API key.](https://docs.befailproof.ai/api-reference/keys/revoke-an-api-key.md): Revocation is permanent — there is no re-enable, so mint a replacement rather than expecting to undo this. It takes effect immediately on the replica handling the request and within 5 seconds on every other replica. Keys seeded from configuration (the bootstrap admin, the dashboard assistant) are pr… - [Rotate a key's secret.](https://docs.befailproof.ai/api-reference/keys/rotate-a-keys-secret.md): The server generates the new secret and returns it in `key`; this is the only time it is shown. The old secret stops working immediately on this replica and within 5 seconds elsewhere, so anything still using it starts failing as soon as you rotate. The key's id, name and permissions are unchanged.… #### Permission sets - [The permission sets defined for your organization.](https://docs.befailproof.ai/api-reference/permission-sets/the-permission-sets-defined-for-your-organization.md): Each entry carries its grants and a `user_count` of the enabled members currently assigned to it, so a UI can show "applied to N users". Builtins (`read-only`, `standard`, `admin`) sort first and are marked `is_builtin`. - [Define a new permission set.](https://docs.befailproof.ai/api-reference/permission-sets/define-a-new-permission-set.md): `permissions` is the list of grants the set confers; every entry must be a known permission and the list may not repeat one. Names are trimmed, capped at 64 characters, and unique within the organization. The set is created with no members — assign it to users through `/users`. - [One permission set, with its grants.](https://docs.befailproof.ai/api-reference/permission-sets/one-permission-set-with-its-grants.md): Reuses the list's gate (`users:read`) for the same reason: the create-user form needs it and reading a set exposes nothing beyond its name and grants. Writing one, by contrast, needs `settings:write`. - [Replace a permission set's grants and description.](https://docs.befailproof.ai/api-reference/permission-sets/replace-a-permission-sets-grants-and-description.md): `permissions` is the complete new list, not a delta — anything you omit is removed. Every member of this organization assigned to the set has their effective permissions re-materialised as part of the request, so the change takes effect on those users' next call rather than after a cache expiry. - [Delete a permission set.](https://docs.befailproof.ai/api-reference/permission-sets/delete-a-permission-set.md): Members assigned to it keep the permissions they already have — they simply stop being linked to a set — and the response reports how many were unlinked as `affected_users`. #### Queries - [List the organization's saved queries, alphabetically by name.](https://docs.befailproof.ai/api-reference/queries/list-the-organizations-saved-queries-alphabetically-by-name.md): Saved queries belong to the organization, not to whoever wrote them. Each entry includes its full `sql_text` and declared `params`. The list is cached for up to 30 seconds, so a query saved a moment ago may not appear on the very next call. - [Save a query.](https://docs.befailproof.ai/api-reference/queries/save-a-query.md): `sql_text` is stored exactly as written and is NOT checked here — the restrictions on what may run are applied at execution time, so SQL that saves cleanly can still be refused by `POST /queries/run`. Names must be unique within the organization. - [Run SQL and get rows back.](https://docs.befailproof.ai/api-reference/queries/run-sql-and-get-rows-back.md): Send either `sql` (free-form) or `query_id` (a saved query to execute); if both are present `sql` wins and `query_id` is recorded only as the run's origin. `params` is a JSON array of positional values substituted for `$1`…`$N` in the statement. - [Describe the tables a query may read.](https://docs.befailproof.ai/api-reference/queries/describe-the-tables-a-query-may-read.md): Returns `{schema, tables: [{name, columns: [{name, type}]}]}` — the same listing the query editor's autocomplete and schema browser are built on. Types are normalised display names (`string`, `int`, `float`, `timestamp`, `uuid`, `bool`), with a trailing `?` when the column is nullable. - [Fetch one saved query, including its SQL.](https://docs.befailproof.ai/api-reference/queries/fetch-one-saved-query-including-its-sql.md) - [Replace a saved query's name, description, SQL and params.](https://docs.befailproof.ai/api-reference/queries/replace-a-saved-querys-name-description-sql-and-params.md): A full replacement, not a patch: send every field you want to keep, because an omitted `description` is written back as empty. Dashboard tiles that render this query pick up the new SQL the next time they are run — editing a query changes every chart built on it. - [Delete a saved query.](https://docs.befailproof.ai/api-reference/queries/delete-a-saved-query.md): Needs `queries:delete`, a separate grant from the `queries:write` used to author one. Refused with 409 while any dashboard tile still renders it — remove those tiles first. #### Sessions - [List agent sessions with their latest evaluation headline.](https://docs.befailproof.ai/api-reference/sessions/list-agent-sessions-with-their-latest-evaluation-headline.md): Requires `evaluations:read` rather than `events:read` — the row a session returns is its evaluation summary, not its events. - [Download one session's full event log as a file.](https://docs.befailproof.ai/api-reference/sessions/download-one-sessions-full-event-log-as-a-file.md): The response is a single JSON object — session metadata plus every event in the session, oldest first, each with its full payload — served as an attachment (`Content-Disposition: attachment; filename="session-.json"`). These are the exact bytes an evaluator receives for the session, so a downloa… #### Settings - [Every registered setting, with your organization's current value.](https://docs.befailproof.ai/api-reference/settings/every-registered-setting-with-your-organizations-current-value.md): Settings are per-organization: each entry is this org's stored value, or the registry default for a key the org has never set. Every entry carries the setting's `schema`, so a UI can render a typed input without re-declaring the catalogue. Secrets never leave the server in the clear — their `value`… - [The context-window overrides configured for your organization.](https://docs.befailproof.ai/api-reference/settings/the-context-window-overrides-configured-for-your-organization.md): One entry per model id this org has overridden; models left on AgentEye's built-in defaults do not appear here. To see what a specific model actually resolves to, override or not, use `/settings/model-context-windows/resolve`. - [Set the context window for one model, for your organization.](https://docs.befailproof.ai/api-reference/settings/set-the-context-window-for-one-model-for-your-organization.md): Body: `{ "model_id": "...", "context_window": N }`. Creates the override or replaces the one already stored for that model id. `context_window` is in tokens and must be between 0 and 1000000; 0 means "unknown", so no context-fill percentage is computed for that model. The new value applies from the… - [Remove one model's context-window override.](https://docs.befailproof.ai/api-reference/settings/remove-one-models-context-window-override.md): The model falls back to AgentEye's built-in default from the next ingested event. 404 if this organization has no override for that model. - [Preview the context window a model resolves to.](https://docs.befailproof.ai/api-reference/settings/preview-the-context-window-a-model-resolves-to.md): Returns the effective limit and its authoritative `source`: `override` (this org configured it), `default` (AgentEye's built-in table), or `unknown` (no limit known, so no context-fill percentage is computed). Deliberately does not consult the cache, so it reports the true source rather than a cache… - [The setting catalogue — key, type, and constraints, with no values.](https://docs.befailproof.ai/api-reference/settings/the-setting-catalogue-—-key-type-and-constraints-with-no-values.md): Registry metadata only, so it reads nothing per-organization and returns the same document for every caller. Use it to render typed inputs without re-declaring the catalogue; use `GET /settings` when you also need values. - [Change one setting for your organization.](https://docs.befailproof.ai/api-reference/settings/change-one-setting-for-your-organization.md): Send either `{"value": ...}` or the bare value — both are accepted. The value is validated against the setting's schema before it is stored, and the updated row is returned. The change applies only to your organization. #### Usage - [Usage totals for the organization's current billing window.](https://docs.befailproof.ai/api-reference/usage/usage-totals-for-the-organizations-current-billing-window.md): Defaults to the window containing now; pass `window_start` to read a specific one. Window boundaries follow the organization's own billing anchor, so they are not calendar months. - [Historical billing windows, most recent first.](https://docs.befailproof.ai/api-reference/usage/historical-billing-windows-most-recent-first.md): One entry per completed window plus the one in progress, so a caller can chart usage over time without knowing the organization's billing anchor. ## OpenAPI Specs - [openapi](/reference/openapi.json)