Skip to main content
This is the Cloud route reference for Jev policies. Jev, TypeSafe’s classifier, reads each tool call against what you actually asked for and answers alongside your policies, never instead of them. Through FailproofAI Cloud, a connected machine uses Jev with the same key it already connects with: no TypeSafe account, no second key, no endpoint to configure. Each call is charged to your organization’s existing plan allowance. Everything Jev does is unchanged from the bring-your-own-key setup: hard policies stay final, a reviewable policy’s deny is cleared only when Jev was asked about exactly that concern, and any failure falls back to the regex result for that call.
Requires failproofai 1.0.8-beta.0 or later. 1.0.7 has no Jev, even though it sorts above the 1.0.7 betas. Without a Jev config nothing changes: hooks run the regex policies exactly as they always have.

Before you start

Install Failproof AI on the machine where your agent runs and attach its hooks to a supported harness. If you are starting from scratch, follow the quickstart through hook installation. Check the installed CLI with failproofai --version; update it if it predates Jev. You also need access to your organization’s Administration → Keys page to create a machine key. Jev reviews named tool calls at the PreToolUse or PermissionRequest gate. It does not review every event in a session. To see Jev clear a policy deny, you need an installed policy marked reviewable; all other policy denies remain final.

Turn it on

  1. Create a key with Jev. In the FailproofAI Cloud dashboard, open Administration → Keys → Create key and pick the machine preset. It grants the three permissions a machine needs: events:add (send activity), policies:pull (receive policies) and jev:evaluate (Jev, charged to your organization’s plan). A key cannot carry jev:evaluate without the other two.
  2. Connect the machine with that key. Read its one-time secret at a prompt, then run the full setup command:
    failproofai config installs the daemon, attaches hooks for the agent CLIs it finds, and connects the machine. The environment variable keeps the key out of the command’s arguments and your shell history. If your harness was installed later, attach it explicitly. If your organization runs its own FailproofAI Cloud rather than the hosted one, add its address: --url https://<your dashboard host> (or export FAILPROOFAI_CLOUD_URL). Without it the key is checked against the hosted service and the connection fails. If that host’s certificate comes from a private CA, install the CA in the machine’s system trust store (for example with update-ca-certificates), not only in NODE_EXTRA_CA_CERTS: the daemon that sends events and pulls policies reads the system store. See Troubleshooting.
That is all. Connecting stores the key and, when the machine has no Jev config yet, writes one for FailproofAI Cloud. From the machine’s first deployment (within about 30 s) FailproofAI Cloud sets the mode: enforce while a Jev policy is deployed to it, off when none is (see Who sets the mode). The output says so:
Checks FailproofAI Cloud deploys run on Cloud. To add checks of your own on the machine, install a pack:
With --no-transcripts, connecting does not turn Jev on. Jev sends each checked tool call and the recent prompt to FailproofAI Cloud, which is more than a decisions-only connection asked to send. The key is still stored, and the output says Jev is available and how to switch it on:
It also turns Jev through FailproofAI Cloud off. A jev.json that runs Jev through FailproofAI Cloud (left by an earlier connection with transcripts on) is removed, by the same rule config --disconnect uses, and the output says Jev via FailproofAI Cloud stays off on this machine. A jev.json for your own Jev endpoint is left as it is, and so is a FailproofAI Cloud one you switched off (--mode off).
Connecting never overwrites an existing ~/.failproofai/jev.json. If you already use your own Jev endpoint, it keeps being used, and the output says the file was left as configured — and, when that file leaves Jev off (refused, or switched off), says so and how to fix it. To switch that machine to FailproofAI Cloud, run failproofai jev setup --provider failproofai.

Who sets the mode

On a machine connected to FailproofAI Cloud, FailproofAI Cloud sets the Jev mode; nothing on the machine does:
  • enforce while any Jev (or regex + Jev) policy is deployed to the machine: every checked tool call from an agent one of those policies targets goes to FailproofAI Cloud, which asks its checks together with the checks of the packs installed there. A call from an agent no Cloud Jev policy targets, with no pack check of its own, is not sent at all. In enforce, Jev’s own deny checks block calls as well as clearing the regex verdicts they review.
  • off when none is: no Jev at all, whatever jev.json says.
To try a Jev policy before it acts, deploy it with effect observe: its checks are still asked, and what they would have done is recorded as a would-have on the policy page, but it never blocks a call and never clears one. failproofai jev setup --mode and the dashboard’s Settings → Jev switch say the mode is managed by FailproofAI Cloud and change nothing. jev.json is not used for Cloud’s checks, even if it names another provider. failproofai config --disconnect hands Jev back to the machine’s own jev.json. failproofai jev status then says Jev checks run on FailproofAI Cloud (mode: enforce). It also names the checks that review each both policy. The fleet page shows why a machine with a mode cannot ask:
  • transcripts_disabled: the machine was connected with --no-transcripts, which never switches Jev on.
  • jev_unconfigured: the machine’s key does not carry jev:evaluate.
  • jev_rate_limited: FailproofAI Cloud refused some of the machine’s calls for the organization’s Jev rate limit, and those calls were decided by the regex policies alone.
  • jev_unavailable: several calls in a row failed, so the machine skips FailproofAI Cloud’s Jev for a minute at a time. It retries on its own.
If FailproofAI Cloud does not answer within 5 seconds, the regex policies decide alone, as for any Jev outage. After 3 failures in a row the machine stops asking for a minute at a time, so a Cloud outage does not slow every tool call. A session pause (failproofai config --pause) does not stop FailproofAI Cloud’s Jev checks, just as it does not stop Cloud-managed policies. It does pause the Jev checks of packs installed on the machine.

Check what it is doing

status shows the provider as FailproofAI Cloud, the Cloud host the machine connected to, the mode, and the key source as FailproofAI Cloud connection, never the key. When a FailproofAI Cloud jev.json is in place but Jev cannot run, it says why: After failproofai config --disconnect there is no FailproofAI Cloud jev.json any more (unless it was switched off, which is kept), so status simply reports Jev as off. status --json carries the same facts (provider: "failproofai", keySource: "cloud", cloudConnected, keyCarriesJev), also when the config is absent or refused. permissions is always jev.json’s; a refusal about credentials.json adds credentialsPermissions, and fix when one command fixes it. test sends one live request and reports its latency and the Jev version that answered. It exits 1, and says so in its title, when the answer arrives after the hook timeout (hooks would record timeout) or answers its check question wrongly. The dashboard’s Settings → Jev panel also shows the FailproofAI Cloud connection: which organization the machine reports into and whether its key carries Jev. It is read from the machine’s own files, with no network call.

Verify a real call

Start a new session in the hooked agent. Ask it to use its file-reading tool on README.md and report the title. Confirm that the session contains that tool call, then run failproofai jev status again: its recent evaluated-call count should increase. Open Policies → Activity in the local dashboard to inspect that call’s Jev verdict and mode. In Cloud, the organization’s Policies page shows Jev outcomes for delivered activity. In observe mode, the verdict is recorded as a would-have and the policy result still decides the call. A clearance appears only when a reviewable policy matched and Jev cleared its named checks.

What reaches the policy page

The machine already sends its hook activity to FailproofAI Cloud (events:add). With Jev on, each gated call’s record also says which evaluator ran, what Jev decided, which policies it cleared, why it fell back when it did, its latency and the model that answered — decisions, codes and names, never the command or your prompt. On your organization’s Policies page:
  • a call Jev’s own verdict decided (enforce mode) is attributed to Jev, and when the deciding check came from a pack, the record also names that pack and its version;
  • in observe mode, Jev’s deny or warning appears as a would-have, next to the rollouts you are observing;
  • the policies Jev cleared, or would have cleared in observe mode, are counted per policy.
  • a decision Jev took part in (a block, a warning, a clear, or a would-have in observe mode) lists each check it asked and what that check concluded: verdict, why (not-detected, exempt, fired, op-requested, …), its evidence p, the exempt answer and each probe’s answer. A check of a policy deployed to watch (observe) is listed too, marked watched; it never blocks. Jev answers with probabilities, not prose, so there is no free-text reason. Plain allows roll up into counts and do not carry this list.

When Jev cannot answer

Every one of these falls back to your policies’ result for that call, and is recorded with its reason:

Where the key lives, and where it goes

  • The key is stored once, in ~/.failproofai/credentials.json (0600, in an owner-only directory), beside the other FailproofAI Cloud credentials. jev.json holds no key for this route; one written there makes the config invalid.
  • If credentials.json carries any permission for anyone but you (group or other, read or write), or its directory can be written by anyone but you, it is refused, not read, and Jev is off until you fix it: chmod 600 on the file, chmod 700 on the directory (or reconnect, which rewrites the file at 0600 and makes the directory owner-only). A directory others can only read is fine; one they can write lets them swap the file.
  • The key counts only while the connection it came with is on the machine: a policy or reporting credential for the same FailproofAI Cloud with the same key, in the same file. A Jev key left behind without one is ignored, and Jev stays off. That happens when an older failproofai’s config --disconnect leaves the Jev key in place (it does not know to remove it), or when an older failproofai’s config --token connects with another key, which on FailproofAI Cloud may belong to another organization. To switch Jev back on, connect again with a machine key.
  • The key is only ever sent to the Cloud origin it was verified against. A jev.json pointing anywhere else is refused.
  • An agent on the machine can read it. credentials.json is owner-only, and the agent runs as that owner. Reading failproofai’s own files is allowed on purpose (only changing them is blocked, by block-failproofai-commands), so the only thing between an agent and this file is block-read-outside-cwd — a reviewable policy — and from a session started in your home directory, nothing. A key with jev:evaluate spends your organization’s Jev allowance (up to the daily cap) from wherever it is used, so treat a machine key like any other spending credential: if an agent may have read it, disable it on the Keys page and reconnect with a new one.
  • Only your global files decide this. A repository cannot turn Cloud Jev on, point it elsewhere or supply its key, and FAILPROOFAI_JEV_API_KEY is ignored for this route.
  • For each call Jev evaluates, one request goes to FailproofAI Cloud, carrying what the bring-your-own-key page lists (secrets redacted). FailproofAI Cloud forwards it to TypeSafe and does not log or keep it.

Turn it off

On a connected machine, Jev is off when FailproofAI Cloud deploys no Jev policy to it: remove or archive the Jev policies there. From the next tool call, hooks run the regex policies exactly as before.