The Risks of Autonomous AI Agents Taking Actions

Autonomous agents fail differently than scripts: they act on inferred intent, not fixed logic, so a bad inference becomes a real refund, email, or deploy before anyone notices.

Why this is a different failure mode

A traditional script does exactly what its code says, every time. An agent decides what to do based on a prompt, a context window, and whatever tools it has been handed. That decision can be wrong for reasons that have nothing to do with a bug: a support ticket contains adversarial text, a tool result is ambiguous, the model over-generalizes from a similar-looking past case. The code runs fine. The judgment underneath it doesn't.

The risk isn't hypothetical for any agent wired to a real side effect — sending email, issuing a refund, posting content, modifying a database row, pushing a deploy. Once the agent has a working credential and a reason (however flawed) to use it, nothing stops it from acting immediately.

Categories of risk, concretely

The naive fix doesn't hold

The common first instinct is to add a line to the system prompt: "Always ask for confirmation before doing anything irreversible." This fails for a boring reason — it's advisory, not structural. The model can still decide, correctly or not, that a given case doesn't count as "irreversible," or a jailbreak can talk it past the instruction entirely. There's also no record of what almost happened, so you can't audit near-misses.

A structural gate instead of a prompt instruction

The fix that actually holds is making the side effect itself unreachable without an external decision — not asking the model to self-police. Concretely: the agent calls an API to propose the action, then polls for a decision, and only the approved branch of the code can call the real tool.

import os
import time
import requests

API = "https://api.impri.dev"
HEADERS = {"Authorization": f"Bearer {os.environ['IMPRI_API_KEY']}"}

def propose_refund(order_id: str, amount_cents: int, reason: str) -> dict:
    resp = requests.post(f"{API}/v1/actions", headers=HEADERS, json={
        "kind": "payment.refund",
        "title": f"Refund ${amount_cents/100:.2f} for order {order_id}",
        "preview": {"format": "plain", "body": reason},
        "idempotent": False,
        "undo": f"No automatic undo — re-charge order {order_id} manually if reversed in error",
        "expires_in": 3600,
    })
    return resp.json()

def wait_for_decision(action_id: str) -> str:
    while True:
        r = requests.get(f"{API}/v1/actions/{action_id}", headers=HEADERS).json()
        if r["status"] != "pending":
            return r
        time.sleep(10)

action = propose_refund("ord_4821", 5000, "Customer reports item never delivered")
decision = wait_for_decision(action["id"])

if decision["status"] == "approved":
    issue_refund_via_stripe(decision["decision"]["final_preview"])  # your real call
    requests.post(f"{API}/v1/actions/{action['id']}/result", headers=HEADERS,
                  json={"status": "executed"})
# rejected or expired: issue_refund_via_stripe is never called

The refund function is only reachable inside the approved branch. A prompt-injected ticket can still get the agent to propose a bogus refund, but it cannot get the money moved without a human clicking approve. That's the actual improvement over "please confirm first": the model's mistake becomes a support ticket a human reviews, not a completed transaction.

Where this stops helping

Be precise about the boundary. This pattern only holds if the approval-gated path is the agent's only route to the side effect — see how to add human approval to an AI agent for the wrapper-around-the-tool detail that makes that true. If the agent also holds a raw Stripe key it can call directly, the gate is decorative. Impri itself doesn't evaluate whether a refund is a good idea — it stores the proposal, notifies a human, and holds the decision. The judgment still belongs to the person who taps approve, same as it always did; what changes is that the judgment happens before the money moves, not after.

Next step

If you're wiring this into a real agent, the quickstart covers getting an API key (cloud or self-hosted), and the MCP integration gives Claude-based agents the same three-call flow without writing raw HTTP.