We nearly sent a plumber to the wrong city on a job worth around £2,000. The AI had, in one developer’s words, “the confidence of a senior salesperson and the attention span of a goldfish.” That incident set the policy for every agent we build now: guardrails before autonomy, every time.

The real danger isn’t that it talks. It’s that it acts.

A chatbot that gives a bad answer is embarrassing. An AI receptionist that books the wrong address, double-books a slot, or confirms a price you never agreed to is a real business problem. The moment an agent can touch your calendar, your customer records, or your payment flow, the risk profile changes completely.

This came up during a build for a trades business - an AI that answers calls, qualifies leads, and books appointments. During prototyping it nearly confirmed a complex job at an address it had pulled from the wrong part of the conversation. No human was in the loop to catch it. We caught it ourselves in testing. That was the last time we shipped an agent without the four rules below.

Rule 1: Tool gating

Dont give the agent access to booking until it has earned it. Start it on qualification only - it asks the caller the right questions and hands a structured summary to a human. The human books. Once you have a week of summaries and none of them are wrong, you consider opening the next gate.

In practice this means your agent starts with one tool:

tools_allowed:
  - qualify_lead # read-only; produces a structured JSON summary

Booking is not in the list. It cant book what it cant touch.

Rule 2: Action whitelisting - three tools, not ten

Most agent frameworks let you attach as many tools as you like. Resist that. An agent with ten tools will find creative ways to use all of them. Give it three, chosen for the lowest-risk actions first.

A sensible early allowlist for a trades receptionist:

tools_allowed:
  - qualify_lead      # collects job type, address, urgency
  - check_availability # read-only calendar lookup, no writes
  - create_draft_booking # writes to a staging table, not live calendar

The draft booking goes to a human for one-click confirmation. The live calendar doesnt move until a person says so. That staging step costs you about thirty seconds per booking and saves you the wrong-city incident.

Rule 3: Evaluate actions, not output

The tempting way to test an AI receptionist is to listen to a call and think “that sounded good.” Thats the wrong test. A confident-sounding agent that writes the wrong postcode to your calendar has failed, regardless of how smooth the conversation was.

Check the artefact, not the tone. After every booking attempt, pull the structured record and verify it against the call:

{
  "job_type": "boiler_service",
  "address": "14 Crane Street, Maidstone, ME14 1AB",
  "slot": "2026-07-22T09:00:00",
  "confirmed_by": null # null = still in staging, awaiting human sign-off
}

If address is wrong or slot conflicts with an existing job, the human catches it before it becomes a problem. Build that check into your review flow from day one.

Rule 4: Human-in-the-loop by default

Every stateful change - anything that writes to a real system - gets a second pair of eyes until it is boring and predictable. “Boring and predictable” has a definition: your reject log is nearly empty for a full week.

Keep the log. Every time a human overrides or corrects the agent, record it:

rejected_actions:
  - timestamp: 2026-07-10T11:23:00
    action: create_draft_booking
    reason: wrong_address
  - timestamp: 2026-07-11T14:05:00
    action: create_draft_booking
    reason: slot_conflict

That log tells you when it is safe to loosen the leash. Run a week with three tools. Review the rejects. If the pattern is clean, consider tool four. If it isnt, fix the prompt or the data before you expand the allowlist. The reject log is your evidence base - not your gut feeling about how the calls sound.

How we roll this out in practice

The trades build followed a staged rollout:

  • Week 1-2: qualify-only. Agent produces summaries; humans book everything.
  • Week 3-4: draft bookings enabled for low-risk jobs - small repairs, fixed-price callouts. Human confirms each one.
  • Ongoing: large quotes and complex multi-visit jobs stay behind human approval indefinitely. The upside of automating a £200 callout does not justify the downside of confirming a £2,000 job at the wrong address.

The trade-off worth knowing

Too tight a gate and you have built an expensive chatbot. The goal is to move the gate as the evidence supports it - not before. The reject log is what moves it, nothing else.

One question worth asking any AI vendor before you sign anything: “What happens when it gets it wrong, and who approves the risky actions?” If they dont have a clear answer, the leash isnt on yet.

If you want to see how we structure this for a specific business, take a look at what we’ve built and get in touch for a conversation.