Keeping a human in the loop for important AI decisions
By Keith Hamilton · · 4 min read
Human-in-the-loop AI means a person can check, understand and change an AI decision when it really matters to a customer or a member of staff. AI is good at sorting, scoring and drafting, but it does not know when it is wrong. This guide explains which decisions need human review, what the rules on automated decision-making under UK GDPR expect in general terms, and how to design a review step that is real rather than a rubber stamp.
Which AI decisions need a human in the loop?
AI decisions that have real consequences for someone need a human in the loop. Routing an email to the right inbox or suggesting a reply for a colleague to edit carries little risk. AI decisions about customers and staff that need care include:
- refusing a service, an account or credit
- setting an individual's price or terms
- shortlisting or rejecting job applicants
- flagging a customer as high risk or suspected of fraud
- deciding whether a complaint or claim is upheld
- pausing a service or passing a debt on for recovery
The test is simple: if this decision were wrong, would the person affected be harmed, out of pocket or treated unfairly? If the answer is yes, plan for human review from the start.
What does UK GDPR say about automated decision-making?
UK GDPR gives people protection around decisions made solely by automated means that have legal or similarly significant effects on them. The Data (Use and Access) Act 2025 changes some of these rules and is being brought into force in stages, so the detail is moving.
The direction is consistent, though. Where significant decisions are automated, people should be told, be able to put their side, be able to challenge the outcome and be able to get a person involved. Stricter rules apply where sensitive information, such as health data, is involved.
Other rules apply too. Equality law applies to decisions made with AI just as it does to decisions made by people, and regulated businesses must meet their regulator's expectations on treating customers fairly.
This is general guidance, not legal advice. If you are automating significant decisions, take specialist advice and carry out a data protection impact assessment. Our UK GDPR and AI checklist explains when one is needed.
How do you make human review of AI decisions meaningful?
Human review of AI decisions is meaningful only when the reviewer is able to change the outcome, not just click "approve". The ICO's guidance makes the same point: human involvement has to be active and able to change the outcome, not a token gesture.
For review to count, the person needs:
- Authority. They can overrule the AI without needing permission or having to justify it at length.
- Information. They can see the data the AI used and the main reasons for its suggestion, not just a score.
- Time. Their workload allows them to look properly. Fifty decisions an hour is not review.
- Training. They understand what the tool does well, where it goes wrong, and what to look for.
What is automation bias?
Automation bias is the habit of trusting a computer's answer, especially when it is usually right. Over time, reviewers stop checking and start agreeing. It is the main way a sound review process quietly fails.
These habits counter it:
- Show the reviewer the evidence before the AI's recommendation, so they form their own view first.
- Keep track of how often reviewers change the AI's suggestion. If the answer is almost never, find out why.
- Check a sample of decisions in depth regularly, including ones the AI was confident about.
- Rotate reviewers or pair them on difficult cases.
How do you design the review step?
Decide in advance which cases go to a person, then make the hand-over smooth. The standard triggers are:
- any decision that goes against the customer, such as a refusal
- cases where the AI's confidence is low or the information is incomplete
- signs of vulnerability, distress or financial difficulty
- disputes, complaints and anything unusual
Our article on AI credit control shows how this works in practice: the AI handles routine reminders, while disputes and customers in difficulty go to a trained person.
The reviewer should see the full history in one place, and their decision and reasons should be recorded. Those records show that review is happening, and they show you where the AI is getting things wrong.
How do people ask for a review of an AI decision?
Give customers and staff an easy way to ask a person to look again at a decision about them. Explain how in your privacy information and in the message that tells them the outcome. When someone asks, have the review carried out by someone who can genuinely change the result.
How do you keep human review working over time?
Check regularly that the process still works as intended. Look at the decisions that were changed on review, complaints and appeals, and any patterns suggesting certain groups are treated differently. When the AI tool or your process changes, revisit the design.
Where Forwardcycle fits
Our safe adoption service includes data protection impact assessments and staff guidance that set out where people stay in control of AI decisions and how review works in practice. If you are planning to use AI for decisions that affect customers or staff, start with the free AI Readiness Assessment or arrange a free call.