AI Decision Making with Human Oversight
The AI returns a decision from a fixed set of options, with a confidence score and a reason, not free text. Your rules decide what runs automatically, what needs approval and what a person decides.
What automated decision making with AI means
Automated decision making with AI means describing a recurring decision so that the AI makes it from a fixed set of options and your systems can process the result. In 2025, 26% of companies in Germany used AI, and 27% of them used it to automate workflows or for decision making (Destatis).
Fixed set of options
The answer comes from a predefined list such as “approve”, “review”, “reject” or a responsible team – not free text as with an AI chatbot.
Confidence score
Every answer carries a value your systems can evaluate. It is calibrated on real cases before any threshold applies.
Reason
The features that led to the decision are shown in readable form – for the person who reviews or approves.
Where structured decisions help
Suitable decisions recur often, have a fixed set of possible answers and consequences that can be described.
Assigning requests and tickets
Emails, forms and tickets go to the responsible team or the right queue, and urgent cases are recognised.
- Team or queue as the answer
- Urgency classified
- Background: AI ticket system
Approval and escalation
Orders, credit notes or discounts above a limit go to the responsible person for approval.
- Limits per process
- Approval with a reason
- In the B2B shop as order approval
Scoring enquiries
Fit and urgency of an enquiry determine the order in which sales handles it.
- Score with a reason
- Sales decides on the offer
- Whether GDPR Art. 22 applies is clarified with your data protection officer first
Detecting duplicates
Duplicate customers, products or suppliers are detected and proposed as a pair with a confidence score.
- Customers, products, suppliers
- Proposal with a reason
- Merged only after approval
Classifying
Return reasons, complaints, product data and document types are assigned to a fixed category.
- Returns and complaints
- Products via category automation
- Documents via document processing
Safeguarding agent steps
An AI agent only proposes actions from an allowed list; anything with an external effect runs only after approval.
- Allowed actions as a fixed set
- Payments, deletions, customer emails only after approval
- Background: prompt injection
Thresholds decide who decides
Your rules turn a prediction into a controlled decision. You set the thresholds, per decision and per direction of a possible error.
Above the threshold
The case runs automatically. Spot checks cover confident cases too, so nobody trusts the machine too much.
Grey zone
A person approves, seeing the confidence score and the reason.
Below the threshold
The complete case goes to the responsible team, which decides itself.
Audit log
Input, model version, result, confidence, threshold and approver are recorded; corrections overwrite nothing.
Legal limits: GDPR Art. 22 and the EU AI Act
Where a decision affects people, the law sets limits. This overview is not legal advice.
GDPR Art. 22
- Data subjects have the right not to be subject to a decision based solely on automated processing which produces legal effects concerning them or similarly significantly affects them (Art. 22(1)).
- Exceptions, for instance where the decision is necessary for a contract or based on explicit consent, require at least the right to obtain human intervention, to express one’s point of view and to contest the decision (Art. 22(2) and (3)).
- Data subjects can request meaningful information about the logic involved (Art. 15(1)(h)).
- As of September 2026, Art. 22 applies as worded in 2016; the European Commission has proposed amendments, none has been adopted.
EU AI Act
- High-risk systems must be designed so that people can oversee them effectively; oversight must be able to disregard, override or reverse the output (Art. 14).
- The regulation explicitly names “automation bias”, the tendency to over-rely on a machine’s output (Art. 14(4)).
- Deployers assign human oversight to people with the necessary competence, training and authority (Art. 26).
- High-risk systems include those listed in Annex III, such as recruitment or creditworthiness assessment of natural persons (Art. 6, Annex III).
- Under the Digital Omnibus (Regulation (EU) 2026/1744), these obligations apply to Annex III systems from 2 December 2027.
Art. 14 applies to high-risk systems; we build the same intervention options in below that level as a matter of principle. The legal assessment of an individual case is for your data protection or legal adviser; our consulting supports the planning.
Decisions where the work happens
The decision is connected to the system where the case arises, and its result is written back there.
Connection
Shop, ERP, CRM and ticket system provide the case and receive the result.
- Via existing integrations or custom programming
- Shop events as triggers, for example with the Shopware Flow Builder
- Customer data from the CRM: CRM integration in your online shop
Cost and operation
A fixed set of options instead of free text keeps every answer short.
- Model choice per decision: small models for simple classifications, larger ones only where needed
- Hosted or on your own infrastructure
- Background: controlling AI costs in your shop
From individual decisions to rules
Describe the decision
The possible answers, the consequences of an error in either direction and the people responsible are defined.
Measure on historical cases
Hits per answer; the confidence score is compared with the actual hit rate.
Shadow mode
The AI decides alongside, people keep deciding. The thresholds follow from the comparison.
Release step by step
Confident cases run automatically first, with ongoing spot checks and an audit log.
How we run projects in general: project process.
Frequently asked questions about AI decision making
A chatbot answers in free text. A structured decision returns an answer from a fixed set of options, with a confidence score and a reason. The result can be processed by machines and tied to thresholds.
A model’s scores do not match the actual hit rate by themselves. They are calibrated on your historical cases, the thresholds are set accordingly and checked with spot checks during operation.
Not solely automated where the decision has legal effects on a person or similarly significantly affects them (GDPR Art. 22). Exceptions require that a person can intervene and the decision can be contested. Where that applies, a person decides in our setup; please clarify the assessment with your data protection officer beforehand.
Effective human oversight (Art. 14) for high-risk systems under Annex III, such as recruitment or creditworthiness assessment. Under the Digital Omnibus these obligations apply from 2 December 2027. Whether a use case falls under this is checked beforehand.
Every decision is logged with input, model version, result, confidence, threshold and approver. This also helps answer the question about the logic involved that data subjects can ask under the GDPR.
Yes, as a framework: the agent proposes actions from an allowed list, each with a confidence score. Actions with an external effect, such as payments, deletions or customer emails, run only after approval, and everything is logged.