Key takeaways in 3 minutes
Most AI demos end when the answer appears. Real AI products have to carry that answer through evidence, human judgement, authority, action, recovery and learning.
I use a seven-stage decision flow:
The interface is only one surface of this system. Underneath it sit permissions, policies, provenance, checkpoints, records and recovery paths. Designing those relationships is AI product architecture with a UX lens.
The Seven-Stage Decision Flow
- 01Frame the decision, consequence, boundary and accountable owner.
- 02Assemble evidence, provenance, freshness, gaps and conflicts.
- 03Separate the AI proposal from its rationale, uncertainty and alternatives.
- 04Help a person inspect, compare, question and disagree.
- 05Match authority and controls to consequence and reversibility.
- 06Execute a fixed, inspectable action with interruption and recovery.
- 07Record the outcome, support contest and improve the next decision.
Most AI demos end at the most flattering moment.
The model has gathered the information. The answer arrives. A neat summary appears beside a reassuring confidence score. Everyone nods at the screen.
Then the demo stops.
In a real product, that is often where the consequential part begins.
Someone still has to decide whether the evidence is enough. Someone needs the authority to accept or change the proposal. A system may need permission to move money, update data or contact a customer. The action may fail halfway through. The outcome may need to be explained, contested or reversed.
That is not a recommendation screen.
It is a decision flow.
Most AI demos end when the answer appears. That is usually where the consequential part of the product begins.
The Product Is The Passage From Proposal To Outcome
Imagine an AI system reviewing a supplier payment.
The invoice matches the purchase order. The goods have arrived. The amount is within the manager's approval limit. The system recommends payment with 96% confidence.
Very tidy.
Buried elsewhere is a bank-detail change made two days ago. A fraud warning has not reached the payment workflow. The manager can click Approve, but cannot hold the payment while asking treasury to verify the account. The record will preserve the click, not what the manager saw.
The model's answer may be correct against its inputs.
The decision flow is still badly designed.
A serious AI product has to orchestrate more than model output. It has to connect evidence, uncertainty, human judgement, organisational authority, technical permissions, safe action and the record left behind.
The NIST AI Risk Management Framework treats AI risk as a continuous activity across governance, context, measurement and management. The EU AI Act's human-oversight requirements for high-risk systems similarly point towards interfaces that help qualified people understand limitations, avoid over-reliance, override outputs and interrupt a system safely. These are not merely policy requirements hovering above the product. They are design inputs.
A Seven-Stage AI Decision Flow
I use seven stages to make the whole system visible:
`text
FRAME → EVIDENCE → PROPOSE → CHALLENGE → AUTHORISE → ACT → LEARN
`
The Seven-Stage AI Decision Flow
1. Frame
Before asking what the AI should do, define the decision.
What is changing in the world? Who or what could be affected? What is the scope? What counts as success? What must never happen? Who owns the result?
This is the intent contract for the flow. It stops a broad ambition such as “optimise supplier payments” quietly becoming permission to act on every supplier, account and exception the system can reach.
2. Evidence
A proposal is only as useful as the evidence a person can verify.
The flow should distinguish source facts, retrieved documents, inferred claims, missing data and conflicting signals. It should show freshness and provenance where they matter.
More data is not automatically more context. A reviewer needs to know which evidence supports which claim and what the system could not establish.
3. Propose
The AI should make a bounded proposal, not quietly convert its interpretation into an action.
The proposal needs a clear target, scope and expected effect. Rationale, uncertainty, alternatives and policy constraints should remain distinguishable. A precise confidence percentage is not a substitute for any of them.
4. Challenge
The person needs a feasible way to determine when the AI might be wrong.
That means inspecting sources, comparing alternatives, seeing counter-evidence, asking a question, changing an assumption or recording dissent. Explanation should help verification, not merely make the recommendation sound more persuasive.
Research on human–AI decision-making warns that explanations can sometimes increase reliance on wrong answers as well as correct ones. The practical lesson is simple: design explanations as tools for checking the work, not as sales copy for the model. See Buçinca, Malaya and Gajos and Vasconcelos et al..
5. Authorise
Responsibility without authority is governance theatre.
The product must know whether this person can make this decision, at this value, for this customer, in this jurisdiction and under the current policy. It should support the actions the situation actually requires: approve, edit, hold, request evidence, reject, escalate or stop.
The gate should be proportional to consequence, reversibility and exposure. Requiring approval for every harmless read creates fatigue. Hiding an irreversible payment behind the same button creates risk.
6. Act
Approval is not execution.
Before action, show what will change, on which target and using whose authority. Bind the approval to that exact payload so the operation cannot drift after it has been approved.
During execution, show progress and partial failure honestly. Preserve a safe interruption point, an undo or compensating action where possible, and a useful handoff when the system cannot continue.
Practical agent-security guidance from OWASP reinforces the architecture: separate decision from execution for irreversible operations, use least privilege and do not let model output become the authorisation mechanism.
7. Learn
The record is not complete when the button is pressed.
Did the action execute? Did it achieve the intended result? Was it reversed or contested? Did similar decisions begin to fail? What should change in the policy, context, product or evaluation set?
This is where a decision becomes operational evidence rather than a disappearing interaction.
The UX And The Architecture Are The Same Conversation
It is tempting to divide this work neatly.
Design owns the screen. Engineering owns the agent. Risk owns the policy. Operations owns the outcome.
Unfortunately, the decision does not respect the org chart.
A disabled Approve button depends on a trustworthy policy classification. A meaningful explanation depends on evidence provenance. An Undo control depends on the action being reversible. A useful decision record depends on capturing the brief as presented, the authority applied, the exact action and the later outcome.
Each interaction implies an architectural requirement. Each architectural choice creates a user experience.
That seam is where AI Product Design Architecture earns its keep.
The Practical Move: Map One Consequential Decision
Pick one AI-assisted decision that changes money, access, data, commitments or a person's options.
Map it across the seven stages. For each stage, capture four things:
- User need: what must the person understand or do?
- System state: what does the product know and what remains uncertain?
- Control: what prevents the wrong action or limits its consequence?
- Evidence: what survives afterwards?
Do not average the gaps into one reassuring maturity score. Find the weakest transition.
The most dangerous point is often not a missing feature. It is the handoff where evidence becomes a proposal, a proposal becomes permission, or permission becomes action.
Why This Connects To Accountability Lab
Accountability Lab is where I turn these questions into practical review methods, approval patterns and decision-record artefacts that teams can inspect, challenge and improve.
The point is not to add governance paperwork after an AI product has been designed.
It is to make evidence, authority, intervention, recovery and learning visible in how the product behaves.
Because the interface is not the thin layer on top of the decision system.
It is where the system reveals what kind of decision system it really is.
Sources And Further Reading
- NIST AI RMF Core — voluntary reference framework; version 1.0 is under revision.
- EU AI Act, Regulation (EU) 2024/1689 — see Article 14 for human oversight of high-risk AI systems.
- ICO guidance on human oversight.
- OWASP AI Agent Security Cheat Sheet.
- Guidelines for Human–AI Interaction, CHI 2019.
