Julien Florkin

Artificial Intelligence

The AI Workflow That Needs a Stop Button

An AI assistant drafts a customer reply. At first, a person reads it, corrects it, and decides whether to send it.

Julien Florkin 6 minute read
Responsible AI in Business: Practical Controls Without Bureaucracy editorial illustration

An AI assistant drafts a customer reply. At first, a person reads it, corrects it, and decides whether to send it. Then the draft becomes consistently useful. Someone asks the tempting next question: why not let the assistant send the reply, update the customer record, or choose which cases receive priority?

That is the control point. The technology may be the same on both sides of it. The consequence is not. A suggestion can be ignored. An action changes the world outside the model.

The practical responsible-AI question is therefore smaller than “Do we have enough governance?” Ask: What may this workflow affect when it is wrong, and how much freedom does the system have to cause that effect before a person can intervene?

In this article

The moment assistance becomes action

In recommendation mode, the assistant prepares information for a person. It can summarize a conversation, propose a reply, or suggest a classification. The person is still the actor.

Execution begins when the system sends, records, routes, approves, rejects, purchases, schedules, discloses, or otherwise changes something without a fresh human decision. Some actions are minor. Saving an internal draft in a temporary workspace is different from sending a refund decision to a customer. The relevant boundary is not whether the output contains text. It is whether the workflow can create an external consequence.

This is also why a generic list of “AI controls” is a poor starting point. A spelling suggestion and an autonomous eligibility decision do not need the same treatment. Applying the heaviest process to every use case creates theatre. Applying the lightest process to every use case hides harm.

The NIST AI Risk Management Framework defines AI risk in relation to likelihood and the magnitude of consequences. It is a voluntary, use-case-agnostic framework, and NIST notes that version 1.0 is being revised. Its useful lesson here is contextual: controls should follow what a system can affect, not the fact that it contains AI.

Consequence and autonomy change the control

Keep the customer-reply workflow in view. There are three practical operating positions.

In recommendation-only mode, the assistant prepares a draft and cannot send or update a system of record. This is the right default when the team is still learning what errors occur, the data boundary is unclear, or the action can materially affect a person.

In approval-required mode, the assistant may prepare the action, but a person must explicitly approve it with enough context to judge the case. This can be proportionate when the task is understood and the reviewer can catch meaningful errors before execution.

In narrowly bounded action mode, the system may execute only a small class of low-consequence, easily reversible actions under explicit constraints. For example, it might add a non-final internal label from an approved list while remaining unable to send a customer message, change entitlement, or move money.

The boundary moves with four facts: the consequence of a wrong action, the system’s freedom to choose or sequence actions, the sensitivity of the information it can reach, and how difficult the effect is to reverse. More autonomy is not simply more convenience. It is a larger path from model output to consequence.

If sensitive data is central to that choice, resolve the data readiness and access boundary before expanding the system’s reach. If the proposal uses tools across several steps, first understand what AI agents in business workflows can actually do. Neither link changes the local decision: this workflow stays below the action threshold until its own boundary is clear.

Set the boundary before the system acts

A useful boundary can be stated in ordinary language.

For the customer-reply assistant, it might be:

The assistant may draft replies from approved knowledge and suggest one internal category. It may not send messages, change customer status, grant or remove access, issue credits, or prioritize a case without explicit human approval.

That sentence does several jobs. It separates preparation from execution. It names allowed and forbidden actions. It keeps high-consequence decisions on the human side. It also gives a technical team something testable: the system’s credentials and tool permissions should make the forbidden actions impossible, not merely discouraged in a prompt.

The information boundary belongs in the same decision. The assistant should reach only the data required for the approved task. “It might be useful later” is not a reason to grant broad access now. If a draft needs the customer’s current request and an approved help article, access to unrelated accounts, staff files, or full historical exports adds exposure without improving the task.

This is operational judgment, not a compliance finding. Laws, contracts, sector rules, employment obligations, consumer protections, and data-protection requirements may impose other duties. A local workflow boundary neither certifies the system nor guarantees that its use is lawful or safe.

Make intervention real, not ceremonial

Putting a person “in the loop” can sound reassuring while changing very little. Review is effective only if the person can understand the proposed action, has time to assess it, is allowed to reject it, and can stop execution before the consequence occurs.

A reviewer shown only a polished reply may miss the source material, uncertainty, or missing customer context. A reviewer processing hundreds of approvals under time pressure may become a confirmation step. A reviewer who cannot edit, reject, or pause the workflow is not exercising meaningful control.

NIST’s human–AI interaction guidance stresses that human roles should be clearly differentiated and that some systems require oversight while others may not. The EU AI Act goes further for systems legally classified as high-risk: Article 14 says oversight measures should be proportionate to risk, autonomy, and context, and should enable appropriately authorized people to understand limitations, avoid over-reliance, interpret outputs, and intervene or stop the system. That legal requirement is specific to the Act’s scope and high-risk classification; it is not a universal compliance checklist for every workflow.

The broader operational test is simpler: can this person prevent this action, in this interface, before it matters? If the answer is no, keep the workflow in recommendation-only mode.

Keep the experiment inside a reversible line

Teams often expand autonomy one convenience at a time. Drafting becomes sending. Suggesting a category becomes assigning priority. Looking up a record becomes editing it. The safest moment to set the line is before those permissions are connected.

Start with the smallest useful mode. Let the assistant prepare work while a person remains the actor. Observe the kinds of cases that require correction, but do not treat a good average as permission to automate the exceptions. Move one action across the boundary only when its consequence is low, its data access is narrow, its constraints can be enforced outside the prompt, and intervention occurs before harm.

The stop condition should be equally plain: if the task touches an unapproved data source, produces an action outside the allowed set, lacks the context required for review, or raises the consequence beyond the agreed boundary, it does not execute. It returns to a person.

When the problem grows beyond one workflow, organization-level AI governance for business leaders becomes the better next context. This article’s job ends earlier: at the moment assistance becomes action.

Responsible control is not the amount of paperwork surrounding the assistant. It is the clarity of the line it cannot cross. Name the consequence, limit the autonomy, make intervention real, and keep the system on the safe side of that line until the workflow—not the enthusiasm—earns a change.

Sources checked

Related reading