Artificial Intelligence: What It Can Do, Where It Fails, and How to Use It Well

A customer writes to say they were charged twice. Before anyone replies, an AI system could classify the message, find the relevant account records, retrieve the refund policy, draft an answer, and perhaps trigger the refund itself. We often compress that whole chain into one sentence: “AI handled the ticket.”

But those are not one job. They are five different jobs with different evidence, different failure modes, and different consequences. Misclassifying the message may delay a response. Retrieving the wrong account could expose private information. Inventing a policy could mislead the customer. Sending the draft without review could turn a plausible mistake into a company promise. Issuing a refund gives the system authority over money.

That small ticket contains the central lesson of artificial intelligence. AI becomes most useful when we stop asking whether it is intelligent in the abstract and start asking what role it plays in a real system. What does it receive? What does it produce? How will anyone know when it is wrong? Who can intervene before the consequence becomes difficult to reverse?

This article will help you answer those questions. You will see what modern AI systems actually do, why an impressive demonstration can fail inside ordinary work, where AI creates genuine leverage, and how to test a use case without confusing speed with value or fluency with judgment.

In this article

Artificial intelligence is a broad term for machine-based systems that infer from inputs how to produce outputs such as predictions, recommendations, content, or decisions. That wording is close to the OECD’s updated definition of an AI system, and its restraint is useful. It describes what a system does without pretending that every AI tool thinks, understands, or acts like a person.

Most AI that people use today is narrow. A fraud model estimates whether a transaction resembles suspicious activity. A vision model identifies patterns in an image. A recommendation system ranks items. A language model generates a sequence of words that fits its input and learned patterns. One system may perform impressively on its intended task and remain useless outside it.

Machine learning is one way to build these capabilities. Instead of writing every rule by hand, developers use data and optimization to shape a model’s behaviour. Deep learning refers to machine-learning methods built from many computational layers; it has driven much of the recent progress in language, image, audio, and multimodal systems. Generative AI is the family that produces new content—text, images, software code, audio, or other media—rather than only assigning a label or score.

These distinctions matter, but the model is never the whole story. The support-ticket system also includes the customer message, account database, retrieval method, instructions, permissions, user interface, human reviewer, refund process, logs, monitoring, and correction path. Change any of those parts and the same model can become safer, more useful, or far more dangerous.

This is why “Which AI should we buy?” is usually an early question disguised as a strategic one. The better starting point is: “Which capability would improve this specific flow of work, under these conditions, for these people?” A model may be technically strong while the surrounding system is badly designed. Conversely, a modest model inside a careful workflow can create meaningful value because its job is clear and its mistakes are caught before they travel.

The useful question: which part of the work changes?

Return to the duplicate-charge ticket. The work can be separated into a sequence:

1. Recognize what the customer is asking.
2. Find the correct customer, transaction, and policy.
3. Propose an explanation or next action.
4. Check the proposal against the evidence and the customer’s situation.
5. Communicate or act.
6. Record what happened and correct mistakes.

AI might help at several points, but “help” means something different at each one. Classification reduces sorting effort. Retrieval reduces search time. Generation creates a candidate response. A decision model may recommend whether a refund fits policy. An automated agent may carry out the approved action. The farther the system moves from proposing toward acting, the more authority it acquires—and the more important it becomes to understand the consequence of an error.

That gives you a practical unit of analysis: not the tool, but the task-output-consequence chain.

• Task: What narrow piece of work is the system being asked to perform?
• Input: What information can it see, and what relevant context is missing?
• Output: Is it producing a draft, a score, a ranking, a recommendation, or an action?
• Consequence: Who or what changes if the output is accepted?
• Control: How will an error be detected, challenged, reversed, and learned from?

This chain prevents two opposite mistakes. The first is to reject a useful low-consequence aid because “AI makes mistakes.” The second is to automate a consequential step because the model looked accurate in a demonstration. A spelling suggestion and a credit decision may both use AI, but they do not deserve the same evidence or oversight.

Where AI creates real leverage

AI creates leverage when it changes the economics or quality of a task without hiding more costly work downstream. Four task characteristics are especially promising.

Volume makes a first pass expensive. A team may need to sort thousands of messages, inspect many documents, or review recurring records. AI can reduce the pile to a smaller set that deserves attention. The value is not that the machine “understands the business”; it is that people can spend more time on exceptions and decisions.

Variation makes rigid rules brittle. Customers describe the same problem in many ways. Documents use inconsistent layouts. Images and conversations do not arrive as neat database fields. Models can often handle this messy variation better than traditional rules, provided the range of real inputs is represented in testing.

Patterns are hard to see at human scale. Forecasting, anomaly detection, quality inspection, and recommendation can benefit when there is relevant historical data and a repeatable relationship between inputs and outcomes. Yet a pattern is not an explanation. If the environment changes, the relationship may weaken while the score still looks precise.

Candidate generation lowers the cost of beginning. Generative AI can offer draft text, code, designs, scenarios, questions, or alternative explanations. This is particularly useful when the output can be inspected and revised. A blank page becomes a set of possibilities; the person still owns truth, fit, taste, and consequence.

The duplicate-charge ticket has all four. Support volume is high, customer language varies, recurring patterns can aid routing, and a draft can save time. But the best first use may not be full automation. It might be simply grouping likely billing issues and presenting the original message beside the classification. That narrow intervention reclaims attention while keeping the evidence visible.

The strongest use cases often follow this pattern: the system absorbs repetition or expands the option space, while people concentrate on context, exceptions, relationships, and accountability. That is business leverage, not because the tool replaces thought, but because it moves thought to where it matters more.

Why impressive AI fails in ordinary workflows

A demonstration controls the input, the moment, and the success criterion. Work does not. Real systems encounter incomplete messages, unusual customers, changing policies, malicious instructions, missing data, conflicting goals, and people who assume the output has already been checked. Four gaps turn many impressive demos into disappointing operations.

The first is the truth gap. Generative models produce plausible content by extending learned patterns. They do not automatically establish whether a claim is true. The NIST Generative AI Profile uses the term confabulation for confidently presented false or erroneous content. A fabricated refund rule can read exactly like the real one. Polished wording may make verification less likely, not more.

The second is the context gap. The model sees what the system provides, not the whole situation. A customer may have two accounts, an earlier promise from an employee, a disputed delivery, or a legal right that the standard policy does not capture. Adding more data can help, but it also increases privacy, security, and access-control burdens. Context is not free.

The third is the objective gap. A system can optimize the measure it is given while damaging an outcome nobody measured. If support automation is rewarded for closing tickets quickly, it may produce short replies that technically end conversations but leave customers confused or repeatedly reopening cases. Fast completion is not the same as resolution.

The fourth is the handoff gap. People tend to trust outputs that arrive with confidence, speed, and professional formatting. A reviewer asked to approve hundreds of drafts may become a rubber stamp. “Human in the loop” then exists on paper while the pace and interface make meaningful review impossible. Oversight is a work-design problem, not a checkbox.

Our ticket therefore needs tests beyond “Did the model generate a good answer?” We must ask whether it found the right account, used the current policy, showed its evidence, signalled uncertainty, handled unusual cases, protected sensitive information, and made correction easier. We also need to observe the reviewer: did the system sharpen their attention or quietly exhaust it?

This broader view explains why responsible AI cannot be reduced to model accuracy. Reliability lives in the interaction among model, data, interface, incentives, people, and consequence.

Match oversight to the consequence

Not every AI output deserves a committee. Not every output deserves one click. Proportionate oversight begins with three properties: how serious the harm could be, how easily an error can be detected, and how reversible the result is.

Use level	Typical output	What can go wrong	Sensible control
Explore	Private ideas, questions, rough alternatives	Time is wasted or a weak direction looks attractive	User judgment; do not treat output as evidence
Assist	Draft, summary, classification, or recommendation	Error influences work but has not yet reached the affected person	Compare with source material; sample errors; keep editing authority human
Influence	Score or ranking shapes access, priority, price, work, or treatment	Bias or error changes a consequential decision	Validated data and measures; qualified review; explanation and challenge route; monitoring
Act	System communicates, approves, rejects, pays, publishes, or changes a record	Error becomes an external commitment or material harm	Explicit authority limits; pre-action controls; strong authentication; logs; rollback; incident response

The levels are not labels for technologies. They describe uses. The same language model may be in Explore when it proposes workshop questions and in Act when it sends a binding customer message. The model did not change; its position in the workflow did.

For the ticket, private drafting is easy to reverse. Sending the draft affects trust. Retrieving account details affects privacy. Issuing a refund affects money. Each step needs a different control, and the safest architecture may grant the model access only to the information and actions required for that step.

Escalation becomes necessary when the system touches rights, safety, employment, credit, health, legal status, sensitive personal data, security, or other high-consequence domains. In those cases, general AI literacy is not enough. Bring in the people qualified to interpret the domain, applicable law, privacy and security duties, and the lived impact on those affected.

This is also where geography and timing matter. In the European Union, the AI Act now applies progressively, with different duties for different roles and risk categories. The European Commission’s current AI Act overview and implementation FAQ describe the live timeline, including transparency and general-purpose-AI obligations and later dates for certain high-risk systems. A blog article cannot determine your legal classification. It can tell you not to treat yesterday’s summary as today’s compliance plan.

How to test an AI use case without fooling yourself

A useful pilot is not a smaller launch. It is an experiment designed to expose where the idea breaks before the organization depends on it.

Begin with a decision baseline. For the support queue, record how the work happens now: time to first useful handling, rate of correct routing, rework, reopened cases, escalations, customer corrections, and the effort required from experienced staff. If you do not understand the current process, a faster new process may simply hide its costs.

Next, choose a bounded role. Let the system classify tickets or draft replies while people retain authority. Fix the relevant data sources and instructions. Define which cases are out of scope. Keep the old process available. A pilot that simultaneously changes the model, data, interface, team roles, and policy teaches very little because no one knows which change produced the result.

Then build a representative test set. Easy examples are comforting and nearly useless. Include routine cases, rare cases, ambiguous language, missing information, multiple languages where relevant, policy changes, adversarial content, and situations where the correct action is to abstain or escalate. Protect personal data and use the least sensitive information that can answer the question.

Measure the whole workload, not only the model output. Did routing improve? How often did a person correct the answer? Were the mistakes obvious? How long did verification take? Did experts spend less time on repetition or more time repairing subtle errors? Did customers need fewer follow-ups? What new privacy, security, vendor, or documentation work appeared?

Finally, define the release rule before seeing the results. Decide what performance is acceptable, which errors are intolerable, who can approve expansion, and what would pause the system. Without a pre-agreed rule, a charismatic demonstration or sunk cost can quietly lower the standard.

For our duplicate-charge ticket, a disciplined first pilot might classify messages into a small set of queues and show the predicted category beside the original text. Staff can correct the label. The organization learns which language confuses the system and whether routing time actually falls. Only after that evidence is stable would retrieval or drafting enter the test—and an action such as refunding would remain a separate decision with a higher bar.

The point is not permanent caution. It is cumulative trust. Each expansion earns its authority through evidence from the real workflow.

Governance is part of the design

Governance often sounds like a meeting held after the interesting work. Good governance is the design work that keeps the system useful when the demo is over.

The NIST AI Risk Management Framework organizes this work around governing, mapping, measuring, and managing risk. The verbs matter. A policy document alone does none of them. Someone must understand the use context, measure performance and harm, decide what to change, and retain the ability to stop or correct the system.

In practice, every consequential AI workflow needs clear answers to a small set of operational questions:

• Who owns the business outcome, not merely the vendor relationship?
• What data and tools may the system access, and what is forbidden?
• Which sources are authoritative, current, and visible to the reviewer?
• What can the system suggest, and what may it do?
• Which cases require abstention, escalation, or qualified review?
• How are decisions, versions, overrides, and incidents recorded?
• How can an affected person question or correct the result?
• What signal would trigger retraining, redesign, rollback, or retirement?

These controls can improve performance rather than obstruct it. Clear source ownership reduces contradictory answers. Permission limits prevent accidental overreach. Logs reveal recurring exceptions. A challenge route surfaces mistakes that internal tests missed. Monitoring catches drift when customers, products, policies, or model behaviour change.

Governance should also follow procurement. If a vendor changes a model, retention policy, training-data practice, price, integration, or service boundary, the risk profile can change even though your interface looks the same. Record the version and intended use; test material changes; keep an exit or fallback path proportionate to the dependency.

For further implementation resources, the site’s AI & Automation hub continues from this decision framework into adoption, workflows, and practical business use. The sequence matters: select and bound the use case first, then choose the tool and operating controls.

What the next wave changes—and what it does not

AI systems are becoming more capable across text, images, audio, video, software, tool use, retrieval, and multi-step tasks. Interfaces are also disappearing into ordinary products. A person may use several models without choosing an “AI tool” at all: a meeting becomes notes, a document becomes a queryable source, an application suggests the next step, or an agent moves information between systems.

This expands the opportunity. More work can be expressed as combinations of perception, retrieval, generation, prediction, and action. Smaller teams can explore alternatives faster. Specialists can build prototypes without waiting for every supporting skill. Organizations can make previously buried information easier to use.

It also makes boundaries harder to see. When generation, retrieval, and action are bundled, users may not know which source produced an answer or which permission enabled a change. An agent that completes several steps can multiply a small early mistake. Synthetic content makes provenance more important. Greater capability increases the need for good system design because more authority can be exercised before a person notices.

Several things do not change.

Goals still come from people and institutions. Data still reflects how it was collected and what it left out. Measures still reward some outcomes while ignoring others. Security still depends on access, incentives, and operational discipline. A customer still experiences the company’s decision, not the sophistication of its model. Responsibility still has to land somewhere a real person can answer, intervene, and repair harm.

The future of AI is therefore not one forecast about machines. It is the accumulation of ordinary design choices: which tasks we delegate, which evidence we demand, which people can object, and which consequences we refuse to automate merely because automation became possible.

Keep the judgment where the consequence lands

The duplicate-charge ticket began as a simple demonstration of efficiency. It now looks more interesting—and more manageable. AI can help recognize the issue, retrieve relevant information, and draft a response. Each contribution can save time. None removes the need to know whose account is involved, which policy applies, what promise is being made, or how the customer can correct a mistake.

That is the mature way to use artificial intelligence: neither as a mind to obey nor as a threat to reject, but as a set of capabilities placed deliberately inside human systems.

Before giving an AI system more authority, name five things in plain language: the task, the evidence, the consequence, the reviewer, and the correction path. If you cannot name them, the workflow is not ready—however impressive the output appears. If you can, you have moved beyond the spectacle of AI and into the work of making it useful.