AI in Finance: Where It Helps, Where It Breaks, and How to Govern It

A suspicious payment arrives at 02:13. It is not exceptionally large. It comes from a familiar account, uses a device seen before, and follows a route that looks ordinary when each transaction is viewed alone. The concern appears only when the payment is placed inside a wider network: three recently opened accounts, a rapid sequence of transfers, and a destination linked to a pattern that investigators have learned to distrust.

This is where artificial intelligence can earn its place in finance. It can examine more relationships, documents, and events than a person can hold in working memory. It can bring the unusual cases to human attention sooner. But the same system can also miss a new pattern, reproduce a biased decision, expose sensitive data, or create the illusion that a numerical score has settled a question it has merely reframed.

The useful question is therefore not whether AI is “transforming finance.” It plainly affects payments, lending, insurance, asset management, compliance, and customer operations. The useful question is narrower and harder: where does a model improve a financial decision, what new failure does it introduce, and who remains accountable when the model is wrong?

AI is a decision component, not a decision maker

“AI in finance” describes several different technical jobs. A model may classify a document, estimate the probability of default, detect an unusual payment, forecast liquidity, extract clauses from a contract, propose a customer-service reply, or generate a summary for an analyst. These systems do not share one capability or one risk profile. Treating every form of artificial intelligence as a single technology encourages vague promises and equally vague controls.

A better starting point is the decision chain. What signal enters the system? What output does the model produce? Who uses it? What action follows? What happens if the output is late, biased, fabricated, or unavailable?

Consider the suspicious payment. The model is not “finding crime.” It is ranking patterns for investigation. A useful output may reduce the haystack, but an investigator still needs evidence, context, and a defensible process. The distinction matters because the quality of the model cannot be measured only by how many suspicious cases it flags. A system that identifies everything as suspicious overwhelms investigators. One that learns only yesterday’s schemes may miss tomorrow’s. One that appears accurate overall may fail for a particular customer group or payment channel.

This framing fits the broader evidence. The Bank for International Settlements’ 2024 analysis describes opportunities across payments, lending, insurance, and asset management while also identifying explainability gaps, third-party dependence, cyber risk, privacy concerns, discrimination, herding, and model error. The benefit and the risk come from the same property: models can impose structure on large amounts of data, but the structure they find is shaped by the data, objective, and environment they were given.

That makes AI less like an autonomous employee and more like a powerful, fallible instrument inside a controlled process. A calculator gives the same result for the same arithmetic. A learned model gives an estimate whose reliability depends on conditions that can change. The institution using it must therefore know not only whether the model performed well during development, but whether the present case still resembles the conditions in which that performance was established.

The strongest uses begin with pattern-heavy work

The most credible financial uses share three qualities. They involve more observations than a person can examine efficiently; the output can be checked against an observable outcome; and the workflow can tolerate escalation to a human when uncertainty is high.

Fraud and anti-money-laundering analysis are examples. Rules remain useful for known patterns, but machine-learning systems can examine relationships across accounts and transactions. The BIS Innovation Hub’s Project Hertha tested network analytics on a large synthetic retail-payment dataset. The project reported improved identification of illicit accounts, especially for previously unseen behaviours, while stressing that the result was an experiment using simulated data and that practical, legal, and regulatory questions remained outside its scope. That qualification is not a footnote. It marks the distance between a promising analytical result and a deployable control.

Document-heavy operations are another plausible area. Financial institutions process contracts, customer communications, policy documents, filings, and compliance records. Classification and extraction can reduce repetitive reading, provided the workflow retains traceability to the source. A summary that cannot show the clause or document behind its conclusion creates review work rather than removing it.

Forecasting and risk monitoring can also benefit from additional signals. Cash-flow prediction, liquidity monitoring, insurance claims triage, and market surveillance all contain repeatable patterns. Yet a forecast is not valuable because it is produced by a complex model. It is valuable when it improves an actual decision relative to a simpler baseline, remains calibrated under changing conditions, and gives the operator enough warning to act.

The practical test is not “Can AI do this task?” It is “Does this output improve the next decision after accounting for false positives, missed cases, review time, data cost, control cost, and the consequences of error?” A model that is slightly more accurate but impossible to challenge may be worse in a regulated workflow than a simpler model whose limits are understood.

Credit decisions reveal the fairness problem

Lending shows why predictive power and responsible use cannot be separated. Additional data may help identify applicants whose conventional credit histories are thin. The BIS notes that transaction, rental, utility, or telecommunications information can add signal and may improve assessment for some underserved borrowers. But a variable can predict repayment while also acting as a proxy for disadvantage, location, disability, family circumstance, or another factor that the institution should not use in that way.

The model does not know the moral or legal meaning of a correlation. It optimizes the target it was given. If the historical target contains earlier exclusion, the model can learn that exclusion efficiently. If the outcome is measured only for people who previously received credit, the training data say less about those who were denied. If an institution changes its product, customer mix, or economic environment, the relationship between input and outcome may drift.

This creates at least four separate questions:

• Prediction: Does the model estimate the chosen outcome accurately for the intended population?
• Decision: Is that outcome the right basis for the action being taken?
• Distribution: How are errors and benefits distributed across relevant groups and circumstances?
• Recourse: Can a person understand the material reason for an adverse decision, correct wrong data, and obtain meaningful review?

An overall accuracy figure cannot answer all four. Nor can explainability be reduced to a colourful chart of feature importance. A useful explanation has to serve the person and purpose involved: a model developer debugging behaviour, a risk manager testing controls, a regulator reviewing compliance, or a customer challenging a decision.

The safest pattern is to separate prediction from policy. The model may estimate a probability, but accountable people define which evidence is acceptable, where thresholds sit, when review is mandatory, and how exceptions and appeals work. This keeps the institution’s values and obligations visible instead of hiding them inside a score.

Generative AI changes the operational risk

Predictive models usually return a score, class, or forecast. Generative systems produce language, code, images, or structured output that may look complete even when it is wrong. That fluent surface changes the risk. An employee can detect a blank field; a plausible invented clause, citation, customer fact, or market explanation is harder to notice.

Generative AI can still be useful in finance. It can help locate information across approved documents, propose a first draft, explain a process in simpler language, or prepare a structured handoff. The safe design makes retrieval and verification part of the task. The system should point to the source, distinguish quotation from synthesis, respect access controls, and make uncertainty visible. High-consequence outputs should require review before they reach a customer, trader, credit file, compliance report, or public statement.

Data boundaries deserve equal attention. A general-purpose assistant may receive confidential client information, material non-public information, credentials, or personal data if the interface makes pasting easy. The control cannot rely only on annual training. It should combine approved tools, technical restrictions, data classification, logging proportionate to risk, and clear escalation when the task does not fit the approved environment.

Automation also changes error scale. A person can send one mistaken explanation. A connected system can send ten thousand, update records, or trigger downstream actions before the problem is noticed. Every generative use case therefore needs an answer to a simple question: what can this output cause? The more consequential the next action, the stronger the identity, review, testing, and rollback requirements should be.

Financial stability turns local errors into shared risk

Some AI failures remain inside one workflow. Others can align behaviour across firms. If many institutions use similar models, data providers, cloud services, or foundation models, they may produce correlated decisions. A shared vendor outage can interrupt multiple firms. Similar trading signals can contribute to crowded positions. Synthetic content can make fraud cheaper. Automated agents can act faster than existing controls can observe.

The Financial Stability Board’s 2024 assessment highlights four channels in particular: third-party dependencies, market correlations, cyber risk, and model risk. It also notes a monitoring problem: adoption is moving quickly while reliable data about the use of AI in finance remain limited.

This does not mean every model threatens financial stability. It means an institution cannot assess risk only at the model level. A model that passes its local test may still depend on a concentrated provider, encourage the same behaviour as competitors, or fail during the market conditions when it is needed most.

Resilience questions therefore belong in the AI review:

• Can the workflow continue if the model or provider is unavailable?
• Can the institution reproduce the decision evidence later?
• Is there a simpler fallback for critical operations?
• Can operators recognize when conditions have moved outside the model’s tested range?
• Does an automated action have a rate limit, approval boundary, or kill switch?

These are ordinary operational-risk questions applied to a new component. That is an advantage. Finance already has disciplines for model risk, change control, segregation of duties, audit, resilience, and accountability. AI governance should extend those disciplines rather than create a theatre of special committees and slogans.

Third-party review has to follow the same decision chain. A vendor may provide a model, a hosted interface, training data, retrieval infrastructure, or all four. The institution needs to know which parts of the service can change without approval, where customer information is processed, how subcontractors participate, which logs are available, how incidents are reported, and whether outputs can be reproduced after a dispute. A contractual promise of “enterprise-grade AI” answers none of these questions.

Ask the vendor for evidence tied to the intended use: evaluation methods, relevant population and time period, known limitations, security responsibilities, update policy, and operational availability. Determine whether the institution can test the system independently and whether data can be exported if the relationship ends. For a model embedded in a critical process, understand the fallback before negotiating performance targets. A service-level credit is little comfort when payments or compliance operations cannot continue.

Concentration may be hidden. Different applications can rely on the same cloud platform or foundation model. A change or outage at that layer can affect customer support, document review, monitoring, and development at once. Build a dependency view across use cases, not one vendor questionnaire per purchase. Where a shared dependency is material, stagger changes, maintain tested alternatives for critical work, and prevent one provider identity from becoming privileged access across the institution.

Procurement should also resist an easy confusion: a vendor’s general benchmark is not evidence for the institution’s decision. The relevant test uses the institution’s data conditions, workflow, people, error costs, and control environment. Purchasing transfers a service; it does not transfer accountability.

Govern the workflow around the model

Model documentation is necessary, but it is not the operating system. Real governance connects ownership, evidence, controls, and response throughout the life of the workflow.

Begin with a named decision owner. This person is not merely the technical owner of the model. They are accountable for the business outcome, the acceptable error, and the response when evidence changes. Add a model owner for performance and change, a data owner for provenance and permitted use, and an operational owner for day-to-day monitoring. In a small organization, one person may hold several roles, but the responsibilities should still be explicit.

Then define the evidence that would justify deployment. Compare against the existing process and a simpler baseline. Test on data that reflect the intended population and current conditions. Inspect false positives and false negatives, not only averages. Run realistic failure cases. For high-consequence decisions, independent validation should examine the data, assumptions, and workflow—not merely rerun the developer’s metric.

Deployment is the beginning of evidence, not the end. Monitor input drift, outcome quality, override patterns, complaints, operational delays, and subgroup performance where lawful and relevant. Record model and prompt versions so a decision can be reconstructed. Review vendor changes and downstream dependencies. Set thresholds that trigger investigation, restricted use, or rollback.

Measurement deserves particular suspicion because the easiest metric is rarely the whole job. A fraud model may raise the proportion of reviewed cases that become investigations while shifting criminals toward channels the model observes poorly. A collections model may improve short-term recovery while increasing complaints or damaging long-term customer outcomes. A customer assistant may reduce average handling time because difficult cases are abandoned or transferred without resolution. The metric can improve while the service deteriorates.

Use a balanced evidence set. Pair model performance with workflow performance and human consequences. For detection, examine recall, false-positive burden, investigator delay, and the value of cases found. For credit, examine calibration, decision quality, distribution of errors, appeals, and corrected data. For an assistant, examine factual corrections, source traceability, escalation quality, customer resolution, and incidents prevented by review. Keep a baseline period so improvement has a real comparator.

Also inspect overrides instead of treating them as operator failure. Frequent overrides may reveal that the model lacks context, the threshold is wrong, or employees do not trust the explanation. Rare overrides may indicate excellent fit—or a culture in which challenging automation feels unsafe. Sample decisions continuously, including apparently successful ones, and ask operators what they see that the dashboard cannot.

Finally, design recourse. Operators need a way to disagree with the output without creating an invisible side channel. Customers affected by material decisions need an appropriate correction and review path. Investigators need a way to label outcomes so the system can improve without treating every human override as ground truth.

Good governance makes a model challengeable. It tells people what the system is for, what it is not for, what evidence supports it, how failure appears, and what happens next.

A bounded experiment is better than an AI programme

Many organizations begin with a platform search. That reverses the useful order. Start with one decision where delay, inconsistency, or information overload creates a measurable problem, then decide whether the workflow deserves a bounded machine-learning experiment.

Define the current baseline before introducing AI: time to complete, error type, missed cases, review load, customer impact, and cost. Choose a use case where the output can remain advisory during the experiment. Freeze the dataset and evaluation period. Decide in advance what would count as improvement and what safety result would stop the test.

A strong experiment might ask whether a model can prioritize suspicious transactions for investigators without increasing missed high-risk cases or overwhelming the team with false positives. It would not claim to “automate compliance.” It would compare review queues, investigator time, case yield, explainability, and failure patterns under controlled conditions.

Keep the first deployment narrow. Restrict data access. Log the model version and evidence source. Require human confirmation for consequential actions. Test fallback. Include the people who will operate, audit, and challenge the system. A technically impressive model that creates an unusable review queue has failed the workflow test.

Run the experiment long enough to encounter ordinary variation: busy and quiet periods, common and borderline cases, staff absences, upstream data delays, and at least one controlled failure of the model connection. An evaluation that sees only clean historical data measures analytical potential, not operational readiness. Document which cases were excluded and why, because the excluded edge may become tomorrow’s most important customer.

At the end, choose among four honest outcomes: deploy within the tested boundary, revise and retest, retain only a low-risk assistance function, or stop. Stopping is not an innovation failure. It is a valid result when the added signal does not justify the added risk and complexity.

The question that keeps responsibility visible

AI can make finance faster, more observant, and more adaptive. It can also make a weak assumption travel farther and look more authoritative. The dividing line is rarely the model alone. It is the design of the decision around it.

Return to the 02:13 payment. A responsible system does not announce that it has found a criminal. It shows why the pattern deserves attention, preserves the underlying evidence, routes the case to an authorized investigator, records the outcome, and remains open to correction. It helps a person see more without pretending to replace judgment.

Before approving an AI use in finance, ask one question that is difficult to evade: If this output is wrong, who will notice, who can stop the consequence, and what evidence will they have?

If the answer is specific, tested, and owned, the model may be ready to earn a bounded role. If the answer is “the AI team will monitor it,” the institution has not yet designed a financial control. It has only purchased a prediction.