A traveller logs on to an airline website late at night, searching for a last-minute fare. The company’s AI chatbot greets him politely and tells him, mid-conversation, that he qualifies for a 30% discount. The answer is confident. The tone sounds official. He books the ticket.
Days later, the airline says the discount does not exist. The chatbot, a spokesperson explains, had “misspoken”. The company does not honour the price. The traveller pays full fare for a promise no human is willing to stand behind.
It is the kind of dispute consumer-protection lawyers are beginning to see more often. But it also points to something larger than a customer-service failure. Artificial intelligence systems are increasingly speaking with authority, making consequential decisions and leaving people with little recourse when those decisions are wrong.
That problem does not always arrive as a false promise. Sometimes it arrives as silence.
A qualified young woman applies for a management role. Within seconds, an AI hiring system rejects her CV. She is given no reason. Buried inside the model’s logic is a brutal pattern: at that employer, managers had historically been men. The system learned the pattern and converted it into a verdict.
It did not hate women. It copied history.
These are the twin failures of modern AI: the confident falsehood and the inherited bias. One invents what was never true. The other repeats what should never have been. Both operate at the speed of a click. Both leave ordinary people to absorb the cost.
The first problem belongs largely to generative AI: the chatbots and writing assistants now embedded in airline websites, banking apps and government portals. These systems are built to produce fluent responses, not guaranteed truth. When they do not know an answer, they do not always say so. Instead, they predict the most plausible reply.
That distinction matters. Plausibility, delivered in a company’s brand voice, can easily be mistaken for policy. The airline chatbot did not lie in the human sense. It guessed, but it guessed confidently, grammatically and with the authority of a uniformed agent. The customer heard a promise. The system produced syntax.
Predictive AI systems pose a different risk. Rather than generating language, they study historical data and project it forward. When the past is fair, that may be useful. When the past is discriminatory, the system can reproduce discrimination at scale.
This is how bias becomes automated. A single prejudiced recruiter may affect dozens of applicants. A biased model can affect millions before anyone notices. The danger is not that the machine has prejudice in the human sense. It is that the machine can convert yesterday’s inequality into tomorrow’s procedure.
Regulators have begun to respond. The European Union’s AI Act, now being phased in, requires high-risk systems to undergo bias testing, maintain documentation and provide for human oversight. Courts and tribunals are also becoming less receptive to the idea that companies can avoid responsibility by blaming a chatbot. In one widely cited Canadian ruling, an airline was held responsible for information its chatbot gave to a customer.
Yet regulation is only beginning to catch up with a question companies should already be asking: who is accountable when the machine is wrong?
Trustworthy AI is not a model, a tool or a policy document. It is a practice and it begins before any code is written, with a clear definition of the decision the system will make and who could be harmed if it fails.
That practice starts with the data. Organisations need to know where their training data came from, who it represents and who it leaves out. They need to test outcomes across gender, age, ethnicity, disability, and geography, as well as the intersections between those categories. A system that appears fair in the aggregate may still fail badly for a smaller group hidden inside the average.
The same discipline is needed at deployment. New AI systems should not be launched all at once into live decision-making. They should be tested in stages, first running quietly in the background, then with a limited group of users, before being trusted at full scale. The point is not to slow innovation for its own sake. It is to ensure that failure is detected before it becomes widespread harm.
Explanation is just as important as testing. A rejected job applicant should not be handed a confidence score or a technical abstraction. They deserve a clear reason in plain language. A customer who is misled by a chatbot should not be trapped in an automated loop. They should have a visible route to human review, with a named process, a clear timeline and a person empowered to act.
There is also a final test that every organisation should apply before deploying AI: can the system be shut down safely? If the answer is uncertain, the system is not ready. An off-switch is not a sign of weak technology. It is evidence of responsible governance.
A trustworthy AI system is not one that never fails. It is one that fails visibly, fails accountably and is corrected before it fails again.
That standard will not be met by slogans about innovation or vague commitments to ethics. It will be met by careful design, honest testing, human accountability and the willingness to stop a system when it causes harm.
Until that becomes normal practice, chatbots will keep speaking with confidence. Hiring models will keep deciding in silence. And the cost — measured in airfares paid, jobs lost and trust eroded — will keep falling on the people least able to challenge the machine.
The author is an IT consultant and an Associate Director at Deloitte.
