AI’s Biggest Bottleneck Isn’t the Model: Somnath Banerjee on Trustworthy AI

AI’s Biggest Bottleneck Isn’t the Model. It’s Trust

Somnath Banerjee on autonomous agents, deepfakes, cybersecurity, data quality, sustainable AI and why recoverability may matter more than raw model capability.

INTERVIEW FEATURE

Artificial intelligence has spent the past few years in a race for capability.

Bigger models. More capable agents. Faster deployment. More ambitious claims about what AI will automate next.

Somnath Banerjee believes much of that conversation is focused on the wrong constraint.

“The binding constraint upon the value of artificial intelligence has never been model capability,” he says.

For Banerjee, whose work spans AI, data engineering, cloud technologies, cybersecurity and sustainability, the harder problem is what happens around the model: whether the underlying data can be trusted, whether an automated decision can be reconstructed, whether an action can be reversed, and whether the people using a system understand when to challenge it.

It is a decidedly less glamorous view of the AI revolution. It may also be one of the more consequential ones.

In this wide-ranging conversation, Banerjee discusses when enterprises should trust autonomous AI agents, why deepfake detection alone cannot solve AI-generated deception, the data mistakes undermining Generative AI projects, how AI is changing the economics of cybercrime, and why provenance, evaluation and small edge models may matter far more over the next five years than their current share of attention suggests.

His argument throughout is consistent: build AI not around the assumption that it will never fail, but around the ability to understand, contain and recover when it does.

Autonomous AI Agents Are Ready. Most Enterprises Aren’t.

AI agents are one of the biggest technology trends today. Are autonomous AI agents ready for enterprise adoption, or are organisations moving faster than the technology?

“Autonomy converts an ordinary error rate into an unbounded liability.”

Banerjee challenges the premise of the question.

In his view, model capability is no longer the primary barrier to deploying agents across a meaningful range of enterprise workflows. The real question is whether the organisation has built the infrastructure required to make autonomous action safe.

“Current models are already sufficient for a considerable range of enterprise workflows,” he says. “What most organisations lack is the substrate that makes autonomous action safe: the ability to reconstruct, after the fact, what was done and on what basis.”

His analogy is instant retail payments. The success of modern payment infrastructure is easy to attribute to a seamless interface: tap, transfer, done. But trust in those systems does not come from the interface alone. Underneath sit settlement mechanisms, dispute resolution, reversals and accountable escalation paths. AI agents, Banerjee argues, need their equivalent.

Before allowing an agent to operate autonomously, he asks two questions:

Is every action the agent can take logged in enough detail to reconstruct what happened later?

Is every action reversible, or at least bounded by something that is?

If the answer to both is yes, he sees little reason to delay deployment. If either answer is no, improving the model will not solve the underlying problem.

There is another trap: confusing a successful AI pilot with a functioning system. Banerjee has seen pilots work because a highly capable employee was quietly examining outputs behind the scenes. The organisation then scales the technology while assuming that person was incidental to the process.

They were not. “That person was, in fact, the control.”

It is a useful warning for companies racing from AI proof-of-concept to production: sometimes the most important component of an apparently automated system is the human nobody included in the architecture diagram.

Deepfake Detection Is an Arms Race Defenders Are Unlikely to Win

Deepfakes are becoming increasingly sophisticated. What can businesses, governments and individuals actually do to protect themselves?

“The detector must anticipate all of them in advance.”

Banerjee comes to the problem with experience on the detection side.

His research combined YOLO-based facial segmentation with Neural Architecture Search and achieved 99.04% accuracy on the Celeb-DF v2 benchmark.

It is a strong result. It also helped convince him that detection cannot be the durable solution.

Fixed deepfake detectors face a structural disadvantage: they perform well against manipulation techniques they understand, but can deteriorate when confronted with techniques absent from their training data.

“The party generating the forgery needs to identify a single method the detector has not anticipated,” Banerjee says.

Instead of asking people to become increasingly sophisticated judges of whether a voice, photograph or video “looks real,” he argues that security systems should be designed so authenticity does not depend on appearance in the first place.

For families: create a verification mechanism before you need one

Voice-cloning scams make the traditional signals of fraud increasingly unreliable. A frightened relative calling with an emergency may sound exactly like the real person.

Banerjee recommends that families agree in advance on a private verification question or piece of information that cannot be recovered from public profiles. Dates of birth, pet names and other easily discoverable facts are poor choices.

Just as important is recognising urgency itself as a security signal. A fraudster wants the victim to act before thinking. The appropriate response is therefore not necessarily to detect the artificial voice. It is to recognise the pressure and deliberately slow the interaction down.

For businesses: put the control where the money moves

Any request to change bank details, credentials or payment instructions should be verified using a second channel that the requester did not nominate. And, Banerjee stresses, seniority should not create an exemption.

This is partly a cultural problem. In strongly hierarchical organisations, asking an employee to challenge an apparently senior executive can impose a social cost. A better system removes the need for personal courage. The rule applies to everybody.

The crucial advantage is that the control works whether the call, message or video was fake or genuine.

For governments: invest in provenance

Banerjee believes governments should increasingly shift attention from detecting fabricated media toward establishing the origin of authentic media.

That means technologies such as cryptographic signing at capture and widely adopted content-authenticity standards.

The question changes from “Can we prove this was fabricated?” to “Can this content demonstrate where it came from?” The second, he argues, has a much better chance of producing a durable answer.

Sustainable AI Has to Change a Decision — Not Just Produce a Dashboard

Sustainability is becoming a strategic priority. How can AI move beyond reporting and dashboards to create measurable environmental and commercial impact?

“A dashboard upon which no decision depends carries a carbon cost and delivers no benefit.”

Banerjee is wary of the phrase “AI for sustainability.” Not because AI has no role to play, but because the label is frequently applied without specifying the measurement, dataset or decision that the technology is expected to improve.

A claim that cannot be tested cannot be evaluated. At the same time, the energy consumption of large AI models deserves to be counted as part of the equation rather than treated as an externality.

The most defensible applications, he argues, tend to be narrower: predictive resource allocation that demonstrably reduces energy use in a cloud environment, monitoring that identifies a specific contaminant before it develops into a crisis, and systems tied to measurable operational outcomes.

Consider agriculture. A cultivator deciding when to sow does not necessarily need a sophisticated visualisation of every available soil parameter. The decision may ultimately be much simpler: sow this week, or wait ten days?

If the technology does not improve that decision, the elegance of the dashboard is irrelevant.

That is the dividing line Banerjee draws between reporting and impact. Start with a decision currently being made badly, inconsistently or too late. Then determine what measurement would improve it.

He applies the same principle to sustainable data engineering. Efficiency, he argues, should be reviewed alongside latency and accuracy from the beginning of system design. That does not mean minimising energy consumption at all costs. It means refusing to treat energy as somebody else’s problem to optimise later.

The Four Data Problems Undermining Generative AI

Companies are investing heavily in Generative AI while continuing to struggle with data quality. What mistakes do they make before deployment?

Banerjee’s answer begins upstream.

“The principal error,” he says, “is treating artificial intelligence as a technology procurement rather than as a data and trust problem.”

1. Nobody agrees what the entity actually is

Ask five departments to define a “customer” and you may receive five different answers. Each definition can be perfectly rational in its own context. The problem begins when an AI system is built on top of that disagreement and its outputs circulate among teams that believe they are discussing the same thing.

The model has not necessarily failed. The organisation failed to establish shared meaning.

Banerjee contrasts this with the ledger of a small neighbourhood shopkeeper. There may be no formal schema, governance council or sophisticated data platform, yet the proprietor knows exactly who owes what and since when because the information was recorded for a known purpose at the moment the transaction occurred.

“He possesses lineage,” Banerjee says. “We possess scale.”

2. Data is reused for a purpose it was never designed to serve

A dataset collected for regulatory reporting may be perfectly adequate for regulatory reporting. That does not automatically make it appropriate for statistical inference or an AI model.

Missing information, collection methods and historical biases that were tolerable under the original use case can become serious errors when the same data is repurposed. The result can be a technically well-trained model that is quietly wrong.

3. Organisations retain data indefinitely

Cheap storage created an understandable instinct: keep everything. But the value equation changes over time. Old data may become progressively less useful to the organisation while remaining extremely useful to an attacker.

Banerjee therefore treats retention limits not merely as a compliance exercise but as a security control.

4. There is no lineage

When an AI system produces an unsound result, the essential question is how it arrived there. Which data contributed? Where did that data originate? What happened to it along the way?

Without lineage, the organisation can neither properly diagnose a failure nor establish confidence that the same failure will not recur.

The common thread across all four problems is organisational tolerance for ambiguity. Teams accept approximate definitions with the assumption that inconsistencies can be reconciled later.

AI changes the consequences of that habit. “Artificial intelligence does not reconcile them later,” Banerjee says. “It amplifies them.”

Where Should a CEO Spend the First AI Budget?

If a CEO handed you the company’s first serious AI budget today, where would you invest it?

“The greater part of a first AI budget should not be spent on artificial intelligence.”

Banerjee’s answer is unlikely to delight an AI vendor.

His reference point comes from outside technology: the surgical safety checklist. A small series of mandatory confirmations before surgery produced meaningful improvements without introducing sophisticated new technology.

For Banerjee, it illustrates a broader lesson: good process design can outperform technology spending, especially when the process removes the need for an individual to decide whether speaking up is worth the social risk.

First: establish identity and lineage in one high-value data domain

Not every database in the company. Pick one domain tied to a consequential business decision and reach the point where the organisation can say what each entity means, where important fields came from and who owns them.

Second: choose one narrowly scoped problem and measure the baseline

If today’s performance cannot be stated, tomorrow’s improvement cannot be demonstrated.

Third: build evaluation and monitoring before expanding model development

Companies frequently become good at deployment before becoming good at determining whether deployed AI still works. But drift is normal.

A model without adequate monitoring will eventually deteriorate, and the organisation may struggle even to explain what happened.

Fourth: invest in the domain experts receiving AI outputs

Much AI training is focused on engineers. Banerjee sees considerable value in teaching the people who consume AI-generated recommendations how to interrogate them, challenge them and identify when something does not make sense.

The logic can be reduced to one line: AI is a multiplier, and a multiplier is indifferent to the sign of what it multiplies. Before scaling AI, make sure the underlying organisation deserves amplification.

AI Has Changed the Economics of Cybercrime

Are organisations ready for AI-powered cyber threats?

“The organisation purchases silence at precisely the moment it requires speed.”

Mostly not, according to Banerjee. But he thinks much of the discussion focuses on the wrong change.

The most important shift may not be a new category of cyberattack. It is the collapsing cost of personalisation.

A convincing attack tailored to one executive once required meaningful research time. That economic cost restricted highly personalised attacks to high-value targets. Generative AI dramatically lowers that cost.

The attack that was once economical against the chief executive can increasingly be directed at everyone.

There is also a less-discussed global implication. Speakers of languages with smaller populations previously benefited from a kind of accidental security barrier: creating convincing fraudulent messages in those languages required expertise that attackers often did not possess.

Generative AI erodes that protection. Fluent and idiomatic scam messages can increasingly be produced across languages at comparable cost.

The old advice to spot phishing through spelling mistakes and awkward grammar therefore becomes much less useful. Poor grammar was never the attack. It was merely an accidental by-product of how the attack was produced.

Banerjee believes the controls that survive this change are the ones that never depended on appearance: navigate directly to important systems rather than following unsolicited links, verify consequential requests using an independently chosen channel, and treat urgency as a warning signal.

He also urges companies to rethink how they measure security awareness. Instead of obsessing over phishing click rates, pay far more attention to reporting rates.

An employee who clicks a malicious link and reports it four minutes later creates a very different level of exposure from an employee who clicks and remains silent.

Punishing employees for simulated phishing failures can therefore produce the opposite behaviour from what the organisation needs during a real incident.

He also urges companies to recognise their own AI deployments as part of the attack surface. Prompt injection, AI agents with access credentials and systems automatically ingesting external content all create new exposures.

Three Underestimated AI Trends to Watch Over the Next Five Years

Which technologies or trends are currently underestimated but could reshape industries?

AI evaluation as an engineering discipline

Benchmarks tell developers something. They do not answer the harder operational question: is this deployed system still performing the function for which we built it?

Domain-specific evaluation, drift detection and mechanisms capable of identifying degradation before it causes serious harm will, Banerjee predicts, become a discipline in their own right.

A significant proportion of AI’s practical value may ultimately reside not in creating models but in reliably determining whether those models continue to work.

Provenance and content-authenticity infrastructure

If appearance can no longer prove authenticity, infrastructure capable of establishing origin becomes increasingly foundational.

Banerjee compares the opportunity to major digital identity and payment infrastructure programmes: unglamorous while being built, transformative once everything begins depending on them.

Provenance may follow the same pattern.

Small specialised models at the edge

The AI industry’s attention naturally gravitates toward scale. Banerjee sees another important frontier in the opposite direction: models compact enough to run where data originates — on a sensor, a handset, agricultural equipment or an industrial device.

These systems can operate with lower latency, preserve privacy by keeping data local and reduce dependence on connectivity.

He compares the engineering mindset to spacecraft design, where severe limitations on mass, energy and heat are not regarded merely as disadvantages. Constraints themselves create an engineering discipline.

Small-model development, he believes, deserves to be treated as a strategic capability rather than an inferior version of large-model AI.

The Most Important Lesson: A System Is Valuable Only When People Will Act on It

Across AI, data engineering, cloud and sustainability, what lesson matters most?

“Trustworthiness is a property of how a system was produced, not an attribute that can be added subsequently.”

Technical correctness is not enough. A system creates value only when people trust it sufficiently to act on what it tells them.

Trustworthiness therefore cannot simply be attached at the end through a policy document, interface or governance label. It emerges from how the system was produced: the pipeline, the data, the accountability structure, the engineering decisions and the ability to inspect what happened.

The principle also explains why he believes expertise can travel between apparently unrelated domains. Rendering a large, disordered stream of data trustworthy enough to support action requires many of the same disciplines whether that data represents network traffic, financial transactions or soil measurements.

But technical methods travel more easily than domain authority. And that distinction matters.

Advice for the Next Generation of AI Researchers and Technology Leaders

What advice would you give researchers, engineers and entrepreneurs trying to build trustworthy AI with real-world impact?

Develop real depth in one discipline

Breadth has value, but depth teaches rigour. Attempting to accumulate expertise across multiple fields too early risks producing what Banerjee calls “confident amateurism.”

Look for problems where your method is valuable even when the domain is unfamiliar

Some of the most interesting opportunities occur when a rigorous approach developed in one field is applied to another. But enter as the person contributing a specific technical capability, not as someone claiming immediate authority over the entire problem.

Be precise about where your expertise ends

A data engineer may identify patterns in environmental data. That does not make the engineer qualified to determine what level of a contaminant is biologically or ecologically meaningful.

Pattern recognition can travel across disciplines. Interpretation often cannot. That is why serious interdisciplinary work requires genuine domain experts.

Submit your work to people who have no incentive for you to be right

Peer review, publication and conference scrutiny matter because internal validation alone is comfortable. And comfort is not the same thing as examination.

When entering an unfamiliar field, Banerjee recommends not beginning with “How would I solve this?” but with a better question: “What makes this difficult for the people already working on it?”

“Regulate for Recoverability, Not Capability”

If there is one message business leaders and policymakers should remember about the future of AI, what is it?

“Build for the failure from which you can recover.”

Do not assume better AI models will remove the organisational problems surrounding them.

Give a company unlimited access to advanced AI tomorrow and the underlying bottlenecks remain: trust, governance, data quality, definitions, accountability and a clear understanding of the problem being solved.

For business leaders, that means the investments most likely to determine the success of AI may not look much like AI expenditure at all. They are investments in making data, processes and accountability good enough to withstand amplification.

For policymakers, Banerjee advocates a different regulatory principle: regulate for recoverability rather than capability.

Model capability evolves rapidly. Rules written around the technical characteristics of today’s systems risk ageing quickly.

A more durable set of questions is possible: Can a consequential decision be explained? Can it be challenged? Can an error be traced? Can the damage be contained? Can the decision be reversed?

Those questions remain relevant even as the underlying technology changes. And they become increasingly important as AI moves deeper into decisions concerning credit, healthcare, insurance claims and other areas capable of materially affecting people’s lives.

Public trust will not be secured through announcements that a system is trustworthy. It will be earned through sound data, transparent processes, credible oversight and repeated evidence that failures can be addressed.

About Somnath Banerjee

Somnath Banerjee works across artificial intelligence, data engineering, cloud technologies, cybersecurity and sustainability. His work and research have included deepfake detection, sustainable data engineering, high-volume data systems and interdisciplinary applications of data and AI. His perspective centres on trustworthy systems, data lineage, evaluation, recoverability and the practical conditions required for artificial intelligence to produce reliable real-world outcomes.

Disclaimer: Somnath Banerjee is speaking in his personal capacity. The views expressed in this interview are his own and do not necessarily represent the views of his employing organisation. No confidential or proprietary information is discussed.