Deploying Business-Critical AI: How to Build the Confidence and Accountability to Go Live

Executive Summary

• Most organisations now have AI in production. Almost none have it where it matters; in client-facing experiences, business-critical decisions, or processes running with minimal human oversight. That gap is where competitive advantage is being won and lost.
• The barrier is typically one of confidence, not capability. In high-stakes AI deployment this is the product of four specific criteria being met.
• Those criteria span how you test AI behaviour, how you build and govern solutions at enterprise scale, and whether your leadership has genuinely bought into why this matters.
• Satisfying these confidence criteria builds muscle memory and accountability and gives you less reason to shy away from deploying AI to turn the dial in meaningful ways. Without it, AI becomes stuck saving £s in peripheral uses without a material impact on how the organisation operates.

Companies Are Moving On From POCs…

This time last year, decades in AI-years, most companies were in “AI POC purgatory”: a lot of interesting POCs that showcase what AI could do, but struggling to productionise any of them. One year on, many organisations now have at least some AI solutions in production, which is a great step forward.

However, often this is AI applied in peripheral processes: it’s easier to do, an Innovation team could develop it with a keen business user, and if it goes wrong ‘it is not the end of the world’. The process does not really matter, there is a manual back up process, or there are multiple ‘humans in the loop’ to limit any negative consequences.

The result of that understandable but conservative approach is that the impact of those solutions is also conservative: it is unlikely to have made any notable dent in growth opportunity, client experience or cost profile.

…But Often Not Applying AI Where It Really Matters, Unlike The Winners.

That is what separates the future winners and the rest: the progressive companies are investing in leveraging AI where it matters to the performance of the business.

The laggards are also “doing AI”, just not where it will make a meaningful impact on the business.

And yes, putting AI in investment decisions, client experience, product development, research & design, ‘no-human-in-the-loop-automations’ will all feel more risky but that is where the real rewards are.

The companies getting ahead in this space may not be visible yet to the market, but by the time this happens, not only is the gap substantial, the difference in velocity in deploying AI means the gap will simply widen further still while laggards realise what they had wrong this whole time.

So the question is: what does an organisation need, to be confident in applying AI in the processes that really matter?

First, let’s assume you are already progressed from ‘We have an AI tool like ChatGPT’ to ‘We have POCs’ to ‘We have 10 simple AI-solutions live’. If you haven’t, you need to sort that out as a priority. Those initial steps may be trivial but are essential to start building enterprise AI literacy, better use case ideas and flushes out the steps internally to get any AI released into production.

At this point, the organisation should be getting more ambitious and progressing to deliver “meaningful applied AI solutions with human oversight” and then the differentiating “we apply AI in our core processes and client experiences” .

Those last steps of progress are rarely restricted by technology, but by the organisational confidence. Because there is always a reason to delay or shy away from the hard stuff:

‘What if it gives the wrong insights?’, ‘What if it behaves incorrectly and we end up in the news?’, ‘What if it gets the calculation wrong and we make the wrong decision?’, ‘What if it abuses the clients?’, ‘What if the model becomes unavailable?’, ‘How do we support the updates on this?’, ‘What if regulation changes?’, ‘Should we wait on a the new model’?, etc.

Four Key Confidence Criteria

In our view, there are typically four conditions that provide the confidence to press that ‘Go Live’ button on a high impact AI-enabled change.

• [1] The Why: Is the organisation, the c-suite specifically, genuinely bought into the “why AI matters to our business success” narrative? Because if they’re not, the organisation is more likely to find reasons to step away from the difficult decisions.
• [2] The Enablement: Is there an AI Change/Transformation/Enablement team in place that supports the organisation as it evolves with AI? That handles adoption, education, regulations, new ways of working, challenging assumptions, provide required funding, etc. AI is not a ‘tech to sort it out’ change, like Cloud or Digital initiatives often were. It has to be business led and well supported.
• [3] The How: Is there an established enterprise and solution architecture for applied AI? While this is a continuously evolving domain, having a view forces the thinking and growing knowledge on the right guardrails, cost, observability, controls, model access, etc. That starts building credibility internally that the organisation ‘knows what it is doing’ when it comes to AI and prevents multiple, varying ways of building.
• [4] The Go-Live Confidence Test: Solution owners can endlessly procrastinate on “is it good enough to go live?”. Traditional testing-by-the-QA-teams does not provide that confidence and sense of accountability for non-deterministic AI system. Evaluations do.

Lets go through these, in reverse order because that is the order closest to decision making around go live confidence.

[4] Evaluations: Test Properly to Confidently Press “Go”.

For decades, organisations have relied on the QA team to decide if a release is good enough. Testing on paper was done by the business stakeholders, in reality it was done by the QA team. It was either a pass or fail, around countless if this then that rules. A lot of work, but straightforward.

That does not work for AI, where we have to evaluate behaviour.

Evaluations, or evals, are how you make qualitative AI behaviour measurable. Instead of pass/fail, you define criteria (such as correctness, tone, factual grounding, refusal behaviour) and assign scores. Track those scores across the project duration as you improve the prompts, the models, the guardrails etc and you get something the project team, change committees and business leadership can confidently act on: a correctness score that moved from x to y over the course of the project.

That number does two things. First, it builds confidence by demonstrating improvement over time and giving stakeholders tangible evidence that the system is behaving reliably consistently within the set expectations.

Second, it creates accountability. It’s the numbers they can point to and the lineage of those evals. A documented evaluation set, tested across a well defined range of scenarios, is the AI equivalent of the test report that gives the sign-off committee something that shows due diligence: does it do (only) what it should be doing, and does it (only) do it in the way it should be doing that.

Of course, those numbers are only meaningful, if the evaluation criteria are rich enough and that requires an operating model shift that most organisations get wrong initially: the ‘business’ must own the evaluation criteria, not the tech team, qa team or delegated to a single junior user. The product owners and specialists understand the domain, the edge cases, the desired tone of voice, the types of interactions users may throw at it, the reasoning steps that make sense, the failure modes, the correct Q&A pairs, etc. Those using and deploying the solution should be the ones to decide what good looks like.

That means that if you need to sign off an solution that applies AI right at the heart of the customer experience, you just want to ask two questions:

(1) who designed the evaluation criteria, and

(2) had the eval performance improved against those criteria over time and now passed the minimum threshold?

If the answer is "the tech team built them but we are happy with the results", the application is not ready. If the answer is “yes, we tested last week and it passed”, it is probably is not the level of Eval maturity that instills the confidence needed, as it is unlikely the evals were the starting point of the build.

But if the business product owner has been defining, reviewing, and iterating on evals throughout the build, and can show you the performance trajectory heading in the right direction, you have something worth signing off.

[3] Enterprise AI Architecture: Have a View, Even if it Changes Monthly

Another layer of confidence comes from alignment to AI architecture principles and standards. While it suited your first five AI applications to be built in whichever way was suitable just to get going, a more structured approach with a reference application architectures and an enterprise AI platform starts to become necessary to scale and achieve high velocity of safe, well-governed AI deployments across the organisation.

All engineering teams need to be able to rapidly ‘inherit’ model access, guardrails, observerability, scaling, cost management etc so they can focus on engineering the solution.

Having that view on the enterprise AI platform and architecture approaches, even if it has a short shelf life given the rapid pace of change, further builds the confidence within the organisation that it can safely deploy AI in critical systems. Because it means someone will have thought about all of the key factors and decided on an optimum route to deployment and that too adds confidence that the solution will support those important processes.

[2] AI Transformation: Not Just ‘a Tech Thing’

This is not ‘doing the same thing, but with access to an AI tool’. It may be cliché, but transformation means re-designing processes, re-thinking product offerings, evolving the client experience, and changing the ways of working and most likely team structures.

That does not happen as a side-of-desk activity and cannot be driven by Technology (and no, not your 3 FTE “Innovation” team either). And if you take the ‘it will work itself out’ approach you are not going to achieve the pace of change that the winning competitors may achieve.

Additionally, to surface the real ‘game-changing’ use cases, and have the support for those, requires an enterprise-wide, thorough knowledge of AI as well as well as a positive mindset towards it.

So your AI-transformation needs those who know what the organisation needs, how to enable change and understand AI, to have a remit from the board to make it happen and provide the coordination, energy, funding, influencing, governance, prioritisation and decision making.

Especially in the first phases, a central AI Hub, AI Transformation, AI Office type function works well. This central team builds the confidence that the organisation has a strategy to execute, the big decisions can be made and federated quickly, and set the quality gateways for AI transformation. Without, the approach risks being too fragmented resulting in inconsistencies in decisions, product/solution choices and quality, ultimately undermining the confidence to implement AI in high stakes areas.

[1] Can Each Employee (and the CEO) Articulate the “Why”?

As called out earlier, the companies that gain real competitive advantage from AI will be the ones who built the enterprise experience, expertise and confidence to deploy AI frequently and rapidly at the heart of the important processes and experiences that improve client experiences, product performance, pace of production, etc.

Good evaluation practises, an architecture that adapts rapidly and scales, a “can do” transformation management attitude helping the organisation to learn, adapt and establish new ways of working are all required.

However, none of these will happen successfully ‘bottom up’. The changes and implications are simply too rapid and too impactful for that. ‘Board approval’ or ‘management support’ is not sufficient either.

If your employees can't see clearly why changes are necessary, what it means for them personally (good or bad), and what they gain, you will get transformation through the back door with plenty of collateral damage.

So the fourth element to gaining true advantage from adopting AI, it that the company leadership genuinely understands and believes investing in AI is essential for the ongoing success of the company. It is their AI strategy that is incorporated to the mission statement and goals, one they repeatedly evangelise and are seen to make decisions backed up with action.

That provides the incentives, support and sense of urgency for teams throughout the organisation to upskill and make those slightly uncomfortable but important decisions to choose the more difficult, more rewarding path. It gives the confidence to press “Go-Live” on that AI-powered client experience or investment decision process.

Confidence to Go-live on High-Risk, High-Impact AI

The companies that win from AI in the next two years won't be the ones with the most deployments. They'll be the ones who made AI work for the important, client-facing, decision-influencing, business-critical systems and processes and built the enterprise muscle to do it consistently and safely.

Your call to action? If your organisation is “doing AI” but it’s having no impact on the success of the company, try to answer these questions:

1. Is the role AI plays in achieving your company’s mission clear? Do you feel leadership actually believe AI adoption is essential to the future success of the company?
2. Is there a central function that helps push the AI agenda, educate the enterprise and coordinate activities?
3. Is the technology team building up expertise, capacity and patterns that facilitate more rapid and safe AI engineering?
4. Are releases making it to production free from blockers caused by unanswered questions on AI behaviour?

If the answer to any of these is no, then you know where to start to grow organisational confidence that AI can be deployed, with velocity and in a consistent manner, to gain you competitive advantage in the areas where it really matters.