Target AI at the Reviews, Not the Work
Every AI deployment produces some amount of wrong output, and something has to catch it. The deployments that stick are the ones where that something was already there.
Attention Is Not a Control
Two kinds of rules governed what the machine was allowed to do. Both were in force the whole time. One was prose. The other was a set of AI guardrails. One was ignored completely, the other obeyed without exception.
Your AI Business Case’s Blind Spot
I have worked on hundreds of business cases. Every one was written inside a department or a division, and written for the benefit of that department or division. That is not a criticism. It is what a business case is for. The only one I ever saw that mentioned disaster recovery was a business case about disaster recovery. That was fine for thirty years. It is not fine now, and the reason has nothing to do with the quality of the cases or the people writing them. The AI business case and the continuity plan never meet Two owners, one assumption, no shared document An AI business case is built by a business unit that wants to be better at the thing it does. Faster lease abstraction, cleaner forecasts, fewer touches per invoice. The case compares the world with the project against the world without it, over three or five years, and both of those worlds are assumed to keep operating. Continuity belongs to somebody else. IT owns it, writes the plan, tests it on its own cycle, and reports on it to a different committee. The continuity plans I have read all have a section saying the work will be done manually and reconciled when service returns. Underneath that section sits an assumption: somewhere in the business are people who know how to do the work without the system. Not named. Not budgeted. Not tested. Present. That assumption survived mainframes, client-server, ERP, the web, mobile and the cloud. Every one of those waves changed how the work was done and left behind the people who knew how to do it. The process moved. The knowledge stayed in the building. The gap is not a missing number. It is a seam between two owners. And a second seam opens after the deployment goes live. The business has reduced its human fallback and increased its dependence on a provider it does not control. Two dependencies moving in opposite directions, neither of them recorded in the document that authorized the move. Nobody skipped a step. No step exists. Stated as a CFO would state it: the case captures the labor avoided. It does not capture the replacement cost of the capability being consumed, because that capability has never appeared on any ledger. The first wave that can consume the fallback Earlier automation moved the process and left the practitioners Earlier waves removed people too. Mainframes removed clerks, ERP removed accounting staff, self-service removed transaction processors. The distinction is not headcount. What those waves left behind was enough capability to reconstruct the process. The people who understood the work were still in the building, and parts of the job were still done by hand, so the understanding stayed current without anybody maintaining it. An AI deployment is funded on the basis that it absorbs volume. That is the case, it is usually a good case, and the savings land. What lands with them is the headcount change that made the numbers work. Some of those people leave. Others are redeployed, which sounds gentler and often is, though it depends on how completely and for how long they leave the work. The relevant question is not whether they are still on the payroll. It is whether anyone is still doing the thing by hand, and how recently. Keeping a human in the loop does not solve it What thirty years of research says about reliable systems The obvious answer is that the people are still there, supervising rather than doing. A human stays in the loop, the process gets reviewed, the capability is preserved. Two things are wrong with that, and both are established rather than argued. Reliability degrades the reviewer. Parasuraman, Molloy and Singh ran a multitask flight simulation in 1993. Participants monitored an automated system while handling other jobs. For one group the automation was consistently reliable. For the other it varied. After about twenty minutes, the group watching the reliable automation was substantially worse at catching its failures. The reliability itself degraded the monitor, and the effect disappears when monitoring is the only job, which matters because enterprise reviewers rarely have one job. Rarity degrades detection. Wolfe, Horowitz and Kenner published in Nature in 2005. Observers searched simulated baggage x-rays, and the only thing that changed between conditions was how often a target was actually there. At fifty percent prevalence they missed seven percent. At one percent they missed thirty. Think of it as Where’s Waldo, except Waldo is only on one page in a hundred. You would find him on page one. By page forty you would be flipping. By page ninety-nine you have stopped looking in any way that counts, and when he finally shows up you turn straight past him. Nobody decides to stop looking. The absence of a decision is the whole problem. The two research traditions rarely cite each other, and they describe the same condition from opposite ends. Ninety-nine percent accuracy, from the reviewer’s side of the desk, is an environment where an error shows up once in a hundred. So two things govern whether a reviewer is a control or a formality: how often errors appear, and whether the reviewer is doing anything else. Errors frequent Errors rare Reviewer does nothing else Works. Frequency keeps them calibrated, attention is undivided Prevalence pulls detection down, though single-task monitors held up regardless of reliability Reviewer has a queue and a day job Degraded and still functioning. Finding things regularly keeps them in the task Both effects compound. The quitting threshold falls and attention goes to the work that visibly needs it This is a way of locating the problem rather than a model of it. Bottom right is where many enterprise review roles end up, and it is where a successful deployment can put you. The deployment moves you right, because the errors the reviewer sees become rarer as the model improves. It moves you down, because the headcount that justified the project is
Lumerai Technology Value Realization Series | Part 3
Why Better Discovery Doesn’t Always Lead to Better Decisions The Confidence Gap in Technology Investments Technology investment decisions rarely fail because organizations lack information. They fail because organizations often mistake more information for greater decision confidence. By the time a technology initiative reaches executive leadership, months of work have often been completed. Business cases have been drafted. Requirements have been documented. Vendors have been shortlisted. Demonstrations have been conducted. Scorecards have been completed. Research from organizations such as Gartner and McKinsey has consistently highlighted the difficulty organizations face in translating major technology and transformation investments into expected business value. Despite months of rigorous preparation, executive leadership is frequently left asking the exact same fundamental question: “How do we know this is the right investment?” That is the Confidence Gap: the distance between having enough information to make a technology decision and having enough evidence to make a confident decision about business outcomes. In the first article of this series, I argued that organizations often solve the wrong problem because they skip structured discovery. In the second article, I introduced the Lumerai Executive Discovery Framework, a disciplined approach to validating business opportunities before evaluating technology solutions. Discovery improves the quality of the problem definition. It does not, by itself, guarantee the quality of the investment decision. Transforming discovery into realized value requires structured executive decision governance. When Good Analysis Produces Poor Decisions One of the most persistent misconceptions in enterprise IT strategy is the belief that more analysis produces better decisions. It does not. During enterprise software evaluations, organizations routinely generate an extraordinary amount of analysis and documentation: Ironically, this abundance of data frequently induces decision paralysis rather than strategic clarity. Executives become overwhelmed by competing departmental priorities, conflicting vendor claims, and technical minutiae. As a result, executive discussions regress from strategic outcome validation to tactical feature grading, pricing negotiations, and deployment timelines. Technology selection becomes the primary objective, while business value realization fades into the background. Confidence Is an Executive Asset Enterprise technology investments are rarely approved simply because a vendor offers the longest feature list. Successful investments are approved when leadership establishes high decision confidence that the deployment will produce its intended business outcomes. Executive decision confidence relies on validating five fundamental governance questions: When these questions are answered with evidence rather than vendor assumptions, governance transitions from passive procurement to active strategic leadership. The Lumerai Executive Decision Framework The primary objective of the Lumerai Executive Decision Framework is not to select software. Its objective is to identify the investment decision that yields the highest confidence of achieving target business outcomes. The framework evaluates major capital investments across six progressive decision gates: Stage 1: Validate Strategic Alignment Establish whether the initiative directly accelerates enterprise revenue, cost efficiency, or competitive differentiation. Initiatives lacking clear strategic alignment should not advance to subsequent investment gates, regardless of technical capability. Stage 2: Define Decision Criteria Before engaging vendors, leadership establishes non-negotiable success criteria driven strictly by operational imperatives. Business outcomes drive technology selection—never the reverse. Stage 3: Evaluate Solution Architecture Assess integration complexity, scalability, security posture, and operating model alignment. Technology is evaluated as a capability enabler within the broader ecosystem. Stage 4: Assess Enterprise Risk Evaluate operational, financial, vendor stability, cybersecurity, and organizational change risks collectively rather than in isolation. Stage 5: Compare Future Scenarios Test how candidate solutions perform across distinct future business scenarios—such as rapid scaling, M&A integration, regulatory shifts, and emerging AI capabilities. Stage 6: Establish Decision Confidence Final investment sign off should occur only when the evidence demonstrates that the business problem has been validated, alternatives have been objectively evaluated, risks are understood, and clear value realization metrics have been established. Confidence does not mean certainty. No major technology investment comes with certainty. It means leadership understands the evidence, the assumptions, the risks, the alternatives, and the expected outcomes well enough to make a defensible decision. Five Biases That Can Distort Technology Decisions Even disciplined organizations are susceptible to cognitive bias. Five biases appear repeatedly during enterprise technology evaluations: Recognizing these biases does not eliminate them. The objective of governance is to make them visible, challenge their influence, and ensure that major investment decisions are ultimately supported by evidence rather than assumption. Executive Example Consider an enterprise modernizing its service architecture following a structured discovery phase. Three finalists remain: Vendor A: May create the greatest long term capability, but introduces significant organizational and implementation risk. Vendor B: May offer the strongest operational fit, but requires a higher initial investment. Vendor C: May produce the strongest financial case today, but could constrain future growth through customization requirements. All three can satisfy the requirements. The decision therefore cannot be resolved by requirements compliance alone. Which investment gives us the greatest confidence that we will achieve the business outcomes we defined? Under the Lumerai Executive Decision Framework, the decision shifts from “Which platform scores highest?” to “Which capital allocation provides the highest decision confidence for our strategic outcomes?” By evaluating each vendor against future operating scenarios and risk tolerances, leadership converts an ambiguous software comparison into a confident strategic decision. Better Decisions Require Better Governance Technology investments should never be authorized simply because a platform tops a feature matrix. They should be authorized because executive leadership has evidence based confidence that the investment aligns with strategic objectives, addresses a validated business problem, appropriately manages enterprise risk, and establishes a credible path to measurable economic value. True decision confidence is not executive intuition. It is the outcome of disciplined decision architecture. Conclusion Technology investment decisions are often portrayed as exercises in software selection. In reality, they are exercises in executive judgment regarding business outcomes. Organizations that consistently outperform their peers are not necessarily better at evaluating technology. They are better at making decisions. They understand that every technology investment is ultimately a business decision supported by technology, not the other way around. Discovery identifies the right problem. Decision making determines the right investment. Execution delivers the solution. Value realization measures success.
AI Capability Doesn’t Always Arrive as Features
Most organizations are running less AI capability than they own. The entitlement arrived, the gate is working as designed, and part of what accumulates now cannot be counted at all.
Nobody Decided to Let Software Decide
Decision rights have been migrating out of people and into business rules nobody owns. Real estate got the bill first. What every executive should know.
The Hidden Risk of AI: Building Transformation Programs for a Future That May Not Exist
For decades, enterprise transformation followed a familiar model: define a future state, build a multi-year roadmap, and execute with discipline. That model is now under pressure—not because transformation is no longer necessary, but because the assumptions behind it are rapidly breaking down. Large transformation programs rely on a core premise: that we can reasonably predict the future operating model. In an AI-driven environment, that premise is increasingly fragile. Capabilities are evolving faster than planning cycles, and organizations are designing multi-year programs against a moving target. McKinsey & Company has highlighted that while generative AI could create significant economic value, organizations are capturing impact fastest through incremental deployment embedded in existing workflows, not large, monolithic transformations. Value is emerging in shorter cycles, not long-duration bets. At the same time, execution risk was already high. Boston Consulting Group estimates that 70% of digital transformations fail to meet their objectives, even in more stable environments. AI does not reduce that risk—it amplifies it. But there is a more fundamental issue emerging—one that is still underappreciated. We are designing transformation programs without a clear understanding of what AI will actually be capable of in two to three years. This introduces a new form of exposure. Not just whether a program will be delivered successfully, but whether it is solving the right problem at all. Entire layers of functionality being built today—workflow orchestration, decision support, even elements of system integration—may be simplified, automated, or eliminated as AI capabilities mature. The risk is no longer just execution failure. It is design risk leading to solution irrelevance. Programs that are delivered on time, on budget, and exactly as designed—but no longer aligned with the business or technology landscape by the time they go live. This is why we are seeing leading organizations shift away from “big-bang” transformation toward more adaptive models. Composable architectures, incremental modernization, and shorter investment cycles are becoming the preferred approach—not as a compromise, but as a strategic response to uncertainty. The question is no longer, “What should our business look like in five years?” It is, “How do we remain adaptable over the next five quarters?” For CEOs, CIOs, and private equity leaders, this has direct implications for capital allocation and risk management. Transformation programs must be structured with optionality—designed to evolve as capabilities evolve, rather than locking into a fixed future state. In many cases, the right answer is not to stop transformation—but to deconstruct it. Rob Purks is a Founding Partner at Lumerai Advisors, a technology strategy advisory firm. Lumerai Advisors provides an unbiased perspective which is not influenced by vendor relationships. With over 150 years of CIO and technology experience, the founding partners bring an honest and complete perspective on technology strategies and challenges. #Telecom #DigitalTransformation #PrivateEquity #ITStrategy #CIO #EnterpriseArchitecture #OPEX #AI #Cloud Lumerai Advisors
The AI ROI Panic Is About to Create the Next Legacy System
By Rob Purks, Founding & Operating Partner, Lumerai Advisors There is a quiet panic spreading through boardrooms right now, and it has nothing to do with whether AI works. It has to do with whether anyone can prove it. The numbers behind that panic are real. Hyperscalers are on track to spend roughly $675 billion on AI infrastructure this year alone. Meanwhile, MIT’s Project NANDA found that 95% of enterprise AI pilots deliver no measurable P&L impact. S&P Global reported that 42% of companies abandoned most of their AI initiatives last year which is more than double the rate of the year before. Forrester is now predicting a market correction with enterprises deferring a quarter of their planned 2026 AI spend into 2027. Even the debt markets have weighed in: Citi identified a measurable credit spread penalty for companies classified as AI adopters without evidence of return. Spending without proof is now literally priced into the cost of capital. I am not here to argue with the data. The data is right. I am here to argue with the response. Key Takeaways The Control Reflex: Why AI Governance Backfires When boards see numbers like these, they react the way boards have always reacted: with control. AI steering committees and per-use-case business cases. ROI attestation before funding. Approval gates between pilot and production. Governance councils that meet monthly to review initiatives that move weekly. Every one of these mechanisms is individually defensible. That is exactly what makes them dangerous. I have spent my career on both sides of this dynamic, as a CIO running technology for telecom operators in Latin America, and later advising enterprises on transformation at Accenture, IBM, and Ericsson. And I can tell you what happens next, because I have watched it happen with every major technology wave: the control structure built to manage today’s uncertainty becomes tomorrow’s constraint. It outlives the problem it was created to solve. Nobody is ever promoted for dismantling a governance committee. I learned this the hard way by inheriting the aftermath of one. At a Latin American operator a major CRM transformation had been put in front of a review board and killed on grounds that were individually hard to argue with: the projected budget was steep, the internal skills weren’t fully in place, and the business wasn’t deemed ready. Every objection was reasonable. The board did exactly what it was designed to do. And while we sat on a defensible “not yet” a competitor moved, modernized its customer platform, and took ground we never fully recovered. The control worked perfectly. The company lost anyway. That is the trap: the most dangerous governance failures don’t look like failures at all, they look like prudence. We have seen this exact movie before. Cloud and agile both promised speed. In most enterprises they delivered something closer to “the same speed with more meetings.” The technology arrived, the operating model absorbed it, neutralized it, and carried on. The gains didn’t disappear, they were quietly strangled by slow decisions and diffused accountability. The ROI panic is now rebuilding that machinery, at speed, with the best of intentions. Except this time there is a difference that should worry every executive: when the constraint is a legacy system you can eventually migrate off it. When the constraint is a legacy operating structure there is no migration project. It just becomes how the company works. I call this Governance Debt: the accumulated drag of controls that outlast the uncertainty they were built to manage. Like technical debt it compounds quietly and like technical debt the interest is paid in speed. The Wrong Diagnosis: It’s Not an AI Adoption Problem Here is the tell that we are solving the wrong problem. In one of the most striking findings of this cycle 97% of executives report personally benefiting from AI yet only 29% see significant organizational ROI. Read that gap carefully. It is not an adoption problem. It is not a model-quality problem. And it is emphatically not a control problem. It is a compounding problem. Value is being created at the level of individuals and teams and the organization has no mechanism to aggregate it, redirect it, or build on it. Adding oversight to that situation does not create compounding. It adds friction to the one place value actually exists. Look at what the successful 29% actually have in common: AI tied to revenue outcomes, business teams owning the workflows, and the whole effort treated as organizational redesign rather than technology deployment.[1] Notice what is not on that list: more approval gates. The organizations seeing returns did not out-govern their peers. They out decided them. What to Build Instead: A Velocity-First AI Operating Model If the answer isn’t heavier oversight, what is it? Three structural moves none of which require a committee: Replace approval gates with kill cycles. Don’t make initiatives prove their worth before they start, make them prove it on a clock. Every AI initiative launches with pre-agreed kill criteria and a fixed time box. The discipline shifts from “may we begin?” to “did we learn enough to continue?” That is governance measured in velocity, not meetings. Measure ROI at the portfolio level, not the use case. Demanding a business case from every individual experiment guarantees you will only fund the safe, incremental, and ultimately unimportant. Venture and private equity investors figured this out decades ago: the portfolio carries the math so the individual bets can take real risk. It is the same logic that drives value creation across a portfolio of companies, you manage the aggregate, not the average. Boards should hold leadership accountable for portfolio-level return and learning rate, not for the survival of any single pilot. Put a sunset clause on every control. Any governance mechanism created to manage AI uncertainty should carry an expiration date and a renewal test: what decision did this body accelerate this quarter? If the honest answer is none, it isn’t governing, it’s accumulating. This is how you keep
The 95% AI Pilot Failure Rate Is Good News
Why the MIT number is a portfolio management finding, not a technology verdict, and the three governance mechanisms that separate an AI experiment pipeline from a capital leak.
AI Isn’t Your Competitive Advantage; Better Decisions Are
Organizations don’t compete on technology. They compete on the quality and speed of their decisions. Introducing the Enterprise Decision Stack™, a new executive framework explaining why strategy, governance, operating models, trusted data, and AI must work together to create sustainable competitive advantage.