5 min read
We’ve invested considerable effort in improving the predictability of software delivery, but are we applying the same rigor to the business outcomes that determine what we build in the first place?
Most organizations can tell you whether a feature shipped on time. I’m curious how many can tell you what business outcome they anticipated before starting, how confident they were in that expectation, what actually happened, and whether that learning influenced their next investment decision.
That question came to mind while reading Marty Cagan’s recent article, Predictability vs Outcomes.
Marty argues that our continued emphasis on output over outcomes may stem from something deeper: the desire for predictability. Organizations commit to features and dates before sufficiently validating their assumptions, then measure success by whether they delivered what was promised. Discovery and learning become secondary to fulfilling the plan.
I agree with much of his assessment. I’ve seen how easily an early estimate becomes a commitment, and how difficult it can be to change direction once expectations are established. Marty is also right that product discovery requires acknowledging what we don’t know and accepting that many ideas won’t produce the results we anticipate.
My work has largely focused on flow, improving delivery efficiency and how we forecast, communicate uncertainty, and build confidence in commitments. I believe a similar discipline should extend to realization, with Product and business leaders accountable for documenting anticipated outcomes, confidence in those expectations, and actual results.
Closing that loop gives us evidence to improve both delivery predictability and our ability to anticipate and realize business value.
What Delivery Confidence Has Taught Me
Software product development is inherently nondeterministic. We cannot know everything before we begin, and attempting to create certainty through detailed plans and fixed dates doesn’t change that reality.
During my years at my previous organization, we invested heavily in improving flow, engineering practices, and delivery reliability, eventually exceeding 3,000 production changes annually while maintaining approximately 99.99% availability.
We also evolved how we approached delivery commitments. Having spent much of my career accountable for engineering delivery and the confidence organizations place in those commitments, I’ve worked to move away from treating early estimates as deterministic predictions toward probabilistic forecasting.
Our expectations should reflect what we know, the remaining uncertainty, and our current level of confidence. As work progresses, new evidence helps us update forecasts, reconsider priorities, and communicate changes before they become surprises.
Executives and customers still need reasonable expectations to make business decisions. Communicating updated forecasts, confidence levels, and changes helps maintain trust while allowing teams to learn and adapt.
I explored this recently in Software Delivery Confidence: How to Build Trust in Commitments.
I’ve increasingly wondered why we shouldn’t apply a similar approach to anticipated business outcomes.
The evidence supporting these confidence levels is different. Delivery confidence draws on historical performance, capacity, dependencies, and technical uncertainty. Confidence in anticipated outcomes should come from product discovery, customer research, and validation of the assumptions supporting the expected business impact.
The underlying discipline is similar, even though the methods and evidence differ.
Can We Build Confidence in Outcomes Earlier?
In Profitable Engineering, I emphasize documenting anticipated outcomes at the initiative or epic level before investing in a solution, then returning to measure actual outcomes. I believe there’s value in also documenting our confidence in those anticipated results.
Consider an organization anticipating that a new capability will reduce customer onboarding time by 20%. How much confidence should we have in that expectation before committing significant engineering capacity?
With demand almost always exceeding available capacity, anticipated business impact and our confidence in achieving it should be central to investment and prioritization decisions.
This reinforces Marty’s point about product discovery. Customer research, prototypes, experiments, and limited pilots can validate assumptions before we commit substantial resources. We may discover that the underlying problem differs from what we initially believed, or that another solution could deliver better results at a lower cost.
We should also distinguish an aspirational business target from an evidence-based expectation. Wanting a 20% improvement doesn’t mean we have sufficient evidence to expect it. How confident are we, and what supports that confidence?
Lower-confidence opportunities may warrant additional discovery or a smaller experiment before making a larger investment. As evidence accumulates, we should revisit the anticipated outcome and our confidence in achieving it. That may happen less frequently than delivery forecast updates, but it remains important.
This is where I believe predictability and outcomes can reinforce each other. Better discovery improves our understanding of potential business value, helping us prioritize investments and make more informed decisions about where to allocate limited capacity.
Who Is Accountable for Closing the Loop?
During my engineering career, I increasingly recognized that improving delivery performance alone couldn’t tell us whether we were realizing the business value we anticipated. That realization shaped my focus on both flow and realization.
If Engineering is held accountable for delivery confidence, shouldn’t Product and business stakeholders be equally accountable for the confidence they place in anticipated outcomes, the evidence supporting those expectations, and ultimately measuring what was realized?
We hear plenty about measuring product success through revenue, adoption, retention, and other business metrics. But measuring overall product performance is different from determining whether a specific investment achieved its anticipated outcome.
That requires capturing the anticipated outcome and its supporting assumptions as part of the initiative itself, along with how success will be measured. When the actual outcome becomes known, we should return to that same initiative and document the result.
Some outcomes become apparent quickly. Others may take weeks, months, or even a year to measure meaningfully. That shouldn’t prevent us from closing the loop. Even when results arrive long after delivery, the comparison provides valuable evidence about our original assumptions, the quality of our discovery, and how accurately we anticipated the business impact.
Over time, that evidence should improve our ability to evaluate opportunities, communicate confidence, and make better investment decisions.
Without it, how do we know whether we’re actually getting better at choosing the right problems to solve?
Marty raises an important concern about prioritizing predictable feature delivery at the expense of outcomes, and I share that concern. I’d extend his argument by suggesting we apply a comparable discipline to anticipated business outcomes: establish an expectation, communicate confidence, validate assumptions early, adjust as we learn, and eventually compare what we anticipated with what actually happened.
Perhaps predictability is a two-way street. Engineering is accountable for communicating delivery confidence, while Product and business leaders are accountable for communicating anticipated outcomes and measuring the value actually realized. Both contribute to the same objective: better investment decisions and improved business value.