A slim chance and a fat chance mean the same thing. A wise man and a wise guy are very different. One must do something to undo what has already been done. Unto means the same as to in most usages. Quite a few means quite many. Pineapples have nothing to do with pines or apples. A house can burn up or burn down with the same outcome. Filling in forms and filling out forms produce the same results. We have noses that run and feet that smell. Many women get a permanent wave about every six weeks. We fire employees who hardly work and praise them if they work hard. We make amends but cannot make one amend. Folk and folks are both plural and mean the same thing.

Being a word pervert, I have spent entirely too much time thinking about the following curiosities. Nonword is a word. We can discuss something and still cuss. Furious means full of fury and joyous means full of joy, but gorgeous does not mean full of gorge. A man with hair sounds much hairier than one with hairs. Why can’t we be chalant, plussed, combobulated, or gruntled? Why can’t we use oderants and perspirants? Why can we remember things when we never membered them to begin with? Of all the odds and ends in your drawer, is the largest one an odd or an end? And finally, observe that stifle is an anagram of itself.

These peculiarities might confuse some people, but to me they represent the spectrum of opportunities for surprise and wonder that make English so powerful, puzzling, humorous, and entertaining. It is a remarkably beautiful language.

1. Measure change and minimize overhead.
2. Steer using economic governance.
3. Plan and predict using Bayesian analytics.

Applying these three principles in context is the crux of honest, measured improvement in business agility and continuous delivery of smarter software-intensive systems.

Measure. Continuous measurement in executable software baselines is a cornerstone of agility. Measurements must illuminate the progress and quality indicators needed to steer projects to more successful outcomes. The agile principles of continuous integration and test-driven development lay a strong foundation for measured improvement. The core metrics for steering can be extracted from the release baselines undergoing continuous integration testing. These metrics quantify the validated learning for more honest reduction of uncertainty and trustworthy assessment of progress and quality.

Agility means quick to react. Change speed must be an asset, not an anchor. Agility is best achieved by demanding that integration testing be performed earlier and continuously. Agility is best measured by quantifying the costs of change over time. Software teams that have improved their cycle time from a few weeks to a few days have clearly become more agile. Instead of being mired in late scrap and rework, and consumed playing defense, they can go on the offensive by innovating more, adding more features or more quality, improving performance, or delivering earlier.

Steer. Economic governance is not a new idea. The principles of the spiral model, iterative development, and risk management; the foundations of modern agile methods; and The Lean Startup all exemplify a process spirit that emphasizes the early resolution of uncertainty. Steering projects with economic governance demands a different outlook on leadership priorities:

• Manage target costs and schedules as a narrowing distribution of outcomes
• Predict outcomes using Bayesian reasoning and ever-improving information
• Optimize quality as a narrowing distribution of defect classes and density
• Communicate progress and quality trends transparently and honestly
• Prioritize integration testing of the forest over unit testing of the trees
• Enhance collaborative teamwork first, then enhance individual production
• Measure trends in the cost of change
• Evolve the precision of requirements and plans as a continuous negotiation

Plan and predict. Software estimates, proposals, and plans are essential predictions that represent the primary information exchanges among governance stakeholders. Trust is earned when integrity and performance are combined. Integrity is improved with more honest predictions that quantify uncertainties. Performance is improved by measurably reducing uncertainty early and continuously negotiating and steering.

Bayesian reasoning for software delivery analytics ensures that steering judgments and negotiations have a strong mathematical foundation and an accepted basis of theory and practice. They exude trustworthiness when we demonstrate a track record of benchmarks (scales for judging appropriateness) and capture our experience in precedent results, references, and experience in the field. Since our industry does not have enough accepted theory and practice, we all suffer from an aura of distrust and skepticism.

One breakthrough insight of Eric Ries’ The Lean Startup is that entrepreneurial progress is measured by validated learning rather than by the volume of output produced. In this paper we have identified techniques for quantifying validated learning in software delivery by predicting and measuring the reduction of uncertainty in the planned outcomes. Like entrepreneurial startups, software delivery is inherently a creative process with high levels of uncertainty. Both endeavors require economic governance: a foundation for quantified planning, decision-making, and measuring that resolves uncertainty earlier and unifies constituencies on managing a shared set of expected target outcomes.

Transforming. In most of today’s software engineering cultures, which are anchored in deterministic planning, middle management governance competes with practitioner freedom. Here are two recurring observations from such cultures:

1. Where there is a perception of good governance and control, practitioners feel choked by repetitive manual reporting.
2. Where practitioner agility and morale are positive, management feels out of control with dynamically changing baselines.

To accelerate software delivery cycles and improve the predictability of economic outcomes, we need to integrate governance and measurement with agility so they are not perceived as competing forces. Practitioners must provide management with improved steering mechanisms such as progress and control measures, automated instrumentation, and real-time development analytics. Management must give practitioners more freedom to innovate through automation of measurement, traceability, progress reporting, documentation, and change propagation. The platform of process know-how and automation must deliver this critical quid pro quo:

More freedom for practitioners < — > Better measurement for stakeholders

Practitioners need to maximize their productive time to create product or service capabilities that deliver more and more value to their users. They must demand that overhead tasks be minimized or automated. Management stakeholders, on the other hand, need more insightful measures and a feedback control loop to steer progress and quality with better predictability of business outcomes. Trust is the secret to achieving this measurement-for-freedom exchange.

Economic governance provides this balance and encourages the collaboration of practitioners and governors to achieve complementary objectives. Integrating economic governance with agile software delivery can be a sensitive transformation for many organizations. Too many process owners advocate extreme positions on both ends of the process rigor spectrum. The traditional extreme imposes an overly comprehensive process with excessive overhead that stifles efficiency. The progressive extreme leans toward overly lightweight process description that cannot scale up appropriately, resulting in significant lost opportunity through inefficient reinvention. A lot of waste can be avoided by targeting a set of economic governance principles for prediction, measurement, and planning, and then a set of economic governance practices that scale the necessary overhead activities up or down as needed.

Every instance of rework introduces a sequential set of tasks that must be redone. For example, a team completes the sequential steps of analysis, design, coding, and testing of a feature, and then uncovers a design flaw in testing. Now a sequence of redesign, recode, and retest is required. These rework activity sequences are the primary obstacles to schedule compression. When designs need to be reworked late in the life cycle, when schedule pressure for delivery is high, projects are forced into shortcuts where they shoehorn in suboptimal fixes rather than subject themselves to the lengthy process of refactoring the architecture. The primary impact of good process improvement has proven to be in reducing the amount of late scrap and rework across the life cycle. This has been a primary thrust of agile and lean methods.

Although overhead activities include some necessary, value-added efforts, like management and planning, in general, the less effort devoted to these activities, the more effort that can be expended in productive activities. The thrust of productivity improvement is to maximize the allocation of resources to productive activities by minimizing the impact of overhead activities on resources such as personnel, computers, and schedule. These improvements in overhead can be achieved by reducing complexity, improving processes, improving team collaboration, or adding automation.

Some simplistic analyses of project resource expenditures in empirical models like SEER and COCOMO suggest that typically, most successful projects spend about 60% of their resources on productive activities and 40% on overhead. Unsuccessful projects have an even higher level of overhead due to excessive scrap and rework. Successful agile projects tend to operate with about half as much overhead, at an 80-20 ratio. This is mostly as a result of reduced late scrap and rework rates and increased automation of change management, regression testing, and metric collection and reporting. There is still a “healthy” amount of scrap and rework—the natural by-product of necessary refactoring as architectural tradeoffs are understood and reconciled in an increasingly flexible code base—and it is counterproductive to try to drive scrap and rework rates too low.

It could be argued that personnel training cannot be considered overhead, but you won’t hear this side of the debate from a project manager. Any project manager who is burdened with training people in processes, technologies, or tools is far worse off than a project manager with a fully trained work force. In practice, a fully trained work force on a project is rare, but employing trained people is always better than employing untrained people, other things being equal. In this sense, training is considered a non-value-added activity.

In a perfect software engineering world with a precise problem description, an obvious solution space, a development team of experienced professionals who work well together, adequate resources, and stakeholders with common goals, we could execute a software development process in one pass with very low overhead and no scrap and rework. Since we don’t live in such a frictionless world, we need to optimize the engineering activity sequence so that overhead is minimized. This is the basis for reprioritizing our early activities toward resolving the largest sources of uncertainty first: They are the primary cause of late scrap and rework.

Developers and engineers will embrace the automation of most overhead activities like automated measurement of the engineering artifacts (models, code, and test bases). Automated instrumentation eliminates the overhead drudgery of metrics collection, progress reporting, change propagation, and traceability, and increases transparency and accountability. Likewise, when governance authorities are enabled with more honest measures that correlate better (and well) with dynamically changing progress and quality trends, they will encourage more change.

Here are some words that are not words but seem like they should be.

• chalante (with purpose; the opposite of nonchalant)
• foreleast (least distinguished; the opposite of foremost)
• gruntled (happy; the opposite of disgruntled)
• inlandish (ordinary; the opposite of outlandish)
• squeam (an ill sensation that results in one feeling squeamish)
• heveled (neat; the opposite of disheveled)
• grue (the ugly root of gruesome)
• ruth (the virtue one lacks if one is ruthless)
• tinguish (the same old stuff; the opposite of distinguish)
• venge (so that revenge, avenge, vengeance, and vengeful have a proper root)
• aster (a great time; the opposite of disaster)
• oderant (the target of deodorant)
• perspirant (the target of anti-perspirant)

There are many more non-words that should be words if the language were designed by rational human beings. Alas, it was not. Therefore, we should just enjoy some of the asymmetries and anomalies. They are beautiful.

The six metric patterns summarized below can be combined with conventional cost and schedule tracking to provide more honest EVM.

1. Planning progress is measured against an architecture of planned features, user stories, and capabilities for demonstrating progress to the user community.
2. Technical progress is measured through the window of integration testing by quantifying the percentage of the product that is currently demonstrable in executable code.
3. Economic progress is a measure of the variance in the cost- or time-to-complete estimate. This is a measure of the uncertainty remaining; it requires that project management periodically quantify an estimate-to-complete and explicitly address the error sources in that prediction. In Lean Startup terms, this is a measure of validated learning.
4. Maturity is a measure of the defect trends and of the suitability of a system or software baseline to be released to its user community
5. Modularity or scrapped code trends over time. This is a measure of the volume of change in the executable code base. It provides insight into the benign or malignant character of software changes.
6. Adaptability or rework trends over time.  This is a measure of the cost of change in the executable code base.

The defining characteristic of software is that it is soft. The easier software is to change, the easier it is to achieve any of its other required characteristics. Both architectural integrity and process effectiveness will drive the cost of change. Therefore, agility is not solely an attribute of a process. It is equally, if not more, an attribute of good design. When architectures are resilient and platforms support change automation and measurement, then projects can optimize their resource investments among defensive efforts (such as bug fixes, cost commitments, and schedule commitments) and offensive efforts (such as new integrations, new innovations, improved performance, earlier releases, and higher quality), and steer toward better outcomes for all stakeholders.

The most important quality metrics are therefore derived from measurements of change trends (defects, scrap, and rework) in the software release baselines throughout the life cycle. The first three metrics listed above quantify how much progress has been accomplished. This is not enough information to steer systems and products to a state where they are suitable for release to their users. You also need three quality metrics to gain insight into how close the current release is to meeting user expectations of both quality and content.

Each of the six proposed metrics has two dimensions: the static value and the dynamic trend. While a discrete value provides one dimension of insight, how these values change over time provides the more important perspective for assessing the current situation, forecasting future expectations, and continuously improving the steering of a project or an organization. The change trends provide insight into whether situations are getting better or worse, and by how much. Because economic governance is focused on managing change, measuring change is a necessity.

Scientists define measurement as quantified observations that reduce uncertainty. We use measurement to advance our understanding and reduce uncertainties. Any significant reduction in uncertainty is enough to make a measurement valuable. Most “best practices” for improving software productivity reduce uncertainties earlier in the life cycle and thereby increase the probability of success.

Improving the distribution to successfully delivering software in a predictable and profitable manner requires an agile leadership style that expects an emerging mixture of discovery, production, assessment, and refactoring. IBM has been advancing agile methods and deploying agile practices for two decades, and we have observed the natural complement of these dynamic development techniques with the dynamic steering principles of economic governance.  What we want to emphasize here is the spirit of these techniques and how they can be reinforced with the right measures and exploited to achieve better business outcomes.

All stakeholders must collaborate to converge on moving targets and manage uncertainties, as illustrated in the figure below. Software schedules, scope, designs, and plans must be treated as predictions with inherent uncertainty and not contracts with implied certainties and requirements. A target date is a prediction, typically the mean of a probability distribution of the release dates. Requirement specifications and models are evolving predictions of the negotiated need. They emerge from a coarse vision with a wide variance to a precise, testable specification with relative certainty.  Nearly everything is negotiable early in the life cycle, and much of the feature set and operational characteristics remain negotiable as tradeoffs evolve from speculative debates to factual decisions. A project plan can be captured as the predicted set of state changes en route to the desired end state. Precision in these life-cycle artifacts should be added commensurate with the uncertainties that have been resolved.

Figure 4. Important steering perspectives to manage uncertainty

For decades, the software industry has been searching for better methods that attack uncertainty. Iterative and agile development methods have emerged organically from diverse software development communities to improve the navigation through uncertainty. This steering requires measured improvement with dynamic controls, instrumentation, and intermediate checkpoints that permit stakeholders to assess what they have achieved so far (their as-is situation), what perturbations they should make to the target objectives (their predicted to-be situation), and how to refactor what they have achieved to adjust and deliver those targets in the most economical way (the roadmap forward). The key outcome of these modern agile delivery principles is reduced overhead and a significant reduction (perhaps as high as 50%) in scrap and rework.

A significant technical catalyst for this transformation is to force integration activities to precede unit test activities, thereby resolving the riskier architectural challenges before investment in unit test coverage and complete implementations.  As the architecture is prototyped, demonstrated, tested, and refactored across the primary usage models and performance scenarios, these early changes will resolve the significant uncertainties and accelerate stakeholder understanding of the optimal scope. When continuous integration is executed effectively, the later you are in the life cycle, the more predictable and straightforward things are to change.

One critical need for better predictions is a more honest and objective assessment of the uncertainties we face so that leaders can steer projects toward better outcomes. This is particularly true in the domain of software development and delivery, where we confront significant uncertainty in specifying what is needed (the requirements), how it will be built (the plan), and how it will behave (the design).  Agile methods explicitly encourage the measurement of progress and quality from evolving releases of demonstrable capability. This is the critical basis for objective assessment.

Measurements in most software organizations are more like the sleight-of-hand statistics quoted by politicians than the matter-of-fact statistics quoted by engineers and scientists. Capers Jones says, “Software has perhaps the worst measurement practices of any ‘engineering’ field in human history.” Politicians and the software industry have similar track records for disingenuous statistics and under-delivering on commitments. Most stakeholders in the software business are justifiably cynical because our collective experience with software productivity improvement is plagued by hyperbole and spin. Although many conventional software metrics approaches, such as earned value management, are well-founded, they are typically implemented in a way that is (unintentionally) dishonest. These statements may seem hyperbolic, but decades of first-hand experience across hundreds of projects have led us to this blunt observation.

To improve the predictability of outcomes, we need to integrate measurement and agility so they are complementary objectives and not competitive forces. Management must empower practitioners with more freedom to change and a platform that automates overhead tasks. In exchange, practitioners must provide management with improved steering mechanisms through transparent progress and control measures that better correlate to economic outcomes.

• Don’t go without me! (Wait for me!)
• I am not dissatisfied. (I am satisfied.)
• I don’t dislike that person. (I like that person.)
• It is not infinite. (It is finite.)
• I am not independent. (I am dependent.)
• I won’t ask you to not go. (Please leave.)
• It is not unusual. (It is common.)
• I don’t disagree. (I agree.)

In most of these cases, the reasonable-looking double negatives are used by people who can’t bring themselves to say the positive form because it feels too strong to them. They resort to the double negative form, which feels softer. My favorite is the expression, “I don’t disagree.” We have all heard this a jillion times in business meetings, community meetings, church meetings or private discussions. It is usually secret code for, “I don’t totally agree, but I don’t want to say I don’t agree.”

The speaker usually follows this up with a sentence that begins with the word but. Try stopping someone right after they say, “I don’t disagree,” and ask, “But do you agree?” The person will usually squirm and stall, think over their answer, and respond either “Yes, but…” or “Well, partly,” and then state the points that they don’t agree with.

I found my all-time favorite example of a double negative about 15 years ago in the sports section of a newspaper. A Cal Berkeley alumnus had just been traded to an NBA basketball team that had the worst record in the league. Asked about his team’s chances in the upcoming year, he was quoted as saying, “We are going to turn this team around 360 degrees.” Although this is not a double negative, it is a great example of a statement that has the same effect. This alumnus meant to turn the team around 180 degrees. He doubled the negation unintentionally, ignorantly setting the team on the same dismal path as the year before. I was not happy that my alma mater had not produced a basketball player who was not illiterate.

Triple and quadruple negation can also occasionally be seen. It results in total obfuscation of the author’s real intent. The last sentence of the previous paragraph is a good example, as is my parenthetical comment regarding Dylan, in a previous post.

While double negatives are frowned upon in English, the French commonly use two negatives to make a stronger negative, particularly in informal French. The Romance languages and Greek and Slavic languages routinely use double negatives. If you are a native English speaker and converse with non-English speakers using either spoken or written communications, you may notice that they use double negatives in their English.

Imagine how hard it is to learn English as a second language when your native tongue routinely uses semantic structures that are considered incorrect in English.

The difference between precision and accuracy is an enlightening lens for focusing on the crux of honest predictions. Accuracy is a measure of truth and freedom from error. Precision identifies the degree of accuracy and implies repeatability or elimination of uncertainty. In practice, more precise random variables have narrower distributions. Unjustified early precision — in requirements or plans — has proved to be a substantial and recurring obstacle to success. The pursuit of early precision is alluring but usually serves only to provide a counterproductive façade for portraying illusory progress and quality. Trust among stakeholders erodes as the divergence between reported progress and true progress inevitably reveals itself. Many stakeholders demand early precision and detail because it gives them (false) comfort in the progress achieved. Software management is full of gray areas, situation dependencies, and ambiguous tradeoffs. Understanding the difference between precision and accuracy is a fundamental skill of good software managers, who must accurately forecast resource estimates, risks, and the effects of change. There are several ways to present estimates, and trust is established with honest communications. Emphasizing accurate estimates with honest qualified precision will help stakeholders engage in a more fruitful discussion of the uncertainties remaining and establish more trust.

Uncertainty can be quantified and tracked by periodically measuring the reduction in variance in the distribution of resource estimates to complete. These estimates are random variables and should be represented by their probability distributions, not just the most likely (the mean) value.  In a healthy software project, each phase of development produces an increased level of understanding by reducing uncertainty in the evolving plans, specifications, and demonstrable releases. At any point in the life cycle, the precision of the subordinate artifacts, especially the plans and estimates to complete, should be in balance with the evolving precision in understanding.

The definition of truth by the American justice system provides an illuminating metaphor for understanding the difference between accuracy and precision. Consider these three familiar components:

1. The truth (be accurate),
2. The whole truth (be precise enough and include relevant evidence), and
3. Nothing but the truth (don’t be overly precise and add irrelevant evidence).

In the software development world, providing an accurate estimate is the main target dimension of the truth. Providing a credible measure for telling the whole truth requires backup evidence such as precedent experience, a prototype, or a resource estimate from an empirical model. Using honest precision (such as ranges or probability distributions) ensures that you are focused on nothing but the truth and are not misleading anyone by implying more certainty than is justified. This is best accomplished by openly disclosing the uncertainties in your forecasts and providing some quantitative indication of the variance of your estimates.

We have all seen proposals for software projects that forecast the cost with incredible precision, such as $2,320,541. Can anyone really forecast the cost of a project that they plan to deliver a year or more down the road, to the dollar? This is dishonest precision. Although precise commitments are frequently necessary in the business world, accurate representations of the uncertainty will establish more trust among stakeholders. A better way to capture this prediction is that with a 95% probability, it will take between $1.7M and $2.6M and 10-13 months to deliver this project. The more fruitful discussion with stakeholders is on why there is so much spread in the estimates and how will the first few milestones reduce that spread by the maximum amount.

Managing uncertainty with more transparent, quantified estimates as summarized above is exactly the sort of process where Bayesian statistics help us improve our predictions at each intermediate stage.

Stephen Covey coined the term “the speed of trust” to illuminate the necessary ingredient in reducing overhead and improving efficiency: trust. Trust is not just a matter of ethics; it is a matter of competence: track record, transparency, and accountability. The speed of trust can also be appreciated by it’s logical opposite: the slowness of distrust. Overhead activities in most enterprises are directly proportional to the amount of distrust. Some distrust is healthy and necessary because humans make errors and poor judgments and a certain level of oversight, planning, reporting, documentation and assurance is necessary to deliver outcomes with a level of certainty and precision. However, when overhead activities are made overly burdensome—i.e., where the value achieved is out of balance with the cost and inconvenience—then the system becomes inefficient. If you want good examples of this, just look at getting through airport security nowadays or getting corporate expense reports processed. If we “trusted” people to fly without weapons or if we trusted people to spend company money frugally, we would implement far less overhead. Besides the additional expense of these non-productive activities, overhead tasks also demoralize the practitioners and introduce new sources of error.

So, how do we improve trust between project teams and stakeholders and thereby minimize the unnecessary overhead activities? More honest communications is step one. Deterministic planning kills trust, because we all know it doesn’t match the reality of the software development world and the uncertainties therein. Communicating plans and targets probabilistically and explicitly quantifying the uncertainty builds trust because it accurately portrays our understanding of where we are and how to reason about the future.

Bayesian predictions are based on centuries old mathematics, and while it was historically a controversial way of thinking about the nature of probability, it is now accepted best practice. The interested reader can learn more about the history of Bayesian reasoning in The Theory That Would not Die. Recently Bayesian reasoning has emerged as central to all sorts of predictions. For example, In The Signal and the Noise, Why Most Predictions Fail and Some Don’t, Nate Silver explains how Bayesian reasoning is central to reasoning about elections, weather, poker and other societal interests.

Bayesian reasoning requires a change of mindset from how most of us approach project estimation. We abandon the idea that a project has a fixed duration or a fixed cost. The Bayesian mindset reasons about predictions as a range of possible values called outcomes. Some outcomes are more likely than others, so we assign each outcome a likelihood value.

Specifying a quantity, not as a fixed single value, but as probabilities of the possible outcomes is called a random variable. The function that assigns to each outcome its probability is called the distribution. The figure below shows some examples of probability distributions of a random variable. These examples all come up in development analytics, serving different purposes. Normal distributions like the two upper graphics are the most common target profiles for estimating future outcomes. Triangular distributions like the lower left graphic represent the simplest step up from estimating values, to estimating with random variables. Triangular distributions simply require that you identify three outcomes: a worst case, expected case and best case.

Some simple examples of probability distributions

Once you have the distribution, it can be used to predict the outcome as a probability. Briefly, for any interval pair of possible outcomes (a, b), the probability of the outcome being between a and b is the area under the curve between a and b. So, for example, the probability of finishing somewhere between 130 days and 170 days in the normal distribution of Figure 3 is 95%. The probability of finishing in 150 days or less is 50%. Probability distributions must satisfy one more condition. The area under the entire curve must equal 1. This ensures that we have identified all the possible outcomes.  With this condition, all probabilities fall between 0 and 100%.

The range of applications for random variables is truly mind boggling. The Bayesian perspective is that all predicted values are random variables. In development projects, important random variables should be defined for: time to complete, cost to complete, ready for release (defect density), time to system failure, the project return on investment, and the financial impact of technical debt [Ref??].

After you quantify your key project concern (e.g., time-to-complete) as a random variable, you can measure uncertainty and risk more meaningfully as attributes of the distribution. Wider distributions are more uncertain than narrow distributions. How one measures the width of a distribution varies with its shape. For example, the probability of shipping between day 140 and 160 in the wider upper left normal distribution is about 50%. However, in the narrower upper right normal distribution, the probability of shipping between days 140 and 160 is about 99%.

Finding the distributions. The challenge for project leadership is to find the distribution for a given random variable such as time to complete, and then continuously update the distribution as the project proceeds. The shape of the distribution provides insight into the probability of meeting the schedule target is improving and if the uncertainties have been resolved.

There are mathematical techniques for doing this that rely on conditional probabilities and the application of Bayes’ theorem.  You start with an initial distribution perhaps from expert opinion of high, low, and expected values to specify triangular distributions⁵. Then you gather data as evidence of the values of the random variable. Bayes’ theorem provides the method to update the initial distribution so it reflects the data, evidence and knowledge of experience to date. The initial distribution is called the prior and the updated distribution is called the posterior. In Figure 1, the baseline estimate at the top would be the prior and the result after pursuing option 3 at the bottom of the figure would be a posterior. In summary, the process is rather simple:

• Make an initial, but informed guess of a prior distribution
• Gather evidence
• Use Bayes’ theorem to update a posterior distribution
• Continue updating the posterior distribution as more evidence comes in.

One of the great features of this approach that it converges to the same posterior no matter what prior was used to feed the process. This is important. We need not build the perfect specification or the perfect plan to improve predictability. It is far better to make a reasonable estimate and get started refining those estimates. This is exactly the spirit of iterative and agile development that differentiates it from conventional waterfall model thinking. To summarize, here are the key differences of Bayesian reasoning:

• The goal is not to predict the eventual outcome exactly, but to just make a good guess and start computing the probabilities based on the evolving evidence.
• Probabilities are quantified using distributions
• Uncertainty is reflected in the width of the distributions
• Bayesian refinement is used to improve the distributions

A traditional run-of-the-mill project manager, let’s call him Ben Dover, would instruct his team to lay out a detailed plan for 12 months, nail down the requirements more precisely, and plan on posting up an early design review with a demonstration of the components that are easiest to mock up. He wants to demonstrate early progress to keep stakeholders happy and avoid any late changes in scope. Ben feels confident that he can deliver since he has a month of “extra” schedule.

A more savvy and contemporary software manager, let’s call her Shirley Nimble, understands that the model’s output is just a point estimate and simply the mean or median of a more complex random variable. She suspects that they have about a 50% chance of delivering the mean and asks to see the full distribution of possible outcomes and the planned sequence of intermediate release content. She wants to go into the project with a 95% chance of delivering on time. Her leadership team uses empirical cost and schedule estimation models along with agile planning techniques and comes back to show her the complete distribution illustrated as the baseline estimate at the top of the Figure below.

 A baseline estimate and 3 alternatives

Examining the baseline estimate, they realize that about half of the outcomes will take longer than 12 months and they actually have only about a 50-50 chance of delivering on time. The reason for this dispersion is the significant uncertainty in the various input parameters reflecting their team’s lack of knowledge about the scope, the design, the plan, and the team capability. The input parameters to the estimation models are also predictions (random variables) with some significant uncertainties. Consequently, the variance of the distribution of outcomes is rather wide. They decide that there are three paths that they can take. These are also depicted in the Figure.

Option 1: Ask the business to move out the target delivery date to 15 months to ensure that 95% of the outcomes complete within the target date.

1. Option 2: Ask the business to re-scope the work, eliminating some of the features or backing off on quality so that the median schedule estimate moves up by a month or so. This ensures that 95% of the outcomes complete in 12 months.
2. Option 3: Explicitly reduce the uncertainties in the scope, the design, the plans, the team, the platform, and the process. This narrows the dispersion in the distribution and increases the probability of delivering within the target date.

The first two options are usually deemed unacceptable by external stakeholders, leaving the third option as the commonly preferred alternative—and the foundation of most of the iterative and agile delivery best practices that have evolved in the software industry. If you examine the best practices for requirements management, user stories, architectural modeling, automated code production, change management, test management, project management, architectural patterns, reuse, and team collaboration, you will find methods and techniques that reduce uncertainty earlier in the life cycle.

Shirley and her leadership team lay out a sequence of demonstrable capabilities, starting with the architectural foundation and resolving the largest sources of uncertainty first. They define an explicit decision checkpoint that is their prediction of the minimum viable product. Each intermediate milestone results in a measurable checkpoint from which they can reduce the variance in their estimate to complete and improve the predictability of delivering on time—or knowing that they cannot.

Now, who is more likely to succeed? Mr. Dover has focused on what he knows and mostly ignores the big uncertainties, postponing their resolution until after he builds up some early project momentum by tackling the straightforward tasks. Ms. Nimble focuses on understanding the uncertainty sources and resolving the harder challenges first, probably showing less early progress, but a increasing the probability of long term success. It’s a stark comparison, but a good illustration of the difference in mindsets between the plan-and-track mentality of conventional engineering governance and the predict-measure-steer-and-adjust mentality of economic governance.