Structured processes are a logical, evolutionary advancement that has emerged from the painful lessons of applying unstructured, ad-hoc methods to problem solving in product development. When products are produced by craftsman, there is a high reliance on their experience and its relevance to a particular circumstance. As the impacts of complexity in producing these products increased, the teams required to build those products became larger. This made it more difficult to control who could work on the teams, and the expertise and discipline that was expected of them. As a result, a cry arose for more systematic, repeatable methods that would produce more reliable outcomes regardless of who was on a team.

Yet an over-emphasis on process is frequently cited as a burden that constrains developers from pursuing their passion. 

Today, countless development and manufacturing standards have been refurbished to attempt to address process yield that is less than 100%. Too often, organizations have adopted such standards as protectionist legislation to reduce competition, rather than because they guarantee higher quality outcomes.

The unfortunate effect of this adoption of process management has been a perception which would seem to refute the claim that clean pipes can produce contaminated water.

There is a simple model of governance that takes account of the effort required to govern: Q = min (#laws, #regulators).

Development processes are heavily based on manual effort and fallible tools. Believing that clean pipes cannot produce contaminated water amounts to accepting the position that flawed tools or flawed human efforts will somehow offset each other and produce good results. Still, this myth lives on in the minds of both novices, some experts, and particularly bureacrats engineers.

This is further complicated because the direct measurement of quality cannot generally be accomplished in process management.

Would you drink water from a glass that was dirty, or drawn from a pipe that had a visible scum coating the inside of it? Probably not. Yet this insight can be helpful in understanding the importance of having a well-designed process to produce a high-quality product.

As Salmonella outbreaks have demonstrated, routine egg consumption can be risky to one's health, since approximately one in 50 consumers are exposed to a contaminated egg each year. When such outbreaks occur, it is critically important for investigators to be able to evaluate each step in the processing of the product as it move through the production processes, so the underlying root causes of problems can be isolated and corrected.

Whether the product is software, eggs or drinking water, the operations necessary to transform inputs into acceptable outputs require multiple steps, locations, and people. If we stick with the water analogy a bit, we can consider requirements to be similar to the source of a water supply for drinking water. The value stream that is involved in developing such requirements into a product is analogous to the links of pipe that fit together to carry water from its source to whoever is consuming it. In this analogy, each link in the pipe is like a process in a value stream. These processes may be geared toward refining the requirements, performing some step or verifying that the software complies with its requirements. Certain processes may be reapplied dant links.

If the interconnections are catch basins, buckets, and funnels, then someone must periodically empty

Eventually something will exit from the pipeline, and with a little luck, that will be quality software. The quality of the pipeline does impact the quality of the code, but to what degree and at what cost can vary. There is no guarantee that even the most superb software development processes will manufacture correct code. Software quality has become a catch-all buzzword for a huge family of various methods geared either toward achieving better software or toward assessing how good software is. Each of these methods represents a different pipe link, and there are various ways of integrating these links that make sense; other combinations are nonsense. For example, putting the system level testing link in front of the processes for checking the requirements for ambiguity is foolish. Not all problems with water (or other products) are self-evident. Management of Water quality is like other types of Quality Management. The water supply network can represent 

Today’s software quality methods can be generally labeled as either process-oriented or product-oriented. Process-oriented methods are focused more towards achieving quality than assessing it. Process-oriented standards would include entities such as the CMMI, DO178, or ISO-9003. Product-oriented approaches presume that you can test quality into a system, and include approaches like software metrics and system-level testing.

If we return to our analogy of a water system, product-oriented methods function like connections that are distributed at different locations along the pipe that allow the developing software, whether in a design language or specification format, to be sampled in order to assess its correctness. Those product-oriented methods that are specifically for software assessment would be clustered near the exit from the pipeline, since in the earlier phases, code will not be available. The process-oriented methods would attempt to ensure that the original requirements were clean, and that every transformation through the development cycle did not inject dirt. Clearly if the water is dirty at point A, then at a later point B, it will still be that dirty and possibly even dirtier, unless some filter is employed between A and B. Dirty water doesn’t clean itself any more than incorrect software fixes itself. Water treatment has it's limits.

You might believe that to assess software quality, all you need is accurate reliability estimation. Because we can almost never know the true reliability of a piece of software, most software reliability models employ error history information to predict future failures. However different error-history based reliability models often compute different reliability predictions for the same data, making it impossible to determine which model will be the most accurate for a specific system. This implies that today’s state-of-the-practice in software reliability assessment is deficient. Even direct measurement of software quality is less than practical. Consider the fact that the testing effort required to establish a certain mean-time-to-failure (MTTF) with 90% confidence is at least twice that MTTF. Even ignoring all problems of test generation, test oracles, test-administration overhead, and the use of overspeed execution and parallel hardware for testing, it is difficult to see how more than about 10 tests/second for complex software could be performed. Current state-of-the-practice is perhaps three orders of magnitude less quick.

Many of the remaining software assessment approaches focus on metrics. Over 100 software metrics are in widespread use today that mainly measure structural or static properties. However only a handful of approaches attempt to measure behavior dynamically. The key problem with structural metrics is that they do not capture the essence of software. Software is implemented by a process that reduces the steps by which an input is transformed into an output through a series of mechanical operations. This transformation is the essential characteristic of software development. A program execution is a series of state transitions, where the machine state contains the output. Exactly what effect a particular instruction has on the mapping between program inputs and program outputs is determined by the program’s input distribution and the program’s instructions. Structural metrics cannot capture this dynamic aspect of software behavior. This lack of connection with the behavior of the software makes structural metrics especially poor as software quality assessors.

Some groups have focused on fault-injection methods as an approach to achieve software quality. Such techniques purposely inject faults into your software, and checks to see whether such injections are detected by other verification methods. However, the types of defects that affect most systems are broad, and the interactions that such defects produce simulates some of these events, and by doing so, it predicts how the software will behave in the future if these anomalous events were to occur.

This entire process of fault-injection involves a lot of statistics and pseduo-random fault-injection methods. Also, the actual process of instrumenting for the anomalies is quite complex. But the results can be most informative, particularly when you learn that your code doesn’t handle problems quite as well as you thought it would. This information provides an immediate quality improvement opportunity. Interestingly, software fault-injection methods, while they directly assess the behavior (i.e., goodness) of your code, also indirectly assesses the goodness of your pipes. By this, we mean that if software fault-injection methods determine that software is intolerant to either faults within itself or anomalies that can attack the software from external sources (such as human factor errors, failed external hardware, or failed external software), then we learn that the other software development processes failed to build in the necessary water filtering mechanisms. Code must be designed with the proper water filtering systems to ensure that the end result from the pipe is crystal-clear water. If software is incapable of producing the outputs we want under even anomalous circumstances, then we cannot claim that the pipes are clean. And if over time it can be demonstrated (via the results of fault-injection) that the set of pipes you have used over and over again do result in clean water, then you at least have anecdotal evidence that correlates your processes with the quality of your software.

The quality of your pipes is only one part of what determines the purity of potable water. It is myth to believe differently. Sadly enough, persons at the highest levels in governments and corporations have all swallowed this lure, hook, line, and sinker. Quality software does not fail often, and never fails in hazardous ways. Software development processes do not define software quality; software behavior does. Behavior is an intrinsic characteristic of software, and it can be viewed without regard to the software’s developmental history. Before we can have faith in software standards and process models, it must be demonstrated that they have a quantifiable relationship to the behavior of the software produced according to them. Parnas once said: "It seems clear to me that not only is a ’mature’ process not sufficient, it may not even be necessary."

The current popularity of process-oriented assessment techniques is, in part, a reaction to the intractability of performing adequate software assessment. Unfortunately, the relationship between development processes and the attainment of some desired degree of product quality is not well-established. This problem is particularly acute for formal methods: forming the required processes does not give a quantifiable confidence that the software, when released, will have the required reliability. In short, testing measures the right thing - product reliability - but it often cannot measure it to the desired precision. Process measurements are more tractable, but they are more effective at managing the flow of work than they are at answering questions like 'how many problems remain'.