We often wish, after going to the trouble of standardizing our work, and training our people, that we will see consistent outcomes as the fruit of our labors. In practice, variation prevails until we can eliminate its causes. Consider the picture on the right, which is an artist's depiction of the frequency and position of urban crimes in South London, Manchester, East London, and Eindhoven. Each individual incident of violent crimes in each area adds to the height of the model at that location. The result forms a mountainous terrain of the variation of these crime incidents by location. Such variation is the nature of populations, and a challenge to solution designers who must deal with those populations.

For an example drawn from our health care domain, consider this situation, from Health Dialog:

If you live in northern Idaho, and you develop back pain, chances are good that you’ll undergo surgery to treat your pain. Move to the southern tip of Texas, however, and the chances that you’ll undergo that same surgery will drop by a factor of 6. The surgery is no more effective in Idaho than it is in Texas. It’s just that doctors in the northwest are more likely than those in southern Texas to recommend surgery. This phenomenon, in which doctors practice medicine differently depending on where they’re from, is called practice pattern variation. And it isn’t limited to treating back pain, or even surgical decisions.

The above situation is called unwarranted variation. It is only one of many different types of such variability, which have their basis in:

1. the nature of the situation (the number and types of diagnoses)
2. the condition of the subject (medical history, severity of illness, age of the patient, receptiveness to recommendations, etc)
3. the novelty of this situation, and the resulting ability to use experience and draw from relevant knowledge in the course of treatment
4. the individual skill and diligence of the medical team
5. slips (making the right decision, but making one of many mis-steps along the path of implementing it) and mistakes (making the wrong decision at a particular point)

We'd like to think that outcomes will be uniform, with the exception of a few unusual cases. What you find instead is that nearly all distributions follow a bell curve, with a handful of quite poor outcomes, a handful of exceptional outcomes (some that may appear nearly miraculous), and a fairly broad, undistinguished middle ground. For example, after an ordinary hernia operation, the chances a patient will have a recurrent hernia are one in ten in the lower group, one in in twenty in the middle, and under one in five hundred for the top performers. For newborns admitted to neonatal ICUs, the risk-adjusted death rate varies from 6 to 16 percent, depending upon the hospital. The likelihood of successful pregnancy for in vitro fertilization varies from under 15 percent to over 65 percent, depending on where you go. Within a particular diagnosis, type 2 differences from the above list, combined with variations in acceptance of high-risk patients for treatment, will account for some of these differences, but many are simply due to types 3-5, above. Over time, many of the sources of this variation can be identified and sytematically reduced, but no member of the medical team is free from mistakes and slips; diagnostic errors average about 10%; errors in reading radiology films are well over twice that.

Turning again to the book Better:

In medicine, we are used to confronting failure; all doctors have unforeseen deaths and complications. What we’re not used to doing is comparing our records of success and failure with those of our peers. I am a surgeon in a department that is, our members like to believe, one of the best in the country. But the truth is that we have had no reliable evidence about whether we’re as good as we think we are. Baseball teams have win-loss records. Businesses have quarterly earnings reports. What about doctors?...

What one really wants to know is how we perform in typical circumstances—some kind of score for the immediate results, perhaps, and also a measure of the processes involved. For patients with pneumonia, how often does my hospital get them the correct antibiotic, and on the whole how do they do? How do our results compare with those of other hospitals? Gathering this kind of data can be difficult. Medicine still relies heavily on paper records, so to collect information you have to send people to either scour the charts or track the patients themselves, both of which are expensive and laborious propositions.

Let's explore one particular disease, cystic fibrosis. This disease is relatively infrequent, occurring in about 1000 patients per year, and given it's low frequency (about 1 in every 3800 births), it is remarkable that we have made the progress that we have. Much of what we know and have accomplished with this disease is due to the diligence of the Cystic Fibrosis Foundation, which has been supporting research and patients for over 50 years. It has invested in countless research studies, helped decode the gene for the disease, and has underwritten countless innovations - new therapies, drugs, and devices. All of these could not have been done by themselves, but required a partnership with dedicated practitioners, working in a synergistic relationship. And their collective track record of successes has been astonishing.

In 1957, the average patient with CF died by age 3. In 1964, the CFF gave a pediatrian at the University of Minnesota, Warren Warwick, ten thousand dollars to collect data on every patient at the thirty one CF centers that year. Warwick discovered an example of positive deviance, a physician named Leroy Mathews at the Babies and Children’s Hospital in Cleveland. Leroy Mathew's patients had a median age at death that year of twenty one years. Turning back to the book Better:

Unlike pediatricians elsewhere, Matthews viewed CF not as a sudden affliction but as a cumulative disease and provided aggressive preventive treatment to stave it off long before his patients became visibly sick from it. He made his patients sleep each night in a plastic tent filled with a continuous aerosolized water mist so dense you could barely see through it. This thinned the tenacious mucus that clogged their airways, enabling them to cough it up. Using an innovation of British pediatricians, he also had family members clap on the children’s chests daily to help loosen the mucus. After Warwick’s report came out, Matthews’s treatment quickly became the standard in this country. The American Thoracic Society endorsed his approach, and Warwick’s data registry on treatment centers proved so useful that the Cystic Fibrosis Foundation has continued it ever since. Looking at the data over time is both fascinating and disconcerting. By 1966, mortality from CF nationally had dropped so much that the average life expectancy of CF patients had already reached ten years. By 1972, it was eighteen years—a rapid and remarkable transformation.

At the same time, though, Matthews’s center had got even better. The foundation never identified individual centers in its data; to ensure participation, it guaranteed anonymity. But Matthews’s center published its results. By the early 1970s, 95 percent of patients who had gone there before severe lung disease set in were living past their eighteenth birthday. There was a bell curve, and the spread had narrowed a little. Yet every time the average moved up, Matthews and a few others somehow managed to stay ahead of the pack. In 2003, life expectancy with CF had risen to thirty-three years nationally, but at the best center it was more than forty-seven.

Today, patients get care at one of 117 specialized centers across the country, each having undergone a rigorous certification process, which assures that they follow the same guidelines for treatment, (which are far more detailed than for most other diseases). All of these centers support ongoing research to find new and better treatments. Yet individual differences continue to be dramatic enormous, and the leader of the pack is at the University of Minnesota. Returning, again, to Better:

The director of Fairview-University Children’s Hospital’s cystic fibrosis center for almost forty years has been none other than Warren Warwick, the pediatrician who had conducted the study of LeRoy Matthews’s suspiciously high success rate. Ever since then, Warwick has made a study of what it takes to do better than everyone else. The secret, he insists, is simple, and he learned it from Matthews: you do whatever you can to keep your patients’ lungs as open as possible. Patients with CF at Fairview got the same things that patients everywhere got—some nebulized treatments to loosen secretions and unclog passageways (a kind of mist tent in a mouth pipe), antibiotics, and a good thumping on their chests every day. Yet, somehow, everything Warwick did was different. Warwick's combination of focus, aggressiveness, and inventiveness is what makes him extraordinary. He thinks hard about his patients, he pushes them, and he does not hesitate to improvise.

Like most other medical clinics, the Minnesota Cystic Fibrosis Center has several physicians and many more staff members. Warwick established a weekly meeting to review everyone’s care for their patients, and he insists on a degree of uniformity that clinicians usually find intolerable. Some chafe. He can have, as one of the doctors put it, "somewhat of an absence of, um, collegial respect for different care plans." And although he stepped down as director of the center in 1999, to let a protégé, Carlos Milla, take over, he remains its guiding spirit. He and his colleagues aren’t content if their patients’ lung function is 80 percent of normal, or even 90 percent. They aim for 100 percent—or better. Almost 10 percent of the children at his center get supplemental feedings through a latex tube surgically inserted into their stomachs, simply because, by Warwick’s standards, they were not gaining enough weight. There’s no published research showing that you need to do this. But not a single child or teenager at the center has died in almost a decade. Its oldest patient is now sixty-seven. In medicine, we have learned to appreciate the danger of ad hoc experimentation on patients—of cowboy physicians. We endeavor to stick to established findings. But with his unblinking focus on his patients’ actual results, Warwick has been able to innovate successfully. And he has become almost contemptuous of established findings. National clinical guidelines for care are, he says, "a record of the past, and little more—they should have an expiration date."

This example also highlights a dilemna of systematic improvement. If no one is free to innovate, or has the time to do so, little will change; but if everyone innovates, there will be no collective learning, or systematic improvement. The key is to progressively eliminate the common causes of variation over time, and isolate those from the special causes - the errors that arise in execution, which can be tackled with training and increased oversight and diligence. There are two keys to any of this happening  - measurement and transparency.