When a team jointly develops a depiction of the causal aspects of historical events, and how those led to their current situation, they need to be aware of the effect that their own beliefs can have on their depiction of these events.  - Robert Jervis

Narrative and story-telling can be powerful psychological influences within group settings. We are 'wired' to desire to have our individual experiences become a part of our culture's folklore, and respond to the compelling stories of others in such contexts:

By a mental mechanism I call naive empiricism, we have a natural tendency to look for instances that confirm our story and our vision of the world- these instances are always easy to find. Alas, with tools, and fools, anything can be easy to find. You take past instances that corroborate your theories and you treat them as evidence. For instance, a diplomat will show you his 'accomplishments' not what he failed to do. Mathematicians will try to convince you that their science is useful to society by pointing out instances where it proved helpful, not those where it was a waste of time, or, worse, those numerous mathematical applications that inflicted a severe cost on society owing to the highly unempirical nature of elegant mathematical theories....

Of course, it is not so easy to 'falsify,' i.e., to state that something is wrong with full certainty. Imperfections in your testing method may yield a mistaken 'no.' The doctor discovering cancer cells might have faulty equipment causing optical illusions; or he could be a bell-curve-using economist disguised as a doctor. An eyewitness to a crime might be drunk. But it remains the case that you know what is wrong with a lot more confidence than you know what is right. All pieces of information are not equal in importance.

We like stories, we like to summarize, and we like to simplify, i.e., to reduce the dimension of matters. The first of the problems of human nature that we examine in this section, the one just illustrated above, is what I call the narrative fallacy. (It is actually a fraud, but, to be more polite, I will call it a fallacy.) The fallacy is associated with our vulnerability to over-interpretation and our predilection for compact stories over raw truths. It severely distorts our mental representation of the world...

The narrative fallacy addresses our limited ability to look at sequences of facts without weaving an explanation into them, or, equivalently, forcing a logical link, an arrow of relationship, upon them. Explanations bind facts together. They make them all the more easily remembered; they help them make more sense. Where this propensity can go wrong is when it increases our impression of understanding.... The problem of narrativity, although extensively studied in one of its versions by psychologists, is not so 'psychological': something about the way disciplines are designed masks the point that it is more generally a problem of information...

The way to avoid the ills of the narrative fallacy is to favor experimentation over storytelling, experience over history, and clinical knowledge over theories... Being empirical does not mean running a laboratory in one's basement: it is just a mind-set that favors a certain class of knowledge over others. I do not forbid myself from using the word cause, but the causes I discuss are either bold speculations (presented as such) or the result of experiments, not stories.

- The Black Swan

When individuals or a team attempts to use such narratives to build models whether they are to be used in predicting future outcomes, to justify some action (a path particularly prone to self-deception), or just to explain their model to others who were not involved in weaving the underlying narrative together, the implications of the dynamics of their models, whether explicit or implicit, may escape their grasp:

Even if our cognitive maps of causal structure were perfect, learning, especially double-loop learning, would still be difficult. To use a mental model to design a new strategy or organization, we must make inferences about the consequences of decision rules that have never been tried and for which we have no data. To do so requires intuitive solution of high-order nonlinear differential equations, a task far exceeding human cognitive capabilities in all but the simplest systems. In many experimental studies, including Diehl and Sterman, the participants were given complete knowledge of all structural relationships and parameters, along with perfect, comprehensive, and immediate knowledge of all variables. Further, the systems were simple enough that the number of variables to consider was small. Yet performance was poor, and learning was slow. Poor performance in these tasks is due to our inability to make reasonable inferences about the dynamics of the system, despite perfect and complete knowledge of the system structure.

- Systems Dynamics

One of the reasons for such challenges ties back to the logical principle that correlation does not imply causation. Quantifying the strength of causality in such models is difficult to evaluate in the best of circumstances. But when combinations of factors are involved, the probabilities of each potential factor must be evaluated relative to how they contribute to each possible outcome. In such situations, the mathematics and robustness of the underlying data must be significantly more rigorous, and typically will stretch the information available to the team. Disagreements about how to interpret sources of such information from the past may also arise, especially if the underlying process to collect such data was not controlled. While algorithms exist to factor in such uncertainty, it is important for corresponding uncertainty and risks to be factored in as well, for any decision-making that such models are to be based upon. While this can yield effective treatment modalities for rare diseases, it is not nearly as helpful for decision-making on projects, where individual and team performance is often the dominant factor.

Even when teams believe there are useful insights available from such models, it is also typical that their models, at least in initial form, will be incomplete or inaccurate:

Much of the information we receive is ambiguous. Ambiguity arises because changes in the state of the system resulting from our own decisions are confounded with simultaneous changes in a host of other variables. The number of variables that might affect the system vastly overwhelms the data available to rule out alternative theories and competing interpretations. This identification problem plagues both qualitative and quantitative approaches. In the qualitative realm, ambiguity arises form the ability of language to support multiple meanings. Rich, ambiguous texts, with multiple layers of meaning, often make for beautiful and profound art, along with employment for literary critics, but also make it hard to know the minds of other, rule out competing hypotheses, and evaluate the impact of our past actions so we can decide how to act in the future; In the quantitative realm, engineers and econometricians have long struggled with the problem of uniquely identifying the structure and parameters of a system from its observed behavior. Elegant and sophisticated theory exists to delimit the conditions in which one can identify a system from its behavior alone. In practice, the data are too scare and the plausible alternative specifications are too numerous for statistical methods to discriminate among completing theories. The same data often support wildly divergent models equally well, and conclusions based upon such models are not robust...

Humans are not only rational beings, coolly weighing the possibilities and judging the probabilities. Emotions, reflex, unconscious motivations, and other non-rational or irrational factors all play a large role in our judgments and behavior. But even when we find time to reflect and deliberate we cannot behave in a fully rational manner (that is, make the best decisions possible given the information available to us). As marvelous as the human mind is, the complexity of the real world dwarfs our cognitive capabilities. Faced with the overwhelming complexity of the real world, time pressure, and limited cognitive capabilities, we are forced to fall back on rote procedures, habits, rules of thumb, and simple mental models to make decisions. Though we sometimes strive to make the best decisions we can, bounded rationality means we often systematically fall short, limiting our ability to learn from experience. Experimental studies show that people do quite poorly in systems with even modest levels of complexity. These studies led me to suggest that the observed dysfunction in dynamically complex settings arises from misperceptions of feedback. The mental models people use to guide their decisions are dynamically deficient and are insensitive to nonlinearities that may alter the strengths of different feedback loops as a system evolves.

- Systems Dynamics

Such teams thus may find that their efforts may unfortunately not be as meaningful to outsiders as to those involved in the original synthesis. Being able to discern key learnings from such a complex set of information can be quite challenging, and very subject to cognitive bias, even in the best of situations - such as when the information has been designed for effective communications. 

One of the contrasts between science and politics is that the former makes heavy use of relevant facts, data, feedback, measurements, and iterative refinement, whereas the latter is more selective in communicating filtered information to reinforce positions which have already been taken. In science, progress is accomplished by collecting measurements, then using them to develop hypotheses, design experiments, and collect more data to test those hypotheses. In this context, dialog, discourse, and peer reviews are essential elements, both to clarify the structure and content of the hypothesis, and assure the repeatability and validity of these experiments.

In contrast, politics, as Nathan Myhrvold eloquently highlights in this post, is primarily driven by the passion and commitment of the participants, rather than a foundational body of knowledge, and controlled observations. This is why, in a political context, dialog too often devolves into attempts to protect territory, leverage perceived strengths of proponents, and attack weaknesses of opponents. As Nathan indicates, "Ancient Romans watched gladiators in much the same way that we read angry bloggers today." Even when there model formulation and assertions are not controversial, they also may not be useful with other stakeholders in shared decision-making that results in predictable outcomes.

Within science, peer reviews also play a critical role, and such feedback is exchanged as new players are added to communicating groups. This is most effective when communications occurs in written, rather than oral, form (or with verbal clarifications). Using such means, misunderstandings can be identified and given due process, and have the opportunity to be resolved. In politics, instead, dominant voices are often instead raised to drown out dissenting voices, and exploit perceived power.

Within this realm, when an underlying communications medium or representation of competing ideas is graphical rather than textual, refinement towards meaningful decision-making can be much more difficult to do, since such representations may often be read in many different ways, or be based upon unproven assumptions or contrary to key elements relative to the experiences of others. If incremental refinements are not performed in response to constructive criticism or probing of such perspectives, one may unfortunately be left with an model that will not correlate well with actual measurements, which may not hold together under closer scrutiny, or which may suffer from perception problems that so frequently dominate human discourse. The efficiency and noise tolerance of such communications channels is one of many challenges associated with change management, but these factors are very important to overcome. One's protection from having rose colored glasses may only come from running experiments that involve independent observers, as indicated above. Such validation and refinement is an important part of any mature modeling effort.

As an example, consider the debates which occurred in the late 1800s and early 1900s about martian canals and whether they were an indicator of life on mars. It is important to realize that this was a time of significant canal building, including the Suez canal, and early work on the Panama canal. A number of scientists disputed the idea that canals existed on Mars, and demonstrated that when viewed through telescopes of that era, objects with point features can appear to the observer to join up to form lines, which is an artifact of our visual system. Percival Lowell, who had popularized the idea of these canals in three different books around the turn of the century, sponsored and financed the Alvan Clark telescope at the Lowell observatory in order to make his own observations. This allowed him to use instruments equivalent to those being used by others at the time. When he made these observations, he reported that he again saw canals - even better than before. Yet during the 1909 opposition, separate measurements were taken which which began to change the predominate view. Despite this, the idea of canals continued to persist as popular folklore for years, and was echoed for another 50 years in movies, novels, and depictions of the Martian landscape.

For another example, consider the classic historical debates of the last century between John Maynard Keynes and Friedrich A. Hayek on economic theory. Keynes had a vision of an inherently unstable market economies. You can visualize the dynamics of income and expenditures under Hayek's mental model, and that visualization may help reveal the perceived market mechanisms that  take the economy from a state of high employment into deep depression and back again, then subsequently into inflationary spirals. In Keynes's judgment, given this environment stability and prosperity can only be achieved through the external interventions of fiscal policy, i.e., discretionary spending and taxing. Alternatively, under Hayek's mental model, it was believed that an explicit recognition of the effects of delays in economic activity would lead to a means-ends reckoning of production and consumption. The market rate of interest would keep production in line with people's willingness to save, and allow for sustainable economic growth. Accordingly, attempts to override the market rate with artificially low, growth-inducing rates would send the economy onto an unsustainable growth path and eventually a 'bubble' that requires correction. According to Hayek, this manifests itself as the boom and bust phases of the business cycle.

When one attempts to reconcile these two views, it becomes evident that the critical discriminator between their points of view is the level of saving (which is simply the absence of spending). For Keynes, savings dampens economic activity . For Hayek, saving is a prerequisite to economic growth. Given enough time, even small differences can have significant impacts on such elements, and the debates about which of these world views is accurate continue even today. Thus, when such interactions are complex, multi-faceted, and dynamic, beliefs will easily color our perceived reality, and thus, affect future outcomes.

The above examples reinforce the importance of calibrating, validating, and documenting any model of a situation with real-world information before deciding it is fit for use in decision-making or other purposes. Historical assessments within project environments reinforce how susceptible we are to such factors, even in well-scoped individual project environments in which 'plans' are our own 'models' of the future. This is how unfortunate outcomes can occur despite the best efforts of experienced and well-intentioned people.