Pathways

Groove or ruts?

Patterns are solutions to common situations that provide value within a particular context. Patterns provide a structured, but tailorable means of shaping how to respond to varying system complexity, risk, and and project challenges.

Patterns are identified through reflective activities that recognize that some decisions and approaches are better than others in these different situations. Why are these patterns better? Through experience, they are found to contribute to overall system coherence, desired architectural attributes, or some cultural aspiration.

In The Benefit of Patterns, Linda Rising (sometimes called the 'patterns princess') recognizes that patterns, like any other approach, is best applied in moderation:

I wonder if the real power.. is not to hand us exoctic solutions, but give us a way to remember the simple, ordinary, basic solutions that we know but forget in the heat of battle... cognitive scientists say that our ability to make good decisions... is hampered by the stress of the moment.

This makes particular sense once we understand models of intelligence such as Jeff Hawkin's memory prediction framework of the human neocortex, , and understand how memory works in our life experiences:

We structure our experience of time around memories. We remember events in relation to other events. But as we get older, and our experiences become less unique, our memories can blend together. If yesterday's lunch is indistinguishable from the one you ate the day before, it'll end up being forgotten. If you're not doing things that are unique and different and memorable, this year can come to resemble the last, and end up being just as forgettable as yesterday's lunch. That's why it's so important to pack your life with interesting experiences that make your life memorable, and provide a texture to the passage of time.

Patterns are thus tightly bound within our lateralization of brain functions in our our ability to process language, recognize patterns, recall past circumstances, synthesize information, and react to situations. in this way, patterns can be thought of as the semantics through which our decisions and actions (or syntax) is given meaning (whether in personal or work situations). These rules must assure that patterns are themselves structured a certain way, in order for their desired behaviors to emerge. This overall structure consists of descriptive shortcuts of selected behaviors, within a broader language that recognizes the interactions of these relationships. As a result, the actual path traversed depends on both circumstances, history, and the choices made in shaping organizational behaviors along the way. This is not a static situation, since patterns depend on context, and as groups apply patterns, the context of their work changes. 

Patterns are thus collection of behaviors that can enhance the ability of a team or organization using those patterns to overall system coherence, desired architectural attributes, and cultural meaning. Of course, like other things which are labeled as best practices which do not have a body of evidence backing them up, embracing the right patterns themselves can be quite subjective, and thus leave you exposed to self attribution errors.

Patterns are identified and adopted when one class of behaviors is found to be more beneficial than others, and are a best practice that strives to counter the problems that arise in process-based approaches to organizational improvement, and replace them with a role-based approach.

Recognition of such patterns is the essence of knowledge and intelligence. 

Alexander believes that order in any system fundamentally depends on the process used to build the system, which is why the fundamental process is important. It is important that each step preserves structure and gradually adds local symmetries and that the organization unfolds over time. The process thus involves step-by-step adaptation with feedback. Simply following the pattern language doesn't give you a clue about how to handle the feedback. So that's why the fundamental process exists: to give complete freedom to the design process to attack the weakest part of the system, wherever it may be.

So, a sequence does the following:

·       Preserves structure.

·       Keeps you doing one thing at a time.

·       Takes the whole organization into account at each step.

·       Allows itself to be repeated tens of thousands of times.

Patterns are expected to live within the context of an organizational process, and rely this foundation to establish the consistent structure as encouraged by Senge. Patterns thus depend upon the initial steps of process improvement (documenting standards of behavior and enforcing execution), and rely on such frameworks to reduce practice variability. Patterns are identified in understanding the frequency and importance of role-based exchanges provide learning opportunities that can be used to enhance value within the context of development activities.

anthropologists like A. L. Kroeber [Rroeber 1948]. Kroeber writes the following:

Patterns are those arrangements or systems of internal relationship which give to any culture its coherence or plan, and keep it from being a mere accumulation of random bits. They are therefore of primary importance ([Kroeber 1948], p. 119).

He talks about several levels of patterns. Universal patterns are those that fit a general culture scheme (i.e., that more or less fit all human cultures). Systemic patterns talk about broader, more normative groupings around collections of beliefs, behaviors, alphabets, or economics. Of systemic patterns, Kroeber writes:

A second kind of pattern consists of a system or complex of cultural material that has proved its utility as a system and therefore tends to cohere and persist as a unit; it is modifiable only with difficulty as to its underlying plan. Any one such systemic pattern is limited primarily to one aspect of culture, such as subsistence, religion, or economics; but it is not limited areally, or to one particular culture; it can be diffused cross-culturally, from one people to another . . . What distinguishes these systemic patterns of culture—or well-patterned systems, as they might also be called—is a specific interrelation of their component parts, a nexus that holds them together strongly, and tends to preserve the basic plan ... As a result of the persistence of these systemic patterns, their significance becomes most evident on a historical view ([Kroeber 1948], pp. 120-121).

 anthropologists like A. L. Kroeber [Rroeber 1948]. Kroeber writes the following:

Patterns are those arrangements or systems of internal relationship which give to any culture its coherence or plan, and keep it from being a mere accumulation of random bits. They are therefore of primary importance ([Kroeber 1948], p. 119).

He talks about several levels of patterns. Universal patterns are those that fit a general culture scheme (i.e., that more or less fit all human cultures). Systemic patterns talk about broader, more normative groupings around collections of beliefs, behaviors, alphabets, or economics. Of systemic patterns, Kroeber writes:

A second kind of pattern consists of a system or complex of cultural material that has proved its utility as a system and therefore tends to cohere and persist as a unit; it is modifiable only with difficulty as to its underlying plan. Any one such systemic pattern is limited primarily to one aspect of culture, such as subsistence, religion, or economics; but it is not limited areally, or to one particular culture; it can be diffused cross-culturally, from one people to another . . . What distinguishes these systemic patterns of culture—or well-patterned systems, as they might also be called—is a specific interrelation of their component parts, a nexus that holds them together strongly, and tends to preserve the basic plan ... As a result of the persistence of these systemic patterns, their significance becomes most evident on a historical view ([Kroeber 1948], pp. 120-121).

 If following the process produces high quality software, then what is the cause of failures in the software? It must be that the process is not being followed or that the process is deficient somewhere. The process must have missed something. If you think this idea is far-fetched, think again. One of the authors once attended a briefing session for an upcoming ISO 9001 audit. The leader stated that everyone should remember that the person was not being audited, but rather the process and compliance with that process were being audited. This statement was intended to reassure people, and it probably did provide comfort—temporarily. For, after all, if processes can be tweaked to handle every eventuality, then developers stand in grave danger of being replaced by programming robots.

But we all know that software developers are in no danger of being replaced by robots, because software design is a highly creative activity done by intelligent people. As a result, process becomes much less relevant. In fact, highly intelligent people have been known to ignore or subvert the official process when it does not apply to them. Every organization has two processes: the official process and the one actually followed. One of the authors once interviewed a group of key developers from several different projects. Although they all used processes that called for design documents and design reviews to precede coding, each one admitted to completing the code first and then writing and reviewing the design document— "so we can check off the design review box." They had found that for them, design documents were not relevant; as such, it was easier to write the documents once the design was instanti¬ated in the code.

A thread that runs through all of our patterns (and that is fundamental to the principles of the pattern languages presented here) is the focus on product. Neither the development processes nor the internal documentation are delivered to the customer. The customer does not pay for elaborate project plans or architecture documents, or, if they do, it is probably a sign that something is more deeply wrong than such measures can ever address. These patterns don't focus on the development of a process, nor do they advocate the creation of a process organization. Instead, the patterns strive to manage the resources that go into internal documentation. The focus is always on product. A product has strong structural elements, and that structure reflects itself in the organization. We believe that aligning these structural elements through good communication practices offers the key to effective development.

Once in a great while, great minds think alike. But for the rest of us, it takes communication to align minds, motives, and methods in order to build an effective team. Communication is a complex human activity filled with social context, psychological complexity, and emotion. So, building a communication structure that in turn supports the building of a product is also a complex activity that requires a cultural setting conducive to effective communication. There is no guarantee that any set of guidelines can produce such an environment. However, these patterns can provide one foundation for an effective communication environment by defining a development culture suited to the needs of software development and taken from projects attentive to human needs. They also provide structures that can contribute to an organization's ability to reduce time to market, solve the right problem, and meet other customer expectations.

Shared patterns can contribute to a high(er) context culture, a shared vocabulary and shared culture, rather than a mechanical or bureaucratic environment full of rules that are either inhuman or arbitrary. The focus explicitly is not on process. Process is a good tool in mature domains with predictable steps. In other domains, it's not clear what good process portends for product quality. A flexible process can contribute to good communication in a domain that must deal with change. But one can't just install a process; instead, a process must emerge from the structures of communication and production beneath it. And the structures, in turn, are held in place by the values of the organization. Thus, the values of an organization are the foundation of not only what is done, but also how it is done.

Consider the performance of a large professional sports team.

Patterns are thus tightly bound to architectures in operational form, as they become rule for how to configure activities. But patterns must also work together with other patterns in order for desired structure and behaviors to emerge. As a result, patterns tend to be expressed in a pattern language, which is an outline of how patterns may be put together. This is not a static situation, since patterns depend on context, and as we apply patterns, the context changes. As a result, the actual path traversed depends on both circumstances, history, and the choices made in shaping organizational behaviors along the way.

Much as a tailor uses different patterns to design and assemble raw fabric into different garments, a pattern in other types of development is simply a way to solve a specific problem by bringing conflicting forces into balance. In Organizational Patterns for Development, Jim Coplien describes how this structure is shaped by natural forces:

Human endeavors share common elements of human nature, which to some degree limits the range of human undertakings. For example, most organizations have leaders, and cliques, as well as their own physical space and organizational structure, and hundreds more. The organization of any major human endeavor follows basic laws of efficiency of communication, span of control, xenophobia, specialization, and other sociological forces that drive similar undertakings to implement similar practices and structures.

Systems of nature have common rhythms and trends that underlie their emergent complexity. These properties come from the structure of their organization: the deeply held relationships that define the organization as a social entity.

In Becoming a Learning Organization, Swieringa and Wierdsma indicate that this structure manifests itself in two forms:

The organizations… arose as communities of interest developed within a project. The “real” organizations in any culture can be defined in terms of coupling between actors or roles brought together by a common interest or objective. Such organizations are called instrumental organizations and should be distinguished from the formal organizational structure. An instrumental organizations is the "instrument which regulates organizational behavior'

The concept of patterns results from reflection upon prior approaches. These reflections have focused on behaviors that result from attentiveness to quality of life, evolutionary growth, incremental repair, and experience-based learning. In the classic review of the IBM/360 development effort (A model of large software development), Belady and Lehman describe this decomposition in this way:

The scientific method has made progress in revealing the nature of the physical world by pursuing courses other than studying individual phenomena in exquisite detail. Similarly, a system, a process, or a phenomenon may be viewed from the outside, by acts of observing; clarifying; and by measuring and modeling identifiable attributes, patterns, and trends. From such activities one obtains increasing knowledge and understanding, based on the behavior of both the system and its subsystems, the process, and its subprocesses.

Many of the ideas behind patterns in design and development originated in the book A Pattern Language: Towns, Buildings, Construction, and were further refined though a set of Design Patterns used by high-performance software organizations. Visualizations and stories are typically the means of illustrating patterns.

Individual performance is known to vary widely, and organizational cultures are highly resistant to change. The patterns of interaction between these individuals must have been highly suited to the product they were building, and the environment they were building it in.

Patterns are applicable to architecture, sports, engineering, and organizational design. Talk about playbooks in sports.

While patterns offer a language to describe idealized behaviors, this language will only achieve its potential when the relevant roles learn to embrace the patterns which are relevant to their responsibilities. This means that these patterns must become the basis of status reporting and decision-making, and serve as a rich, semantic communications token for a complex set of behaviors. Otherwise, they are just one more abstraction of behavior that is disconnected from doing the 'real work' of the organization.