DO-178 is the primary document used by aviation certification authorities throughout the world. The document is used to assess the adequacy of practices used in developing software for commercial aircraft. Certification is a complex topic, described in the latest version of DO-178 as follows:

Legal recognition by the certification authority that a product, service, organization or person complies with the requirements. Such certification comprises the activity of technically checking the product, service, organization or person and the formal recognition of compliance with the applicable requirements by issue of a certificate, license, approval or other documents as required by national laws and procedures. In particular, certification of a product involves: (a) the process of assessing the design of a product to ensure that it complies with a set of standards applicable to that type of product so as to demonstrate an acceptable level of safety; (b) the process of assessing an individual product to ensure that it conforms with the certified type design; (c) the issuance of a certificate required by national laws to declare that compliance or conformity has been found with standards in accordance with items (a) or (b) above.

There are many unique challenges associated with development of safety-critical software. Such projects typically have to deal with issues such as constrained resources, novel system architectures, and real-time performance constraints, which amplify the already difficult challenge of successfully developing software within fixed time frames and to very demanding requirements.

The introduction of regulatory oversight and conditions into this environment can either help or hinder, depending upon how well this oversight is implemented. If insufficient attention is given to key details during execution, or if there is a lack of understanding of the underlying requirements necessary for implementation, the risk to the resulting system development activities and products will be high. The purpose of DO-178 guidance is to mitigate these risks, by providing sufficient information for experienced software certification professionals and airframe manufacturing applicants to successfully plan the work so that certification can be achieved, and then make consistent determinations of compliance regarding design, development, and operational practices for aircraft and related ground systems.

DO-178 is neither a software development process, or a prescriptive standard for such a process. Instead, the document serves to structure and guide negotiations between applicants and certification authorities on the activities and deliverables which must be accomplished during product development and support activities. The guidance strives to balance the need for tangible material that provides a consistent basis for these determinations against the range of existing airborne software and company processes that are used in actual practice across the industry. As a result, while the document provides key compliance criteria, it does not provide specific, useful information on how to apply this information on development programs, in a cookbook, 'how-to' manner. The document will only be effective when correctly interpreted and applied by experienced professionals, as they approve or recommend approval of technical data for submitting to certification authorities. This style is consistent with the mentoring and selection process for Airworthiness Representatives that is used across the industry.

The complexity and extent of the compliance data required by DO-178 depend upon the characteristics of the system/software, associated development practices, and the interpretation of DO-178, especially when it is applied to new technology and situations in which there is little or no precedent available to draw from. Two different Airworthiness Representatives, working from the document, may still reach different decisions about the acceptability of a particular artifact, according to their particular experience, preferences, and situation. Like any standard, DO-178 has good points and bad points (and versions even contain a few errors). However, careful consideration of its contents, taken together with solid engineering judgment, are intended to produce better and safer software, and insights about how the document is applied in the real world that can then be fed back to accomplish continuous improvement of the standard itself.

However, some believe the enhanced rigor and completeness of verification evidence required by DO-178B is a significant burden to the avionics industry. Representatives have complained that producing this evidence requires an inordinate amount of effort, and question the value resulting from this effort. Since the original emphasis of the DO-178 document has always been safety rather than cost effectiveness, companies have widely varying approaches in implementing DO-178B, and some are clearly more cost effective than others.

The initial version of the document was published in 1982, just prior to certification of the first models of Boeing's 757 and 767 aircraft. When the first revision, DO-178A, was published only 3 years later, the avionics industry had just completed a major transition from analog to digital systems, systems in which software played a much larger and more safety-critical role. As a result of this transition, a growing number of companies were subject to certification oversight and compliance determinations. During this initial use, an unacceptable portion of these companies struggled because these versions of DO-178 were not sufficiently detailed to provide them with sufficient implementation guidance.

Because of these shortcomings, the industry quickly came to a conclusion that still another version of DO-178 was needed. The new version was expected to incorporate experience from these initial applications of prior versions of the guidance, and to address a number of questions that were being raised as a result of emerging technical issues and proposed new approaches. In some situations, these questions required new rules to be established, such as in identifying mechanisms for certifying reusable software components, qualifying tools, assessing object-oriented design impacts, and dealing with inoperative code in operational systems.

To pick just one of these examples, DO-178 activities are expected to produce rigorous and complete verification and traceability evidence, and the efforts required to produce this information can be quite substantial. As the complexity of software has increased, the cost to produce this verification information has also grown proportionally. While tools have always been used in many situations during development, emerging tools were envisioned to play an increasing role far beyond what had previously had been attempted. This reliance upon such tools, potentially with limited reviews of their results and artifacts, motivated the industry to consider a more explicit definition of tool qualification criteria, and to integrate those criteria into the development practices and descriptions of the rest of the document.

The DO-178B revision was also expected to align its concepts and terminology with emerging guidance for systems development which was concurrently being written. This guidance, DO-254, provides guidance for design assurance of airborne electronic hardware, and describes the expected information required throughout project conception, planning, design, implementation, testing, and validation. For example, through joint agreements between the two standards efforts, it was agreed that the terminology used to describe the criticality of safety considerations needed to change from "critical, essential, and nonessential" to "catastrophic, hazardous/severe-major, major, minor, and no effect". To reflect these changes, the software levels which had originally been described in DO-178A as "Level 1, Level 2, and Level 3" would also need to be changed to "Level A" through "Level E" in DO-178B. This change also enabled changes between system and software levels according to the system design and implementation techniques that were used during development.

More significantly, a means of clearly defining and communicating the effect of these changes needed to be established for these corresponding requirements as these levels changed. For example, with respect to the types of verification test coverage required for Levels C, D, and E software, only statement coverage needs to be achieved, but for Level B software, decision coverage is required, and for Level A software, each part of a condition must be demonstrated to correctly affects the outcome. The standards body working on the standard needed to determine how these differences would be defined, and the document then had to clearly delineate them in a way that was clear, concise, and appropriate to the situation.

After consideration of all of these changes, it became apparent that a complete rewrite of DO-178A was required, despite the fact that this would be the third revision developed for this community in only 5 years. Because of concerns about the direction which this new version could take, literally hundreds of participants from across industry were involved in authoring, reviewing, and integrating the resulting DO-178B content. There was significant pressure to 'get this one right'. In light of the volatility of prior efforts. With the benefit of hindsight, it is worthwhile reviewing the keys of success to this effort, which collectively enabled us to successfully manage the scale and nature of participation for the overall standardization, and to improve the quality and usability of the resulting guidance, and its acceptance by the affected stakeholders:

1. There was skin in the game for the participants

   The development of aircraft systems and the software embedded in these systems relies heavily upon a 'safety culture' that actively focuses on producing software with dramatically fewer safety-related defects than traditional practices would normally produce. In such cultures, with lives and company reputations at stake, the opinions of practicing colleagues tend to be much more highly valued than outsiders. The engineers involved in these development efforts must pay meticulous attention to detail, and act deliberately as safety-related requirements and issues are identified. As they do this, other needs like cost effectiveness, schedule predictability, or adoption of the latest technology innovations, while important to the business, are understandably less important than the focus of DO-178B, which is on assuring the development of software which will operate in life-critical systems.

   In this situation, the committee participants recognized that the guidance which would be produced by committee activities would be of primary importance when performing their duties in their 'day jobs', but would not be the only guidance they would need to deal with. While DO178 is described as guidance throughout the document, this is because regulatory authorities describe it as an acceptable means of determining compliance for aircraft certification, without closing the door to other means. In all other ways, however, this document provides the minimum acceptable provisions (i.e. the standards) which must be taken into account in planning and implementing software-based development activities for products that are to receive regulatory approval. For many other industry standards, such a 'mandate for use' is either not in place, or is not effective because of the way the guidance has been expressed or invoked. Further, compliance in these other situations often does not need to be demonstrated in any formal sense (though groups may still claim they are compliant, without saying by how much), or to whom and how that demonstration has been secured and recorded.

   Decisions about the content of DO-178B in this context were made by striving for consensus by all participants. Consensus does not mean agreement, but rather acceptance, and required a give and take by all participants to find common ground, and . Achieving this consensus across many interrelated issues required many meetings to realize, but paid off in the quality of the resulting document. When consensus could not be achieved, minority decisions could be documented and were accepted by the Special Committee chairs, but this path was only found necessary in very limited cases, and all these concerns were resolved by final publication.

   Such consensus-based standards may be biased towards guidance which retain the ability of industry members to refine (rather than replace) their current practices, and requires judgment by experienced individuals to make detailed compliance determinations. To reinforce, although a standardized format for documentation might be preferred by regulatory groups, any such format selection would likely favor some companies or groups over others. In a team of competing industry participants, any such specific recommendation would favor some over others, and thus would fail to achieve the necessary consensus buy-in. The relevance of the resulting guidance with respect to the diverse implementation approaches used over this period is a direct consequence of the team environment which was fostered as the standard was developed and ratified.

   Over time, this requirement for compliance demonstration has resulted in forming a community of practitioners from many different perspectives . These practitioners can come together and have meaningful discussions about development practice trade-offs and their consequences. Such trade-offs are a part of life of every system development. At the end of the day, DO-178 and it's processes don't certify safe systems; this community does. The importance of their engagement in this reflection and self-regulation cannot be over-emphasized, and such involvement by practitioners has often been missing from other efforts to crafting a body of knowledge for other occupations or industry segments.

2. Terms of reference were established to scope and provide accountability for the effort

   A set of terms of reference was established for the DO-178B Special Committee by the RTCA board in order to focus and stabilize the efforts of the working groups. These objectives called the group to:

   This direction served as the charter for the team's activities; these terms of reference were reviewed, revised, and accepted by the entire Special Committee before being finalized. This helped to assure that the wording of this charge was understood and accepted by the members who would be implementing this mandate.

   • Examine existing industry and government standards and consider for possible adoption or reference, where relevant
   • Assess the adequacy of existing software levels and the associated nature and degree of analysis, verification, test and assurance activities. The revised process criteria should be structured to support objective compliance determination
   • Examine the criteria for tools to be used for certification credit - for example, tools for software development, configuration management, or verification
   • Examine the certification criteria for previously developed software, off-the-shelf software, databases, and user-modifiable software for the system to be certified
   • Examine the certification criteria for architectural and methodological strategies used to reduce the software level or to provide verification coverage, for example, partitioning and dissimilar software
   • Examine configuration control guidelines, quality assurance guidelines, and identification conventions, and their compatibility with existing certification authority requirements for type certification, in-service modifications and equipment removals
   • Consider the impact of new technology, such as modular architecture, data loading, packaging, amd memory technology
   • Examine the need, content, and delivery requirements of all documents, with special emphasis on the Software Accomplishment Summary
   • Define and consider the interfaces between the software and system life cycles
   • Review the criteria associated with making pre- and post-certification changes to a system
   • Consider the impact of evolutionary development and other alternative life cycles to the model implied by DO-178A
3. Working group responsibilities for developing content were defined

   When dealing with hundreds of participants from many different sectors - regulatory agencies, airframe manufacturers, and suppliers - and many perspectives - systems engineering, quality assurance, software engineering, and management - it was necessary to organize committee activities into parallel groups. A working group structure was established so that participants could self-select where they could best contribute. Working group chairs were then selected who were committed to working together to resolve cross-group boundary issues, and who could provide the necessary leadership and guidance to help drive issues to closure within each of these sub-groups.

4. Governance to manage information exchanges and decision-making was established

   Governance is often an afterthought when groups are themselves developing new governance. However, because of the number of people involved in developing and reviewing DO-178B, effective governance practices were essential to establishing trust and managing the evolution of the material being developed. A table of contents was established early on to describe the intended architecture of the document, and provide a basis for assigning responsibilities to the working groups. A template for working papers was also established as a means to identify, track, and communicate progress on critical technical decisions within each working group over time, in a trade study like environment. Key concepts and features that affected multiple sections of the document were then identified, to ensure that agreed-to approaches for these concepts were developed before tackling other related issues. For example, the group wanted to avoid specifying any particular structure for required artifacts and documents that were to be produced during a DO-178B-compliant development effort, and instead of using these traditional artifact descriptors, made a conscious commitment to describe information in abstract containers that could be implemented in a variety of ways; this decision rippled through many other sections of the material that was developed. Finally, a means of identifying and tracking issues for existing content was established. This mechanism satisfied multiple purposes for the teams' authoring efforts, and served as problem reports, change requests, and requests for the resulting information. The communications, tracking, and disposition of these issues was provided centrally in support of all working groups as a service for all participants.

5. A terminology foundation was documented and maintained

   Different groups, and individuals from different backgrounds, often use these different wording in developing content they are familiar with, even though they had the same underlying meaning in mind. These same groups, in somewhat different circumstances, may also use the same wording even though they intended their wording to mean different things. Such communal differences needed to be resolved in order for the resulting guidance which they produced to be coherent.

   As a classic example, consider the terms 'verification' and 'validation', which traditionally have many different definitions and interpretations. In DO-178B, verification is defined to be the 'evaluation of the results of a process to ensure correctness and consistency with respect to the inputs and standards provided to that process'. In DO-178B, verification is one of the integral processes (configuration management, quality assurance, and certification liaison), which means it cuts across all the development processes, rather than being a serial activity which begins after a predecessor activity. Validation is defined as 'the process of determining that the requirements are the correct requirements and that they are complete'. Since validation is considered a system-level activity, it is not explored further in DO-178B. Definitions such as these form a solid foundation which are used to tie the other sections of DO-178B together, but need to be established up front so that the content built upon these definitions could be integrated together with a minimum of rework.

   Such verification is a part of the checks and balances which DO-178B provides to protect against mistakes. Added protection is required when this verification must be performed independently, which is defined in DO-178B as follows:

   Separation of responsibilities which ensures the accomplishment of objective evaluation. (1) For software verification process activities, independence is achieved when the verification activity is performed by a person(s) other than the developer of the item being verified, and a tool(s) may be used to achieve an equivalence to the human verification activity. (2) For the software quality assurance process, independence also includes the authority to ensure corrective action.

   Once definitions such as the above were established, key concepts could then be built from this foundation. As an example, consider the document's treatment of software requirements, which are described as "a description of what is to be produced by the software given the inputs and constraints". They take two forms in DO-178B: 'high-level' and 'low-level' requirements. High-level requirements are defined as "software requirements developed from analysis of system requirements, safety-related requirements, and system architecture", and low-level requirements are defined as "software requirements from which source code can be directly implemented without further information". This definition can obviously be interpreted differently (and thus require further elaboration of low-level requirements) when applied to a new-hire and an experienced engineer, which could drive up costs unnecessarily if this elaboration proves unnecessary once the coding responsibility is assigned. The importance of independence in verification is also highlighted in this situation to ensure that the low-level requirements are taken to the required level.

6. Development process objectives and interfaces were baselined

   Compliance with DO-178B is intended to establish confidence in the activities used to develop safety-critical software, and assuring that these activities are producing information suitable for certifying the systems which the software is embedded within. DO-178B's life-cycle processes are not intended to prescribe or limit the structure of the actual development processes which are put in place by development organizations ; instead, the processes described in DO-178B are a descriptive vehicle for communicating expectations about the activities actually performed during development. While these actual activities may take many different forms, DO-178B defines its own set of software life-cycle processes so the actual activities can be traced to the DO-178B processes.

   A set of objectives for the DO-178B life cycle processes are also defined. These objectives define the 'intent' behind these processes, and thus describes the value which each of these processes are expected to produce, in a manner similar to how a judiciary may consider legislative intent in determining their rulings.The objective evidence necessary for demonstrating that these objectives have been accomplished is also described. For example, testing objectives are defined as showing that requirements are fully satisfied, and demonstrating 'with a high degree of confidence that errors which could lead to unacceptable failure conditions... have been removed'. There are obviously many different approaches which can be employed to achieve such objectives, but the proof of their pudding is whether they satisfy these objectives or not. DO-178B objectives also help verify the correct implementation of safety-related requirements that flow from the system safety assessment.

   Compliance data is intended to be produced as a natural product of these activities, but unlike many other standards, DO-178B does not prescribe how these activities should be structured. Interpretation and judgment need to be applied in determining the activities and deliverables appropriate to each situation in order to satisfy both DO-178B process objectives and criteria, as well as any business objectives that must also be satisfied (which are outside of the scope of DO-178B).

7. An evolutionary development plan for the document was outlined

   Concepts in any standard need to be developed and communicated in an organized and sequenced fashion so progressive elements can be used to develop the reader's understanding as the document unfolds. A top-level document structure was derived from a concensus view of how certification programs would be described. An outline of the overall document was established, and target page counts were established for different sections, with the overall objective of keeping the length of the document to approximately 75 pages. The thinking was that more writing does not generally lead to clearer writing.

   This content for this structure had to be developed incrementally by the responsible work groups, and this material thus evolved over many meetings in which the working group members held the majority of their team's interactions. As content for the document progressed through position paper development to working-group agreed-to proposals for content, and presented that content to plenary, it was apparent that the underlying concepts often did not yet 'stand alone', and had to be explained by the working group chairs to the rest of the plenary members.

   For example, a key change was made in DO-178B in how requirements for software verification were to be defined, with a greater emphasis on requirements-based testing than in DO-178A. This change was intended to discourage the practice of focusing software testing primarily to achieve structural (i.e., White-box testing) coverage, at the expense of fully testing functional and performance requirements. Requirements-based testing - augmented by a determination of the structure coverage achieved by these functional tests - is a more cost-effective and meaningful way to conduct such verification testing.

   The emphasis on requirements-based testing resulted in a new and significant focus area in DO-178B, which expects consistency between between requirements, code, and the tests that verify that the code satisfies the requirements. Traceability is often used to provide evidence of this consistency and completeness, and such traceability is required to be thorough and bidirectional, demonstrating that requirements trace forward to code and tests and satisfy requirements, and trace backward from tests to code and to the requirements for which the tests were created. At the higher levels of safety and criticality, DO-178B requires that this traceability evidence show 100% structural code coverage. These concepts were each developed within different working groups, and were only able to achieve consensus after multiple iterations in the document test - to introduce the idea, integrate its expression, and mature its elaboration. A roadmap of these concept developments and refinements over time (which initially were communicated in the form of working group papers) was helpful to begin developing realistic projections for how long DO-178B would actually take to complete.

8. A repository was established for sharing information and status

   An accessible means of exchanging and providing access to information was put in place to support the committee activities. This mechanism helped improve configuration control of document elements, provided protected access to working groups for the working group papers that were not ready for broader review, and helped build a community of practice to identify concerns and leverage community knowledge in trying to address them. The repository that was established was funded in a creative manner, with software licenses paid by one airframe manufacturer, the hardware paid by another, and the server and associated operations provided by a university which was actively involved in the committee activities. This repository has been so useful that has been upgraded many times, and remains in use today.

9. An editorial team was established to integrate content into a coherent product

   While the content for the document was developed by many individuals, it was important for the resulting integrated content to be written as if it had been the work of a single author. Since the document was to be international (and would thus need to be translated into other languages), care needed to be taken to ensure that the language and structure of the document was amenable to this translation, and that it could be accomplished without changes in the underlying meaning. As an example, the process of progressively refining requirements was described by some authors as 'decomposing requirements', but such terminology, while often used within the United States, would not translate well into French (essentially becoming 'rotting requirements'. Finally, the team needed to establish trust with the original content providers, who wanted the intent of their original writing to be preserved. Over time, what this meant was that responsibilities for the document sections transferred from individual working groups to the editorial team for final integration.

10. A formalized review process was employed to assure quality criteria were realized

    The editorial team employed a structured review process, including a specific checklist designed for such reviews. As these reviews identified issues with the document, the working groups were initially responsible for addressing these issues and determining if the resolution required a change in a technical concept, or how it was explained. As the document entered its final form, change authority was delegated to a smaller and smaller number of individuals, to ensure the tone and style was consistent, as the number of 'must fix' open issues was driven to zero.

Work on position papers for the document began immediately after the first meeting, using the working group structure that was established in plenary sessions. The first drafts of DO-178B began to be produced in 1989. Work continued through quarterly meetings, with intervening publication of document updates, exchanges of issues and working group papers, and progressive refinement and convergence on the agreed-to guidance. The final release was adopted in 1992, after the responsibility for the document text was transferred to an editorial board who conducted intensive sessions to thoroughly review the document, reconcile outstanding issues, and prepare the document for publication. The resulting product has achieved wide acceptance, and remains in active use after nearly twenty years on all new and derivative commercial aircraft developments around the world. The document's usability, clarity, and compatibility with other standards has been well received.

A new revision, DO178C, was initiated in March 2005, and committee work continues on this revision. Refinements have been requested to provide guidance on emerging technologies since the last revision, including use of object oriented technologies, software modeling, and systems and software boundary issues, but generally, D0178C will retain intact nearly all of the original content and structure from DO-178B. Since this material has now been widely used in practice for nearly twenty years, the above practices collectively offer advantages over less formalized approaches that have been used in other standards efforts. However, practices are only effective when used by the right people, and the leaders and contributors to DO-178B can remain proud of their contributions to the practice of software engineering in safety-related systems.