A team should measure progress by successful completion of explicit integration objectives, including demonstration of desired product quality, not just be schedule-driven. To focus on schedule alone will ultimately create a bow-wave of work, and an overly complex environment in which to troubleshoot problems in which nothing works, and so everything must be suspect in every situation.

Unfortunately, as integration proceeds, schedule pressures naturally increase. As a result, retaining a parallel focus on quality can be very difficult to do. A measurement system that tracks and reports the status on the total quality picture, in light of the state of all testing, is crucial to providing decision-makers with the right information to pick the right path forward as integration proceeds.

During integration, there are several different kinds of verification activities that are being performed:

1. Verification activities driven by requirements (sometimes called 'requirements verification'), and whose primary purpose is to assure that all requirements have been incorporated into the development processes of individual components, and attempt to get the requirements set as stable and high quality as possible going into the design effort.
2. Verification activities used to assess development processes and the resulting products of those processes, and thus assure the requirements are properly being elaborated into robust designs that will be consistently produced, perform as expected, and not have undesirable behavior. Such activities are often called 'design verification', and focus on defects introduced during the development process, attempting to detect such defects as early as possible. Since these development processes often span many different groups, and since the components being developed often interact in unanticipated ways, the primary purpose of this verification is thus to produce information about these development processes and associated component maturity.
3. Verification activities that are driven by manufacturing, deployment, and installation provisions. Such activities are sometimes called build verification, and are intended to detect defects unique to installation of a component into the corresponding system, and that may be injected by that process (for example, improper installation procedures, incomplete execution of installation steps, etc).

Additionally, validation activities are also typically being performed, and should be driven by realistic, end-to-end operational usage scenarios, to tie together the components into a cohesive whole and confirm the collection of development and verification processes are all producing the desired result. Such validation activities are typically defined and performed by members of an independent, but representative user community, rather than by the original development team itself, and thus provide a degree of independent assurance that things are proceeding per plan. All of these activities require careful process management and oversight, to ensure the desired results are achieved; see descriptions of verification practices and validation practices which are typically utilized to perform such oversight.

These verification and validation activities are usually performed in a variety of different planned test sites, facilities, and analysis environments, and are sequenced in a pipelined (but overlapping) fashion, so that the defect detection and removal processes are most efficient and effective. Each of these environments will typically have their own set of stakeholders, perspectives, and challenges. Generally, these activities will be performed according to a pre-planned, overall verification strategy, which provides for a means of looking across all these activities in order to develop an overall assessment of both the effectiveness of each activity, and what the impact is likely to be on what will be produced at the end.

Metrics are typically used to assess progress against these objectives, and often are deployed over time, according to the business goals which are important to consider at different points in the various integration cycles. An organized approach to defining such metrics is to work through a 'Goals - Questions - Metrics' (or GQM) framework, to ensure that the business purpose for the metrics is clear and understood, and that the metrics themselves are designed to meet those business goals.

Below is an example structure and representative metrics set for a large software-intensive integration effort; obviously, not all of these will be relevant to all phases of such an integration effort, and may not all be appropriate to other development efforts, but are offered to stimulate thought about how overall activities might be instrumented for decision-making. A detailed discussion of how metrics can be effectively used in a data-rich environment can be found here.

While the number of measurements in the above list may seem overwhelming, the goals and questions can actually all be satisfied with a relatively small set of information about each defect and how and when it was detected, isolated, and resolved, i.e:

1. Defect characterizations- when and where it was detected, and how serious it is
2. Information about what it took to find and repair these defects - time and effort
3. Information about the component in which the defect was introduced - how big, how complex, and against which version (so statistics like injection and removal rates of defects per thousand lines of code can be calculated)
4. Information to relate the defect to the individual development and verification processes which were used to uncover the defect, to assess their efficiency and effectiveness

Often a single capture of data can thus be used to aggregate information and thereby answer multiple questions and contribute towards gaining insights for multiple goals. This can only be achieved if well planned, however, as even the above minimal set itself can have significant variation in the costs of collection, accuracy and uniformity of information, and resulting usability of the information that is produced. For example, tracking the costs of rework is often difficult to collect over time, without recording per-defect efforts of many people (in hours or minutes) for their individual participation. For this reason, such measures are typically best done as subjective assessments, such as 'easy (<1 person-hr), typical (1-8 person-hrs), substantial (8-40 person-hrs), and major (>40 person-hrs), rather than absolute measures, to ease this burden, while preserving the ability to meet the original business goal. However, even with bucketing, such assessments, while very important, can be difficult to analyze across organizations, tracking systems, and processes, without considerable planning.

Many groups often prefer to just collect and count the minimum set of information (type 1 above). While this may be desirable, if this path is chosen, it will be difficult to use that information to manage the quality of the product proactively and make accurate predictions of the future. Instead, all you will do is report on the past, without being able to predict the future.

Of course, you can always start with this basic information, and expand what you collect over time... though if you make this choice, you may not be able to go back and analyze information from previous phases when you didn't collect this data, and thus the reliability of your predictions will take more time to validate and improve. Always collect at least three points before presuming that a trend has been established.