How good is that performance - really?
This is a great book for all managers that are involved in any measurement program that is intended to influence behavior. The book was developed from an award-winning doctoral thesis at Carnegie Mellon University, and has been a part of SEI's curriculum in their measurement courses, as examples of what not to do. The book is filled with both amusing and disturbing examples of measurement 'dysfunction'. The author identifies three different types of performance measurement - for motivation, process improvement, or process coordination. It is measurement for motivation that causes the greatest dysfunction.
For example, if you record the fact that 10 widgets are produced on machine A and are comparing this against the 10 widget benchmarks for rewarding performance bonuses, or other kinds of individual incentives, it is very likely that other perspectives like quality will suffer in the drive to make the 10 widget goal. If you record the fact that 10 widgets are produced on machine A while only eight widgets are produced during the same time using a competing technology on machine B, this is measurement for process improvement, and can be very useful - but only provided it is limited in scope and used purely for the stated purpose. In practice, this is an extremely difficult thing to restrict, and to do accurately and consistently. Finally, if you record the fact that 10 widgets are produced on machine A and convey this information to the widget packaging department to ensure that enough widget cases are ready, this is measurement for process coordination, and is also potentially useful on its own.
The book offers good recommendations about developing effective performance measurement systems. The focus must be on people and how they react when they are part of organizational systems that are being measured.
Interviews enrich the text, conducted with eight recognized experts in the use of measurement within the software development, including David Card, Tom DeMarco, Capers Jones, John Musa, and Lawrence Putnam, all with interesting insights and great reputations.
The book presents an idealized model of how to implement measurement in a well-managed organization. Instead of looking at an organization made up of thousands of employees and a few hundred managers arranged in a hierarchy, Austin's model consists of three roles: a manager, an agent (employee), and a customer who buys the goods or services provided by the agent under the supervision of the manager.
The model assumes that an agent's job consists of performing two activities and presumes that the customer will only be happy if the agent performs both well. The author examines the cases where the manager can monitor neither of the two activities, where the manager can monitor only one of the two activities, and the situation where the manager can monitor both activities. According to the model and associated research, the agent will behave differently in all three of these cases. If the manager cannot measure either activity, then the situation is effectively delegated management, and the assumption is that agents want to work well, and that they are not trying to work benefits by exerting the least amount of effort (though in practice, over time, this often happens). If the manager can measure both activities, then the activities are fully managed, and success is possible. If the manager can measure only one of the two activities, we have a dysfunctional model. According to the author, such incomplete measurement is not only useless, it is dangerous since it motivates agents to make efforts only for what is being measured (and rewarded), at the expense of what is not. For example, if a help desk line measures performance by the number of calls an employee takes, then employees are motivated to spend very little time per call. The customer is left dissatisfied, but the measurements show that the agent is providing first class results.
The book suggests one method, probabilistic measurement, to mitigate such dysfunction. If such dysfunction comes from being unable to measure everything an agent does, e.g. you just can't have your managers listen to all help desk calls, the manager can carry out random samplings of performance, by recording all the calls, and listen to a random selection of them each day. The agent will then expend effort along those dimensions that cannot be completely measured simply because he knows they might be evaluated through the sampling. This is how many mature call centers now monitor the effectiveness of their support personnel.
All in all, the book is an effectively simplified model of individual and organizational behavior, in response to measurement, which has initiated a healthy and constructive debate about the value of activities such as maturity models in initiating meaningful change within organizations or industries.
