Printer Friendly

Performance measurement and manipulation: Colin Fisher and Bernadette Downes discuss their research into the ways in which metrics are massaged in the public sector.

[ILLUSTRATION OMITTED]

Public-sector organisations commonly use performance measures (or metrics), league tables, star systems and targets for performance management purposes. Occasional newspaper reports and informal discussions with public-sector managers suggest that managers sometimes respond to these systems by engaging in metric manipulation--ie, fiddling the figures.

We investigated the circumstances in which public-sector managers might be tempted to do this and wrote our research report for CIMA. We hoped that identifying the triggers of metric deception could be used to find ways to improve the governance of accounting-based performance measurement schemes.

The research was conducted largely in UK National Health Service trusts, although some was done in local authorities. We wanted to find out what made managers more or less likely to fiddle the figures and what influenced the seriousness of the manipulation. Several forms of manipulation were identified. At the most serious level, people put data they knew to be untrue into performance measurement systems. But other forms of manipulation, starting with the most serious, comprised:

* Manipulation by reclassifying data--for example, delaying adding patients to a waiting list.

* System manipulation (also known as gaming): exploiting loopholes in the rules of the performance management system.

* False implication: presenting information to encourage erroneous conclusions.

* Being economical with the truth.

* Distraction: hiding less favourable statistics in a confusion of other data.

The last two are perhaps not a major ethical concern and may even be an acceptable and necessary part of an organisation's internal processes.

We found that metric manipulation occurred when certain mechanisms were triggered, depending on the particular circumstances of the case. We identified eight mechanisms and five triggers. Rather than describing all the combinations of these, the following story, based on one of our interviews, demonstrates one way in which a situation can trigger mechanisms that result in metric manipulation.

Our respondent was a nurse manager in a hospital trust. She and her colleagues recognised that the performance measurement system in place was intended to help the government monitor and control NHS trusts. But it was also designed to be useful to the trust's management. The trust's informal culture was not sympathetic to metric manipulation: managers believed that it was better to find out the causes of poor results and fix them.

The quality of care for long-term patients was an important issue in the Department of Health's agenda. One statistic used to measure this was the hospital re-admission rate. The primary care trust (PCT) sent our respondent some statistics showing that it had met its target of cutting the re-admission rate by a certain percentage. She was sceptical, though, because her own monitoring indicated that the situation had become worse. But she recognised that when two trusts calculate the same performance measure using different systems and data sets it is likely that their figures won't tally. The sensible thing, she thought, was to contact the PCT and propose a meeting to reconcile the two trusts' figures and produce reliable information.

[ILLUSTRATION OMITTED]

She was shocked, therefore, when the PCT replied that a meeting would not be worthwhile. After discussing this with the PCT manager she formed the impression that he was not interested in using the measures to improve the internal operations of the service. The PCT was interested only in presenting good news to the Department of Health. If the figures suggested good news its managers saw no point in challenging them especially if this meant they might turn into bad news. They wanted to "feed the beast" and so avoid interference and questions from bodies higher up the NHS hierarchy.

In this incident the deception was that the PCT was economical with the truth--managers were dubious about the figures but kept quiet. This mechanism, which we called "avoiding hassle", was triggered by two circumstances: the acceptance of metric manipulation within the PCT and the belief that the measures were an externally imposed nuisance that were of no internal value. In the hospital trust, our respondent would not have manipulated the data because it would have been unacceptable in her organisation's culture, and because she thought that the performance measures could be used to improve services.

Our research was not designed to identify the prevalence of metric manipulation in the NHS, but it did indicate that, although the temptation to manipulate could be high, the manipulation that occurred was mostly at the less serious level.

Of the circumstances that could trigger the mechanisms, the most common were the two in our case study: an organisation's cultural tolerance of metric manipulation and a view that the targets and performance measures were externally imposed and had little internal value.

Our report closes with some suggestions for changes to the governance of performance management systems that could reduce the temptation to massage the metrics. It also suggests that a code of ethics could provide guidance about when actions that are seen as legitimate in the negotiating and inspection game become unacceptable deceptions. Most of the recommendations are not quick fixes, but medium-term approaches designed to make metric manipulation less attractive.

The government has recognised that using performance targets too stringently can have a negative effect. If a reduced number of targets were to be used as indicators rather than inflexible obligations, managers would use them more honestly. This would require trusts to use performance measures as tools for improving internal management as well as for reporting to government. Keeping staff aware that managers are constantly alert to metric manipulation reduces people's temptation to fiddle the figures. Most trusts adopt the idea of balanced scorecard reporting to some degree, but embedding this in trust management culture would also reduce the temptation to manipulate.

Further information

A CIMA Mastercourse entitled "Introducing the Forecasters' Toolkit" will be led by Bijan Tabatabai in London on November 18, For details, visit www.cimamastercourses.com. FM's October article entitled "Improving forecasting" stated that he had worked with IBM to develop the toolkit. In fact, it was Unilever.

Colin Fisher is professor of managerial ethics and values at Nottingham Business School, where Bernadette Downes is a senior lecturer in HR management. Visit www.snipurl.com/48jqx to download their paper, "Performance measurement and metric manipulation in the public sector".
COPYRIGHT 2008 Chartered Institute of Management Accountants (CIMA)
No portion of this article can be reproduced without the express written permission from the copyright holder.
Copyright 2008 Gale, Cengage Learning. All rights reserved.

Article Details
Printer friendly Cite/link Email Feedback
Title Annotation:technical matters
Author:Fisher, Colin; Downes, Bernadette
Publication:Financial Management (UK)
Date:Nov 1, 2008
Words:1031
Previous Article:Accounting in the hospitality sector: Ruth Mattimoe and Will Seal tell Louise Ross that their study of UK restaurants and hotels won't produce a dry...
Next Article:Financial risk management: Alexa Michael sets out the tools and techniques available for firms to mitigate the worst effects of the credit crunch and...
Topics:

Terms of use | Privacy policy | Copyright © 2018 Farlex, Inc. | Feedback | For webmasters