Home > Publications

Measuring up: How good are the Government's data systems for monitoring performance against Public Service Agreements?

Every two to three years the Government undertakes a Spending Review, agreeing the financial settlements for each Government Department. As part of this process Departments and the Treasury negotiate Public Service Agreements (PSAs). PSAs set objectives for the priority areas of Government's work and are a public agreement of the performance to be achieved for Government Spending.

Following the recommendation set out in Lord Sharman's 2001 report, the Government, in March 2002, invited the C&AG to review the reliability of data systems underlying the reporting of progress against PSAs.

The objectives of the work are to:

provide Parliament and the public with assurance that Departments are operating sound data systems for the purposes of monitoring and reporting progress against their PSA and;
help improve the quality of Departments' data systems and support better performance management.

Good quality data is essential to track progress against specified indicators and we review the data systems providing the data.

The review considers four questions:

Does the organisation have appropriate policies and practices in relation to data quality?
Does the data system measure the intention of the objective?
Does the data system operate well?
Does the data get reported in a way that can be fully understood?

On the basis of our review, we provide a 'Traffic Light' rating and a conclusion for each data system, indicating whether the system is:

fit for purpose (green);
broadly appropriate but needs strengthening (amber); or
not fit for purpose (red).

Rating	Meaning …
GREEN (fit for purpose)	The data system is fit for the purpose of measuring and reporting performance against the indicator
GREEN (disclosure)	The data system is appropriate for the indicator and the Department has explained fully the implications of limitations that cannot be cost-effectively controlled
AMBER (Systems)	Broadly appropriate, but needs strengthening to ensure that remaining risks are adequately controlled
AMBER (Disclosure)	Broadly appropriate, but includes limitations that cannot be cost-effectively controlled; the Department should explain the implications of these.
RED (Systems)	The data system does not permit reliable measurement and reporting of performance against the indicator
RED (Not established)	The Department has not yet put in place a system to measure performance against the indicator

Departments can prepare for the Data System Validation by undertaking a Self-Assessment.

Previous findings

Earlier NAO published reports on Validation of Data Systems were:

In December 2007, the Fourth Validation Compendium Report Volumes I and II were published.
In December 2006, the Third Validation Compendium Report Volumes I and II were published.
In March 2006, the Second Validation Compendium Report was published.
In March 2005, the report Public Service Agreements: managing Data Quality - Compendium Report was published.

Validation reports for individual Departments and PSAs under the Comprehensive Spending Review 2007 will be included on this site when they are finalised.

Useful Links:

HM Treasury website: The Comprehensive Spending Review 2007 – HM Treasury website

Independent Reviews of reported CSR07 Value for Money savings