Some 75 per cent of the data systems government departments use to report progress against their Public Service Agreements (PSA) 2005-08 targets are broadly appropriate, but fewer than half of these were fully fit for purpose. The National Audit Office has also found there have been no significant improvements in data systems since previous analysis in 2004.
The NAO has provided details of 65 data systems, used to measure progress against 46 PSA targets, across six departments: Cabinet Office, Department for Culture, Media and Sport, Department for Education and Skills, Department for Environment, Food and Rural Affairs, Ministry of Defence and HM Treasury. Some three-quarters of the systems developed by Departments provided a broadly appropriate basis for measuring progress against their PSA targets, but more than half of these required some improvement to strengthen measurement or reporting arrangements.
23 per cent of systems had weaknesses in either their design or operation; and for 17 per cent of systems, Departments had not explained adequately the impact of unavoidable limitations in the quality of their PSA data. In all departments except HM Treasury, at least as many systems needed improvement as were fit for purpose. Furthermore, in nearly 20 per cent of cases Departments’ systems were not fit for monitoring progress on the key elements of their PSA targets or the systems had not been established at all.
The NAO has recommended that staff responsible for managing PSA data systems should assess the risks to data quality in the systems and also need to consider the implications for measurement and data systems at the time PSA targets are designed. Staff who are responsible for a Department’s data quality policy should raise the profile of PSA data systems by setting out clear expectations and reporting standards. They should also actively monitor PSA data quality and ensure that there is adequate challenge to outturn data.
In addition, the Treasury needs to challenge Departments’ measurement arrangements early in the process of developing new PSA measures and targets; review how Departments respond to NAO validation and recommendations for strengthening PSA data systems; and provide measurement guidance to Departments, setting out HM Treasury’s expectations for PSA data quality.