How the Department for Culture, Media and Sport assessed the BBC’s efficiency as part of the licence fee settlement

Executive Summary

National Audit Office Value for Money Report

1. As part of the process of setting the level of the television licence fee from April 2007, the Department for Culture, Media and Sport (the Department) has to make an assumption about the level of efficiency savings the BBC can deliver. The BBC, as part of its licence fee bid, set out that its ongoing annual efficiency target should be 80 million of new savings each year from 2008 (approximately 2.5 per cent of the baseline expenditure in 2008-09). The savings to be generated through efficiencies at the BBC, coupled with the BBCs estimate of its funding requirements, will strongly influence the level of the new licence fee.
   
   
2. The Departments decision on the BBCs efficiency and the overall licence fee settlement will be based on its own assessment of the BBCs requirements and is not dependent on negotiations with the BBC. The Department has commissioned consultants to advise it on the BBCs efficiency and has welcomed submissions from interested parties, including the BBC. In July 2006 it sought the views of the National Audit Office on the adequacy of the evidence base available to it on the efficiencies the BBC might reasonably be expected to make. We agreed to report on whether we consider the Department has reasonable grounds on which to base its assessment of the BBCs scope for efficiency savings.
   
   
3. We have considered the extent to which the different approaches taken by the consultants are likely to provide the Department with sufficient, relevant and reliable evidence of the nature, scope, ambition and deliverability of the BBCs efficiency and value for money plans. As the basis for our work was to consider the evidence available to the Department, we have not re-performed any benchmarking work on the BBC or carried out any direct review of BBC activities. It has not been part of our work to identify a particular efficiency target for the BBC.
   
   
4. The Department now has available to it a considerable body of evidence:
   
   
   • The BBCs own analysis of what is achievable, in the light of what the BBC will have achieved by 2008 under its three year value for money programme. This is based on internal BBC reviews of its costs across all business areas. The BBCs fundamental assumption is that by March 2008, when it is due to have completed its three year value for money programme, it will have matched best practice in current technology and work practices and therefore be at the efficiency frontier (where best practice in efficiency has been adopted across the organisation), at which point further efficiencies can only be achieved through adoption of technology or working practices yet to be developed. The BBCs analysis is supplemented by the work of Oxera, consultants initially commissioned by BBC management to establish whether its efficiency target beyond 2008 appeared appropriate. Oxera set out that because in 2000 PKF had stated that the BBC was at the efficiency frontier and that the BBC was on track to improve significantly on the targets set at that time by the Department (informed by PKF), there was evidence to suggest that the BBC would be at the efficiency frontier by the end of the value for money programme in 2008. Oxera considered that the BBCs target beyond 2008 was in line with external benchmarks and would be more challenging than many of them.
     
     
   • The work of PKF, commissioned by the Department to consider the BBCs efficiency achievements against targets set in 2000 and efficiency assumptions underpinning the BBCs assessment of its future funding needs. PKF had previous experience of working at the BBC as part of the review leading to the previous licence fee settlement. Based on its review of the BBCs achievements and plans for further savings and its professional judgement and experience, PKF concluded that the BBC would not be at the efficiency frontier by 2008 by way of its value for money programme, and that it could therefore deliver a higher level of efficiency savings than set out in the BBCs bid.
     
     
   • The response of the BBC to the PKF report. The BBC does not agree with some of the conclusions of the PKF report and, in particular, PKFs estimate of the efficiencies the BBC could deliver, which is higher than the BBCs estimate because the BBC believes it is based on judgement rather than evidence or analysis. The BBCs concern is that if its funding were based on unachievable efficiency targets it could face detrimental impacts on the quality and range of its programmes and services.
     
     
   • The work of PA Consulting, commissioned by the BBC Governors to review the achievability and deliverability of the BBCs value for money plans, to review the BBCs licence fee bid and to comment on PKFs draft report. PA Consulting concluded that the BBC was beginning to deliver the value for money programme and that the target of 80 million in annual savings beyond 2008 was stretching but achievable. However it also noted that the changes proposed in the BBCs programme spending divisions and in some of the professional services divisions were not yet transformational (where the BBC would match best practice in current technology and work practices) and the absence of transformational change limited the efficiencies that the BBC could deliver.
     
     
   • The work of Magentum, consultants commissioned by the BBC and some other UK broadcasters to compare the price per hour paid by the broadcasters for a range of UK commissioned television programmes. The data for the programme prices used for comparison pre-dated the BBCs value for money programme which aims to cut the cost of content programming by up to 15 per cent by March 2008. Magentums work showed that, on average, the BBC paid more per hour for programmes than the other broadcaster comparator in nine out of the ten programme categories analysed. However, if prices are adjusted in those categories which Magentum considers are most likely to be affected by the longer programme length needed to fill a BBC hour, then the BBC pays more in eight out of the ten categories analysed. Based on this adjusted price, the price paid by the BBC is also more than five per cent greater than the other UK broadcaster comparator in six of the ten categories (representing 29 per cent of expenditure analysed).
     
     An important part of the Magentum study was to analyse the impact of key production components and editorial decisions on prices paid across all categories. As the analysis was carried out at a programme level, there were no broad conclusions to be drawn about the different prices paid by the BBC in each programme category. However, the BBC asked Magentum to carry out further work on the factual programme categories to compare prices at a more detailed level in order to understand better the factors underlying the differences in price paid in these categories. Magentum considers the results for the categories in this further analysis indicate the BBC is commissioning programmes which have a more expensive set of production components and, that overall, these programmes may comprise a more complex set of production factors and higher level of editorial ambition than those commissioned by some other UK broadcasters. The BBC intends to conduct more work to assess whether the higher prices are justified or whether this represents a further opportunity to drive efficiency.
     
     
   • Work completed for the Department by academics at Nottingham University Business School and Nottingham Trent University looking at productivity trends within the broadcasting sector. The work covered the BBC but did not identify the BBCs position relative to the rest of the broadcasting sector. Although the authors highlighted a number of caveats, all their estimates suggested that the broadcasting sector had experienced positive productivity growth in recent years.
     
     
5. In summary, the evidence available to the Department provides detailed analyses of the BBCs performance against targets set in 2000 and the range and deliverability of its current and future efficiency plans. However, determining how efficient an organisation is and can be, in absolute and relative terms, is fraught with difficulty. It is not a precise science, and ultimately is a matter of judgement.
   
   
6. In considering the evidence available to the Department we have been conscious of the added value brought by credible external assessments of organisations, particularly those by independently commissioned consultants such as PKF and PA Consulting, which have thoroughly considered, from the inside of the organisation, the BBCs current state of efficiency and efficiency plans. The credibility of such assessments is enhanced where they independently arrive at similar conclusions. Such work, combined with the external comparisons brought by Oxera and Magentum, gives the Department a good picture of the BBC.
   
   
7. While it is always possible to do more to inform the judgement, our principal conclusion is therefore that the Department now has adequate evidence from which to assess the broad level of efficiency the BBC could reasonably be expected to achieve during the period of the next licence fee settlement.
   
   
8. A successful efficiency programme must be able to demonstrate that the quality of service delivery has not been adversely affected. None of the consultants attempted to measure the quality of BBC outputs as it was outside their terms of reference. It will be for the Department, in its wider consideration of the licence fee settlement, to consider the quality of BBC output.
   
   
9. While the evidence now available to the Department is adequate for its purpose, some of this evidence was arrived at through work commissioned by the BBC. The Department considers it commissioned an adequate evaluation of the BBCs efficiency but welcomed all contributions to the subject from any source, including those commissioned by the BBC. While there is a proper tension between the Departments roles in knowing enough about the BBC to make a decision and not having an intrusive performance management framework in place, the Departments distance from the BBC may complicate its acting as an intelligent customer when commissioning and evaluating work on the BBCs efficiency.
   
   
10. While the Departments decision on the BBCs efficiency will be informed by the evidence available to it from all sources, there are several ways in which the process of assessing the BBCs efficiency could be strengthened. These would include:
    
    
    • consulting with the BBC and others on appropriate comparator organisations and performance indicators in advance of any assessments and building them in to the terms of reference for consultants to minimise subsequent disagreements; and
      
      
    • the Department alone engaging any consultants thought necessary to take forward assessment or comparator work, although it cannot of course prevent others, including the BBC, from commissioning such assessments.
      
      
11. There may also be wider lessons for the Department and the BBC Trust to learn from economic regulators such as Ofgem and Ofwat, who have to consider similar issues when they determine outputs and set prices for the regulated utilities.