Executive Summary

National Audit Office Value for Money Report

1. The term Civil Society encompasses all aspects of society which are independent of the state and the private sector. Within that broad concept, Civil Society Organisations (CSOs) are bodies organised around shared interests and values. They include non-governmental organisations (NGOs), trade unions, faith groups and business associations. There are many such organisations worldwide which vary enormously in their size and scope. Large international NGOs such as Oxfam or Christian Aid are mainly based in developed countries, while national NGOs and local collaborative or mutual assistance groups are based in developing countries.
    
2. The Department for International Development (DFID) is responsible for leading the United Kingdom governments contribution to reducing global poverty in accordance with the Millennium Development Goals. [Footnote 1] DFID works in partnership with a range of bodies, including CSOs. A white paper Eliminating World Poverty, published in 1997, promoted partnership working between DFID and voluntary organisations to create stronger public and international support for development. DFIDs work with CSOs has since expanded rapidly, with expenditure more than doubling since 1997. By 2004-05, DFID channelled 328 million, some nine per cent of its total expenditure, through CSOs. This consisted of grants to UK and international NGOs (often themselves working in partnership with local CSOs in developing countries), partnership agreements with key NGOs and in-country arrangements with local CSOs (Figure 1 ). This Report focuses primarily on funding through DFID country teams to local CSOs, and on the two largest UK based schemes. These are the strategic level Partnership Programme Agreements, held with 18 key partners at the time of our audit, and the Civil Society Challenge Fund through which DFID engages with a wider range of UK CSOs. Appendix One provides further detail on these two funding schemes.
   
   
   Figure 1 ("Overview of DFID's support to CSOs") is unavailable in this version of the executive summary.
   
    
3. Civil society organisations can play a variety of vital roles in development, which make them important partners for DFID in many countries. They can help to hold developing country governments accountable for poverty reduction, give voice to the concerns of poor people, secure access to government services for marginalised groups, and promote awareness in the UK and globally. Where CSOs empower citizens and challenge host government actions, they can provide some assurance to donor nations on the effectiveness of aid channelled through national systems. This assurance is particularly relevant in countries where donors, including DFID, are aligning their assistance programmes with developing nations own poverty reduction strategies, often through direct support to these nations budgets.

   
   Challenges of engagement
    

4. CSOs bring many development benefits, but also challenges for DFID in engaging effectively with them:
    
   • Some CSOs are keen to maintain their independence from state and donors;
      
   • Local CSOs are often small, informally organised and only have the capacity to spend small amounts of donor funds;
      
   • Host country governments can be reluctant to allow funding in sensitive areas; and
      
   • International donor funding may develop the capacity of CSOs, but it might also cause them to become more distanced from the poorer people they represent.
      

   All of this has led to varied funding arrangements by DFID to support the different needs of CSOs.
    

   Our study
    

5. Against this background, we examined how DFID is seeking to promote effective working partnerships with CSOs in support of poverty reduction objectives. Our review included:
    
   • DFIDs view of the role of CSOs in international development (Part One);
      
   • How DFID engages with CSOs (Part Two); and
      
   • The achievements of CSOs in reducing poverty (Part Three).

   Our methodology included visits to four countries for discussions with DFID staff, CSOs and government officials, as well as visits to 19 projects involving CSOs and file review of another 40 projects and Partnership Programme Agreements. 90 UK CSOs also responded to a survey we commissioned to obtain their views on how DFID engages with them. Our full methodology is at Appendix Two. We have previously reported on DFIDs response to humanitarian emergencies, which is not therefore covered in this report.
    

   Our conclusions and recommendations
    

   Working with CSOs to achieve development objectives
   
   Communicating a role for CSOs
    

6. The Departments objectives and performance targets focus on its efforts towards achieving the Millennium Development Goals for poverty reduction. DFID has no specific performance objectives for its work with CSOs at the level of its Public Service Agreement, as these organisations are seen as a means through which to secure its developmental objectives rather than an end in themselves. It does however, set out some specific objectives at lower levels in the DFID performance framework. The decentralised nature of DFIDs organisation makes it important for the UK office to provide clear guidance to country staff regarding its expectations for working with CSOs. A range of guidance exists on DFIDs intranet, explaining the important role which CSOs can play, and the need for country staff to understand this role. DFID has recently sought to consolidate this through a position paper on engaging with CSOs, with information on relevant funding schemes. However, DFID does not provide in-depth guidance on how to assess civil society in-country.
    
7. The country plans for all four of the countries we visited recognised a role for CSOs. We also identified three DFID country teams which developed specific strategies for engaging with CSOs, in addition to their country assistance plans. Country circumstances vary and DFID has a decentralised and country partner-led approach so it has not provided a fixed framework for country offices to apply. But our review identified that even taking situation-specific factors into account, there was a variable depth of analysis of, and engagement with, CSOs across country programmes and underlying the various support schemes.
    

   Analysing the nature of CSOs
    

8. Our country visits illustrated that staff had a good working level understanding of the importance of CSOs to development particularly in those sectors where DFID country offices had active programmes. More broadly, DFID has a Drivers of Change initiative, looking at key factors which would contribute to poverty reduction in-country. The individual Drivers of Change studies carried out to date have generally identified CSOs as valuable instruments for poverty reduction. However assessment of the capacity and effectiveness of such bodies was often quite limited and did not provide a baseline for detailed monitoring of progress in the development of civil society a limitation also inherent in other assessments of civil society that country offices had made.
    
9. Staff at DFID country offices do not always have current knowledge about live projects and partnerships arranged from the UK through schemes such as the Civil Society Challenge Fund or the Partnership Programme Agreements, nor are they always responsive to requests for comments on individual projects. Of our review of 20 Challenge Fund projects, only 11 country teams responded when asked to review project proposals. DFID has not always had a clear understanding of some of the other donors actions in the civil society sector in-country to reduce poverty. Without an overall picture of the development expenditure in-country, donors contributions may not be complementary, or may not tackle areas of greatest need.
    

   Effectiveness of different support mechanisms
    

10. DFID operates a range of CSO support mechanisms, which have a variety of purposes in promoting awareness, building accountability or delivering services. Some UK-based schemes, such as the Civil Society Challenge Fund, allow DFID to support NGOs and other voluntary sector CSOs based in the UK, and to provide CSOs with funding to deliver projects with specified objectives. Others, such as Partnership Programme Agreements provide longer term, more strategic funding of UK and international CSOs. In addition, an estimated 154 million was given in-country directly by DFID teams to local CSOs.
     
11. The range of funding mechanisms allows DFID to support UK CSOs, encourage innovation and contribute to international debate as well as supporting specific projects on the ground. The various schemes have evolved over time, in response to different and changing needs. While DFID monitors expenditure and results through each UK funding scheme, it does not collect sufficient performance information to evaluate the relative cost effectiveness of the different types of scheme.
     
12. We found it difficult to assess the progress of the Civil Society Challenge Fund against its higher level objectives. DFID sees the scheme as important in supporting UK organisations, promoting innovation, and promoting development awareness in the UK, as well as securing benefits on the ground through projects. It measures the success of individual projects under the scheme but it does not currently have adequate mechanisms for assessing performance against the schemes wider objectives. It cannot, therefore, assess the full impact of the 10 million spent under this scheme.
     

    Ensuring satisfactory governance arrangements
     

13. DFID makes an assessment of the adequacy of the governance arrangements of CSOs prior to engaging with them. This is one of the ways in which DFID gains assurance that its funds will be properly spent. This assessment is particularly important for CSOs holding Partnership Programme Agreements, where CSOs have relative freedom in the use of funds provided by DFID. Our survey on governance arrangements of CSOs holding Partnership Programme Agreements found that partner governance arrangements are generally satisfactory, with board members generally having clear responsibilities and a wide breadth of skills and experience. However, there was room for improvement in a few areas, notably, performance review, ethics policies and openness. Provided a satisfactory standard has been achieved, DFID does not link the quality of CSOs governance arrangements, or improvements in them, with the level of funding they receive.
     

    Measuring development benefits
     

14. The nature of engagement with CSOs, difficulties in how to measure the benefits and the geographically dispersed nature of aid have all created challenges for DFID in assessing the benefits of working with CSOs. We identified some common features:
    
    
    • While all forms of assistance had objectives, their associated performance measures were not always fully developed. With more strategic forms of engagement such as Partnership Programme Agreements, DFID found it hard to specify Agreement objectives that were both measurable and aligned with its development priorities, although it has improved the specification of more recent Agreements. Country and Challenge Fund projects generally have clear objectives which are linked to DFID’s wider development priorities. But we found that in almost half of the projects we reviewed, the indicators established were not sufficiently robust for tracking progress against objectives.
       
    • Choices about DFID’s monitoring and evaluation methods also impact on its ability to target its funds. For Partnership Programme Agreements, DFID consciously relies on a lower level of detail on specific activities. This means that the level of funding has been determined mainly by reference to DFID development objectives (and, at least initially, historical factors), rather than performance.
      
      
      
15. Notwithstanding limitations in assessing impacts, the evidence we collected from project visits and from documentation review indicated that engaging with CSOs is leading to benefits, in line with development objectives. Monitoring reports found that 80 per cent of these projects were largely but not fully successful in achieving their intended outputs. Project evaluation reports commissioned by DFID showed that working with CSOs had been beneficial, through strengthening the institutional capacity of CSOs and others, improving advocacy and delivering services. There is less evidence to demonstrate that the projects were achieving their objectives efficiently, or that the benefits of the projects were sustainable, particularly in terms of continued funding and support for CSOs by local communities after donor funding has been withdrawn.
     
16. DFID and other donors are shifting their emphasis towards building capacity and promoting accountability in developing countries. Given this shift, DFID must develop new tools to measure the effectiveness of its funding of CSOs. This is particularly important as DFID moves to more strategic funding mechanisms with reduced levels of monitoring. In the projects we reviewed, baseline information was rarely available to permit direct assessment of the impact of DFIDs funding, and most projects did not provide measures of economy or efficiency.
     

    RECOMMENDATIONS
     

17. DFID uses a number of different delivery mechanisms to engage with CSOs in support of poverty reduction, including increasing use of innovative basket funding mechanisms in some countries. DFID assistance to CSOs largely achieves intended benefits, but assessments of the effectiveness of its assistance are sometimes limited by its results measurement arrangements. DFID reviews and controls the cost of its projects and Agreements, but has fewer tools with which to probe economy or efficiency. It is therefore difficult to assess the extent to which its assistance represents value for money. We have identified three areas where our recommendations may assist DFID in improving further the effectiveness of its expenditure on CSOs and to promote better scrutiny of value for money:
     

    On understanding the strength and position of civil society
     

    1. DFID does not systematically assess the overall strength, distribution and quality of CSOs in each country, and its knowledge more generally about the strength and capacity of society to challenge government is not complete.
        

       In the UK, DFID should:
        

       • Strengthen and expand its current guidelines on how country teams should assess civil society capacity and monitor progress; and
          
       • Assign clear central responsibilities for reviewing policy and monitoring implementation.
          

       At country level, DFID should work in co-ordination with host governments and other donors to strengthen its understanding of the strength and capacity of CSOs through:
        

       • Building on existing planning arrangements to assess more systematically the capacity of CSOs to contribute to poverty reduction. The analysis should be of an appropriate level of detail to inform country planning but need not include analysis of every individual CSO. It would include assessment of CSOs maturity and capacity to play an active role in society, and the potential benefits and risks of engaging with them.
          
       • Maintaining an up to date view of in-country engagement with CSOs, including centrally run DFID schemes such as the Civil Society Challenge Fund, and sufficient awareness of the activities of other donors and the host government to secure effective coordination.
          

       On DFIDs use of different funding mechanisms
        

    2. Funding to CSOs has emerged over time in response to changing priorities and demands. DFID should review its funding mechanisms to determine their cost-effectiveness, and to better evidence the circumstances in which different approaches are likely to be best. This could include a specific focus on evaluating innovations such as local Partnership Agreements and multi donor joint funding agencies, which offer the prospect of a more strategic approach to long-term development challenges, better co-ordination and reduced transaction costs where they can be based on well-specified and monitorable strategic objectives.
        
    3. The Civil Society Challenge Fund has a range of objectives such as increasing the capacity of UK NGOs and encouraging innovation, but DFID assesses the development benefits only of individual projects, not the overall success of the scheme against its broader objectives. DFID should be more explicit about the rationale behind the Challenge Fund scheme, and set up appropriate mechanisms to measure the wider impact of the scheme.
       
       

       On DFIDs funding appraisals and performance monitoring
        

    4. DFID has managed support to CSOs by reference to established procedures for selecting and monitoring projects, which ensure their relevance and oversee delivery of benefits. Across the projects and programmes we reviewed, however, there was scope to improve the focus on value for money and performance by:
        
       • Making sure indicators and baselines against which to judge progress are well-specified, while remaining manageable for CSOs;
          
       • Linking CSO performance more closely to future funding level decisions for the strategic Partnership Programme Agreements; and
          
       • Making greater use of competition by specifying the changes it wants to engender, and letting CSOs bid for associated support. Where full competition is not possible, DFID should carry out appraisals of project costs aimed at securing economy.

       ---

       1.  [back from footnote 1] The Millennium Development Goals aim to combat poverty in developing countries and achieve a fairer international system by 2015. They were adopted by member countries of the United Nations in 2000, and provide a global consensus on objectives for poverty reduction. For more information, see www.un.org/millenniumgoals.