Section 1: The General Approach of the SAI
| 1 | SAI responsibilities |
| 2 | Acquiring the necessary skills |
| 3 | Involvement of the SAI |
| 4 | Planning the Audit |
Section 2: Scoping the project
Section 3: Project Management
Section 4: Contract
Section 5: Value for Money Evaluation
| 39 | Achievement of objectives |
| 40 | Evaluation of alternatives |
| 41 | Ensuring service delivery |
| 42 | Confirmation of affordability |
|
Albania |
India |
|
Antigua and Barbuda |
Israel |
|
Argentina |
Lithuania |
|
Australia |
New Zealand |
|
Austria |
Norway |
|
Bahamas |
Paraguay |
|
Chile |
Peru |
|
Czech Republic |
Poland |
|
Denmark |
Russia |
|
Ecuador |
Saudi Arabia |
|
Egypt |
Turkey |
|
El Salvador |
United Kingdom (Chair) |
|
Estonia |
Uruguay |
|
Germany |
Yemen |
|
Hungary |
Zambia |
These parts are contained in INTOSAI’s published guidance [INSERT EVENTUAL TITLE OF INTOSAI PUBLICATION]. This document gives more detailed advice on how to follow the guideline and is only available on the INTOSAI website. The guidelines themselves are grouped in five sections.
Section 1: The general approach of the SAI
7 If the SAI is to carry out well thought through performance audits, leading to worthwhile recommendations and the spread of good practice, it needs to determine its remit in examining a public/private finance and concession contract and plan its audit thoroughly. It will also need access to a wide variety of relevant skills which are unlikely to be available within the SAI itself.
Section 2: Scoping the project
8 This section covers the SAI’s examination of how a procuring organisation decided the scope of a public/private finance and concession deal. In reviewing this subject the SAI will need to have a clear understanding of how the organisation chose this project ahead of other possible uses of its resources and what its objectives were in doing this. The SAI will also need to examine how the organisation structured the project to meet its needs and the role the organisation’s assessment of the private sector’s capabilities had in shaping this structure.
Section 3: project management
9 This section covers the SAI’s examination of how a procuring organisation managed the process of awarding a public/private finance and concession contract. Thus the SAI will need to examine whether the organisation established a team, with the necessary skills and expertise, with responsibility for delivering the project successfully and whether it established a proper system of cost control. The SAI will also need to review the planning that the organisation undertook to ensure that there was an effective competition for the award of the contract and that any agreed deal offered value for money.
Section 4: Contract
10 This section covers the SAI’s examination of how a procuring organisation selected a private sector partner and then negotiated the final contract with these. The SAI will need to examine that the organisation properly evaluated all aspects of the bids received, such as the allocation of risk between itself and the private sector and the price quoted, and then chose as preferred bidder that which offered it best value. The SAI will also then need to review the negotiations between the organisation and its preferred bidder to identify the effects any changes agreed in these negotiations had on the deal’s value 4for money.
Section 5: value for money evaluation
11 This final section covers the SAI’s examination of how a procuring organisation gained reassurance, before committing itself by signing the contract, that the deal met its objectives, was affordable, and offered value for money.
12 Since the public/private finance and concession approach is a fairly recent development, and bearing in mind that good practice in successfully implementing such projects is still evolving, the guidelines proposed should be seen as suggestions and advice, put forward in the light of experience so far. They are not laws or set procedures which every SAI should apply in their entirety to every study. Their purpose is essentially to offer guidance to ensure that an audit of such deals follows a professional and structured approach and identifies useful lessons for future deals. We believe nevertheless that many of the issues addressed are likely to remain valid and relevant, however the public/private finance and concession approach develops.
The SAI should identify what its audit responsibilities are in relation to public/ private finance projects and concessions and decide how to carry these out.
Supplementary guidance
None.
The SAI should identify and secure the core in-house skills it needs to carry out authoritative studies of public/private finance projects and concessions, and supplement these skills with expert external support as necessary.
Supplementary guidance
Where the SAI carries out performance audits of the contract letting process, the skills required can be divided into two groups. First, there are the general performance audit skills required to undertake the effective review of any complex or high-risk assignment. Since public/private finance and concession contracts are a recent development and good practice in the area is still evolving, auditing these deals will require the exercise of considerable judgement. Therefore the SAI will generally need to put its ablest performance auditors on this work.
Second, because of the complexity and specialist nature of many of the areas involved in achieving a successful public/private finance and concessions deal, the SAI needs to have access to specific skills relating to the service being put out to contract and the marketplace. These will include, in addition to contractual and legal skills, knowledge of the financial markets, in particular to project financing and the conditions and information required by financiers, together with financial, analytical and economic skills.
It is unlikely that the SAI will have in-house all the necessary skills and expertise in these specialist areas to discharge its responsibilities. In this situation the SAI will need to buy in the particular specialist skills. Figure 1 lists some of the consultancy skills the SAI may need to draw upon when examining public/private finance and concession deals and the areas in which consultants may be able to help. The actual consultants used will depend on the nature of the project being examined and the SAI’s own analysis of what skills are required and the extent to which these are already available in-house.
Experience suggests that the selection of such consultants should place at least as much emphasis on the individual consultants who will be providing the advice as on the consultancy firms who employ them. Individuals with good prior experience of the public/private finance and concessions approach are preferable. The consultants selected will need to have the respect of the audited body as well as the SAI if their findings are to carry weight. Regardless of the size and prestige of the firm selected, they will still require careful management by the SAI.
It may also make sense for SAIs to recruit experts in some or other of the areas of advice covered by consultancies, especially for example where the state is embarking on a major programme of service procurement using the public/private finance and concessions approach, because SAIs will also need to develop their range of core in-house audit skills in response. The SAI’s consultants could help with this. SAIs have also found it very helpful to exchange experiences and information with each other, and to exchange staff on secondments, as part of this learning process. It can also be valuable for the SAI to second some of its staff to external specialist firms (and vice versa) to obtain the necessary expertise.
|
Figure 1: Sources of consultancy skills when auditing public/private finance and concessions deals |
|
Financial consultants |
|
|
|
Legal consultants |
|
|
|
|
Property consultants |
|
|
|
|
Technical consultants * |
|
|
|
|
|
* The nature of the technical expertise the SAI may need to obtain by using external consultants will differ depending on the nature of the services involved; for example, IT, telecommunications, or facilities management. |
The SAI should examine the public/private finance and concessions deal as soon as is practical, usually after the contract is awarded.
Supplementary guidance
None
In planning the audit of a public/private finance and concessions contract, the SAI should plan to cover all major aspects of the deal that have a bearing on value for money, as set out in the following guidelines, to identify the key parties to the deal and, where possible, to take evidence from them, and to be alert to identifying lessons for the future.
Supplementary guidance
The SAI will need access to the public sector body letting the contract as soon as possible after the contract award since the contract team may be dispersed to other work, especially where the contract is a unique event for the body concerned. Without such access the SAI will be unable to examine such issues as the project’s objectives, key criteria for success and the views of those who negotiated the contract on lessons for the future.
It will also be important for the SAI to plan to seek, where possible, the views of other parties with an interest in the project. public/private finance and concessions projects typically involve a wide range of third parties in addition to the public sector body letting the contract. These parties can include for example those representing the end users of the new facilities (for instance, doctors and nursing staff on hospital projects, and road user groups on roads), local organisations such as local government bodies, and representatives of staff providing the service prior to the deal being concluded. A particularly important group will be the firms involved either successfully or unsuccessfully in tendering for the contract. Figure 2 (over) lists potential topics for discussion with these firms. The SAI must quickly establish who the interested parties are and plan effective means of taking evidence from them about the project. Techniques to identify the views of these parties include interviews, structured questionnaires and attitudinal surveys.
If there is a gap between the award of the contract and the SAI’s report, the SAI will need to be alert to post-contract award events which may cast a light to the extent to which the audited body met their project objectives. For example, the private sector contractor may fail to begin delivery of the services required by the date given in the contract or fail to meet the required performance standards when service delivery begins.
As part of its planning of the audit the SAI will need to consider how best to present any lessons identified during the examination so that they are acted on in future. For example the SAI could choose to present any lessons learned in the form of a management letter to the audited body. Alternatively, it could publish these as a formal report to parliament. In deciding on the method of disseminating these lessons, the SAI will also need to take into account the effect this might have on its relations with parliament and the government.
|
Figure 2: Topics for discussion with bidding firms |
|
The SAI should examine how the audited body prioritised potential projects and whether it implemented projects in that priority order.
Supplementary guidance
In many public/private finance and concessions projects, the private sector finances the construction of a major physical asset, operates it, and recovers the cost over time through charges either to the client organisation for services rendered or through charges to the general public for their use of the asset involved (eg toll bridges). Consequently the client does not need to find all the money for the capital asset up-front during its construction. For this reason, the public/private finance and concessions approach can be very attractive to public sector organisations since it allows them to undertake projects which they might be unable to finance conventionally.
In this attraction, however, there lies the risk that the organisation’s priorities may be distorted in favour of those projects which are capable of being run as public/private finance and concessions and against those which meet the body’s real needs. Therefore, when selecting any project to be undertaken, organisations need to establish some means of choosing projects by reference to wider programme objectives which offset such potential distortions. They need to think hard about the volume and quality of services they require and the full costs of their provision over the long-term, regardless of the eventual method of procurement.
The SAI should therefore identify whether the audited body established as part of its strategic planning a clear need for the services eventually procured. Since it is possible that such planning may have identified many potential projects, not all of which could be afforded, the SAI may also need to consider whether the audited body prioritised and selected the projects identified in accordance with criteria which reflected its policy objectives and overall business strategy.
The SAI should examine whether the audited body stated its requirements clearly from the start and expressed these in output terms.
Supplementary guidance
Although at the early stages it is not usually necessary for an organisation to specify its requirements in detail, there should be formal objectives for the project setting out in outline the requirements. The SAI should therefore examine whether an audited body set such objectives and whether these requirements were based on a proper analysis of its needs. This analysis should cover not only current needs but also some consideration of how these needs might change over time.
The objectives set should have specified the body’s needs in terms of deliverables (ie outputs, not inputs) if they are to maximise the scope for private sector innovation. The SAI should therefore examine whether the deliverables specified by the audited body focussed on what was required and not on how they should be achieved. This can be difficult for the audited body since traditionally many such bodies have themselves been managers or providers of services and directly responsible for the construction of capital assets. This means that they have in the past necessarily focussed on inputs (numbers of staff, physical specifications of buildings and so forth). So the audited body needs to ensure that those of its staff responsible for negotiating public/private finance and concession deals are trained to do so.
As part of its setting of objectives for the project, the audited body should have identified what money it had available to pay for the services to be procured and set cost limits accordingly. This is important as public/private finance and concessions deals represent a contractual commitment by a procuring organisation to pay for services, provided they are delivered. As such, these deals reduce the future flexibility of the organisation to manage its expenditure. For example, it will be harder than under traditional procurement for the organisation to cut back on the level of maintenance of the assets involved, when faced with demands to reduce its spending, as such cutbacks will need to be agreed with the contractor. The SAI should therefore check that the audited body formed a view early in the project of its likely cost and whether it would have sufficient funds to meet this.
Finally, as part of specifying the outputs required, the audited body should have given close consideration to the level of their likely use of the services being sought. In a public/private finance and concessions deal an organisation is usually committed to a certain level of service under the contract for a number of years. Gaining release from this obligation will need the contractor’s agreement and may well prove expensive. In comparison, under traditional procurement, if the organisation ceases to need a particular service, it can stop it immediately and sell any assets involved in the provision of this. (There are however limits to the level of flexibility available under traditional procurement. The ability to dispose of unwanted assets, which, under traditional procurement are paid for up-front, was often constrained because of the poor condition of these due to the cutbacks in their maintenance. Public sector assets are often highly specialised and, in the case of public sector accommodation, often in locations which are not attractive to the property market. Such assets can therefore be difficult to dispose of or have little market value.)
It is possible to build into a public/private finance and concessions deal some flexibility on usage. For example, on some past property deals, the public sector have negotiated the right to vacate in the future a certain percentage of the accommodation currently occupied for no extra cost on top of their usual service charge. The audited body should therefore have identified in advance the likely level of flexibility in respect of its usage of services it required from the private sector. The body’s examination of this issue should have been thorough since, if the body’s actual level of usage is different to that forecast, it could end up either paying for unnecessary services as its level of usage is lower then expected or for unrequired flexibility if usage is higher.
The SAI should review whether the audited body made a preliminary assessment of the private sector’s capabilities for delivering the requirements.
Supplementary guidance
None
The SAI should identify whether the audited body made a preliminary evaluation of the benefits it sought.
Supplementary guidance
The SAI should examine whether the audited body prior to going out to tender undertook a preliminary assessment of the benefits it was seeking from the project and used the results of this to establish criteria for the evaluation of the benefits offered by the bids. Figure 3 lists some of the relevant criteria for such evaluations. The SAI should check that the audited body used these criteria as the basis of its evaluation of tenders and did not draw up new criteria at the end of the procurement process simply to justify the deal reached.
|
Figure 3: Examples of evaluative criteria |
|
The SAI should review how the audited body assessed the impact any wider policy objectives might have on the project.
Supplementary guidance
None
The SAI should examine whether the audited body examined a range of alternative ways of meeting its needs, such as public sector traditional procurement or privatisation, before choosing the public/private finance and concessions option.
Supplementary guidance
Figure 4 gives a number of alternative project mechanisms which may be available to an audited body when considering exactly what form of partnership with the private sector would best meet their objectives. The choice of partnership will in part be influenced by who is to pay for the service provided. For example the project may involve a public sector body procuring and paying for a service itself. Alternatively it may involve the granting of a concession to a contractor who then charges the general public for the use of the services provided.
|
Figure 4: Alternative project mechanisms |
|
The SAI should review whether the audited body identified the room for innovation in advance in areas such as design and construction, operation, and project financing.
Supplementary guidance
Figure 5 lists possible areas for innovation and what points the SAI needs to be aware of in each when considering the scope for innovation in a project and the limits placed on this by an audited body.
|
Figure 5: Areas for innovation |
|
Design and construction |
|
Operation |
|
Financing |
The SAI should review whether the audited body investigated in advance the appropriate allocation of projects risks between the public sector and private sector parties affected by the project.
Supplementary guidance
The SAI should check whether an audited body identified at the start the risks associated with the project and the scope for their transfer to the private sector. This preliminary identification of risk will be closely linked to the body’s early consideration of the contractual terms they will be seeking from the private sector (Guideline 17) as it is the contract which enshrines the eventual agreed allocation of risk between the public and private sectors.
On the scope for risk transfer, the body should have made some initial contact with the market in order to identify what risk transfer the private sector were likely to be willing to accept. The SAI may need the help of an experienced external consultant in assessing whether the transfer of risk initially proposed was reasonable. A good indication of whether the original proposals were unreasonable would be if the actual risk transfer differed greatly from that originally identified.
There are a number of individual risks the audited body should have paid particular attention to. For example, on demand risk, Guideline 6 mentioned the need for the body to consider the likely level of the services delivered under a public/private finance and concessions contract. As part of its examination of risk allocation, the body should have considered the extent to which it would seek to transfer this risk to the private sector and the basis on which it should do this. The basis of the risk transfer can vary. For example, payments to the contractor can be linked in whole or in part to the level of usage of the services supplied, rather than to their availability. Alternatively the audited body could require the right to reduce the scope of the services supplied without incurring a financial penalty (Guideline 6). The audited body could also negotiate the right to exit the whole of the contract, regardless of whether the contractor has defaulted, at set points within the contract period without the need to pay the contractor compensation. It is particularly important that the audited body sounds out the market early on the extent of transfer of demand risk and the basis for this as, on many projects, the private sector will refuse to accept this risk or will only do so by including a large premium for it in the quoted service charge. The audited body should therefore have considered very carefully whether its plans to transfer demand risk are likely to result in value for money.
The audited body should have considered closely the risk of fraud. The body should therefore have included in its planned contract terms provisions covering the occurrence of fraud and corruption.
The SAI should also examine how thoroughly the audited body considered the extent of risks retained in the public sector as for certain risks this may offer the best value for money. In identifying risks remaining with themselves the audited body will need to have examined closely any warranties or indemnities sought by the private sector from themselves as these can effectively limit the extent of any risk transfer to the private sector. (There is relevant guidance also in Guideline 12 of the Guidelines on Best Practice for the Audit of Privatisations.)
One risk retained by the public sector is contract renewal risk. This risk here is that the audited body is unable to relet the contract without proper competition at its natural expiry or early termination. The SAI therefore should check that the audited body considered, even at an early stage, what was to happen at the end of the contract. The body should have identified in advance the core skills which needed to be retained in-house, and the provisions that needed to be included in the draft contract, to ensure its ability to relet the contract competitively if necessary.
The SAI should also be aware that often the steps taken to manage an identified risk can result in the creation of a further, secondary risk. The SAI should therefore check that the audited body’s identification of the risks associated with the project also included an examination of such secondary risks.
Effective risk management involves not only the identification of each risk involved and who will take responsibility for this but also the formulation of a strategy for dealing with that risk. The SAI should therefore examine whether the audited body, after identifying the risks involved, considered the arrangements for managing the risks that it retained. These arrangements will be closely linked to those introduced to manage the public/private finance and concessions contract, when awarded, and the contractor’s performance under this.
For their management arrangements to be effective the audited body should have identified the core skills that it needed to retain in-house to perform this management function. The body should have put these management arrangements in place prior to the award of the contract to ensure that the staff who are responsible for monitoring the contract and managing retained risks are familiar with these and receive any necessary training. The audited body should have also planned for the arrangements to come into effect not only once service delivery begins but also prior to this during the development or construction stage. This should help ensure that service delivery begins on time.
The SAI should examine the extent to which the audited body considered, before starting the procurement process, whether the project was likely to be affordable and offer value for money.
Supplementary guidance
The SAI should examine whether before going out to tender the audited body prepared an outline appraisal which demonstrated the project’s value for money. This appraisal should have included the following elements:
The SAI should also check that, in addition to the above financial factors, the audited body identified the non-financial benefits and disadvantages of each option in order to make a complete assessment of the value for money of each.
The audited body should also have confirmed that it could still afford to pay for the project by comparing the costs identified above for each option with the likely level of funding available in the short and long term (Guideline 6). It should also have identified that it was likely that the private sector could provide the services required (Guidelines 7 and 12).
The SAI should examine whether the audited body prepared a proper business case to support the decision to begin the project’s procurement.
Supplementary guidance
The SAI should examine whether an audited body produced an outline business case before proceeding with the project. This case should summarise the planning work previously undertaken and which the SAI should have already examined in depth. Figure 6 lists the contents usually found in such a case.
|
Figure 6: The contents of an outline business case |
|
project objectives
|
|
Procurement strategy |
|
Project timetable |
|
Evaluation criteria (Guideline 7) |
|
Commitment
|
The SAI should examine whether the audited body assessed the skills it would need to deliver the project successfully and where it could obtain these, for example from in-house staff or from external advisers, and whether the body then assembled its project team in good time.
Supplementary guidance
The first step in the thorough planning of the procurement of a public/private finance and concessions deal is to assemble in good time a properly qualified project team. This is crucial if any contract is to be awarded successfully. Ideally the team should be set up once the procuring organisation has decided that the project is feasible and that procurement should proceed.
An organisation should only put such a team in place after it has identified what skills it requires at the various stages of the project in order to ensure its success. project management is important to the successful completion of any project since a project manager not only sets timetables for all stages of the project, and monitors progress against these, but is also responsible for seeking to resolve problems as they arise so that they do not cause delay. The team should also include at least one member with knowledge of and responsibility for ensuring compliance with any relevant procurement rules. The SAI should therefore review what skills and expertise the audited body made use of in the procurement.
Having considered what skills it is likely to require, the procuring organisation should identify the sources of these skills. In a complex procurement like a public/private finance and concessions deal it is highly unlikely that all the necessary skills and expertise will be available in-house. To the extent that they are not, there will therefore be a need for external advisers with previous experience of negotiating public/private finance and concessions or other partnership types of deals to complement the in-house expertise (Figure 4: Alternative project mechanisms - Guideline 10). Such advisers should be appointed early in the project since their commercial experience should help prevent the project being developed in a way which is unattractive to the market. Good legal and financial advice is particularly important, and the SAI should check whether the audited body actively took up the references of prospective advisers.
In certain circumstances, for example where an organisation is procuring a series of public/private finance and concessions deals, it may not be necessary to set up a completely new team for a project as one may already exist from a previous deal. The team will therefore be experienced in many of the issues involved in negotiating such deals. However an organisation will still benefit from an analysis of the expertise required for a new project since each public/private finance and concessions deal will have unique features. For example, although the new project may involve the provision of accommodation as on previous projects, the ownership of the site in question may be less straightforward than before, requiring legal advice in new areas.
Finally the audited body should have established reporting and decision making procedures within the team to ensure proper control of the procurement. To ensure speedy decision-making, clear levels of delegated authority should have been established not only within the project team and but also with users. A member of the body’s staff should have been designated as "project sponsor" to provide the focal point for the body’s input into the project. There should also have been a project champion within the body’s senior management to help the team cut through problems which could not be resolved without reference to senior management.
The SAI should examine how the audited body investigated the market prior to beginning the formal procurement, to establish that there were suppliers who were willing to tender for the project.
Supplementary guidance
The SAI should check whether the audited body investigated the market sufficiently early in the project to ensure that there were suppliers who were willing and competent to bid for the work. In examining this consultation, the SAI should also take into account any investigation of the market that the audited body undertook when assessing the project’s feasibility (Guideline 7). If the body has undertaken such consultation, it will need to have avoided conferring preferential treatment on any potential bidder or bidders by such action.
The SAI should examine whether the audited body identified the contractual issues that were likely to arise during the procurement and drew up a draft contract, setting out initial proposals on each issue.
Supplementary guidance
Figure 7 lists the main contractual issues that one would expect a procuring organisation to address early on. The SAI should examine whether the audited body addressed these issues sufficiently early on and what legal advice it took when doing this. The SAI will also need to form a view as to whether the source of this advice had sufficient expertise to give good quality advice on the issues concerned. Guidance on the likely acceptability of their contractual proposals to the private sector should have been available to the audited body from its initial contacts with the market on the likely allocation of risk (Guideline 12).
|
Figure 7: Main contractual issues |
|
In deciding on the opening position on the above contractual issues, a procuring organisation can save effort and expense by drawing, where relevant, on previous experience with similar public/private finance and concessions projects or on standard terms and conditions for such deals issued by any relevant central body. There is however a risk that in doing so that they may include contract terms that are not relevant to this particular project. The audited body should therefore have considered carefully whether it should replicate the provisions of past or standard contracts in their entirety, given possible changes in circumstances.
The SAI should identify whether the audited body prepared a tendering strategy, covering the number of tender rounds to be held, the number of bids to be invited at each tender stage, the body’s approach to communicating with bidders, and a realistic timetable for the tender process.
Supplementary guidance
The SAI should check that, as part of its planning of the procurement, the audited body drew up a tender strategy before going out to tender. Since in many cases there will need to be several rounds of bidding, the body will need to have considered how many rounds it intended to hold and the number of firms it proposed to invite to tender at each round, and the overall tender timetable in which these tender rounds were to be held. Although it is good practice to map out such matters in advance, the body should be sufficiently flexible to vary the strategy in the light of changing circumstances, and possibly at short notice, if it believes that better value for the taxpayer will result.
In drawing up its tender strategy the audited body should have been sensitive to the time and costs that bidders incur in preparing bids and accordingly limited the number of rounds to the minimum consistent with a successful competition. Because of the substantial costs involved it should similarly have limited the number of bidders expected to submit full bids. The tender timetable should also be sufficiently brisk to maintain competitive tension, but long enough to permit the important contractual issues to be settled before the selection of the winning bidder.
As part of its tender strategy the audited body should have considered its plans for communicating with bidders. Such a communications strategy should cover how reasonably probable and competent bidders are to be made aware of the project, and what information will be released to them. As regards the possible release of the tender strategy itself to bidders with its details on the planned number of tender stages and bidders, there is much to be said for letting bidders know what is likely to be involved. This will enable them to make informed decisions on their own bidding strategy. Releasing such information however could cause problems at a later date should the procuring organisation decide to vary its plans. The SAI should therefore check whether the audited body took steps not to restrict unduly its freedom of action in this, for example by making it clear to bidders that it reserved the right to vary the tender strategy if necessary. Changes in tender strategy should be communicated to bidders in an even handed way.
In preparing the tender strategy the audited body should have considered how to structure the tendering plans in such a way as to allow early discussions with bidders. Such discussions are necessary if the public sector is to benefit from private sector innovation in the shaping of its requirements and the proposed solutions to these. Not allowing scope for such discussions may result in the audited body having to request further, unplanned, bids from tenderers in order to secure the services required. In the European Union, for example, allowing for such discussions will mean that the body should carefully consider using the "negotiated" or "concession" procedures, rather than the "restricted" procedure, because the former provide greater scope for negotiations with bidders earlier in the process.
Finally, in cases where an audited body is letting more than one contract to a common timetable, the SAI will need to examine whether the body considered at an early stage offering more than one contract to a bidder. Awarding contracts to the same bidder in this way may result in lower prices as the successful firm may pass on savings arising from economies of scale. The audited body will however need to have checked that its plans complied with any relevant procurement regulations and that potential bidders had the capacity to handle more than one contract simultaneously. Alternatively the audited body may have taken a policy decision to award its contracts to different bidders in order to strengthen the market for public/private finance and concessions deals if for example initial soundings of the market had shown that it was weak in the particular sector concerned.
The SAI should identify whether the audited body prepared a credible project timetable which identified milestones against which progress could be measured, and points within the process at which the body was to review the project’s continued viability.
Supplementary guidance
None
The SAI should examine whether the audited body assessed the project against any publicly funded alternative, and review the adequacy of this public sector comparator.
Supplementary guidance
The audited body will need to establish the public sector comparator as part of the planning of the procurement if it is to be used later as a benchmark against which to judge bids received. The outline business case prepared earlier in the process (Guideline 14) will have set out the key basis for comparison. The body should then revise the figures in this as the procurement progresses, to reflect more up to date information on public sector costs, and if necessary revisit the assumptions underlying the comparator. As much of the comparator as possible should be completed before the receipt of best and final offers from bidders if it is to help in the assessment of bids.
When reviewing the calculation of the public sector comparator the SAI should focus in particular on the assumptions underlying the comparator and the variables which will have the most significant impact on the total cost of the publicly funded alternative. Variables likely to be important, and therefore worthy of closer examination, include the completeness and reasonableness of the valuation of the possible financial impact of the various risks involved (Guideline 12), the discount rates used to calculate the net present values, and the forecasts of the volume of services required. It may be necessary when examining these assumptions and variables to commission the help of external consultants with relevant technical or financial expertise.
When examining the publicly funded alternative the SAI will need to consider the amount of detail required in this comparator. The level of detail required will depend on the impact that the comparator is likely to have on the assessment of value for money. For example, where private sector provision clearly offers much better value for money, there is less need for a precise calculation of the comparator’s value.
The SAI should examine whether the audited body succeeded in creating a good tender list.
Supplementary guidance
The SAI should first examine what action the audited body took to stimulate the market. The previous investigation of the market (Guidelines 7 and 16) should have identified the likely interest in the project and what, if any, stimulation might be necessary. If some stimulation of the market were necessary, this should have been done before formal procurement started in order to maximise the number of good quality bids submitted. Stimulation can involve direct mailing of potential bidders, advertisements in the trade press, contacts with trade associations, and the holding of bidders’ conferences.
The SAI should also check how the body advertised the project. It should pay particular attention to the wording of any advert of formal announcement of the project since too specific a statement of requirements in such notices can deter potential bidders and restrict the opportunity for innovation by the bidders during the competition. An examination of the guidance on expected quality and technical standards provided to bidders will also be necessary. Clear guidance will aid bidders in the compilation of their bids. If however this guidance is too restrictive, innovation could be stifled
A competition will only have been effective if it included good quality firms. The SAI will thus need to examine how the audited body selected the list of firms invited to tender. In doing this, the body should have focused on firms’ technical, financial and managerial capabilities. The body should have been concerned with not only whether a firm was capable of providing the required services should they be awarded the contract but also whether, prior to the award of the contract, they are likely to remain in the competition and not withdraw at some point. Assessment of a firm’s management of its workflow will be particularly important. Such an assessment should cover not only work on other contracts that the firm will have during the construction or development phase of this contract but also the other contracts that the firm is bidding for at the same time as tendering for this contract. Because of the complexities and costs of preparing a public/private finance and concessions bid bidders have sometimes withdrawn from one competition to concentrate on another which they consider they have more chance of winning.
Because of the size and complexity of many public/private finance and concessions deals, firms often come together as consortia to compete for this work. Where a consortium expresses an interest in their proposed deal, the audited body’s assessment of capabilities should cover all members of the consortium as often these possess different skills and expertise. For example, on accommodation projects it is common to find construction and facilities management firms in a consortium. Once a consortium has been included on the tender list the audited body should continue to monitor the composition of this as it can often change with members leaving and new ones joining. Where there are changes of membership the audited body should have satisfied themselves again that the consortium still possesses the capabilities needed to deliver the required services.
A consortium may decide that it does not need to possess itself all the skills and expertise that are needed to deliver the required service but will instead sub-contract these in when necessary. If the audited body is content with this approach, it will still need to assess whether the sub-contractor, when eventually chosen, possesses the required capabilities to ensure service delivery. The body should continue monitoring the use of sub-contractors which are not members of the consortium during the asset construction / development stage. The project’s successful implementation could be jeopardised by a change of sub-contractor in this late stage
The SAI should also identify what help from its external advisers the audited body received when drawing up the tender list. These advisers are likely to have much more knowledge of the market place than the audited body on matters such as the possible existence of cartels between bidding firms.
Finally, the best check on whether the tender list did indeed provide sufficient competition is to verify the number of bids actually received, compare this with the number that the body planned to receive, and seek explanations for any difference between the two. If the number of actual bids was less than expected, it will be useful to seek the opinions of bidders or potential bidders as to why this was the case. Another indication of the strength of competition is the quality of the bids received, as evidenced for example by the audited body’s own evaluation of bids received.
The SAI should consider whether the audited body set out a clear specification of the requirements.
Supplementary guidance
In determining whether the specification of requirements clearly stated the audited body’s real needs, the SAI will need to examine how the specification was developed. The body should have involved the users in the identification of `requirements but controlled their input to ensure that users’ expectations were realistic and matched the organisation’s real needs. Discussion of requirements with bidders will also be useful to identify whether these requirements were practicable and whether there was scope for improvement.
This consultation should have resulted initially in the production of an outline specification of requirements. These documents are often highly complex and the SAI may well need external advice in understanding them. The SAI should verify that the contents of this specification were consistent with the project deliverables identified when the objectives for the project were set (Guideline 6) and were themselves sufficiently output-based and covered the level of performance required of the contractor when eventually awarded the contract. In deciding whether the specification was sufficiently clear, it will also be necessary to seek the views of all the firms which bid for the contract, and perhaps also any who decided to withdraw from the bidding.
The SAI should identify whether, through the establishment of a competitive final shortlist, the audited body succeeded in maintaining competitive tension to contract award, and managed the negotiations with the preferred bidder well.
Supplementary guidance
A good way for a procuring body to achieve competitive tension is to maintain time pressure throughout the procurement. Such pressure can be maintained by notifying the procurement timetable to bidders so that all parties are aware of the target dates that need to be met, and also providing bidders with model frameworks for contracts and other guidance on the audited body's requirements so that time is saved. The SAI should therefore check the information the audited body released to bidders on its timetabling requirements.
One indicator of the successful maintenance of competitive tension is the creation of a competitive shortlist of firms invited to submit final bids. The SAI should therefore verify the number of final bids actually received, compare this with the number that the audited body planned to receive and seek explanations for any difference between the two. If the number of final bids was less than expected, it will be helpful to seek bidders’ opinions as to why this was the case. The views of the bidders on the procurement will also provide evidence as to whether competition was effective.
Given the lack of competitive pressure during the negotiations with the preferred bidder and the changes to the deal’s terms often experienced in this stage, the audited body should have minimised the scope for negotiation by keeping this period of exclusive negotiations short and by achieving as much improvement to the terms of the bid as possible before the preferred bidder are informed of their selection. The SAI should therefore be concerned if these exclusive negotiations took many months and/or there were changes during this period to the proposed price, scope of services, allocation of risk, or contractual terms.
The SAI should identify whether during the procurement the audited body regularly reassessed that the project continued to offer value for money.
Supplementary guidance
The audited body will find it useful if it continued to assess the deal’s value for money periodically once the contract has been awarded. It will then be better informed as to whether or not to exercise any right it may have negotiated in the contract to exit it, regardless of whether the contractor has defaulted (Guideline 12).
The SAI should examine whether the audited body set realistic budgets for all project costs, including internal and external resources.
Supplementary guidance
None.
The SAI should check that the audited body appointed good quality external advisers after competition.
Supplementary guidance
None.
The SAI should review how the audited body monitored and managed its project costs, including internal and external resources.
Supplementary guidance
None.
The SAI should examine whether the range of solutions proposed by bidders included design variants.
Supplementary guidance
None.
The SAI should review whether bidders put forward operational variants in their proposals.
Supplementary guidance
One of the government’s wider policy objectives (Guideline 9) may be the protection of the rights of those public sector staff transferring to the private sector under any public/private finance and concessions deal. If this is the case, the SAI will need to check, as part of its review of bidders’ operational proposals, whether the audited body, when evaluating bids, assessed the proposed arrangements for employees’ pension and employment rights and union recognition.