The World Bank and Public Procurement—An Independent Evaluation Appendixes to Volume II: Achieving Development Effectiveness through Bank Procurement Contents ABBREVIATIONS ..................................................................................................................................................................... I APPENDIX A. BANK GUIDELINES AND PROCESSES ........................................................................................................ 3 APPENDIX B. CDD PROCUREMENT ................................................................................................................................... 31 PPP PROCUREMENT: COMPARISON OF THE BANK AND OTHER ORGANIZATIONS.................................................. 55 APPENDIX C. PROCUREMENT RISK CONCEPTS AND THRESHOLDS ........................................................................... 61 APPENDIX D. PROCUREMENT TRACKING SYSTEMS, TIME ANALYSIS, AND AFRICA REGION REVIEW ................. 89 BIBLIOGRAPHY .................................................................................................................................................................. 110 ENDNOTES .......................................................................................................................................................................... 119 ii Abbreviations ADB Asian Development Bank ISR Implementation status report AfDB African Development Bank MAPS Methodology for Assessing APEC Asian Pacific Economic Cooperation Procurement Systems CDD Community-driven development MDB Multilateral development bank CPAR Country procurement assessment MEAT Most economically advantageous report tender CPIA Country policy and institutional MRRD Ministry of Rural Rehabilitation and assessment Development DfID Department for International NCB National competitive bidding Development NEPE State Technical Unit FMIS Financial management information OPRC Operational Procurement Review system Committee HDI United Nations Human Development OPCS Operations Policy and Country Index Services IAD Internal Audit Vice Presidency PAD Project appraisal document IAEA The International Atomic Energy PPP Public-private partnership Agency P-RAMS Procurement risk assessment ICB International competitive bidding management system. process PROCYS Procurement Cycle Tracking System— ICR Implementation completion report Africa Region ICRR Implementation completion and RPM Regional procurement managers results reports SEPA Procurement plan execution system ICT Information technology SME Small and medium-sized enterprises IDA International Development Agency TOR Terms of reference IDB Inter-American Development Bank UNCITRAL United Nations Commission on IEG Independent Evaluation Group International Trade Law IFC International Finance Corporation WBI World Bank Institute INT Integrity Vice-Presidency WTO World Trade Organization ISDB Islamic Development Bank Appendix A. Bank Guidelines and Processes Overall Perceptions Table A.1. Bank Procurement Methods—Adequacy of Competition Bank Country proc Bank Country Private Civil Question Avg mgmt staff TTLs clients sector society Do the Bank's Procurement procedures ICB, NCB, and Consultants Selection Procedures in particular) lead to adequate competition in 2.9 2.9 3.2 2.8 3.3 2.9 2.5 terms of numbers of bidders, openness and fairness, geographical reach, etc.? Source: IEG questionnaire. Note: 4 = more than adequate; 3 = adequate; 2 = only somewhat adequate; 1 = inadequate; TTL = task team leader. Table A.2. Bank Procurement—Support to Integrity, Transparency, Cost, Quality, and Timelines Bank Country proc Bank Country Private Civil Question Avg mgmt staff TTLs clients sector society To what extent do the Bank's procurement requirements positively support the integrity, transparency, cost, quality, and timeliness of 3.1 2.6 3.6 3.0 3.5 3.0 2.8 delivering public sector projects? Source: IEG questionnaire. Note: 4 = more than adequate; 3 = adequate; 2 = only somewhat adequate; 1 = inadequate; TTL = task team leader. Table A.3. Bank Procurement—Capturing Bidder Interest Bank Country proc Bank Country Private Civil Question Avg mgmt staff TTLs clients sector society Are bidders s more likely to show interest if the project is under Bank procurement policy 2.8 3.6 3.0 3.2 3.2 3.2 and procedures, as opposed to a project 3.2 under national guidelines and processes? Source: IEG questionnaire. Note: 4 = more than adequate; 3 = adequate; 2 = only somewhat adequate; 1 = inadequate; TTL = task team leader. APPENDIX A BANK GUIDELINES AND PROCESSES Table A.4. Bank Procurement—Emphasis on Price Factors Question Bank proc Bank Country Private Avg staff TTLs clients sector To what extent does the Bank's oversight (no-objection) of prequalification/ short-listing, bidding documents/RFP, 2.9 2.9 2.7 2.9 3.0 bid/proposal evaluation, and contract award emphasize price? Source: IEG questionnaire. Note: 4 = more than adequate; 3 = adequate; 2 = only somewhat adequate; 1 = inadequate; TTL = task team leader. Table A.5. Bank Procurement—Flexibility in Response Bank proc Bank Country Question Avg staff TTLs clients Is the Bank able to respond quickly to special circumstances such as those requiring deviations from policies and procedures, or changes to 2.3 2.7 2.1 2.2 procurement provisions of the financing agreement, or changes to previously agreed arrangements? Source: IEG questionnaire. Note: 4 = more than adequate; 3 = adequate; 2 = only somewhat adequate; 1 = inadequate; TTL = task team leader. Table A.6. Bank Procurement—Incidence of Delay in Contract Award Bank Ctry Proc Bank Ctry Private Civil Question Avg mgmt Staff TTLs clients sector society How frequent is the incidence of delay in the award 2.9 3.2 2.5 2.9 2.7 2.9 3.2 of contracts? Source: IEG questionnaire. Note: 4 = more than adequate; 3 = adequate; 2 = only somewhat adequate; 1 = inadequate; TTL = task team leader. Methodology for Assessing Procurement Systems Indicators and Scores Table A.7. MAPS Indicators and Passing Scores Under the Piloting Program UCS required score Pillar I: Legislative and Regulatory Framework Indicator 1: Public procurement legislative and regulatory framework achieves the agreed standards and complies with applicable obligations. Sub-indicator 1(a): Scope of application and coverage of the legislative and regulatory framework. 3 Sub-indicator 1(b): Procurement Methods 2+ Sub-indicator 1(c): Advertising rules and time limits 3 Sub-indicator 1(d): Rules on Participation 3 Sub-indicator 1(e): Tender documentation and technical specifications 3 Sub-indicator 1(f): Tender evaluation and award criteria 3 Sub-indicator 1(g): Submission, receipt and opening of tenders 3 4 UCS required score Sub-indicator 1(h): Complaints 3 Indicator 2: Existence of Implementing Regulations and Documentation. Sub-indicator 2(a): Implementing regulation (sic) that provide defined processes and procedures not included 2 in higher-level legislation Sub-indicator 2(b): Model tender documents for goods, works, and services 2 Sub-indicator 2(c): Procedures for pre-qualification 2+ Sub-indicator 2(d): Procedures suitable for contracting for services or other requirements in which technical 2+ capacity is a key criterion. Sub-indicator 2(e): User's guide or manual for contracting entities 2 Sub-indicator 2(f): General Conditions of Contracts (GCC) for public sector contracts covering goods, works 3 and services consistent with national requirements and, when applicable, international requirements Pillar II: Institutional Framework and Management Capacity Indicator 3: The public procurement system is mainstreamed and well integrated into the public sector governance system. Sub-indicator 3(a): Procurement planning and associated expenditures are part of the budget formulation 2 process and contribute to multiyear planning. Sub-indicator 3(b): Budget law and financial procedures support timely procurement, contract execution, and 2 payment. Sub-indicator 3(c): No initiation of procurement actions without existing budget appropriations. 2 Sub-indicator 3(d): Systematic completion reports are prepared for certification of budget execution and for 2 reconciliation of delivery with budget programming. Indicator 4: The country has a functional normative/regulatory body. Sub-indicator 4(a): The status and basis for the normative/regulatory body is covered in the legislative and 3 regulatory framework. Sub-indicator 4(b): The body has a defined set of responsibilities including the following… 2 Sub-indicator 4(c): The body's organization, funding, staffing, and level of independence and authority (formal 2 power) to exercise its duties should be sufficient and consistent with the responsibilities. Sub-indicator 4(d): The responsibilities should also provide for separation and clarity so as to avoid conflict of 3 interest and direct involvement in the execution of procurement transactions. Indicator 5: Existence of institutional development capacity. Sub-indicator 5(a): The country has a system for collection and disseminating procurement information, 2 including tender invitations, requests for proposals, and contract award information. Sub-indicator 5(b): The country has systems and procedures for collecting and monitoring national 2 procurement statistics. Sub-indicator 5(c): A sustainable strategy and training capacity exists to provide training advice and 2 assistance to develop the capacity of government and private sector participation to understand the rules and regulations and how they should be implemented. Sub-indicator 5(d): Quality control standards are disseminated and used to evaluate staff performance and 2 address capacity development issues. 5 APPENDIX A BANK GUIDELINES AND PROCESSES UCS required score Pillar III: Procurement Operations and Market Practices Indicator 6: The country's procurement operations and practices are efficient. Sub-indicator 6(a): The level of procurement competence among government officials within the entity is 2 consistent with their procurement responsibilities. Sub-indicator 6(b): The procurement training and information programs for government officials and for 2 private sector participants are consistent with demand. Sub-indicator 6(c): There are established norms for the safekeeping of records and documents related to 2+ transactions and contract management. Sub-indicator 6(d): There are provisions for delegating authority to others who have the capacity to exercise 2 responsibilities. Indicator 7: Functionality of the public procurement market Subindicator 7(a): There are effective mechanisms for partnerships between the public and private sector. 2 Sub-indicator 7(b): Private sector institutions are well organized and able to facilitate access to the market. 2 Sub-indicator 7(c): There are no major systemic constraints (e.g. inadequate access to credit, contracting 2+ practices, etc.) inhibiting the private sector's capacity to access the procurement market. Sub-indicator 7(d) – Clarity and transparency of rules for determining whether to engage international or 2 national markets. *Only analyzed for 3 countries. Indicator 8: Existence of contract administration and dispute resolution provisions Sub-indicator 8(a): Procedures are clearly defined for undertaking contract administration responsibilities that 2 include inspection and acceptance procedures, quality control procedures, and methods to review and issue contract amendments in a timely manner. Sub-indicator 8(b): Contracts include dispute resolution procedures that provide for an efficient and fair 3 process to resolve disputes arising during the performance of the contract. Sub-indicator 8(c): Procedures exist to enforce the outcome of the dispute resolution process. 3 Pillar IV: Integrity and Transparency of the Public Procurement System Indicator 9: The country has effective control and audit systems. Sub-indicator 9(a): A legal framework, organization, policy, and procedures for internal and external control 2+ and audit of public procurement operations are in place to provide a functioning control framework. Sub-indicator 9(b): Enforcement and follow-up on findings and recommendations of the control framework 2+ provide an environment that fosters compliance. Sub-indicator 9(c): The internal control system provides timely information on compliance to enable 2+ management action. Sub-indicator 9(d): The internal control systems are sufficiently defined to allow performance audits to be 2 conducted. Sub-indicator 9(e): Auditors are sufficiently informed about procurement requirements and control systems to 2 conduct quality audits that contribute to compliance. Indicator 10: Efficiency of appeals mechanism. Sub-indicator 10(a): Decisions are deliberated on the basis of available information, and the final decision can 3 be reviewed and rules upon by a body (or authority) with enforcement capacity under the law. Sub-indicator 10(b): The complaint review system has the capacity to handle complaints efficiently and a 3 means to enforce the remedy imposed. Sub-indicator 10(c): The system operates in a fair manner, with outcomes of decisions balanced and justified 3 on the basis of available information. Sub-indicator 10(d): Decisions are published and made available to all interested parties and to the public. 2 Sub-indicator 10(e): The system ensures that the complaint review body has full authority and independence 3 for resolution of complaints. Indicator 11: Degree of access to information Sub-indicator 11(a): Information is published and distributed through available media with support from 2+ information technology when feasible. Indicator 12: The country has ethics and anticorruption measures in place. Sub-indicator 12(a): The legal and regulatory framework for procurement, including tender and contract 3 documents, includes provisions addressing corruption, fraud, conflict of interest, and unethical behavior and sets out (either directly or by reference to other laws) the actions that can be taken with regard to such behavior. Sub-indicator 12(b): The legal system defines responsibilities, accountabilities, and penalties for individuals 3 and firms found to have engaged in fraudulent or corrupt practices. Sub-indicator 12(c): Evidence of enforcement of rulings and penalties exists. 2+ 6 UCS required score Sub-indicator 12(d): Special measures exist to prevent and detect fraud and corruption in public procurement. 3 Sub-indicator 12(e): Stakeholders (private sector, civil society, and ultimate beneficiaries of procurement/end- 2 users) support the creation of a procurement market known for its integrity and ethical behaviors. Sub-indicator 12(f): The country should have in place a secure mechanism for reporting fraudulent, corrupt, or 3 unethical behavior. Sub-indicator 12(g): Existence of Codes of Conduct/Codes of Ethics for participants that are involved in 2 aspects of the public financial management systems that also provide for disclosure for those in decision making positions. 7 National Competitive Bidding Requirements and Compliance Table A.8. Bank NCB Requirements and Country Systems AZ BD ET ID MA PE PH SN Reports Countries Field Visit Field Visit Field Visit Field Visit/UCS Field Field Field Visit/UCS Field Reviewed Reviewed Visit Visit Visit/UCS /UCS 2012 2002 2005 2008 2005 2010 2008 2001 2004 2005 2012 2011 2004 2009 2012 35 26 CPAR 2009 LA CPAR LA LA LA LA LA CPAR LA LA LA LA LA LA CPAR Draft Bank Requirements for NCB There may be no regional / domestic preferences regarding the 18 15 1 1 1 1 1 1 1 1 1 1 sources of labor and material.1 Prospective bidders must be allowed at least 30 days for bid 19 14 1 1 1 1 1 1 1 1 1 preparation.2 Appropriate standard bidding and prequalification documents 15 14 1 1 1 1 1 must be used. Eligibility cannot be restricted based on nationality of bidder 17 13 1 1 1 1 1 1 1 1 and/or origin of goods.3 Award must be made to the lowest evaluated qualified and 15 13 1 1 1 1 1 1 responsive bidder. Minimum requirements must be explicitly stated in the 12 12 1 1 1 1 documents. Bidding opportunities must be advertised in the local press. 11 11 1 1 1 1 Bidders are not generally required to register with a local or 13 11 1 1 1 1 1 1 1 federal authority except under certain conditions.4 The procurement process cannot be cancelled, all bids rejected, 13 11 1 1 1 1 1 1 1 1 and/or rebidding conducted without approval. Public bid opening is required. 11 9 1 1 1 1 1 1 Bid evaluation criteria other than price may be allowed only if 8 8 1 quantified in monetary terms. Price negotiations may not be conducted with “winning” bidders 10 8 1 1 1 1 prior to contract signature.5 Parastatals may only be allowed to bid under certain conditions.6 9 7 1 1 1 Bids may not be rejected based only on a comparison with the procuring entity's estimate; invitations to bid shall not establish 11 7 1 1 1 1 1 1 1 minimums and maximums.7 Audits and inspection of records related to bid submission and 8 7 1 performance of the supplier are permitted.