1. Introduction and Overview
This report has been prepared as part of an evaluation of the BIZ programme, undertaken for the Ministry of Economic Development by a consortium consisting of Business and Economic Research Ltd (BERL), the National Research Bureau and the Centre for SME Research, Massey University.
This part of the evaluation has been undertaken by BERL. Comments from other members of the project team undertaking different parts of the evaluation have been incorporated in this report (in particular, based on the Analysis of BIZ Provider Interviews).
1.1 Context
Before commencing with our examination of the various aspects of the reporting, management and implementation of the BIZ programme, it is useful to place it in context.
The policy for the BIZ programme was developed and implemented in 1998 and 1999 with a philosophy and an approach which was new to the Ministry. It included several features such as delivering services through private providers under contract, with providers themselves defining local needs and the types of services required to meet them. This policy had significant implications for all aspects of the implementation process and management of the programme.
One outcome of this is that both providers and the Ministry have had to do a lot of "learning by doing" to establish effective methods of operating. This report is in large measure about reviewing the lessons which have been learnt during the last two years, identifying the enhancements which have already been made (including ongoing ones), and suggesting directions for the future.
1.2 Terms of Reference
This report focuses on management of the programme by the Ministry, covering the following sections of the specification of Evaluation Services:
| 6 | Analysis of BIZ Provider Reports |
| 6.1 | BERL must undertake an analysis of a representative range of BIZ provider reports and include the results in the evaluation. This analysis will include: |
| (a) | the relevance and usefulness of the information required and its adequacy for managing contracts and ensuring delivery of quality services; |
| (b) | Whether providers are meeting performance standards, targets and guidelines; |
| (c) | Differences in performance among providers and in the uptake of services and possible reasons for this; |
| (d) | The proportion of funding spent on provider administrative costs compared to funding spent on the provision of services; and |
| (e) | Whether providers are meeting reporting and data requirements adequately. |
| 7 | Analysis of management and administration of the programme |
| 7.1 | BERL is to evaluate the effectiveness and efficiency of the management and administration of the BIZ programme by the Ministry, to identify areas for improvement and include these results in the evaluation report. This will include, but not be limited to, considering: |
| (a) | The Ministry's administration and management systems for contract management and for dealing with BIZ providers generally; |
| (b) | The tendering process through which providers were selected; |
| (c) | The adequacy and appropriateness of contracts and performance measures used in them; |
| (d) | What measures are taken by the Ministry to monitor performance of providers and the effectiveness of these; and |
| (e) | The views of providers on the performance of BIZ Unit staff. # |
| (# evaluated as part of the Analysis of BIZ Provider Interviews rather than in this report) |
Each of these topics is considered in the following sections, which provide an overview of relevant information, and a discussion under each of the above topic headings. Sections 2 and 3 correspond with sections 6 and 7 respectively of the specification of Evaluation Services set out above.
We should stress that there is a considerable overlap between the issues relevant to each heading, so we have generally provided an extensive discussion at an early stage of the report, with reference back to the original discussion where the same or a similar issue appears later.
1.3 Methodology
We gathered information for this section of the evaluation in the following ways:
- We examined each contract, and collated a range of information from them.
- We obtained a copy of the database maintained by the BIZ Unit. The information contained in the database is described in section 2.1.
- We undertook three interviews with BIZ Unit staff members, and two interviews with the Economic Development Policy Unit (which has policy responsibility for business assistance programmes), to discuss various issues concerning the implementation and administration of the BIZ programme. We also attended a meeting in which the Ministry's auditors of BIZ providers discussed their findings.
- We examined a sample of provider reports and records of communications between providers and the Ministry. The purpose of this was to get a feel for the nature of the concerns and problems of providers, and how the BIZ Unit deals with issues as they arise. This was to form a basis for the questions we were to ask the BIZ Unit and the Economic Development Policy Unit.
- We obtained and analysed a large amount of information from the Ministry relating to the BIZ programme including, but not limited to:
- Request for Proposal: Business Development Programme;1
- Request for Proposal: National Business Information Service;2
- a copy of the Negotiation Report;3
- BIZ Programme Provider Review;4
- Evaluation Report: Business Development Programme5;
- Assessing Awareness and Understanding of the New BIZ Programme;6
- BIZ Development Unit Business Plan 2000/2001;7
- A variety of Cabinet papers relating to the establishment of BIZ; and
- the BIZ Unit's Key Procedural Manual.
- We examined 36 compliance review reports for 31 BIZ providers. The insights gained from these formed the basis of questions we asked the BIZ Unit (see section 2.1 for a description of the compliance review process).
For the purpose of this evaluation, we did not distinguish between the sole BIZInfo provider and the remaining providers, given the contract management process does not treat BIZInfo differently to the remaining contracts. We have also avoided discussion about any individual provider and its relationship with the Ministry (which would be inevitable considering BIZInfo separately).8
1.4 Acknowledgements
We gratefully acknowledge the assistance of Ministry staff, particularly Kay Switzer, Stephen Knuckey, and Gregory Crott and his colleagues at the BIZ Unit; and of fellow members of the Evaluation team. This report would not have been possible without their insightful comments, contribution of time and positive attitude.
Needless to say, the responsibility for the contents of this report is entirely BERL's.
Back to Top