In the previous section of the report we presented a summary of the conclusions from each of the individual research components. Here we have drawn these separate strands of thinking together and addressed the key questions that the Ministry of Economic Development wished to address in the evaluation of BIZ (as listed in Section 1.3 Project Objectives).

3.1 The Future of BIZ

Under this heading the project team was asked "to assess whether the government should continue to deliver the BIZ programme in its current form, including, but not limited to, whether the current approach is meeting the right business assistance needs, and is the most appropriate way to meet those needs". A number of elements of the project offered perspectives on this question, and the following section draws upon all of the separate reports on these elements.

The first comment that must be made is that all of those who responded in any form to this evaluation were insistent that BIZ should stay. There were many suggestions for improvement, but on the whole there was a highly favourable response to the programme. Because of this strongly positive response in this section of the report the project team has focused on identifying the key issues in "the future of BIZ".

3.1.1 The Current Situation

Currently there are two components of BIZ: BIZinfo and what is described in this report as BIZ training. These are intended to be separate but complementary. BIZinfo offers its services to virtually any client with a business problem, with referrals to a wide range of business information services, including BIZ training providers. BIZ training providers offer more intensive services to a narrower group (existing SMEs), and obtain most clients from their own marketing, networks etc (i.e. are not expected to rely primarily on BIZinfo referrals).

Both should continue as distinct services. As well as the modifications identified below, the complementarity between them could be enhanced, for example though better provision of information between the two in both directions, without affecting their distinct identities.

BIZinfo: Delivery Mechanisms and Scope

At present the key component of BIZinfo is its database, which has entries on some 900 organisations throughout New Zealand that are assessed as contributing to the development of management capability of SMEs. These organisations are in addition to the 46 that were contracted to the Ministry to offer BIZ training programmes.

It is critical for the future of BIZinfo that this database should continue to be enhanced in terms of both its content - the inclusion of up-to-date information about as many competent providers as possible - and its accessibility.

Therefore, a key issue for BIZinfo is the mix of mechanisms used to deliver the information on the database. There are two main ways for clients to use this database; through the website (which can be accessed directly or at a local BIZinfo centre) or by telephone (either via the 0800 number, which is answered at a regional call centre, or by calling a local BIZinfo centre). A hard copy of approximately half of the organisations on the website has also been published recently and is available through BIZinfo centres and BIZ providers (the Industry NZ Small Business Assistance Directory).

During the process of the evaluation there was considerable discussion on whether this mix of delivery mechanisms is the most cost-effective. One of the main questions was whether it is reasonable to expect the local BIZinfo centres to deliver the very specialised function of information provision (which depends on highly effective needs diagnosis) without significantly more input into training. This reflects the quality of service that can reasonably be expected from organisations which receive modest funding to provide BIZinfo as an addition to their primary business assistance functions.

A parallel question was whether there is real value in local and/or regional delivery of information (through BIZinfo centres and regional call centres respectively), given the effort that has been put into building a comprehensive, centralised database.

It is of course axiomatic to state that development of the BIZinfo website needs to be an ongoing activity, given increased use of the Internet and advancing technology.

A further issue for BIZinfo is the scope of the services to be offered. There has been some discussion on the value of providing an advisory function as well as referral information. This is based on the argument that as some diagnosis of the client's business problem is required to make an appropriate referral, it is a logical extension to offer advice to deal with straightforward problems.

However, such an extension would present some problems, including:

defining boundaries between when advice is appropriate and when referral to a specialist adviser is required;
potential liability for consequences of advice given;
training and resources required for advisers; and
potential competition with private sector advisers.

Providing an advisory function is more substantial than a simple extension of BIZinfo. If the BIZ programme were to include business advisory services, this would represent a major expansion of its activities, possibly a distinct third service component. This should be considered by the Government and the parent body of BIZ, Industry NZ.

Industry NZ is presently defining its own package of services and organisational image. Within those, it might or might not see business advice as an appropriate service to offer, and/or BIZ as the preferred mechanism. That is a matter for Industry NZ to resolve, not this evaluation.

BIZ Training: Identifying Needs

The "local autonomy model" chosen as the delivery framework for BIZ assumes that local people and organisations are in a better position than central government to a) identify local needs and b) develop responses to meeting these needs. To a certain extent this seems to have been borne out: in the Provider Survey almost all the organisations visited had been involved in business development or local economic development for some time, and were able to access a store of expertise in preparing their proposals for the Ministry.

The proposals submitted were designed to indicate how each provider could contribute to the overall goal for the BIZprogramme: to improve the management capabilities and performance of existing SMEs. It was assumed that the bidding agencies would be competent in:

identifying the levels of management capability (in different topics) within the various groups of business people in their communities;
identifying existing providers of management training and development services; and
understanding current issues in adult learning.

In practice, although a number of providers were successful in identifying the "correct" needs (in terms of topic and "level") in relation to the particular circumstances of their community, this was not always the case. Some were not able to achieve this due to lack of time, skill or other resources, whilst others chose to focus on delivering training programmes in single topic areas (e.g. computing), rather than on the needs of a whole business community, with individuals at different levels of capability.

Those providers that did attempt to identify the needs of a whole community used a number of different processes. Some consulted widely, accessed extensive client databases, and/or undertook direct research with potential clients. Others were less comprehensive in their processes and possibly focused on developing proposals that included training topics that could be easily delivered by the existing organisation.

It is uncertain whether these organisations identified the management capability needs of a local community more effectively than would have been the case if the Ministry had taken the lead. This was particularly apparent in relation to the target groups. These groups (Maori, Pacific Peoples and women) were identified by the Ministry (rather then the providers). While some organisations were already working with one or all of these groups, others were not. From what we told during the Provider Survey, methods of identifying the needs of these groups (within the short time frame available) were simplistic and mostly based on past experience rather then any real attempt to assess "management capability needs".

The identification of these target groups also had the effect of leaving out the valuable experience that some providers had with other groups (e.g. start-up enterprises and immigrants). There was considerable discussion over the omission of these groups. Some providers had experience with these groups while others commented on this either from a perspective of equity or in terms of the potential value to the New Zealand economy.

In terms of meeting the needs identified it was again over to the provider organisations to decide upon the best way of meeting these needs. Some organisations, experienced in "improving management capability" developed a framework of complementary services, for example including pre-assessment/diagnosis, training and/or mentoring, seminars etc. Others selected a single service (e.g. seminars) and focused all their efforts on this.

Charging for Services

BIZ was established on the basis that the services are provided at no charge. The matter of charging has since been raised, as some providers have experienced significant numbers of people failing to attend courses (although most providers have developed other ways to deal with this). Other providers have put forward the view that charging may make individuals value the service more and be more committed to implementing the new knowledge gained.

An issue of concern about BIZ is the possibility of displacement, i.e. that its free services could displace those offered commercially by other providers. BIZ is implicitly targeted at types of business assistance not offered through existing commercial sources, and it could be argued that risk of displacement is reduced by the clear separation between free-of-charge and commercial services.

The corollary of this is that the absence of charging reinforces the requirement for a robust process for the identification of needs, to ensure that BIZ is supporting those services where there is a gap in commercial provision.

From the Government's perspective, charging may have some attractions in giving it leverage to secure BIZ services for more SMEs within the same funding - albeit with some policy issues about charging rules.

In the Awareness Survey those respondents who were aware of BIZ were asked if they knew that the services offered were free and how this influenced their likelihood to use them. The majority of BIZ aware respondents did know that the service is free, but there was a diversity of views regarding the impact of "free services" on programme uptake: While 32% said that it would make them much more likely to approach BIZ (emphasis added), 25% felt that it would make them a bit more likely to approach BIZ, and 37% indicated that it would make no difference.

Pacific Peoples are much more likely (53%) to approach BIZ knowing it is free than the general population. This was also a characteristic of recently established businesses with 42% indicating "much more likely".

(Related issues, which are outside the scope of this report and not discussed here, are external advertising and sponsorship of BIZ providers and services.)

3.1.2 Modifying BIZ

The project team supports the notion that developing management capability is dependent on the actual providers of services and organisations that can "sign-post" potential users towards these and other providers. Hence the team supports the continuation of the two elements of BIZ.

However it is clear that more thought needs to be given to the delivery mechanisms for BIZinfo, particularly given the time that has already been invested in building up a central database. As this now seeks to provide comprehensive coverage of sources of assistance to SMEs throughout New Zealand, the argument for the need to offer local information, either through the BIZinfo centres or the regional call centres, is weaker. While this is clearly an issue for the BIZ Unit and the successful tenderer, it could be argued that a single, dedicated call centre with an enhanced referral database and further training for call centre staff could provide better quality service delivery and more consistency in meeting client needs.

In terms of delivery of BIZ training, the project team supports the notion that different communities will have different needs in terms of management capability, but we believe that this should not conceal the fact that there are also general concepts, which may translate into specific courses and/or delivery methods (e.g. mentoring).

While we consider that the local autonomy model is appropriate, to be wholly effective it needs a significant investment of time by providers (and to a lesser extent the Ministry and/or Industry NZ). In the initial contract round this was seriously compromised by the very short timeframes for tendering and contract negotiation, and we cannot state with any confidence whether the current mix of services offered under the programme fully meets local demand.

We conclude that while the local autonomy model has much to recommend it, the approach needs to be modified. A revised approach should allow the Ministry/ Industry NZ and local groups to identify local needs in partnership with each other, and other relevant stakeholders. This would also allow a more co-ordinated approach to be taken between the "general providers" (i.e. those delivering a range of topics through a number of delivery methods) and the "subject-specific providers" (i.e. those delivering specific skills, possibly over a wider geographical area).

We suggest that the next bidding round is supported by an explicit framework that assists providers to:

understand the broad concepts of "management capability";
identify the particular needs of groups within the community (whether they are formally "targeted" by the Ministry or not); and
design a programme (whether across a broad range of management competencies or in specialist areas) that meets the needs of the target groups and develops existing strengths at both a community level and an individual level.

In practice this may mean that the Ministry undertakes some new baseline research on management capability in New Zealand SMEs, which providers can use as the basis for developing specific proposals that address needs in terms of target groups (for example business start-ups, immigrants as well as others), the level of management capability and the most appropriate type of service (training, mentoring etc).

We also suggest that in the next contracting round the bidding organisations should describe the process that they have followed to identify the needs of their target clients. Providers offering "general" programmes should also be asked to describe the ways in which they are going to draw upon the expertise within their communities.

The effect of these changes is likely to be more structured services from BIZ providers. The majority of services will be

providing a range of management skills at a basic level;
delivered by local organisations
more comprehensive, with enhanced assessment and diagnosis, leading to seminars and courses (as presently offered) and followed by enhanced one-on-one mentoring and coaching, and networking.

This does not necessarily mean that each provider will be expected to provide all of these services. Rather, we would expect providers to work together at the local level to assemble integrated packages for clients.

There may also be gaps where particular skills beyond the basic level are not sufficiently well provided, or where particular groups have specific needs. Having identified these gaps through a needs analysis, the basic BIZ service offering could be complemented through provision of specialist courses etc. Organisations providing such services would not need to emphasise pre-assessment and post-follow-up to the extent of basic providers; and could operate on a multi-region and national basis, rather than just locally.

In relation to charging, the project team concluded that this is a complex issue which has implications for the quality of services delivered, displacement of private sector providers, (as noted above), and a number of other related issues. At the moment the project team does not support charging (see also the discussion under Section 3.5.2).

3.1 The Effectiveness and Impact of BIZ

Under this heading the project team was asked "to assess the effectiveness and impact of BIZ, including, but not limited to, whether the management enhancement capability and referral services are achieving their objectives".

The starting point was to consider the overall goal for the BIZprogramme, which is to improve the management capabilities and performance of existing SMEs. BIZinfo and the BIZ training component have their own objectives and the following sections address these in turn.

3.1.1 Objectives of BIZ Training

The specific objectives of the BIZ training component are to:

improve the management capability of SMEs;
increase the understanding of SMEs of the value of assistance so they decide to take responsibility for seeking out advice and assistance on improving their capabilities;
improve coverage of management capability enhancement assistance, both in coverage to SMEs and coverage of services; and
improve the management capabilities and understanding of the value of assistance of identified target groups.

In terms of these four objectives, the following comments can be made:

Improving Management Capability

At the end of June 2000, the BIZ training component of the programme had been delivered to some 31,000 participants, representing just over 7,000 individual enterprises. This means that BIZ is reaching a very small proportion of New Zealand's 255,000 enterprises that employ fewer than 50 full-time staff (this excludes the approximately 80,000 enterprises engaged in agricultural production). In addition, the Awareness Survey revealed that only 13% respondents had heard of BIZinfo (unprompted). However, this percentage has improved considerably over the last year (up from 6%), when combined with prompted awareness it totals 53% (up from 33%), and is at a favourable level when compared with other products.

Whether the management capability of these clients has been improved is not easy to determine from the two surveys of SME managers, although satisfaction with the service delivery was viewed as an important indicator. The Client Survey showed that satisfaction with the content and delivery of the courses was high and in the Awareness Survey more than half of those contacting BIZ rated themselves as "pleased" with the experience. The difficulty is that the BIZ Unit only requires providers to report on numbers making use of different services. While some providers do collect information on client satisfaction, the methods of assessing this are not particularly robust.

Another perspective on the question of improving management capability was to ask respondents about the benefit that they saw themselves as having received. In the Awareness Survey nearly three-quarters of BIZ users rated the benefit of using one or more services as "very good" or "fairly good". A quarter rated the benefits as "not good, but not bad" or "not good at all".

Similarly, respondents to the Client Survey believed that the BIZ courses had been beneficial, particularly in helping them to plan for the future. They also reported fairly high beneficial impacts for understanding customers and how to market to them; raising the business skills of staff; handling the financial needs of the firm; reducing waste of time and materials; transferring good ideas from a business network; and raising the quality of the product or service.

Nearly half (46%) of the respondents to the Client Survey reported increases in business turnover over the previous year, 39% reported higher business profits and 32% reported higher market share. The number of employees is higher for one in five (21%) compared to twelve months ago and the level of debt has decreased for one quarter (27%) compared to a year ago.

This provides some evidence for performance improvements of BIZ clients. The results are encouraging in that all of the indicators change in the direction expected. Of course, these data reflect the perceptions of the owner or key decision-maker and may not reflect the reality.

The project team also cautions that these data cannot be seen as demonstrating that BIZ leads to higher performance in client businesses. Rather there is an association between BIZ usage and improvements in performance. This distinction is important, as there are a number of factors that influence business results. It is possible, for example, that businesses undergoing the sort of growth shown above would be motivated to seek out BIZ services as their organisations grow and change.

From the Awareness Survey, all businesses reported similar changes in the previous year, regardless of whether they were BIZ clients or not. While businesses that had attended BIZ training appear to have performed marginally better than non-BIZ clients in terms of turnover, market share, job growth and profitability, the difference did not appear not significant.

Despite these comments, the lack of a systematic approach to assessing a client's management capability at the point at which they entered the BIZ programme made it difficult for the project team to assess whether improvements in management capability had resulted. Until there is an efficient system in place for assessing individuals (and enterprises) as they "enter" and "leave" BIZ, it will always be difficult to assess whether BIZ has met its primary objective.

Increasing the Understanding of SMEs of the Value of Assistance

Based on the available data, the project team was not able to form an opinion on whether this is occurring or not. If this objective is retained as an important element of BIZ it will be necessary for the providers to develop a method of assessing whether it has been met. Currently the sole indicator seems to be whether users of BIZ are willing to undertake further training, and although this is high (70% according to the Client Survey), the project team notes that this does not necessarily indicate an increased understanding of the value of assistance.

Improving Coverage of Management Capability Enhancement Assistance

This objective is concerned both with increasing the type of service available to SMEs and improving coverage throughout New Zealand. The project team is not aware of any providers established solely for the provision of BIZ, and therefore assumes that they were existing organisations, presumably already offering some business assistance. Despite this, it appears that this objective has been met, as the provision of free training courses and seminars on the scale that has been achieved by BIZ is certainly something that is new in business assistance in New Zealand.

What is not clear is whether the BIZ courses are displacing existing providers of management capability training and development. More than half of the respondents to the Client Survey indicated that their attendance at BIZ courses and seminars was not replacing other purchases, although almost 40% indicated that their attendance was partly additional and partly substituting. Another perspective came from the Provider Survey, although this was limited, with only a small number of providers aware of substitution as a potential problem.

However, the level of reported substitution (and the potential for displacement of existing operators) needs to be monitored, especially as BIZ moves into "phase two". It is likely that there will be pressure on BIZ providers from their clients (and therefore on the Government from the providers) to allow and fund them to develop "second tier" courses. These should not be funded by BIZ however; as clients develop an awareness of their own requirements (and limitations) from basic course, they should be more willing to pay for development of higher level skills.

If BIZ is used to fund second-tier courses, there may be genuine displacement in the training and development industry as BIZ competes with those who are already servicing these more advanced needs.

Improving the Management Capabilities and Understanding of the Value of Assistance of Identified Target Groups

The statistics collected by the BIZ providers show that women make up 61% of all participants, Maori comprise 24%, and Pacific Peoples 9%. This represents higher rates of programme participation by these groups than would be implied by representation within the population of SME owners and operators⁴. If participation in BIZ training can be taken as a proxy for improving management capability, then these statistics suggest that this objective has been met.

As was noted above, the project team was not able to form an opinion on whether the target groups are increasing their understanding of the value of assistance, although it is possible to take the number of respondents intending to attend future courses as a positive indicator.

3.1.2 Objectives of BIZinfo

The objectives of BIZinfo are to:

be a first point of contact for SME business assistance enquiries;
improve awareness and reduce confusion of SMEs on the availability and value of business assistance;
improve coverage of the provision of information on available public and private business assistance both in terms of coverage to SMEs and coverage of information; and
ensure effective market penetration for identified target groups.

In terms of these four objectives, the following comments can be made:

Be a First Point of Contact for SME Business Assistance Enquiries

Those seeking the services of BIZinfo have a number of ways of accessing the service, including the website, an 0800 number and via a local BIZinfo centre. The rationale for this variety of routes is that this is the best way of ensuring coverage to all of the potential users of BIZ.

This decision appears to have been borne out by the statistics collected by BIZinfo which show that different user groups show distinct differences in the ways they access BIZ. For example, Maori and Pacific Peoples clients comprise a greater proportion of total clients who visit local BIZinfo centres in comparison to the proportion of Maori and Pacific Peoples callers to the BIZinfo 0800 number.

Whether BIZinfo is actually providing the "first point of contact" for business assistance enquiries was addressed in the Mystery Shopper survey, and an average of 66% of respondents indicated that they understood one of the roles of BIZinfo was to provide "a first point of contact". However, this varied - recording 56% for the 0800 number and 83% for the provincial sites surveyed. It was also not the role most commonly identified - this was the referral capability.

The project team concluded that while BIZinfo is valuable for those who are new to "business assistance", for more experienced users of such services, known routes are likely to be used. For example, during the Provider Survey a number of respondents commented that once a client has a relationship with a particular provider they tend to return to the provider for further advice rather then re-contacting BIZinfo. As more and more SME managers participate in the BIZ programme the Ministry may need to recognise that BIZinfo may only be an appropriate "first point of contact" for those new to business assistance.

Improve Awareness and Reduce Confusion of SMEs on the Availability and Value of Business Assistance

By the end of March 2000, local BIZinfo centres had received 7,854 visits, the BIZinfo call centres had received 16,327 calls, and the website had had almost 3 million hits and about 500,000 page accesses. The effect of these contacts is reflected in the Awareness Survey, which shows that overall awareness for both BIZinfo and BIZis significantly higher amongst SME owners than it was a year ago. This suggests that SMEs are getting the message about "where to go" for business assistance.

It is not clear whether confusion is being reduced, particularly as the local autonomy model means that many organisations delivering BIZ are also operating under their existing names as local development agencies (LDAs). Some of these organisations also offer BIZinfo, and a number provide services on behalf of CEG, some of whom expect to be doing the same for the new Maori Business Facilitation Service.

Improve Coverage of the Provision of Information on Available Public and Private Business Assistance

BIZinfo has established a database of business assistance that is directly accessible to users via the website. The database also provides the call centre operators and the local BIZinfo centres with their main source of information on where to refer callers. This database (which contains approximately 900 organisations and lists over 2,500 training courses and other services) is completely new in New Zealand and represents a major improvement on what information has been provided before.

A printed version of the database which includes some 430 business assistance organisations has just been produced by BIZinfo and Industry NZ, giving users a number of ways of accessing the available information. It should be noted however that keeping the database up to date will always be a challenge and a major expense.

As noted in Section 3.1.2, the project team suggests that further consideration needs to be given to the future configuration of the delivery mechanisms for BIZinfo.

Ensure Effective Market Penetration for Identified Target Groups

As noted above, by the end of March 2000, local BIZinfo centres had received 7,854 visits, and the BIZinfo call centres had received 16,327 calls. Split by target group 37% of the users are female, 16% Maori, and 3% are Pacific Peoples. Users of the call centres are 42% female, 15% Maori and 3% Pacific Peoples. Again, using the proxy of self-employment as a basis for assessing the expected numbers from these groups that are using BIZinfo, this is higher than expected.

3.1.3 Improving Effectiveness and Impact

In terms of improving the effectiveness of the BIZ programme, a number of key issues emerged from the project. Some can be addressed at the provider level, but most have implications for the policy unit of the Ministry responsible for industry programmes and the BIZ Unit itself. In the next contracting round there needs to be a focus on:

Building a "Community" of Competent Providers

Improving the effectiveness and impact of BIZ depends partly on operational issues that can be addressed by the BIZ Unit through the process of negotiation with the providers. However, if this is to be successful it depends on the competency of the providers to deliver the required improvements. If this is to occur then the BIZ Unit must have access to a "community of providers", with organisations that cover all of the potential target groups at all of the levels needed. From the perspective of the project team, this is one of the most important issues that can be addressed: the local autonomy model depends on organisations with the capacity to deliver.

The BIZ Unit needs to be proactive in promoting this development. Therefore, a greater emphasis should be placed on the BIZ Unit working with providers, and providers working amongst themselves, to develop the competencies of providers and improve the quality of services delivered. This needs to be done through a provider development agenda - a deliberate process to identify and share best practices, e.g. performance standards, course frameworks, diagnostic processes, course materials, approaches to marketing etc.

We note that this has resource implications; it may not be possible within current resources.

It is fundamental to effective service delivery that providers must employ highly skilled trainers with access to high-quality resources and effective systems. In the next tender round providers should be required to demonstrate that these are in place and are being maintained (for example, through a budget for staff training and training material development).

Clarifying the Relationship between BIZinfo and BIZ

As the Awareness Survey demonstrated, most SME owners were not able to makea distinction between BIZ and BIZinfo. Confusion between the two components of BIZ was also noted in the Provider Survey, with most respondents suggesting that having a number of different outlets (call centres, local BIZinfo centres etc) providing BIZinfo has not assisted the situation. There seemed to be particular problems with the local BIZinfo centres, which operate within an organisation that may or may not be a BIZ provider.

The project team concludes that the current method of providing access to BIZinfo through different routes needs to be reconsidered. While it is clear that different outlets will appeal to different client groups, (an argument in favour of keeping a variety of outlets), it is also clear that operationally it is difficult to ensure standards are maintained.

The project team proposes that:

BIZinfo and BIZ are retained as separate services;
BIZinfo continues to operate the website and a call centre function; and
prior to the next contract round NBIS (the BIZinfo provider) and the BIZ Unit reassess the delivery mechanisms for BIZinfo, in consultation with BIZ providers.

Effectively Communicating with Potential Clients

In terms of provider issues, there was still some expectation that BIZinfo should be marketing BIZ to prospective clients. However, despite this continuing belief, most providers were undertaking their own promotion, and the Client Survey and the Awareness Survey provided a number of suggestions on how best to communicate with clients.

At present media advertising (for newer business) and direct mail advertising (for established business) are the ways that firms come to hear about BIZ services. Referral from BIZinfo is relatively low. In terms of awareness, television, mail-outs and print advertising were the most common media by which SME owners became aware of BIZ, suggesting that for providers it is possible to segment their target groups and select the most relevant form of advertising.

However, the Awareness Survey showed that the perception of BIZ amongst owners and managers of SMEs is almost exclusively related to BIZinfo. There is little knowledge of BIZ training outside the relatively small number of owners and managers of SMEs who have used its services.

This is potentially a matter of some concern, in that the part of the BIZ programme which uses most of its resources, and is presumably expected to deliver most of its benefits, is so little known amongst potential clients.

(Indeed our use of the term "BIZ training" is a reflection of the fact that this, by far the largest component of the programme, has no official title.)

We do not propose to discuss a communications strategy for BIZ training, especially as BIZ training providers have developed their own individual marketing to a point where they are mostly able to fill courses with eligible clients. Moreover, we note that Industry NZ is considering its own marketing and branding, so any consideration of this for BIZ training needs to be consistent with what Industry NZ does in its own right.

However, it is clear that the effectiveness of BIZ training could be enhanced if it had its own distinctive identity and "brand" which could be communicated to potential clients. The development of this needs to occur as part of Industry NZ's communications strategy.

Assessing Client Needs

There is no systematic approach to diagnosing the needs of individuals (or enterprises) as they enter the BIZ programme. From the Provider Survey it was clear that pre-assessment is only offered by a small number of providers, and the Client Survey showed that pre-assessment or diagnosis of individual firm needs appears to be either not done or not consciously experienced by nearly half of BIZ clients.

If BIZ is to be developed in the future on the basis of its positive impact on individuals and enterprises then pre-assessment needs to be an integral component of the service.

Delivering Quality Services

The effectiveness of BIZinfo depends upon the quality of its base information (the database) as well as the skill of those who interact directly with clients (call centre staff and those answering the telephone at local BIZinfo centres). Similarly, the effectiveness of BIZ training depends on the quality of the material that underpins the delivery, and the skills of those individuals who interact directly with participants in all the different components of BIZ (courses, seminars, mentoring etc).

Given that the BIZ Unit has contracted a single organisation to deliver BIZinfo, it should be straightforward for quality control to be maintained; even allowing for the difficulties raised by the presence of the local BIZinfo centres. But BIZ training presents quite a different situation, and the project team concludes that more could be done to ensure that users of BIZ are offered a high quality service from all providers. While the local autonomy model will encourage the development of different services, it should not make it impossible to provide services that are consistent, both in terms of delivery and supporting materials.

Measuring Effectiveness and Impact

Measuring effectiveness and impact is difficult. It is even more complex in relation to BIZ, where the objectives of the programme are implicitly concerned with affecting both individuals and enterprises. That is, not only do the objectives of BIZ focus on the individual, there is an implicit assumption that improving the management capability of individuals will have a positive impact on the performance of their enterprises. Because measurement is complex it is common for those responsible to assess satisfaction as a proxy for impact.

This current situation (where the focus has been on satisfaction rather than impact) will continue unless those responsible for the next round of contracting take a lead in improving present practice. This should be addressed if the suggestions in Section 3.1.2 are put into practice (i.e. those bidding need to be more explicit about the process that has been followed in order to identify needs).

Developing a "Second Level" for BIZ

The first round of BIZ has generated a cohort of BIZ clients who are now experienced users of business assistance. On the whole they are also satisfied users and likely to continue to see BIZ as an appropriate source of information and a trusted provider of services. However they are clear that their future needs will be different. In the Client Survey respondents rated mentoring as the form of assistance most likely to yield a good result for the client firm. This was particularly the case for Maori clients.

The implication for BIZ providers (and therefore the BIZ Unit) is that they need to a) seek new topics for training (with the caveat already noted about competing with private sector trainers); and b) find ways to deliver services of the type that are required.

Expanding Eligibility to Include Start-Up and Pre-Start-Up Businesses

The omission of start-up and pre-start-up businesses from BIZ eligibility has consistently being identified as an area of weakness with the programme.

In particular, Maori who fall into the category of not yet being ready to describe themselves as being "in business" indicated that the development of business skills is not a high priority. Instead, they need to develop a broader range of "enterprise skills" - a term used to refer to a range of pre-business skills which may include topics such as confidence building, budgeting etc. The end-point of a course of this type will be when the participant assesses whether self-employment is a suitable option.

The history of the omission of start-up and pre-start-up eligibility is that during the implementation of BIZ, the decision on expanding eligibility for the programme was deferred pending reviews of other programmes that catered for these groups. Currently, the only free government assistance for start-up and pre-start-up groups is for Department of Work and Income clients on the Be Your Own Boss scheme (although the scheme is also open to non-departmental clients who pay a small fee to attend). The timing is now appropriate for consideration to be given to this matter.

This issue is one that Industry NZ will need to decide how best to deal with, i.e. whether BIZ should be expanded to include these groups, or if these groups should be catered for through other programmes.

It is acknowledged that any significant provision of services to these groups would require additional funding.

3.2 Efficiency and Value for Money

Under this heading the project team was asked "to assess whether BIZ provides value for money and efficient service delivery, including, but not limited to, whether the approach used to deliver services is as efficient as possible".

In the context of the project we considered these questions in relation to i) the BIZ providers and ii) the BIZ Unit. However we begin by noting that the concept of efficiency is one which needs the evaluator to take a view on the level of resources needed for the delivery of a particular set of services - possibly by comparing service delivery by different agencies and/or over different time periods. In this project there was no benchmark for comparing the allocation of resources, and as a result the comments made below are general rather than specific.

3.2.1 Efficiency of BIZ Providers

When interviewing the BIZ providers the researchers sought their views of their own effectiveness, i.e. how well they were doing their job of delivering services to SMEs. The focus was on the providers' views on this topic rather than measuring effectiveness against a pre-set standard (such as programme objectives). This topic was covered in the previous section of this report. However, whilst carrying out this part of the project (which meant visiting 20 of the providers and talking to another 12 by telephone) it was possible to make some general observations about the efficiency of those organisations being visited.

On the whole the standard was high, and almost all appeared "well managed". Their offices were well equipped, there was evidence of systems for dealing with enquiries, and during their visits the researchers were not aware of any events that could be described as demonstrating inefficiency. However, in the course of preparing the Client Survey, it became evident that there are not good quality systems for registering participants in all cases. Auditing, follow-up performance surveys, and eligibility monitoring are compromised when this occurs and an improvement in the precision of record keeping is desirable.

From another perspective, the BIZ Unit reports that nearly all providers deliver their quarterly/monthly reports on time and over the 18 month duration of the programme it has identified few providers that are "not delivering". Of those that do fall into this category it is notable that the BIZ Unit has identified them quickly and taken appropriate action to minimise the loss to the Crown and the risk to the provider's clients.

The main area of concern is the apparently wide range in providers' administrative costs found in our analysis of provider reports. As already noted, there is no benchmark as to a reasonable level of administrative costs.

Moreover, there is no ability to assess unit costs of services, i.e. costs per course, per participant or per participant-day, which are a fundamental measure of the cost-effectiveness of individual providers, and of the programme as a whole.

3.2.2 Efficiency of the BIZ Unit

From our observations we also conclude that the BIZ Unit can be described as efficient. There is a contract management system in place that is regarded positively by the providers, there is almost constant work being undertaken on improving systems and processes, and the comments about the BIZ Unit from the providers indicates that they rate the staff in the unit very highly in terms of responsiveness (to their enquiries).

However, we again note the absence of a base against which to compare the performance of both the providers and the BIZ Unit. Nor are there any standards for guiding both the providers and the purchasers in terms of "acceptable" unit costs per training course (and/or per seminar or mentoring session).

This would clearly have been a difficult exercise. However there may have been some service delivery models in existence that the Ministry could have reviewed during the planning phases of the BIZ programme. In this context we understand that the Ministry did consider the health system and concluded that there were too many differences between health and business assistance for the health model to be of direct relevance. Although this may be true, the project team ask whether consideration was also given to models of delivering training services that are more similar to BIZ - for example those funded by ETSA and CEG.

3.2.3 Improving Efficiency

The local autonomy model that was selected for the delivery of BIZ was based on the assumption that local expertise would be drawn upon to ensure that ways of enhancing management capability would be relevant to the specific situation. This model meant that the Ministry provided few guidelines to the providers as to the level of costs that would be acceptable. It would now be possible to calculate "unit costs" for the different providers, as a way of assessing "value for money". However, the project team has not done this, and notes that if this exercise is carried out the variation between the providers will need to be considered. For example variation between providers in terms of size, proximity to clients, methods of service delivery, etc., all have an influence on "unit costs".

Another factor that makes the concept of value for money difficult to assess is the absence of a truly competitive bidding round. While only a proportion of those that expressed interest in BIZ were awarded contracts, it appears that in some areas there was little competition. The limited time available for the programme to be launched also suggests that in some areas it may have been difficult to identify the most appropriate providers and negotiate the best available services.

3.3 BIZ Programme Implementation

Under this heading the project team was asked "to assess the conduct and operation of the programme implementation of BIZ, including, but not limited to, whether the implementation of, and operation of, BIZ is effective and appropriate".

3.3.1 Background

As explained in Section 1.1, prior to BIZ, the Ministry of Commerce funded business assistance through the Business Development Programme (BDP), delivered through Business Development Boards (BDBs). A review of the BDP recommended the disestablishment of the BDBs as Crown entities and the launch of BIZ. There was only a short period from the time that this announcement was made and the target date for the first BIZ agencies to be operating, and in hindsight it is obvious that this was not ideal either for the BIZ Unit or for those organisations seeking to be contracted.

However, since the establishment of BIZ there has been a lot of positive activity. The BIZ Unit has become operational, BIZinfo has developed, the providers (on the whole) have delivered to their contracts, and the new entity of Industry NZ has been established (and has assumed responsibility for BIZ).

Based on the experience of the first eighteen months of the programme, there are a number of lessons which should be applied to the operation of BIZ.

3.3.2 Improving Programme Implementation

Change Eligibility Criteria

While BIZinfo offers information to anyone who seeks it, access to BIZ training is restricted to those who are already operating a business. It has already been noted that it is not always easy to determine what constitutes a business, particularly for providers that are relying on information from potential clients. A further question is whether it is helpful to exclude those who may be able to make use of "business skills", even though they are operating in a context that may not be officially described as a business (e.g. a community-based enterprise).

Provide Providers with Time, Information and Resources

The local autonomy model has provided the Ministry with an effective way of maximising the valuable experience of individuals and organisations that have been involved in business assistance or local economic development. These individuals and organisations have built up their experience over a period of time, often by contracting to other government agencies. However it is a new model for the Ministry, and if it is to be as effective as it could be, then the providers need to be supported in a number of ways which enable them to contribute to their own organisations and the programme as a whole.

As already mentioned in Section 3.2.3 Improving Effectiveness and Impact, every effort must be taken to ensure that the competency of providers is developed. Under a service contract approach this is often assumed to be the responsibility of the provider. However, as this evaluation has shown, the lack of a baseline for purchasing the services (and the subsequent negotiation process) suggests that insufficient resources were factored in for this aspect of BIZ.

Improve the Tendering Process

The first round was marred by a lack of time and a general sense of confusion amongst potential providers who were entering into a process that was new for many of them. The next round should remain transparent and competitive, but we suggest that the Ministry allows a certain level of flexibility to exist, so that it can identify particular providers and/or programmes that it wishes to have delivered, during and outside the tendering process.

Modify Contract Structure

In terms of the contract structure, some BIZ providers were happy with the degree of flexibility it offered. However, others suggested a more structured framework, with a "menu" of services that the provider can pick and choose from in terms of delivery. This was seen as providing a useful framework for those involved in bidding for BIZ for the first time, or for those organisations attempting to assess how the provision of BIZ services fitted with those contracted to other funding agencies (for example CEG and/or TPK). It would also enable the Ministry to communicate with current and potential providers about its strategic objectives.

Think in a Conceptual Way about Integrated Programmes

The project team notes that BIZ is now under the umbrella of Industry NZ. We have no formal recommendations about the future role of BIZ within Industry NZ, as this is beyond the scope of the evaluation.

However, the establishment of Industry NZ, and the establishment of its first set of assistance schemes, does provide an opportunity for BIZ to be repositioned as an important part of a fully integrated programme of business assistance.

The development of new initiatives by Industry NZ needs to be managed with care, to ensure that the clear objectives, services and "brand" of the BIZ programme are not lost or subsumed in the broader strategies of Industry NZ. The project team also notes that any developments here will have resourcing implications.

3.4 Target Group Delivery

Under this heading the project team was asked "to assess the efficiency, impact and appropriateness of the BIZ scheme as it relates to Maori and Pacific Peoples and to enterprises owned and operated by women. This should cover conduct and operation of the programme including the mechanisms used to target Maori, Pacific Peoples and women for both the management enhancement capability and referral services".

The target groups selected by the Ministry for the BIZ programme (Maori, women and Pacific Peoples) were devised for the purpose of the Business Development Programme (run by the Business Development Boards) and when BIZ was initiated no new target groups were identified. In addition, a traditional target group for business assistance (business start-ups) was not identified as a focus for BIZ.

The result of this situation is that the BIZ contracting unit specified the target groups already noted (Maori, women and Pacific Peoples) although no specific delivery targets were identified (i.e. numbers attending or outcomes to be achieved). This was left to the organisations that tendered for BIZ funding which had the choice over whether to identify target groups.

The majority of providers did specify delivery to one or more target groups and on the whole these targets have been met: prior to January 21, 2000 18,672 women, 7,994 Maori and 2,517 Pacific Peoples had attended BIZ training. This means that of all BIZ clients to date over 60% of all those receiving BIZ training are women, almost 25% are Maori and 9% can be categorised as Pacific Peoples.

A different picture emerges from looking at BIZinfo clients, where 15% of users of the BIZinfo call centres are Maori, 3% are Pacific Peoples and 42% are women. However, these figures should be treated with caution as identifying ethnicity accurately is more difficult for BIZinfo (clients can refuse to answer questions concerning ethnicity and this is less likely to occur at BIZ training programmes where a closer relationship develops between provider and clients).

However, the fact that BIZ appears to be reaching reasonable numbers of the specified target groups does not mean that no improvements can be made. In fact, the data collected suggests that there are identifiable differences between the different groups using BIZ and that these differences should be considered in the next contract round.

3.4.1 Characteristics of Individuals in Target Groups

As noted previously, the most recent figures on self-employment and ethnicity (derived from the 1996 Census) show that Maori New Zealanders represent 15% of the population and around 6% of the self-employed population.⁵ Similarly, the total numbers of Pacific Peoples resident in New Zealand makes this group 6% of the population. However, they are under represented in the self-employment statistics (at around 1.5%).

These statistics indicate that these two groups at least are different from other SMEs although exactly why this is the case is not clear. In the context of the BIZ programme objectives and in reference to the more recent Closing the Gaps policy, it is clear that greater participation in self-employment from these groups is desirable.

For this to occur, the needs of these groups must be identified and considered in detail. This was the purpose of the focus groups which were conducted with Maori (4 groups) and Pacific Peoples (2 groups). A summary of these findings has already been presented in Section 2, where some of the key themes include the following:

Maori comment on their shorter history of self-employment compared to European New Zealanders. They comment upon a lack of experience in business, a dearth of role models and a lack of business oriented networks.
Partly related to the lack of experience in business is the issue of doing business in a context that is seen as being dominated by European New Zealanders and where the practices and customs of operating appear to be based on different cultural mores for example community rather than individual, which may underpin the frequency with which Maori enterprises appear to focus on collective objectives (such as job creation) rather than individual aims.
A third related issue concerns the legal structures that Maori use for these endeavours, especially in light of the legal requirements and/or restrictions on Maori assets. Individual hapu-based enterprises are common amongst this client group. Often constituted as trusts, these enterprises have different objectives from those usual under a company structure.

In other words, although some Maori clients have exactly the same needs from the BIZ programme as others, there are others who have different needs: those who are a) working in the context of iwi-based enterprises and b) not yet ready to describe themselves as being "in business". Many of those in the former group need "business skills" (although in some cases they are running non-commercial enterprises).

By contrast, those in the latter group, not yet in business, have indicated a desire for training in pre-business skills, elsewhere described as "enterprise competencies".

A similar situation exists for Pacific Peoples. There are few of them in business in New Zealand, which means that they have few networks and role models. Many are also operating in the context of community based value systems, where the collective goal is as important as that of the family.

As the focus groups with Pacific Peoples showed, they are often first generation New Zealanders or if not, are closely associated with communities and churches based on people who emigrated from the islands. However, they are keen to acquire business thinking, practices and routines. They also expressed a need to find ways of having others in their community understand that these are legitimate and necessary ways of operating business, and not a reflection of their abandoning the cultural mores of sharing and giving, nor an expression of selfishness or greed.

From these comments it is evident that Pacific Peoples business people have similarities to that of Maori: they have a shorter history of self-employment compared to European New Zealanders, they are operating in an environment which is dominated by European practices and customs, and they have a tendency to use trading structures which make their eligibility for BIZ in doubt.

This situation is also true to a lesser extent for women. However, the increasing numbers of women in business⁶ and the high numbers in the client survey suggests that in general terms this target group is not being disadvantaged. This is not to say that this group does not have particular characteristics that could be addressed through BIZ: for example women are over-represented as owners of part time businesses. However, on the whole it can be said that women in self-employment are different to the other two target groups.

3.4.2 Implications for Design and Delivery of BIZ

The characteristics of the target groups have implications for almost all aspects of how BIZ is delivered to individuals from these groups.

Building Awareness of BIZ

In the Awareness Survey, enterprises with at least one Maori and/or Pacific Peoples owner or decision-maker were identified and asked how they had heard about BIZ. Compared to all responses, Maori were less likely to mention mail-outs as a means by which they heard about the BIZinfo/BIZ programme and were more likely to be aware of mentoring, information referral and assessment of business needs than the rest of the sample.

Participants at all the focus groups had specific suggestions for ways of reaching their particular community. Pacific Peoples groups were confident that the way to disseminate knowledge of BIZ training to Pacific Peoples business people was after Sunday church services, at the church, and with the co-operation of the minister. The feeling was that this would give both good reach, and credibility to the message. Maori participants commented on the influence of respected members of the community on their decision to attend.

A number of BIZ providers spoke about the need to gain trust with these groups through using a number of avenues that they may not otherwise have considered. They also spoke of the need to invest a large amount of time and effort in building a relationship between a provider and an individual (or a group of individuals).

Contacting BIZinfo

The 0800 number and the website were both seen as effective ways to contact BIZinfo but the local centres ranked a poor third as a means of contact, and even after prompting, most business owners were unaware of these centres. It became obvious in the Awareness Survey that Pacific Peoples clients were less aware of these methods of contacting BIZ. During the Provider Survey there were some comments about the value of a mobile service that goes out into communities where there are high numbers of Maori and Pacific Peoples.

Preferred Services

An issue that emerged during the Pacific Peoples focus groups was an expectation that participants in BIZ would emerge from business training able to do something, rather than merely "knowing things". This may be a useful lever in engaging the interest of Pacific Peoples clients. This group also identified mentoring as their most favoured approach to business training, and commented that one-day courses could provide participants with too much information to absorb.

In the Maori focus groups the most common reaction was that they would like to do more and generally needed to do more. With the exception of where a person had been essentially misplaced into a course (i.e. for which they were too advanced or not advanced enough), people felt the courses had been effective in terms of content and delivery.

Among those at entry level, the presence of a Maori person as trainer was valued, largely in giving confidence and, on occasion, inspiration. This became less of an issue for those who were more established in business - for these individuals the mentor approach was more appealing.

Finance and the difficulty of obtaining it was spoken of as the largest obstacle to developing their small business (Maori focus groups). Participants expressed some frustration at not being able to inspire lenders, or obtain business credit. The project team recognises that the issue of funding is frequently over an individual's lack of skills, and BIZ could usefully include a module to lift the level of thinking and dialogue about raising capital, in order that Maori people in small business then approach lenders with the kind of balanced understanding that will attract the lender's confidence. (The team understands that giving business people skills to attract private sector capital is a key part of Industry NZ's Investment Ready Service).

Certificates

Where the Maori businessperson was already relatively established in business, certificates for the courses were of low interest. Where the person was at entry level, the initial interest expressed was higher, but faded upon closer consideration. Certificates were not seen to be the route of the owner-operator, nor were they regarded as having commercial currency.

Charging

In the Maori focus groups, fees for BIZ courses were resisted largely on the grounds that cash was tight for the entry-level person and the value of a course not easily judged in advance of payment. At the same time, it was acknowledged that people may not value something that was given free and that a charge would ensure more diligent attendance. There was less resistance to the idea of charging for printed resources and other costs (such as food).

3.4.3 Improving BIZ for Identified Target Groups

Members of the identified target groups are not homogeneous, and within each group there are a number of different sub-categories. In the future it is the needs of these sub-categories that should be considered, when deciding whether to offer "mainstream" services or service that have a "cultural context". For example, the impression gained from the focus groups was that Maori and Pacific Peoples might derive the most benefit if providers consciously identify three stages of small business among them and tailor BIZ provision accordingly:

Entry-level people, many of whom are slipping through the current eligibility screen. They need instruction in the concepts of business, help with business planning, and a customer building focus. Members of this group value trainers who have the same ethnicity.
Enterprises in the early stages of set-up, which are already trading and need operational advice and the formalising of business practices.
Established firms with several years trading. They need to widen the base of the business, improve profitability, acquire expertise that will reduce outside spending and deal with one or more of the host of developments that are necessary to grow or even just survive. The ethnicity of the provider is of little interest to managers of these firms.

In addition it is important that the Ministry takes a view on whether BIZ should be made available to those involved in a hapu or iwi-based enterprise. The project team observes that these individuals are already being serviced by BIZ (sometimes by bending the eligibility criteria slightly) and they are clearly making use of the business skills that they are learning. If this group is formally recognised (as we believe it should be) then this may also be a sub-category where cultural context is important.

⁴Up-to-date statistics on ethnicity of SME owners are not readily available, but data on self-employment from the 1996 census provide an approximate baseline. At this time Maori New Zealanders made up 6% of the self-employed population and Pacific Peoples just over 1%. Cameron, A & Massey, C (1999) Small and medium sized enterprises: A New Zealand perspective Addison Longman Wesley, Auckland, New Zealand

⁵Cameron & Massey (1999)

⁶Cameron & Massey (1999)

Jump to main page content.

Printed: 25/11/2009