• Home
• NZ Economic Development
• Industry Development
• Archive
• Evaluation of BIZ Programme
• Analysis of BIZ Interviews
• 2.0 Data Summary

• Industry Development

• Archive

• Evaluation of BIZ Programme

• Overview Report
• Analysis of Programme
• Analysis of BIZ Interviews
• Awareness & Understanding
• BIZ Management Services
• Mystery Shopper Survey
• BIZ among Pacific People
• BIZ among Maori People
• Expressions of Interest

• 2.1 Summary of Data - BIZ Providers
• Section 1: The Delivery of the Organisation That We Are Visiting
• Section 2: The Performance of the BIZ Unit
• Section 3: The Programme Overall
• Closing
• 2.2 Summary of Data - BIZinfo
• Section 1: The Delivery of the Organisation That We Are Visiting

The data set was collected from all respondents using a semi-structured interview format and in the following sections a summary of findings is presented in a way that focuses on its qualitative nature. First there is the material gathered from the BIZ providers through face to face interviews.

In their analysis the researchers have sought to identify the recurring themes and/or points of difference in the comments. Where possible the researchers have commented on any interesting differences in responses, given the size of the organisation's contract, its target groups, and the type of service offered.

2.1 Summary of Data - BIZ Providers

In the following section all the data is summarised and presented in relation to the 38 questions that were asked face to face and the 9 questions that were asked by telephone. No providers or respondents are identified by name, but to illustrate specify points some non-attributed direct quotes have been used.

The data was collected using a semi-structured interview format and is presented in a way that focuses on its qualitative nature. In their analysis the researchers have sought to identify the recurring themes and/or points of difference in the comments. Where possible the researchers have commented on any interesting differences in responses, given the size of the organisation's contract, its target groups, and the type of service offered.

Section 1: The Delivery of the Organisation That We Are Visiting

This question was designed as an "ice-breaker" - to help establish a relationship of trust between the researcher and the respondent, and to help the researcher ask appropriate questions throughout the interview.

The main comment that needs to be made about the responses is that the provider organisations that were interviewed varied widely: Some are community and/or iwi organisations that offer social services as well as assistance with economic and business development. Others offer a narrower range of services, e.g. the Chambers of Commerce are solely concerned with economic and business development. A third group is more narrowly focused again - on the provision of specific training courses, such as computer training. The researchers were not aware of any organisation that had been solely established for the purpose of gaining a BIZ contract.

In terms of BIZ services the organisations also vary: some offer seminars only, others offer seminars and courses (a series of linked sessions), and others offer a broader spread of services, for example pre-assessment of individuals, mentoring etc. There did not appear to be a "standard" set of services available to clients, although most could be described as falling into the categories of i) seminars, ii) courses, iii) mentoring and iv) networking.

There were a wide variety of responses to this question: Some providers had been working with identifiable target groups for a number of years (either through the Business Development Programme or through other government/community-funded programmes), and already had a strong feel for the numbers of attendees they could expect. Providers in this group also made extensive use of previous experience and personal and organisational networks.

Those organisations that were new to working with specific groups had more limited ways of collecting appropriate data to provide a baseline. In terms of identifying actual proportions, most of the latter group used local population statistics, rather than business population statistics (which may have been more reliable).

From those organisations with experience in business development there was be some feeling that not all the potential target groups had been recognised, particularly in reference to business immigrants and business start-ups. In this context one respondent pointed out that it was the MED that identified the target groups, and the providers had just responded to the tender guidelines.

This question was designed to identify whether the respondents were aware of how well they were delivering to different target groups - their actual performance was not the focus, as this had been addressed in the audits that had recently been conducted.

For some respondents the question was irrelevant, as not all had identified specific target groups in their contract: other organisations were delivering to a single group only e.g. Maori.

Of those who answered, most were able to comment on their organisation's performance without consulting their records - indicating that monitoring target groups was a matter of concern, and that they were aware of variations between targets and performance. In this context a number of respondents mentioned issues such as the variability between reporting quarters and the inadequacy of their original forecasts. This supported the researchers' impression that most providers were very aware of their contractual obligations on this issue.

Overall their perception was that delivery to their women clients had been easiest to achieve. Some suggested that if a couple is involved in a business then it is the woman that will come to the training. One commented that their contract could have been extended by four times to deliver to the amount of demand that was being experienced for courses for women.

Maori were perceived as being the next easiest to deliver to, although some providers commented on eligibility issues. For example, there was some discussion about the way in which the eligibility criteria seemed to exclude iwi organisations, and some providers offered an interpretation of the criteria that allowed access to those representing "business-like" organisations, such as those set up to deliver to government contracts (also see Q9 and Q33). Another issue was attendance, with a number of respondents commenting that with a tightly targeted group an event such as a tangi could mean that a course would fail to achieve its target numbers. (Some other issues are explored under the comments on Q25 and Q30).

The main difference of opinion lay in whether Maori clients have different needs from other clients. Some respondents felt this to be the case strongly, while others commented that the difference was not connected to ethnicity- the issue was that Maori clients are often at a different level in their business.

By contrast with the success rate in achieving targets for Maori and women, and the animated discussion that was generated on the issues relating to this, there was very little comment about Pacific People. Almost all respondents commented on the difficulty of achieving Pacific People targets.

The researchers also observed that the organisations that were most successful in achieving targets had long standing relationships with these communities, rather than short term commercial arrangements that exist solely for the sake of the contract.

All of those asked seemed to have adequate systems for monitoring attendance at the various types of service offered, and the respondents were confident in their use. Given the recency of the audit this was not surprising.

However, there were a variety of systems in use, and it appeared to the researchers that their quality varied. This is to be expected given that many of the organisations have a long history outside the BIZ programme and have different sources of funding (and therefore different reporting requirements). There is a perception that there would be value in the BIZ Unit developing a single system for providers to use. (At this point many respondents commented on the changes that have been made to the reporting system, and the suggestion that a single system be developed was clearly made with the hope that this would minimise any future changes).

Most respondents commented that this data was collected primarily as the basis for generating invoices for the BIZ Unit. Some commented that the data would be used for planning if the contract needed to be renewed. A significant number of respondents commented on the value of the data for their own uses (other than planning), but no clear pattern emerged in the organisations that they represented.

A large number of the organisations interviewed had been offered contract variations. Some of these were initiated by the BIZ Unit and others were initiated by the provider. Partly the answer to this question was related to the degree of flexibility in the original contract - where this was already flexible providers were able to reallocate funding after informal approval from the contract manager, whereas those organisations with very specific contracts had to negotiate contract variations. It was notable that very few respondents reported that a contact variation had not been approved when it has been sought.

There was a wide variety in the responses to this question, depending on whether the organisation used contractors or delivered service using its own staff, the type of services offered and the scale of the contract.

Almost half of the organisations with the largest contracts commented on the need for extra administration staff, and almost all others identified extra people offering services on behalf of the organisation (compared to prior to BIZ).

It was noticeable that the providers that have appointed new non-administration staff (such as trainers) have identified very specific skills. While many identified suitable new staff on the basis of content skills (marketing, finance etc), a significant number were able to talk about a variety of aspects of good presenters such as skill in instructional design and desktop publishing.

A number of providers moved people into the BIZ programme from other positions within the organisation. The practical effect was that existing expertise was made available to BIZ that had been developed within other organisations or under other government-funded programmes.

On the basis of the information gained through these interviews the researchers are not able to comment on whether the in-house delivery or contracting out model was more effective. However, it does appear that some providers had a clear appreciation of i) the range of skills required to run BIZ successfully, and ii) the need for high levels of expertise in specific areas (e.g. training delivery, instructional design etc). It is likely that larger providers will possess the range and depth of skills implied.

This question generated a lot of discussion, with all providers able to identify some changes that they would make. These ranged from those that reflected concerns with improving the "big picture" (e.g. "we want to make sure that providers work in a complementary fashion and don't duplicate services") to the pragmatic "we will stop providing food at training sessions".

The majority of answers were focused on improving the services for clients (e.g. a number mentioned the need to develop "second level" courses on topics that the clients had already been introduced to). However, some addressed the problems being experienced by the provider organisations (e.g. several mentioned the benefit of charging at this point, as a way of avoiding "no-shows"). Another issue was the need to build in greater contract flexibility, i.e.some providers commented that contract variations could have been avoided if their contracts had not been so specific. This particular issue may well have been a function of the way in which the original purchase round had been conducted - with both purchasers and contractors being new to the BIZ programme.

A number of respondents took the opportunity to comment on how they would approach the next contract round. On the whole this original negotiation process was felt to have been too rushed, with the providers having little control over what was described as a "shopping trip for the Ministry". One commented (in relation to another round of negotiation); "we would be more outspoken, and would not undersell our capacity to perform".

This question was designed to elicit respondents' views on the BIZ Unit criteria of eligibility for those attending courses, and was also to check that the criteria were known. While almost all appeared familiar with the criteria, there was a large variety in responses once the researchers probed more deeply. For example there was some discussion on what constitutes an SME in New Zealand, i.e. whether it includes part time businesses, and those that may be described as "hobby" businesses. Two respondents mentioned the difficulty of identifying a "beginning point" for an enterprise. For those respondents that were aware of the definition that is based on numbers employed by a particular enterprise, there was considerable variation in the upper limit that was identified. The researchers' impression is that it is sometimes easier for providers to identify an enterprise that is not an SME.

However, most of the discussion touched upon the enterprises that are excluded from BIZ training: start-up businesses and community trusts. The overall response seemed to be that the BIZ Unit has developed the eligibility criteria and to a point the providers are happy to apply them. However, there is a degree of tension here - most of the respondents genuinely wish to help those that approach them and find it difficult to turn them away even if they appear to be outside the eligibility criteria. There was some suggestion that this is a particular issue for Maori providers and/or Maori clients (see discussion under Q25).

This question elicited a large variety of responses, but in general there were three groups of respondent: First are those organisations that employ specialised trainers and expect them to run the entire programme by themselves - from sending out advertising material, taking charge of enrolments, running the courses and administering the post course evaluation. These organisations provide almost no support; "that's what we employ trainers for".

Second, a number of organisations do all the administrative work for the trainers and just expect them to do the face to face sessions. Again, these organisations provide almost no support for those delivering the training.

Third are a smaller group of organisations that take on a high level of responsibility for the entire delivery. Organisations like this take a lead role in generating enquires and enrolments for the course, training and selecting appropriate trainers and tutors, designing and reviewing course content and overseeing the evaluation. Those in this group most often described their role as being engaged in quality control (cf. training support), and a number of these providers could be described as exhibiting best practice.

Although this question was originally designed to capture responses concerning the BIZ Unit, in the field this was omitted, and Section 2 provided the forum for this discussion.

Although there was some variation in the answers to this question, the differences were chiefly in the detail (i.e. between the different evaluation forms). Almost without exception respondents answered that the main form of evaluation for the training was a form filled in at the end of the course. There were differences in practice between the organisations, some of which would be important if the evaluation for a course offered by one organisation was to be compared with that of another, but on the whole the practice was fairly standard. So standard in fact that there seems to be value in adopting a single format for evaluation and an agreed process for administering the forms.

A smaller number of organisations commented on the value of evaluations that were done at some interval after the training. Some had built this practice into their contract, but others were trying to add it on as additional way of evaluating the impact of their training. Several of these respondents commented that they were behind their self-imposed timetables on this follow-up.

Two respondents mentioned other forms of training evaluation such as having an expert review the material. None conducted peer evaluation, although one organisation was considering it as an option.

Most organisations had relatively standard evaluation forms that asked attendees to assess the training under the headings of "content", "delivery" and "resource materials". Some of these were quite detailed and also focused on the longer term impact on the individual ("did the training increase your confidence") or the business ("will you be able to use your new skills in your business?").

Some used the opportunity to ask attendees about future training needs, and included details such as the most suitable time of the day for them to attend training.

Given that most of the forms were very similar in their overall intent (although the level of detail varied widely) there seems to be value in identifying best practice examples and offering them as an resource to all providers.

All respondents commented on the value of the information for improving the course, and most were very explicit about the process that was followed and the specific aspects that would be changed. For example, a number of respondents gave immediate feedback to the trainers involved and also held regular meetings for all the people involved in delivery to discuss feedback. Others had less formal systems but there was no discernible pattern in terms of which organisations were more likely to take this approach.

The answers here depended on whether i) the organisation was delivering to one or several target groups and ii) whether its contract specified that delivery to different target groups would be done through distinct courses. Where separate delivery was the case the researchers found two distinct trends; i) where the course is identifiably different and ii) where it is essentially the same course, with minor changes such as the examples used to illustrate different learning points.

Those promoting the different course approach argued that there is a need to recognise the differences of attendees, whether women, Maori or Pacific People (although there were few actual examples of how a course would be changed to meet the needs of women). These respondents also argued that a major benefit of training is having people of a similar level learn together as a way of encouraging them to form a network. They felt this was enhanced with a separate course.

By contrast those that were using the same course argued that differences between participants were less related to ethnicity or gender, but rather related to their "level" of business knowledge.

These two approaches also emerged in response to Q3, Q25 and Q30, and clearly suggests that there is an issue with delivery to target groups that needs to be resolved in the next contract round for BIZ. Depending on the discussions that are made following this evaluation, there are a number of options for the BIZ Unit: it could be more explicit about the types of service that will be purchased, based on a policy of recognising "attendee difference" or it could focus on "business difference". Alternatively the BIZ Unit could continue with its current policy - of seeking advice on what to purchase based on the expertise of the providers.

While a few respondents commented on changes that they would make, the vast majority were happy that the approach that they are currently taking is adequate. However, the researchers note that this judgement was made in the absence of any independent assessment of their courses and of their support material. There was a huge variation in the standard of the written material we viewed, with some offering substantial workbooks designed specifically for the BIZ programme and others circulating material that appeared out of date (in terms of content) and poorly produced (in terms of print quality and/or layout).

The researchers suggest that the answers to this question may have been quite different if providers were familiar with more material from other providers. Greater awareness of "high quality" material would no doubt have an impact on improving quality, and would also assist in the development of BIZ as a quality brand, with some level of standardisation .

One respondent commented that in another tender round the organisation she represented would not necessarily invest as much time in developing the courses, despite acknowledging that this would probably compromise their quality.

There was a consistent response that charging (a nominal fee) might help minimise the numbers of people enrolling in courses who subsequently failed to attend. However, most respondents also commented on the difficulties of administering the charge, and questioned whether the cost to the organisation might outweigh the benefits gained. Those respondents that recognise the complexity of charging as a policy wee more likely to be those from organisations with larger BIZ contracts.

A smaller number of respondents said no to any charge. They were concerned that this could have a negative impact on numbers attending and that it was inconsistent with promotion that describes the courses as free.

The fact that this question received such a mixed response (and that even those who were keen to charge could also see the problem associated with it) suggests that a policy change could meet with resistance from some providers and clients. However, this could be minimised by helping providers identify specific aspects of the services that could legitimately attract a charge (e.g. food, stationery etc).

There was an almost unanimous perception that the training has had an identifiable impact on the individuals who have attended the training, based on the feedback from the evaluation forms and the direct feedback from attendees. Some providers ran specific follow-up surveys in an attempt to assess the impact more formally, although most were aware of the difficulties in assessing change.

Several commented on the part the training plays in personal growth and in increasing motivation. More than one respondent used the phrase "opening their eyes" in describing the experience of training attendees and spoke of the way in which this sort of training alerts people to different options and choices. This response was likely to come from an organisation that also offered social programmes.

Others spoke more specifically about how the courses contribute to the development of specific skills - which can then be used in the context of the business.

Another theme mentioned by a number of respondents was the way attending training helped attendees build new networks and develop links to other providers of business related services (e.g. accountants).

Several respondents commented that the experience of attending a single course has a long-term value - in alerting the individual to the value of continued education.

A number commented that the impact on the individual is either difficult to measure or that it is almost irrelevant - unless the individual used their new skills within the business. As one noted "the real outcome is the added value to the business".

This question related to the previous one, and was designed to assess the respondents' understanding of the difference between the impact on the individual and the impact on the business of a programme like BIZ. Most respondents did show an awareness of the difference, but as with the last question few organisations had any systems in place to formally assess the impact. One commented that this was because he was not sure how to measure something as complex as this; "I think they are using their new skills, but I'm not sure how to best assess this".

This theme (the complexity of measurement) was addressed by a number of others. One commented that "the impact on businesses (of attendees) will vary and will take different times to be able to see benefits". In a similar vein one respondent commented that the "main advantage of BIZ is that it speeds up the rate of business development, which would have happened anyway but at a slower rate". Others commented on the difficulty of separating the impact on the business from the impact on the individual; "the people and the businesses are related. What you do for the people helps their business"

Others spoke of the constraints of time which meant that they were not able to do the sort of follow-up that they felt would be valuable; "at the moment all the focus is on the delivery"

Section 2: The Performance of the BIZ Unit

This question was designed to help the researchers understand how the provider organisations interface with the BIZ Unit, and we expected many respondents to name more than one person in the Unit, given our impression that those employed here have distinctly different roles e.g. the unit manager, as opposed to the contract managers. However, most did not - naming their contract manager only.

Janet Takerangi's replacement of Jaewyn McKay was mentioned, but otherwise there seems to have been continuity in terms of contract management.

Most respondents commented on the role of the BIZ Unit primarily in relation to contract management. Here they mentioned specific tasks such as paying the providers, considering contract variations, monitoring performance, and sending out new reporting requirements.

A significant number of respondents also commented on the role of the BIZ Unit in disseminating information, liasing between providers, and facilitating networking (e.g. through the meeting held in Wellington in November 1999).

A small number of respondents commented that they were happy with the services provided by the BIZ Unit and were not seeking any extension of these. However, the larger group identified services they would like the Unit to provide, or to extend.

A number of respondents thought that the Unit could be much more active in developing practical resources for the providers. A number suggested specific resources, such as standard courses (see the discussion under Q33), resources for Maori and Pacific People clients (the Maori Business Directory idea was mentioned here).

Others talked about the role of the BIZ Unit more generally in relation to quality assurance.

Another specific suggestion (that was mentioned by several respondents) was that the Unit should develop a computerised client management system (including database and reporting functions). This was consistent with the evident concern about the changes in reporting requirements that has already been noted.

A third theme was the potential for the Unit to be more active in facilitating networking between providers.

A final comment (that echoes those that were raised by others at different points in the interview) was that the Unit needs to be more active in promoting the BIZ programme. The point being made here is that the current promotion focuses on BIZinfo and respondents clearly feel that referrals from this source were not occurring.

This question was designed in this way to help the researcher focus on the satisfaction criteria that the respondents were identifying. The responses to this question are captured under the analysis to Q27, but it is worth noting here that none of the respondents had difficulty in identifying an event when they were satisfied.

An overall summary of the responses would be that they found the BIZ Unit "responsive" and "supportive". Specific examples (that were cited more than once) seemed to be most concerned with channels of communication e.g. "he/she offered to visit instead of answer my questions over the telephone", "he/she came back to me really quickly - with the right information".

As with Q23, the essence of the responses to this question are captured under the analysis to Q27, but it is worth noting here that not all of the respondents could identify a suitable event in answer to this question. Those that could identify a specific event were mostly concerned with changes to the reporting requirements and the way these were communicated.

This question was deigned to elicit comments on the way in which the BIZ Unit deals with those providers that are offering services to Maori or Pacific People (as opposed to Q30 which was included to elicit comment on the design of the BIZ programme overall in reference to these groups). In practice respondents tended not to make a distinction between the questions, and the comments below need to be read in conjunction with those to Q30. The researchers also note that the answers were from providers, who were commenting on their perceptions of the needs of the Maori target group (which is not a homogeneous group).

From the responses it is clear that Maori providers have a diverse range of opinions on clients' views and expectations that are not easily categorised by provider type: Some were of the view that Maori clients want to be part of "mainstream" services so they can network and interact with other business owners, whereas others felt that they want customised services that recognise a Maori cultural context. Here there was the suggestion that the most successful Maori business owners are those who have worked out how "to combine the best of both Maori and western values in their operations". The only distinction between the groups holding the two opinions seemed to be in reference to their own services: some providers offer training that already has a strong cultural context, and they were most convinced that this is necessary. Those providers that offer training that is more generic were equally convinced that this is the most suitable delivery approach.

As already noted, there is also the recognition that Maori clients are often at a different and earlier stage of development in terms of their business skills and need services pitched at a different level (see the discussion under Q3and Q14). One provider stated that Maori are "often less trained and skilled than their Pakeha counterparts" and thought that a greater level of funding could be needed to help close the gaps. It is worth noting that during the period when these interviews were being conducted it became clear that a number of the providers would also be bidding for funds from Te Puni Kokiri, in relation to its new programme for start-up businesses.

Providers pointed out that there are a lot of requests for start-up assistance from Maori and that these are difficult to turn away under Maori kaupapa. BIZ is targeted at the owner/manager, however Maori look beyond the individual to the group (such as hapu or iwi) and this requires greater flexibility in the targeting of service delivery.

Another issue was eligibility: trusts are the structural form under which many Maori conduct their business operations, and these organisations are looking for the kind of services that BIZ provides. However trusts are currently ineligible for assistance.

There was also recognition that the Maori context can sometimes be abused and there was some disagreement about the extent to which funding should follow traditional tribal channels. Whichever approach is used, respondents echoed a comment that was made by almost all providers; "that it is often difficult to identify the people within the business that need training and to get them to attend courses". This was felt to be particularly the case with men.

The answer to this was overwhelmingly no, as this was not seen as a key function of the BIZ Unit. However some respondents commented that their organisation's involvement in BIZ as a programme had helped them to develop.

This was an extremely difficult question for respondents to answer, and of the 18 who were asked this question only half were able to identify a single overall assessment. For this group their assessment was overwhelmingly positive - between 3 and 5.

However, most respondents wished either to differentiate between different people within the BIZ Unit or between the BIZ Unit as an entity and the staff member they have most frequent contact with (usually the contract manager). Some also wished to differentiate between the past and the present, with most commenting that performance seems to be improving (although almost all respondents who had been in contact with Jaewyn commented with regret on her departure).

Another issue was that most respondents wished to identify specific assessment criteria rather than commit to a single overall assessment. However, in answering this question the providers gave a valuable insight into their expectations of the BIZ Unit.

Most commonly mentioned were qualities that described the service that the BIZ Unit provides. Words like "level of support", "responsiveness", "speed" and "accuracy" seemed to point at those qualities that the providers regard highly, and that they felt were demonstrated (to varying degrees) by the BIZ unit.

Another cluster of ideas concerned the way in which the BIZ Unit communicates with and relates to providers, and here there were both positive and negative perceptions. To one provider the unit takes a "big brother attitude". Others commented on the need for the unit to be "approachable", "flexible", "non-bureaucratic" and to "stay out of the way".

A final category of ideas concerned the overall purpose of the BIZ Unit. Respondents talked about a need for the unit to be "skilled in promoting the BIZ programme", display a "high quality of decision making", and be "proactive" and "accountable".

The researchers strongly suggest that these concepts are tested further with the providers and that a specific instrument is then developed to give the BIZ Unit a way of measuring perceptions of their performance over a period of time.

Most respondents commented that they already email their reports to the BIZ Unit (with a backup hard copy) and it was not clear what the implication of on-line reporting would be. In the field this question was modified to focus on issues concerned with reporting, which most respondents had already identified as a real issue.

This was partly connected to the number of reporting changes that had been made by the BIZ Unit (the researchers were told that three significant changes have been made to the reporting to date). All respondents were critical of this: as one commented "each time the requirements change we have to get a computer programmer in to reorganise our system - and we have to pay". Another commented that "the providers need an integrated management information system that produces the required results" (referring to the template format).

However, there were also some respondents that work for organisations that have been in existence for some years, and/or that have contractual relationships with other agencies. For this group the issue of reporting is more complex, as all of the funding agencies require different outputs, and a standardised system would not necessarily improve their situation.

Section 3: The Programme Overall

Here we were seeking the respondent's opinion on changes that they would like to see made to the BIZ programme overall (including BIZinfo and BIZ training) as opposed to the improvements that they are expecting to make to their own organisation's delivery of the programme. A number of specific themes emerged.

Not unexpectedly the first theme that emerged was overwhelming support for BIZ to continue. It was felt that more resources should be allocated and that the government needs to signal a long-term commitment to business development. A specific comment came from the Maori providers; that the programme had been slow to gather momentum with this target group, but now it has the capacity to have a considerable impact.

However, there was considerable concern abut the relationship between BIZ and BIZinfo; "BIZinfo is not working for us" was a comment from more than one respondent and a specific concern (mentioned several times) was that there is no system of tracking referrals from BIZinfo to specific BIZ providers (this was in respect of referrals to specific BIZ providers rather than the general tracking that currently occurs). This lack of a system was raised as an overall problem, and also as an issue in areas where there is more than one provider: there is a perception in these areas that the BIZinfo staff have "favoured" providers, and refer clients to them no matter what their need.

In the absence of a formal system for tracking referrals, BIZ providers are identifying the referral source by asking client as they make contact with them. On the basis of this info, more than one respondent commented "over the duration of the programme we have not had a single referral from BIZinfo".

Several respondents suggested that more resources should be directed at BIZ training and less at BIZinfo. In the discussions on BIZinfo some respondents also questioned the purpose of BIZinfo particularly in relation to marketing BIZ courses. A typical comment was "both BIZ Training and BIZinfo should be retained, but must be much more cohesive".

Another area of concern was provision of services for new businesses. Several respondents echoed the comment made by one; "where do start-up businesses go?"

A different stance was taken by those who were concerned with what BIZ will do next for existing businesses. Thinking about the next phase for BIZ training, a number of respondents commented on the need for the BIZ Unit to be aware of the need to "take it up a level - targeting the businesses that are going to survive or medium sized businesses". In this context they were clear that they would be better able to provide "the next level" of services if the BIZ Unit took a stronger role in providing good quality resources to providers.

A number of respondents also talked about the need to broaden the scope of BIZ beyond the provisions of i) training and ii) information to a more formally integrated package of business assistance, which could include pre-assessments, mentoring and other one-to-one interventions.

They also identified a need for the BIZ Unit to take a stronger role in quality control; updating programmes, helping providers share best practice and designing and implementing better audit systems.

Another theme that emerged was the role of the BIZ Unit in providing feedback to all providers on the impact of the programme; "we need more detail than the bums on seats reports we get now". Specifically mentioned was the need to move away from measuring outputs to assessing outcomes. One respondent commented; "we need more emphasis on quality and less on quantity".

Several respondents commented on the need to improve the contract negotiation process. Specific comments were concerned with the transparency of the negotiation/purchasing process and the ease of preparing for the negotiations; "we need a RFP that is easy to read, unambiguous, clear about what is wanted, and which sets clear performance targets". Similar comments were made about simplifying the reporting process. A number commented on the need for the contracts to remain as flexible as possible, to allow the providers to meet changing customer needs. Others voiced an opinion that in the next contract round there should be fewer providers, and that more thought should be given to assigning providers to specific areas. There was also comment that providers should be assigning to deliver to specific ethnic groups, rather than having targets set within the overall programmes offered by large, mainstream providers. .

Most respondents for whom this question was relevant felt they had already commented under Q3 and Q25. A particular issue seems to be the length of time the programme has taken to gain momentum with these target groups and the need for the providers to take time to build long term relationships of trust. This seems to be the case even for those organisations with existing relationships, but is particularly important for those who are servicing these groups for the first time.

The overwhelming response to this question was "we don't, and we would like to". In this context a number mentioned the Wellington meeting and felt that this provided an opportunity for sharing best practice that was not fully used. However, on the other hand, more of this interaction is desired by many providers.

A common theme was that most providers have their own networks either of other BIZ providers (e.g. one mentioned a regional meeting organised by one provider) or of other business development agencies (e.g. SBECNZ and EDANZ). There is also some informal networking with agencies that are not BIZ providers.

However as one pointed put; "there is no incentive to network - at the end of the day the process is competitive".

While most of the respondents answered no, or that they were not aware of whether this was the case, a number commented that they felt a reverse effect was occurring: People who attended BIZ training may be more likely to attend paid training courses, thus expanding the total being spent on business development in general.

There was also some regional variation here - the Christchurch organisations commented that there had been a local gap that they had been able to fill.

There was an almost unanimous feeling that this would be valuable, with the caveat that the standardisation should be a framework only (i.e. not a whole course in detail). There was strong feeling that this would provide a foundation/template for the providers that they could then tailor to their own local conditions and/or target groups, by for example, adding relevant local content. Most believed that this would add to the quality of the course, and would genuinely enhance the BIZ brand. As one said "the BIZ Unit need to get real - how else can quality be maintained?" One commented that this quality control is particularly important as BIZ moves to the next stage.

There was a strong opinion that this developmental work should be paid for by the BIZ Unit, although there was an equally strong feeling that the work itself should be done by a credible independent organisation.

A smaller number of respondents (4) felt that the value in BIZ lies in its customisation; "this work best as we provide what local businesses need". They were less convinced about the need for standardisation were more likely to be delivering services to Maori and were concerned about the ability of a standard format to fully address the needs of this group.

While the previous question was deigned to seek out the needs of the providers, this question focused on the needs of the clients (as interpreted by the providers). The overwhelming response here was "no - attendees are wanting skills - not certificates". One respondent commented that imposing accreditation of any sort would be unfair on non-accredited providers. A policy of this sort was seen as having the potential to exclude them from the provision of BIZ, thus restricting the range of providers that would be used. There was also a perception that accrediting programmes would have an impact on the ability to provide training for free.

There was an overwhelming "yes" to this question, and a concern that this is causing difficulty for clients. Interestingly this was the response even for those organisations where BIZinfo and BIZ are offered out of the same location.

A number commented negatively on the process of allocating contracts to BIZinfo providers: "It was a big mistake to split this between providers - the two go very much hand in hand". Another said "we don't believe BIZinfo works, in particular the local shop front operations - what is its purpose when people will eventually end up with us anyway? The advertising confuses the business community".

Others focused on the way that BIZinfo is servicing client enquiries: "BIZinfo services often use volunteers or people who don't have a good understanding of business and the way it works" was one comment. Another respondent said " BIZinfo needs to link much more closely with BIZ - BIZinfo gets the enquiries but can't handle them or give a sensible answer, so often they end up referring them to us - so why don't businesses just call us to begin with?"

Several respondents also commented that once they had developed a relationship with a client they were more likely to ring their organisation direct for further assistance. "Clients are already aware of us as a BIZ provider so often call us directly for information - not knowing that BIZinfo even exists. It stretches our resources when we provide this extra service - but it makes no sense to tell them to hang up and call a complete stranger"

A number of comments focused on the future for BIZinfo: "It could be run entirely through the Web page and 0800 numbers rather than 20 individual offices". Others suggested a new name for BIZinfo to differentiate from BIZ training.

Two different answers emerged here: One group was happy with the degree of flexibility the contract offered, while the other suggested a more structured framework. In this context several suggested the approach that WINZ takes to contracting BYOB course, while one mentioned the CEG contract for LEEDS agencies. The common factor in these contracts (as reported to the researchers) seems to be that they provide a "menu" of services that the provider can then pick and choose from in terms of delivery. This was thought to be a particularly helpful option for agencies that are new/inexperienced, or where the individuals responsible for putting together the tender are new to the process.

Closing

Answers here were dealt with directly by the researchers and were not recorded.

Some respondents did not wish to add anything at this point but most wished to emphasise comments made already in the course of the interview. The main themes were:

• BIZ is seen as en effective programme which is an improvement on the previous programme (the BDP) and which is essential in the new era where jobs for life are not guaranteed.
• There is a need for commit to the continuity of BIZ and work on i) extending it to plug gaps (e.g. start-up businesses and ii) integrating it with other programmes (such as the EAP).
• A belief that the Ministry's reporting mechanisms only collect superficial data and a desire to see this improved, to measure success and quality rather than quantity.
• A perception that the flexibility of the programme has been particularly effective in allowing different providers to offer services to particular target groups, such as Maori and women.

2.2 Summary of Data - BIZinfo

The data from the interviews with BIZinfo providers is included in this section of this report. Because of the small number of respondents the commentary has been kept brief in order to maintain the anonymity of those interviewed.

Section 1: The Delivery of the Organisation That We Are Visiting

The four partners of NBIS all have different roles in relation to the BIZinfo contract. In addition they are all involved in business or economic development outside of the BIZinfo contract.

This was not applicable to the respondents as the specification from the MED was that the BIZinfo service had to deliver globally, rather than to specific groups as was necessary for the BIZ training providers. The more relevant question was why the organisation had become involved in BIZinfo. To some extent (and for some of the partners) this was because the members of the consortium had been brought together by the MED, but once this occurred, all could see involvement in BIZinfo as contributing to the organisation's objectives.

All the respondents regarded their organisations' involvement in BIZinfo as having helped them meet their objectives.

Two of the four NBIS partners have a particular interest in the BIZ target groups (namely Pacific People and Maori), and the respondents from these organisations were aware of the statistics on BIZinfo users from these groups. They were not surprised at the lower uptake of these groups, for reasons noted elsewhere (e.g. the lack of targeted advertising).

All respondents indicated their enthusiasm for being involved in another round of BIZ funding, but noted that they would make changes to the bid, especially in the light of the strategic planning exercise that NBIS has recently undertaken.

The key issue here was that the respondents saw the BIZinfo service as being "more than just referral or information". In the words of one respondent: "a 'successful' outcome is considered to be one where the client's problem issue is correctly identified, and choices provided; and where the client takes some action to follow up (or makes a positive choice not to proceed further)".

While the respondents found it easy to describe the roles of the call centres and the website, the BIZ shops proved more problematic, partly because they do not appear to have been completely successful in contributing to the overall design of BIZinfo outlets. While they were designed to provide a service in smaller centres, difficulties have been encountered, as they are usually delivered by agencies with other roles (Chamber offices etc) and which are not under the direct control of NBIS.

One respondent emphatically described the role of BIZinfo in terms of client needs (rather then in relation to the needs of BIZ providers); "BIZinfo is there to facilitate them (clients) getting what they need". This was also the view of others interviewed, however there was a perception that BIZ providers see BIZinfo primarily in terms of a promotional arm for them.

There was no difference here compared to the previous question: the respondents saw their work as meeting the needs of clients, and if they regarded the BIZ providers as the most appropriate for the particular clients' needs then they would be recommended. However, it was clear that non-BIZ providers would also be recommended if it was felt that their service would be more appropriate.

Evaluation seems to be focused primarily on monitoring the meeting targets (numbers of clients seeking help via the various method of delivery). This was an area of concern from some of the partners, who were keen to develop a method of evaluation that would focus on outcomes cf. outputs, and allow NBIS to measure "added value".

We were told that the information is provided to the MED and also used by NBIS to provide: feedback to call centres and BIZ shops; input to internal seminars; feedback to BIZ providers; a base for targeting advertising.

None of the respondents found this easy to answer, although there were opinions expressed on the target groups that "the service is not having an impact on". Again, see the comment on changing the focus from measuring outputs to measuring outcomes.

The respondents all had different views on this question, and it is worth emphasising that this question was not put formally to the NBIS board, but was put to individuals, some of whom serve on the board. On the whole it appears that the partner organisations are enthusiastic about BIZinfo being continued, but they had very specific ideas about what improvements could be made. These included:

• Reconsidering the role of the BIZ shops - either by increasing the resources available to them and/or bringing them under the direct control of NBIS. There was also some discussion over where they should be sited, and the opportunity for a mobile service for remote communities.
• Increasing the scope for offering advisory services through the call centres.
• Changing performance measures to "measure more than numbers".
• Increasing resources for Maori and Pacific Peoples' services, and reconsidering the method of service delivery for these client groups.
• Improving communication with other government departments. This emerged as an issue in relation to the planned service from TPK.
• Improving quality by accrediting trainers and advisors (this was thought to be particularly important in the BIZ shops where the person on the ground has no one else to ask).
• Extending eligibility to start-up businesses.
• Reducing political interference from Ministers' offices.
• Strengthening linkages between BIZ and BIZinfo and also with organisations like BITC.
• Improving marketing and advertising, particularly in relation to Maori and Pacific Peoples.