EDF

Review of
The Enterprise Development Fund
Research, Evaluation and Monitoring Team
Industry and Regional Development Branch
MINISTRY OF ECONOMIC DEVELOPMENT

September 2005

Ministry of Economic Development
Industry and Regional Development
Research, Evaluation and Monitoring
Level 10, 33 Bowen Street
P O Box 1473
Wellington 6140
New Zealand

Contents

Acknowledgments.........................................................................................................................4
Executive Summary.......................................................................................................................5
PART ONE: Evaluation Goals and Methodology. ..................................................................12
9. Goals of the evaluation...................................................................................................12
10. The intervention logic.....................................................................................................12
11. Methodology....................................................................................................................12
12. Methodology of the survey............................................................................................13
PART TWO: Policy Rationale for the Enterprise Development Fund ................................15
1. MED Policy ..........................................................................................................................15
13. Policy objectives for the Foundation Services.............................................................15
14. Policy objectives for the Enterprise Development Fund...........................................15
2. NZTE policy.........................................................................................................................17
15. Design of the Enterprise Development Fund .............................................................17
16. Criteria for projects .........................................................................................................17
17. Scoring of applications ...................................................................................................17
18. Future changes: devolvement and criteria for 05/06..................................................18
PART THREE: Findings and Recommendations on the Implementation of Policy.........20
19. Programme implementation .........................................................................................20
20. Response by MED to the Enterprise Networks implementation.............................20
21. Operational implementation: the Business Evaluation Team ..................................21
22. Operational implementation: the Enterprise Development Team ..........................21
23. Design of the Enterprise Development Grants scheme.............................................22
24. Project activities available under EDG.........................................................................26
PART FOUR: Findings and Recommendations on Delivery of EDG .................................28
3. Uptake of the fund ..............................................................................................................28
25. Total allocations...............................................................................................................28
26. Total number of applications ........................................................................................28
27. Total number of firms.....................................................................................................29
28. Approval rate...................................................................................................................29
29. Calls to the Business Evaluation Team ........................................................................29
30. Change in application numbers over time ..................................................................30
4. Assessment of applications................................................................................................32
31. Change in approval rate over time...............................................................................32
32. Recommendations...........................................................................................................35
33. Time taken to approve grants .......................................................................................36
34. Average time taken to approve grants over time.......................................................36
35. Recommendation ............................................................................................................37
5. Distribution of the EDG .....................................................................................................38
36. Summary of findings on distribution of the EDG ......................................................38
37. Regional spread of firms assisted .................................................................................38
38. Regional spread of allocations ......................................................................................39
39. Sectoral spread of allocations ........................................................................................40
6. Grant allocation and collection by firms..........................................................................42
40. Amounts allocated ..........................................................................................................42
41. Amount claimed by closed projects .............................................................................42
7. Analysis of applicant firms................................................................................................43
42. Summary of findings given below. ..............................................................................43
43. Overall size: turnover and FTE .....................................................................................43
44. Exporting ..........................................................................................................................44
45. Age of Firms.....................................................................................................................45
46. Firm Growth ....................................................................................................................45
8. Types of assistance funded ................................................................................................47
47. Advice and expertise categories .................................................................................................... 47
PART FIVE: Findings and Recommendations on Outcomes ................................................49
9. The framework for analysis of outcomes. .......................................................................49
48. Goals of the EDG.............................................................................................................49
49. Goals of the evaluation...................................................................................................49
50. Specific outcomes sought by the evaluation ...............................................................50
51. Interpretation of the policy intent: improving management or business
capability?..................................................................................................................................50
52. Interpretation of the policy intent: improving management ability or
knowledge? ...............................................................................................................................51
10. Quantitative findings on accepted firms. ........................................................................53
53. Guide to the findings ......................................................................................................53
54. Numbers of firms responding.......................................................................................54
2
55. Numbers of firms completing projects. .......................................................................54
56. Overall satisfaction .........................................................................................................54
57. Proportions of EDG clients seeing changes in the way they manage their
business......................................................................................................................................55
58. Proportions of EDG clients undertaking projects aimed at management level
areas or business level areas ...................................................................................................56
59. Proportions of EDG clients seeing gains in firm activity ..........................................57
60. Proportions of EDG clients seeing changes in firm performance ............................58
61. Proportions of EDG clients gaining facts or advice ...................................................59
62. Do facts or advice projects impact on management practice?..................................60
63. Are facts or advice projects also management or business level projects?.............61
64. Do management level or business level projects impact on management practice?
............................................................................................................................................61
65. Do changes in management practice impact on firm performance? .......................62
66. Do advice or facts projects impact on firm performance? ........................................63
67. Do management or business level projects impact on firm performance?.............64
11. Quantitative findings: interpretation and conclusion ...................................................65
68. Overall ..............................................................................................................................65
69. Activity types...................................................................................................................66
70. Interpreting the results: improving management capability....................................66
71. Resolving the ambiguity between facts and advice...................................................67
72. Declined firms..................................................................................................................68
73. Recommendation ............................................................................................................68
12. Qualitative findings ............................................................................................................69
74. Guide to the findings ......................................................................................................69
75. Overall ..............................................................................................................................69
76. Business/strategic plan development and feasibility studies. ..................................69
77. Intellectual property protection. ...................................................................................70
78. Strategic Design...............................................................................................................70
79. Market research and marketing plans .........................................................................71
80. Prototype ..........................................................................................................................71
81. Mentoring and training ..................................................................................................72
82. Certification and systems evaluation ...........................................................................72
Appendix: Survey Questions .....................................................................................................73
3
Acknowledgments

The evaluation would like to thank Dr Richard Arnold, Senior Lecturer in Statistics with
the School of Mathematics, Statistics and Computer Science of Victoria University of
Wellington, for his valuable and essential advice on the evaluation and data analysis,
and for reviewing many drafts of the report.
The evaluation would also like to acknowledge the openness with which New Zealand
Trade and Enterprise worked with this evaluation, and the over 100 firms who gave
their time to talk with the evaluation.

4
Executive Summary

1. Purpose and evaluation methodology
1. In accordance with the New Zealand Trade and Enterprise (NZTE) Foundation
papers (EDC (03) 54 refers) the Ministry of Economic Development (MED) has
conducted an evaluation of the Enterprise Development Fund (EDF).
2. The focus of the evaluation was to examine:
• Programme implementation
• Programme delivery
• Programme outcomes
And conclude whether the programme should continue, and continue unchanged or
with changes.
3. NZTE split the EDF into ‘Enterprise Development Grants’ and ‘Enterprise
Networks’. Enterprise Networks has now been removed from the EDF. This
evaluation has examined implementation for the fund as a whole, but examined
delivery and outcomes for the Enterprise Development Grants only.
4. The evaluation analysed NZTE data and interviewed NZTE staff responsible for
assessing applicants to the EDG. In order to answer questions about outcomes the
evaluation undertook a telephone survey of a representative sample of EDG
recipients. The sample was a stratified simple random sample of 132 firms with a
response rate of 98%.

2. Summary of the Enterprise Development Fund
5. The EDF was established in 2003 (EDC (03) 54 refers) as part of NZTE foundation
services. Three programmes were amalgamated to form the EDF: the Enterprise
Awards Scheme, the Export Network Programme and World Class New
Zealanders. These three all delivered funding assistance to firms.
6. The EDF was appropriated $8.613M annually (for 03/04 and 04/05).
7. The EDF provides funding of 50% of total costs for businesses and entrepreneurs to:
− Engage the services of a business mentor for a finite period of time
− Undertake more advanced management and technology based training (as
delivered through existing providers such as NZIM, Polytechnics etc)
− Employ specific external advice and expertise in a management area (such as
feasibility studies, e‐business, market research, enhancement and uptake of new
technologies, human resources, intellectual property, strategic planning,
environmental management, production management)
5
− Undertake international market development activities, including new market
investigation, trade fair participation, trade/business missions, business
exchanges, and visiting buyers.

3. Implementation of EDF
8. MED considered the implementation of EN by NZTE sufficiently different from its
policy objectives to remove Enterprise Networks from the EDF. Funding for
Enterprise Networks was moved to operational expenditure (CAB Min (04) 38/4
refers).
9. EN is now part of NZTE’s sector facilitation activities. NZTE have been invited to
give the policy framework for these activities, and MED is currently in discussion
with NZTE on this.

4. Implementation and Delivery of EDG
10. The design of the fund was to mitigate issues present in the former schemes by
improving accessibility, eliminating delays and lack of clarity in the application
process, and by adhering consistently to clear criteria.
11. Some of these issues have not been fully addressed:
− Assessment involves a significant degree of subjective interpretation of criteria.
− Firms may be declined at the end of the financial year not because they fail
criteria but only because the fund has been fully spent.
− NZTE does not advertise the fund widely and this reduces the chances the fund
will encourage those who do not usually seek help to do so, and so limits the
fund’s ability to achieve the goal of changing opinions about seeking help.
− Turnaround of applications is on average a month, from first submission to final
decision.
12. By Quarterly Report figures EDG was fully subscribed for the 04/05 year, and had
an under spend in 03/04.
13. The numbers of firms applying to the EDG has increased 40% over the two years
since implementation, giving a total of 684 firms applying to date. 570 applications
were accepted. This is a small number and the fund at its current size will probably
not have significant impacts on New Zealand SME capabilities.
14. There has been a large change in the approval rate. At inception 80% of all
applicants were approved but the rate has subsequently reduced. Approvals were
at 70% by January 2005, 60% by April 2005 and at 50% at the end of the 04/05 year.
15. Delivery of the EDG is even across regions and sectors. There is no significant
difference in the numbers being accepted or declined across regions and sectors,
and the rate and size of allocations is similar across regions and sectors.
6
16. Firms applying to the EDG and firms being accepted include large numbers of
exporters. New criteria will be implemented in October 2005 to eliminate the export
focus in favour of ‘net benefit to NZ.’

5. Outcomes of EDG
17. 84% of firms completed the projects they had sought funding for, the remainder
decided not to undertake the project or stopped part‐way.
18. 78% of firms have seen benefits from the grant, in terms of having a product ready
for launch or launching it, understanding which segments of the market to target,
improving the way the firm runs, gaining or clarifying a strategy for the future, or
entering a new market. This means of the 84% of firms who complete a project, 93%
see positive gains from it. This is a very good result for the EDG.
19. For the final outcome of improving firm performance, 39% of EDG firms have seen
gains in firm performance, in terms of export or domestic sales. 37% said it is too
early yet to see such gains in performance. This was usually because while the grant
was closed the project was taking longer than one year.
20. For the intermediate outcome of improved management capability, 56% of firms
saw changes in the way they managed the business; 30% of firms said projects had
no effect on management‐ but these firms typically had undertaken projects which
were not geared at impacting on management. 47% of firms gained management
level assistance and 41% gained business‐level assistance.
21. The management changes included altering the internal practices of the firm,
learning from the project about ways to tackle such projects, gaining direction for
the firm and sometimes confirming that their own approach had been right. Firms
also reported an improvement in their overall ability to manage the business.
22. The activity types of intellectual property protection and prototype development
had very low rates of firms reporting changes in management practices,
undertaking management level projects, or gaining advice. Intellectual property
protection and prototype development are not well fitted to the goals of building
management capability.
23. The evaluation undertook a simple random sample of declined firms. However it
had no information on why they were declined, and so if they undertook the project
themselves even while being declined it may be because they had the capability and
resources to do so, and so it was appropriate that they were declined.
24. The evaluation learnt that many were unclear on why they were declined, and this
left them feeling ‘brushed off’ and unwilling to reapply for assistance. In order to
not defeat the goal of improving the perception of the value of external assistance, it
would be appropriate to give more details of why a firm was declined.

7
Analysis of the policy goals: management and business capability
25. The evaluation noted the fund is aimed at assisting both management and business
capability. At present, the fund is roughly split 50‐50 between two types of firms:
those whose goals for the grant were to improve their business’ capability, and firms
who wished to improve their management capability, or who were undertaking
management level projects.
26. The evaluation distinguished projects by whether they were improving business or
management capability, and looked to see the impacts of each project type. The
evaluation has found a significant association between the proportion of firms
having had management level assistance and reporting impacts on management.
This means that results from the evaluation show that firms undertaking
management level projects will usually also see changes in the way they manage
their business. Firms undertaking business level projects are less likely to see such
changes.
27. The evaluation also found that firms undertaking projects defined as management
level had significantly higher rates of gains in performance. This means that results
from the evaluation show that firms undertaking management level projects will
usually also see gains in their performance. Firms undertaking business level
projects are less likely to see such changes.
28. There is a caveat to this finding. Firms undertaking business level projects usually
said they had not yet seen gains as it was too early given the project they
undertook. So time is needed to see if the above finding holds. It may not if many of
the business level project firms see gains in performance from the project in future
months.

Analysis of the policy goals: improving capability via information or via
building skills
29. The evaluation found there is uncertainty in whether the fund’s objective is to be a
channel by which managers can access information, or whether it is to assist them
build their capability. At present, two types of firms are accessing the fund: those
who can themselves implement the project but need information to do so, and those
who need assistance to gain the capability to implement their project.
30. The evaluation distinguished projects by whether they delivered advice to build
capability, or delivered information to firms, and looked to see the impacts of each
project type. The evaluation has found a significant association between the
proportion of firms having advice or not and having management level or business
level assistance; and also seeing changes in management practice or not. In
particular, firms receiving advice had high rates of receiving management level
assistance and seeing changes in management practice; whereas firms receiving
facts had higher rates of receiving business level assistance and not seeing changes
in management practice. This means that results from the evaluation show that
firms having advice will usually also gain management level assistance and usually
8
see changes in management practice. Firms gaining facts are less likely to see such
changes.
31. As management level projects see higher rates of gain in performance, it seems
aiding management via advice is the best way to aid management to achieve gains
in performance.
32. The evaluation also found a tentative association between receiving advice and
seeing gains in performance. This suggests firms receiving advice will see gains in
performance at a higher rate than firms not receiving advice, but this relationship
may change with time.

6. Operational recommendations for EDG.
33. Scoring criteria for assessing applications need to be defined carefully to ensure
they may be applied objectively and filter applicants appropriately. This is a crucial
part of ensuring the goals of policy are met. NZTE are currently reviewing their
criteria and scoring and it would be appropriate for MED to be involved in this.
34. Firms applying when the fund is fully subscribed or close to full subscription, and
which would otherwise pass, should be held over to the beginning of the next
financial year, rather than being declined.
35. If an application is felt to have insufficient information it should be returned with a
request for more information. Only those applications which fail the criteria should
be declined.
36. Consideration should be given to advertising the fund to extend its reach and
improve its ability to improve the perception of external advice in firms.
37. The EDG assessment team needs a better feedback mechanism of the outcomes of
projects to help them in assessing applicants. Currently one report is made by firms
to NZTE at the close of the grant, and it has an optional provision of sales data
which most firms do not complete. A better feedback mechanism would be
requiring in the report the provision of information on difficulties and successes
implementing the project, and what was learnt as a result. There also should be a
request for a second outcome report requiring information on sales six or twelve
months post end of project.
38. An examination is made of the quality of applicants over 2005/6, and the reasons
firms are declined, as part of monitoring the proportions of firms accepted and
declined. The proportions accepted should be around 70%. If this total is
inappropriate given the overall quality of applicants, then the positioning of the
fund can be reviewed.
39. It is recommended NZTE establishes a system of storing electronically data from
the application forms on all applicant firms’ financial and capability needs, taken
from their description in the application form of the project and their goals for the
external advice or expertise. This should be able to be compared to final outcomes
from the project funded by the grant, and would place both an evaluation in the
9
future and NZTE in a stronger position to assess changes in the firms post
intervention. MED should be involved in establishing this system.
40. The evaluation recommends a process is instigated of detailing in a more
personalized way the reasons for a firm being declined, in order to prevent these
firms feeling ‘brushed off’ and deciding against seeking help again, thereby
thwarting the goal of improving the perception of external advice.

7. Policy recommendations for EDG
41. To target the fund more squarely at management level projects the assessment of
proposals and possibly the description of the funds goals to the market, would need
to be revised. This would require a definition from MED for NZTE of at least the
core of what ‘management level’ is (even if the edges of the distinction with
‘business level’ are left somewhat blurry).
42. The fund could be weighted toward firms wishing to gain advice. This can be done
by adjusting the assessment to spot these sorts of firms, and possibly the
description of the fund to the market.
43. The evaluation recommends a second look is taken at the sample in six to twelve
months to re‐assess outcomes. If the 37% who said it was too early to see benefits
from the grant see gains in performance it will raise the success rate of the fund
significantly. The opportunity can be taken of assessing the current situation of the
firms who had reported seeing gains in performance, to see if the gains have
continued. NZTE may be in the best position to undertake this, as part of its on‐
going monitoring of clients.

8. Overall conclusions
44. The evaluation concludes that the EDG is effective in delivering intermediate
outcomes, and thus far has achieved for nearly half of firms the final outcomes of
improved firm performance. Further evaluative work is required to see the
outcomes for the remaining half.
45. Determining additionality, that is, whether the fund was the cause of the changes in
management or business capability and in firm performance, is usually taken to
require establishing that the fund is the sine qua non cause of the effects in question
via testing the counterfactual conditional that the effect would not happen without
the cause. The evaluation was unable to demonstrate this with the methodology at
its disposal. A well‐designed randomized control trial is the established method of
demonstrating such a causal link, but a randomized control trial takes more time
than the evaluation had to implement and evaluate, and would conflict with the
aims and implementation of the fund, since it would mean withholding grants from
firms thought to be worthy of them. The evaluation is satisfied that the evidence
gathered by survey has established a sufficient association between the fund and
outcomes to merit the fund continuing.
10
46. The evaluation concludes that the Enterprise Development Grants should continue,
with some adjustments as specified above. It recommends that MED review the
overall size of the fund and whether it is reaching appropriate numbers of firms.

11
PART ONE: Evaluation Goals and Methodology.

9. Goals of the evaluation
47. The evaluation had three overarching goals.
48. The first was to understand the operational implementation of both MED and
NZTE policy. This would inform the Ministry whether operation design was
consistent with policy and whether the operational implementation was such the
goals of the fund could be achieved.
49. The second was to understand the delivery of the fund, in terms of numbers of
firms assisted and declined, regional and sectoral distribution of funding, and the
types of firms who had received funding.
50. The third goal was to determine whether the outcomes policy wished to achieve
were occurring for the firms undertaking the activities funded by the EDF.

10. The intervention logic
51. The intervention logic describing the EDF is given below.
52. The yellow boxes describe the problems the intervention is to overcome, the pink
boxes the activities the intervention will undertake to overcome the problems, the
blue boxes are the intermediate outcomes the activities should produce, and the red
boxes are the final outcomes that the intermediate outcomes, and so the
intervention as a whole, should achieve.
53. In other words, the activities of engaging the services of a business mentor,
undertaking more advanced management and technology based training,
employing specific external advice and expertise in a management area and
undertaking international market development activities should result in
improving management capability, improving the perception of the value of
external advice, and improving collaboration between entrepreneurs and existing
firms, and this should result in improved firm performance.

11. Methodology
54. The evaluation used three different methods to meet its three goals.
55. To understand implementation the evaluation considered the policy documents of
MED and NZTE, and met with the NZTE teams and personnel who manage the
operation of the EDF, and learnt from them their processes and systems.
56. To understand delivery the evaluation analysed the data collected by NZTE on the
firms who apply and are granted or declined a grant.
57. To understand outcomes the evaluation undertook a survey of a representative
sample of accepted and declined firms and questioned them on their reasons for
applying for a grant, the activities they undertook using the grant, what they learnt,
and what outcomes had been achieved.
58. Determining additionality, that is, whether the fund is the cause of any changes in
management or business capability and in firm performance, is usually taken to
require establishing that the fund is the sine qua non cause of any effects in question
via testing the counterfactual conditional that the effect would not happen without
the cause.
59. The evaluation was unable to demonstrate this with the methodology at its
disposal. A well‐designed randomized control trial is the established method of
demonstrating such a causal link, but a randomized control trial takes more time
than the evaluation had to implement and evaluate, and would conflict with the
aims and implementation of the fund, since it would mean withholding grants from
firms thought to be worthy of them.
60. The evaluation is satisfied that the evidence gathered by survey as it has designed it
would establish a sufficient association between the fund and outcomes to answer
questions of outcomes and the question of whether the fund merits continuing.

12. Methodology of the survey
61. The survey was by telephone interview.
62. The population was all firms who had completed a project by April 2005, giving a
total of 191. The sample was a stratified simple random sample with a sample size
chosen to give a margin of error of 5% for estimates of proportions. This gave a
minimum sample of 129. The evaluation surveyed 132 firms.
63. The sample was stratified by the activity types available under the EDG, and the
activities to be included were selected by MED policy. Policy wished to know the
outcomes for firms undertaking strategic design, intellectual property protection,
prototype, business or strategic plans, feasibility studies, certification, systems
evaluation, training, mentoring, market research and market plans.
64. The sample was selected as a stratified simple random sample but with sampling
fractions which were similar in each stratum, and also very high, the data could be
analysed as a simple random sample.

13
Enterprise Development Fund Logic Model

Problems Activities
Intermediate outcomes Final
outcomes

Lack of early stage

financing
Impediments to
Enterprise Development Fund
provides funding to smaller
collaboration, with firms, entrepreneurs, start-ups, Improved business, management,
resultant inefficiencies exporters and groups of
companies, after an initial
technology, international business,
exporting and leadership capability
capability assessment, to: Improved firm
1. Engage the services of a performance
Lack of international business mentor Improved perception in

linkages/contacts
2. Undertake more
advanced management
the value of developing
business capability Improved
and technology based Improved perception of the value
international
Lack of business
3.
training
Employ specific external
of seeking external advice
competitiveness
management capability advice and expertise in a Increased global of
products/services
(including export,
international
4.
management area
Undertake international
connectedness
management skills). and domestic market Improved collaboration between

Also access issues.
development activities firms
Risk aversion of firms

Undervalue
contribution of
external advice
PART TWO: Policy Rationale for the
Enterprise Development Fund

1. MED Policy

13. Policy objectives for the Foundation Services
65. In April 2003 Cabinet agreed to establish the Enterprise Development Fund as part
of the Foundation Services of NZTE. (EDC (03) 54 refers)
66. The Foundation Services are to assist small, young firms, young entrepreneurs and
start‐ups, via information, training and financial assistance. The rationale was that
running a business requires skills and knowledge that small young firms may not
yet have acquired. Acquiring them may be made difficult by the expense of
purchasing external advice and expertise and a belief that expertise is not required
or is not valuable. The foundation services were established to make acquiring the
skills and knowledge easier and more attractive for firms. The rationale was given
in EDC (03) 54 as:
Small businesses and entrepreneurs often lack the financial ability to
employ all the expertise they need to get new concepts and projects
up and running. They also tend to undervalue the potential
contribution of expert advice on key elements of their business ideas.
Without such advice, they are often unable to satisfy the requirements
of investors, banks and other financiers for information, including
business planning, approaches to preserving the value of intellectual
property, and how they plan to market their products and services.
As a result, small businesses often do not realise their full potential.
These barriers could be overcome if access to information was made
simpler and business operators were provided with better
opportunities to acquire the skills and knowledge to improve their
management capability.

14. Policy objectives for the Enterprise Development Fund
67. Three Industry New Zealand and Trade New Zealand programmes were
amalgamated to form the EDF: the Enterprise Awards Scheme, the Export Network
programme and World Class New Zealanders. These three all delivered funding
assistance to firms.
68. The goal of the former Enterprise Awards Scheme (EAS) became the overall goal of
the EDF. This is to assist innovative firms and entrepreneurs to build capability by
enabling them to employ expertise and advice.
69. Funding is for 50% of total costs up to a maximum of $20,000‐ an increase from the
EAS total of $10,000. This is per applicant per annum, and is for businesses and
entrepreneurs to:
− Engage the services of a business mentor for a period of time
investigation, trade fair participation, trade/business missions, business exchanges,
and visiting buyers.
70. The design of the EDF was to address issues with its predecessor schemes. The
issues were:
− Criteria: the EAS was a relative, awards based scheme, where the top proportion
per month received assistance, and the quality of the proposals accepted for
funded varies from round to round as approval is relative. The EDF was to be an
objective criteria‐based scheme, where any firm which met certain criteria gained
assistance.
− Application turnaround: businesses often did not know for up to two months
whether their application was successful.
− Accessibility: the limited capability of small business owners to self‐appraise their
needs limits the programmes ability to effectively address critical capability issues.
− Overlaps: There were overlaps with the case managed Business Growth Fund.
The limited funding of the EAS lead to a greater focus on high growth and more
established businesses than was intended.
16
2. NZTE policy

15. Design of the Enterprise Development Fund
71. In a paper to the Integration Ministers (Paper to the Integration Ministers 4 July
2003 refers), NZTE gave the detailed report back on its proposed operational design
and criteria for the EDF requested in EDC (03) 54.
72. The paper noted that the EDF was an amalgamation of the three predecessor
programmes and that it would remain similar to them. It also noted that grants
were for $20,000 and for 50% costs. The activities to be funded were also the same.
73. NZTE proposed some additional design features. These were:
− Criteria: the fund would adhere to clear criteria and would centralise all final
approvals within NZTE.
− Application turnaround: the approval processes would ensure a high level of
responsiveness to clients. NZTE anticipated a fast turnaround of applications, and
aimed for a five day turnaround from receipt of applications.
− Overlaps: the fund would remain accessible and responsive to start‐up businesses
and entrepreneurs not yet in business.
− Ease of application: the application forms for the EDG and background material
would be as clear and as easy to complete as possible. Being aware that
applications forms and the information required must still be reasonably
demanding to ensure accountability, NZTE undertook to provide clients with
assistance in completing application forms.

16. Criteria for projects
74. NZTE established criteria for the project or activity firms wished funding for. These
are:
− The project must have good commercial potential and add value to an existing
business or an entrepreneur’s current activities
− Applicants must demonstrate that they have the capacity and capability (including
financial and planning support) to carry the project through to commercialisation
− The project or activity must be capable of generating high returns, or high levels of
growth for the applicant once commercialised or undertaken.
− The project must be consistent with relevant laws and regulations

17. Scoring of applications
75. NZTE noted it expected high levels of demand for the EDG, because it had
experienced high levels of demand for the Enterprise Awards Scheme, where only
17
30% of applicants were accepted. NZTE proposed to score proposals for the EDG to
ration demands for funding, so only those applications which best met the goals of
the programme and of NZTE received funding. Applications were proposed to be
scored against criteria with weightings so certain criteria were more important to
achieving the critical score. The criteria were given as:
− Robustness of the project plan
− financial and organisational stability
− ability of organisation to implement project/undertake activity
− level of innovation or additionality associated with the project or activity
− level of growth within the organisation and likely economic benefits
− level of need for government assistance

18. Future changes: devolvement and criteria for 05/06
76. FRST and NZTE have begun to align the delivery of EDG and Smartstart (EDC Min
(04) 25/5 refers). The alignment involves devolving grants of $5000 and less to an
agency network.
77. This, on numbers to date, is 15% of all EDG grants. A half of these have been
strategic development or product development grants and a quarter have been
mentoring or training. The last quarter are evenly spread across the activities.
78. In July 2005 (subsequent to the evaluation’s timeframe) Ministers agreed to revise
the EDG criteria to make clearer the distinction between programme, entry and
assessment criteria, and to speed up assessment processes (EDC Min (04) 25/5 and
The coordinated services delivery project update of 22 July 2005 refers). NZTE aims to
implement the criteria and scoring system by 1 October 2005.
79. The revised assessment criteria are now:
− Robustness of project plan;
− Ability of business/individual to undertake activity (“previously ability of
organisation to implement project or activity and their commitment and drive
to undertake planned work”. This criteria also picks up “financial and
organisational stability”);
− Potential to improve business performance (previously “potential for growth
and benefit to regional/national economies”)
− Net benefit to NZ of improved business performance (previously “level of
innovation or additionally associated with the project or activity”); and
− Level of difference the funding will make (previously “level of need for govt
assistance/funding assistance and a catalyst for growth”).
18
80. NZTE has removed ‘financial ability’ as a criterion, principally because the firm
must expend the full amount prior to reimbursement by NZTE, and this is a
sufficient step to ensure the firm is financially able to undertake the project.
19
PART THREE: Findings and Recommendations on the
Implementation of Policy

19. Programme implementation
81. The EDF was to be an amalgamation of the former Enterprise Awards, World Class
New Zealanders, and the Export Network Programme, delivered by Trade New
Zealand and Industry New Zealand.
82. NZTE has implemented the Export Networks Programme into the EDF quite
differently from its implementation of the other two, in fact it could be argued it
was not implemented into the EDF at all, but kept altogether separate. It was
renamed: ‘Enterprise Networks’ (EN). The other two predecessors are renamed
Enterprise Development Grants (EDG).
83. NZTE split the appropriation for EDF of $8.613M between the EN and the EDG. EN
received $4.48M, EDG the rest ($4.413M).
84. Export Networks aided firms in their exporting efforts by assisting groups of firms
to target high quality market development opportunities offshore. It did this by
assisting groups of firms to take advantage of market opportunities which
individual firms could not, thereby achieving gains in forex performance
unavailable to individual firms.
85. The primary issue for MED with EN was the order of practice. NZTE selected
offshore opportunities and requested interest from firms in those activities; rather
than establishing a process whereby firms applied for funding to attend offshore
activities they had selected.
86. The selection of offshore activities by NZTE was sector‐led: events were ranked by
their degree of fit to NZTE’s sector priorities, and those which fitted best were
chosen for the year’s EN activity. Client managers were notified and they invited
firms on their books whom they deemed appropriate to apply. Firms not selected
could not apply. Offshore activities which were not selected by NZTE were not
available for firms
87. The design principles of the EN contradicted CER. Targeting export education
breaches the EDF criteria of no common ownership. A number of firms larger than
criteria for EN have accessed the fund.

20. Response by MED to the Enterprise Networks implementation
88. MED considered the implementation of EN sufficiently different from their policy
objectives to remove Enterprise Networks from the EDF. Funding for Enterprise
Networks was moved to operational expenditure (CAB Min (04) 38/4 refers).
89. EN is now part of NZTE’s sector facilitation activities. NZTE have been invited to
give the policy framework for these activities, and MED is currently in discussion
with NZTE on this.
20

The report ends its evaluation of EN here. The rest of the report is on the
evaluation of the remainder of the EDF: the Enterprise Development Grants
scheme.

21. Operational implementation: the Business Evaluation Team
90. The client managers for most firms using the EDG are those of the Business
Evaluation Team. The Business Evaluation Team is also the ‘frontline’ team who
handle the calls to the NZTE hotline phone number. There are 14 members of the
team.
91. As client managers, each member of the team has large numbers of EDF and other
NZTE clients. As they have so many, they are not expected to have a proactive
relationship with all their clients. Individual client managers may choose to ring
firms on their books, but this is not obligatory. Hence EDF clients are called ‘light
touch’ firms. They are not closely watched by NZTE.

22. Operational implementation: the Enterprise Development Team
92. The Enterprise Development Team is the central point of assessment for
applications for the fund. There are four members.
93. Firms can apply for the fund directly to the team using applications from the
NZTE website, or can be guided to the team by the business evaluation team. If
firms apply directly to the enterprise development team and do not have a client
manager, the team finds them a suitable client manager, usually one from the
business evaluation team.
94. The team has two hats. They have a pre‐assessment role, where the team assists
firms with the filling out of firms, reviews applications and may personally advise
firms on whether the applications are sufficiently complete and what additional
information is required. There is potential tension between these two roles, but
careful use of advice on potential is also of benefit to firms who are clearly outside
the criteria. The assessment process is discussed further in paragraphs 102‐133
95. The team’s other hat is assessment. Their process is to divide applications among
team members, and then assess them. The team then meets, usually once a week,
but more frequently as necessary, to discuss their assessment of applications. At
this point they either confirm the assessment, or reassess them.
96. Firms are notified by letters if they are accepted or declined. From December 2004
letters to declined firms were changed to include details on why the application
was declined. If firms were declined only because they lacked information but are
21
felt to have potential they are encouraged to re‐apply with the additional
information.
97. The team is encouraged to become familiar with business, by allowing members to
visit other NZTE client managers based across New Zealand.

23. Design of the Enterprise Development Grants scheme
98. The evaluation examined whether design issues and proposed design (given in
part two) have been met and implemented.

Assessment against criteria
99. Firms are assessed against criteria. However there are some issues with
assessment which means it is not as objective as it might be, and assessment does
not always adhere to criteria in ways clear to firms (even if the criteria themselves
are clear to firms).
100. Firms who apply late in the financial year are declined only because the funding
year is finished, and not because they do not qualify. At the end of the 04/05 year
the score firms had to achieve was raised in order to prevent an over spend,
suggesting firms applying later in the year were disadvantaged. See section 31 for
further details.
Recommendation
101. It would be more consistent with a criteria‐based scheme if firms applying very
late were held over and approved against the criteria in the first weeks of the new
financial year.
102. Scoring should not be raised only to prevent an over‐spend apparent towards the
end of the financial year, as this unfairly disadvantages firms applying late in the
financial year. Having raised the scoring NZTE intends to leave it at this level, but
should not raise it again before the end of the 05/06 year.

Turnaround of applications
103. NZTE aims for a fast turnaround of applications once they are complete. NZTE
stated the turnaround would be 5‐days from receipt of application, and there are
some issues in meeting this. NZTE does not on average make a 5‐day turnaround
from receipt of application.
104. Instead, because it assists firms fill out their applications, and sends applications
back when they need more information, NZTE counts its turnaround from the
date the application form is sufficiently complete. NZTE says it maintains a seven
day turnaround from this point. See section 33 for further details.

22
Recommendation
105. It may be preferable to resolve the inconsistency of returning applications for more
detail and declining applications but with an invitation to reapply with more
detail. The evaluation recommends declining applications only when the firm
actually fails the criteria, and not when there is insufficient information to tell
whether it ought to pass or fail. The latter leaves firms in considerable uncertainty.

Accessibility
106. The accessibility of the fund is limited by its being advertised to economic
development agencies and other consultants, because only firms already seeking
assistance or advice will learn about it. This lessens the chances the fund will
encourage those firms who don’t usually seek external advice to do so. It also
limits the fund to those who have already decided they have a capability gap.
Recommendation
107. Advertising the fund more broadly may solve this issue, however advertising the
fund broadly must be balanced against the potential resulting problem of the fund
selling out very quickly in the year.

Overlaps
108. The delivery of the fund by NZTE has been successful in avoiding overlaps with
the Growth Services Fund. NZTE have added to the eligibility criteria the
requirement that firms have a turnover of less than NZ$5M and/or 20 FTEs or less.
109. EDG clients are typically low growth, low turnover and fairly young. However
there was no definition of what ‘young’ meant, or whether it was to be interpreted
in terms of age, or, for example, growth, or capability. See sections 45 and 46 for
further details.

Ease of application process
110. The evaluation asked the sample of firms it surveyed about the length of
application time and whether firms were satisfied with the time it took them to
apply. The majority of firms said that given what resulted from the grant it was
worth the time and effort.
111. Most firms did go on to say the process was long‐winded and took a great deal of
time‐ for some days, others hours. On the whole however, having received funds
and seen a benefit, they were felt the grant was worth the time it took to apply.
112. Many also noted that the process of writing out their business plan and plan for
the project for the application form was beneficial for their own clarity of business
goals and current situation.
23
113. The evaluation can not tell how many
firms did give up in filling out the Satisfaction with the time taken to apply
forms and never applied. Some firms
told the evaluation they nearly gave Sample of closed projects 1/10/2003 to 4/2005
up. If there were ways of shortening
120
the application forms, it may be

worth considering. 100
80
114. Some firms commented that they
Number
wished for more flexibility in 60
adjusting the project once it had 40
started. NZTE will not reimburse

20
firms for additional tasks on the
project to those in the original 0
worth it Not worth it
application. The claims must match Worth of time taken
the tasks laid out in the original

application form. The only variation which is permitted is to change the external
provider. Many firms find as the project develops that they need to add tasks to it,
and firms have explained to the evaluation that they cannot predict the project
exactly one year out, and often things eventuate which they didn’t anticipate. The
firms argued that this is the nature of undertaking early‐stage projects.
115. To get around this, firms try to have applications accepted with minimal
information, to give them the freedom to add tasks later, and yet still have the
project fit the original broad application.
116. NZTE notes it must balance flexibility in process with the time it must spend
evaluating whether a change in a project is appropriate and whether it changes the
project sufficiently it as a whole no longer qualifies for funding. It feels the costs of
this will outweigh the benefits to firms.

Recommendation
117. With devolvement it may be that issue of adjusting projects mid‐way can be
ameliorated with the greater degree of contact between firms and the delivery
agents. This issue might best be considered when the devolvement is evaluated.

Criteria and scoring of proposals
118. From implementation until October 2005 the criteria which have been used to
assess firms were as initially proposed. (The changes made subsequent to the
time‐frame of the evaluation were given above).
119. The categories for the criteria for the time‐frame of the evaluation were:
− robustness of proposal
− financial and organisational stability
24
− ability of organisation to implement project or activity and their commitment
and drive to undertake planned work
− potential for growth and benefit to regional/national economies
− level of need for government assistance/funding assistance as a catalyst for
growth
− level of innovation or additionality of the product/service of the value added to
existing
120. Under each category have been added descriptions and corresponding scores,
ranging from poor to good.
121. The team is to match the application to the description and score accordingly. For
example under: ‘ability of organisation to implement project or activity’ the
descriptions and scores are:
1‐3 Lack of or limited experience in the industry
Lack of or limited experience in business
Management depth appears light in some areas
4‐6 Demonstrated level and depth of management experience
Demonstration of drive to succeed
Evidence of transferable skills
Mentors/support work engaged as needed
7‐10 Demonstrated depth of management experience
Experience in industry
Obtains external advice or mentoring
122. The higher the score the better an applicant does by that criterion. The scores are
then weighted. The scores an applicant receives under ‘growth potential’ and
‘innovativeness of product’ are multiplied by 20. The others are multiplied by 15.
123. An applicant must obtain a minimum of 4 out of 10 for ‘growth potential’ and
‘innovativeness of product’ and 3 out of 10 for each of the others to be approved.
Multiplied, that is 80 out of 100 and 45 out of 100 respectively.
124. An applicant must get a total of 570 overall to be approved.
125. The team meets regularly to confirm assessment scoring, however they have
reported to the evaluation that there is close consistency in scoring by the
members, and so they have decided to cease meeting regularly to check
consistency of scoring.

Recommendation
126. The evaluation has noted that there is some ambiguity between the criterion
addressing ability of the firm to implement the project they seek funding for,
25
which is measured by management experience, and the capability‐building goal of
the fund.
127. Firms gain higher scores and thereby are more likely to be granted an award the
more experience they have. They gain between 7 and 10 points if they have:
‘demonstrated depth of management experience’ or ‘experience in industry’ or
‘obtains external advice or mentoring.’ They will fail the criteria if they have: ‘lack
of or limited experience in the industry’ or ‘lack of or limited experience in
business’ or ‘management depth appears light in some areas.’
128. It seems inconsistent with building capability if a lack of or limited amount of
experience fails a firm. It may help if the criterion and scoring refer specifically to
the minimum kind of experience firms should have, to qualify, and not the degree
of experience (measured, say, by years). The EDG can then be more easily aimed
at those firms with certain basic skills who lack more sophisticated skills.
129. The tests for net benefit to NZ are improved productivity and efficiency, limited
displacement (displacement would result from funding firms with significant
domestic focus and significant domestic competition) and presence of spillovers.
These are still high level concepts requiring variables which can identify in a
project description when and to what degree productivity would be improved, or
when and where a spillover would occur.
130. Translating high level concepts into concrete variables (operationalising the
concepts) is a crucial part of ensuring the goals of policy are met. For example,
‘level of difference’ must be operationalised in such a way that the variables which
are to indicate funding will make a level of difference, really do indicate the
funding makes the required difference. If they don’t, firms are not truly being
assessed against that criteria.
131. The evaluation recommends MED is involved in assisting with development of
the scoring criteria for the EDG assessments.

24. Project activities available under EDG
132. The full range of project activities available to firms through the EDG is broader
than first proposed. The activities currently available are:
− strategic business development (including business/strategic plan development,
human resource strategic plan development, financial viability planning,
strategic design advice)
− feasibility studies
− product development (including prototype design, development and testing,
intellectual property protection)
− business and operational excellence (including international quality standards
certification, systems evaluation and development, environmental management
system)
26
− e‐commerce and e‐business strategies (including development of a e‐strategy)
− Market strategy development (including market research or new market
investigation, marketing plans).
133. These activities cover a very broad range of possible activities, but are not all
equally popular, and some are rarely taken up by firms. The most popular, and so
most of the allocations have gone to: intellectual property protection, prototype
development, strategic business development, and strategic design advice. See
section 47 for further details.
27
PART FOUR: Findings and Recommendations on
Delivery of EDG

Note: NZTE has some problems with its data system and has difficulties extracting data from it. The
evaluation is aware that details on at least 153 applications are missing from its data set. This is 17.4% of
the total grant data for EDF. The evaluation has no details on these applicants and so has not included them
in its analysis. The evaluation is confident the data it has is sufficient to give an accurate picture of the
delivery of the EDG to date.

134. There are possible impacts of delivery that the evaluation is unable to determine
exactly; for example, the development of a network of consultants able to help
businesses and willing to encourage businesses to seek their help. Conversely, for
example, it is possible that the fund stimulates consultants to adjust their fees
upward to take advantage of the 50:50 funding.

3. Uptake of the fund

Note: for section 3 and 4 only the evaluation reports on data for 20 Months, that is, from October 2003 to
end June 2005. This covers the time from implementation until the end of the 04/05 year.

25. Total allocations
135. NZTE allocated $4.133M to EDG.
136. The total appropriated to the EDG for 03/04 and 04/05 is $8.266M
137. The evaluation has a total allocation for 03/04 and 04/05 of $6,424,798.11.
138. By NZTE quarterly report figures there was an under spend in 03/04 but 04/05 was
fully subscribed. Quarterly report figures have a total allocation for EDG of
$M7.253.

26. Total number of applications
139. In 20 months NZTE has received 809 applications for the Enterprise Development
Grants capability building component.
140. In the 2003/2004 year there were 320 applications.
141. In the 2004/2005 year there were 489 applications. This is a 53% increase from
03/04 to 04/05.

28
27. Total number of firms
142. Note that firms reapply, and so there Firms returning for a second grant
have been more applications for the
grant than there are firms applying 1/10/2003 to 30/6/2005
143. Over 20 months from October 2003 to 30 $20,000.00
June 2005 684 firms have applied for the
2004/2005 Amount claimed

EDG. The small size of the EDG limits its $15,000.00
impact on SMEs.
144. 285 firms applied in 03/04. $10,000.00
145. 399 firms applied in 04/05. This is a 40%
increase from 03/04 to 04/05. $5,000.00
146. The graph right shows firms who

$0.00
reapplied. Firms claiming two grants are
$0.00 $5,000.00 $10,000.00 $15,000.00 $20,000.00 $25,000.00
represented by dots in the middle of the 2003/2004 Amount claimed
graph. Those who did not receive two
grants are the dots on $0 on both axes.
147. 136 firms (16%) have reapplied for the EDG. 68 (8%) reapplied for the same
project, with 20 of those declined again, and 34 accepted. Three were declined
twice then accepted. The evaluation has recommended earlier that a clear
distinction be made in assessment processes between returning applications for
more information and declining applications.

28. Approval rate
148. In the period October 2003 to June 2005 EDF has approved 570 applications and
declined 216 applications.
149. This is 786 in total, and the difference from the earlier total given of applications
(809) is that that was number of applications handled. Some of these have been
returned to firms who did not reapply.
150. In the 2003/2004 year 251 applications were accepted (81%) and 57 declined
(18.5%).
151. In the 2004/2005 year 319 applications were accepted (68%) and 159 were declined
(33%).

29. Calls to the Business Evaluation Team
152. EDG inquiries can be made to the business evaluation team who run the ‘frontline’
NZTE hotline number, and the team may recommend firms to apply for an EDG
grant. The data from the call centre does not give a picture of the interest in EDG
going via the centre.
29
153. The call centre does not tally
distinctly the various NZTE grants, Inquiries to the hotline team
so all grant inquiry by telephone is

under ‘grants.’ From 03/04 to March 1/17/2003 to 30/4/2005
04/05 (see graph right) the number of 3000 Total calls
requests for grants handled by the Web inquiries
Grant advice
2500
hotline call centre looks steady.
2000
154. The data on web hits does not
Number
1500
exclude hits generated by spam, so
the web data does not show whether 1000
genuine hits have increased or 500
decreased. NZTE is working to build
0
firewalls, and notes that this will
DEC 2003
DEC 2004
SEP 2003
SEP 2004
FEB 2004
FEB 2005
JUN 2004
OCT 2003
OCT 2004
MAR 2004
MAR 2005
JUL 2003
JUL 2004
NOV 2003
NOV 2004
APR 2004
APR 2005
JAN 2004
JAN 2005
AUG 2003
AUG 2004
MAY 2004
show as a marked decrease in hits,
but will not be a decrease in genuine Month
hits.

30. Change in application numbers over time
155. There has been an increase in the numbers applying to the EDG over the two years
of its operation. After a small beginning, the interest in EDG grew. Numbers
applying jumped in April 2004 to roughly fifty a month and have remained
around that total.
156. The graphs below show applications lodged (graph below left) and applications
assessed (graph below right). Note the surge in numbers lodged in June and July
2004, with the corresponding surge in assessments in June and August 2004 and
carrying over through September and October 2004.
Applications lodged Applications assessed
1/10/2003 to 7/4/2005 1/10/2003 to 30/5/2005
80 80
Number of assessments
Number of applications
60 60
40 40
20 20
0 0
DEC 2003
DEC 2004
DEC 2003
DEC 2004
SEP 2004
SEP 2004
FEB 2004
FEB 2005
FEB 2004
FEB 2005
OCT 2003
OCT 2004
JUN 2004
JUN 2005
OCT 2003
OCT 2004
JUN 2004
MAR 2004
MAR 2005
MAR 2004
MAR 2005
JUL 2004
JUL 2004
NOV 2003
APR 2004
NOV 2004
APR 2005
NOV 2003
APR 2004
NOV 2004
JAN 2004
JAN 2005
JAN 2004
JAN 2005
AUG 2004
AUG 2004
MAY 2004
MAY 2005
MAY 2004
Month Month
30
157. There was a drop in decisions in July, presumably as a result of yearend 03/04
activities, and many applications lodged in July look to have been assessed in
August.

31
4. Assessment of applications

31. Change in approval rate over time
158. From 03/04 to 04/05 the number of applications which have been declined has
increased from 57 to 159, which is an increase of 179%. The number of applications
accepted has increased from 251 to 319, which is an increase of 27%.

159. This shows that over the two years of delivery there has been a change in the
approval rate. The graphs below show the numbers accepted and declined over
time, with the dark line added to show the trend.
160. The graph below left shows that the numbers of accepted applications are in
decline. The graph below right shows that the numbers declined are increasing.

Declined applications over time Accepted applications over time
1/10/2003 to 30/5/2005 1/10/2003 to 30/5/2005
60 Declined 60 Accepted
applications applications
Moving Moving
50 average of average of
50
declined accepted
applications applications
40
Number
Number
40
30
30
20
20
10
0 10
DEC 2003
DEC 2004
DEC 2003
DEC 2004
SEP 2004
SEP 2004
FEB 2004
FEB 2005
FEB 2004
FEB 2005
JUN 2004
OCT 2004
JUN 2004
OCT 2004
MAR 2004
MAR 2005
MAR 2004
MAR 2005
JUL 2004
JUL 2004
NOV 2003
APR 2004
NOV 2004
APR 2005
NOV 2003
APR 2004
NOV 2004
APR 2005
JAN 2004
JAN 2005
JAN 2004
JAN 2005
AUG 2004
AUG 2004
MAY 2004
MAY 2005
MAY 2004
MAY 2005
Month Month

161. As another way of viewing this change, consider the ratio of acceptances to
declines.
162. If the numbers accepted and declined over the total number of applications is
examined (graph below right) it can be seen that the proportion of applications
accepted has steadily declined from 80% (0.80) in November 2003 to 60% in
August 2004 and 50% (0.50) in May 2005.
32
163. Put another way, the number of
applications declined has risen
Proportion of applications accepted and declined over time
from below 20% (0.20) to around
50% (0.50).
1/10/2003 to 30/5/2005
164. This is a large change. It has
Accepted
resulted in a very large decline applications
0.8 Moving
rate. average of
accepted
applications
165. The change in the proportions Declined
Proportion
0.6
applications
Moving
accepted and declined may have average of
declined
a number of explanations. 0.4 applications
166. First, demand for the fund may 0.2
be higher than the EDG can

currently accommodate, and so
DEC 2003
DEC 2004
SEP 2004
FEB 2004
FEB 2005
JUN 2004
OCT 2004
MAR 2004
MAR 2005
JUL 2004
NOV 2003
APR 2004
NOV 2004
APR 2005
JAN 2004
JAN 2005
AUG 2004
MAY 2004
MAY 2005
the EDG team is forced to fail
more applications than they can Month
pass. Second, it might be that

over time the applications are lowering in standard. Third, there may have been a
change in assessment methods. Fourth, the there may be an issue with the criteria
themselves.
167. Examining the expenditure of the fund (the data the evaluation has will be
sufficiently accurate to allow this) shows it was not in serious danger of being over
subscribed early in the 03/04 year when acceptance numbers first declined, nor
even toward the end of the 04/05 year when declines reached over 40% (see table
right). In March 2005 the fund still had ¾ of a million Date Allocation
dollars left to spend. Decreasing numbers of accepted
October-03 $183,604.23
applications seems inconsistent with an effort to prevent November-03 $236,002.47
oversubscription. December-03 $383,350.52
January-04 $247,223.25
168. The second possible explanation was a change in the February-04 $115,889.17
standard of applications. This may be due to a change in March-04 $225,215.89
April-04 $404,186.75
the types of firms applying. Perhaps a truly representative
May-04 $456,915.74
standard of applicable firms is now being seen by the team; June-04 $703,380.07
because better than average firms applied early in the life of Total $2,955,768.09
the fund.
July-04
$117,510.73
169. As NZTE does not advertise the fund widely, in its early August-04 $593,816.55
September-04 $468,432.43
days only those firms who are proactive about seeking October-04 $465,113.12
assistance would have known about it. These firms may November-04 $334,250.10
have also been better than average. December-04 $361,215.81
January-05 $307,229.22
170. If this is the case, the score to pass the criteria is higher than February-05 $391,522.81
the average firm can manage, and perhaps is too high. March-05 $347,920.68
Total $3,387,011.45
171. Conversely, the change in standards may have arisen
because without advertising, only those firms who approach economic
development agencies or other consultants discover the fund. These firms may be
33
less able than average. If this is the case, advertising may raise the standard of
firms applying.
172. NZTE has recently raised the maximum score firms had to achieve to be awarded
the grant, from 570 to 585. This was in March 2005 and was said to be in order to
prevent an over spend in the 04/05 year. However as shown above there was a fair
bit of money still available in March, and the decline in proportions accepted
began earlier than March, so this cannot be the explanation for the trend as a
whole. If the standards of firms applying have declined, raising the score will
compound the issue. This will be able to be seen in the approval rates next year.
173. Finally, the standard of applications may explain increasing levels of declines if
sufficient numbers are declined for poor quality applications, but are subsequently
approved. However the numbers in this situation are too small for this. Only 8%
were declined and then reapplied for the same project.
174. The third possible explanation was a change in assessment methods.
175. NZTE has told the evaluation that their assessment has become more thorough
with expanded assessment instructions implemented in January 2005. The
expanded assessment instructions contained detailed instructions for the existing
scoring requirements to help the EDG team score applications. These might have
made it easier to determine the standard of the applications, causing higher
numbers to be declined, or they may have made it tougher for firms to pass
assessment and be approved. Again, this change was implemented too late to be
the cause of the decline.
176. The final possible cause was an issue with the criteria and their application.
177. In assessing applications by applying criteria the team makes a decision, and the
evaluation has looked at the information the team has at its disposal to make its
decisions, and how that is translating into decisions. Understanding this may help
in understanding why there is a trend of increasing declines.
178. Considering first the reasons for declining firms, after December 2004 (which is
the date from which NZTE kept detailed records on their reasons for declining
firms), 75% are failed on ‘potential for growth and benefit to the economy’, 62%
are failed on ‘level of innovation of the product or service, or value‐add to the
firm’s activities’.
179. Less than 10% are failed on ‘level of need for government assistance’, ‘ability of
firm to implement the project’ or on ‘proof of funding’. A third is failed on
‘financial stability’. There is a fairly even split on pass or fail on ‘robustness of
proposal.’ This criterion is general‐ the overall feel for the application itself.
180. The main reasons for failure then are that the applications show low or poor
potential for growth, from the idea, product or service proposed; or show low or
no benefit to the economy. This suggests that the EDG has not had firms with
lower ability, as was speculated above; else firms would be being failed on ‘ability
to implement the project’ or ‘financial stability’.
34
181. Rather, the team is rejecting firms because they feel they have low or poor
potential for growth and low benefit to the national economy.
182. Looking next to the information the team has available to make these decisions,
the team has the information from the story woven by the firm in their application,
information from other NZTE client or sector managers, and information from
their own experience of assessing applications.
183. The team is limited in information from assessing previous applications because
there is very little feedback for them on their previous decisions. The final reports
firms file usually have no information in them on sales. There is no request in the
forms for information on issues faced by the firm in implementing the project.
There is no other process of follow up. As a result, the team is not learning from
the outcomes of their decisions which of their decisions were right or wrong.
184. So with limited information, the team must make one of the hardest decisions to
make about firms‐ the potential of a small and young business and a new product.
This decision is genuinely uncertain. As the EDG team noted to the evaluation,
‘it’s a real punt’.
185. Furthermore, with the fund going to young inexperienced firms, NZTE faces
genuine risk that funds go to firms who will fail anyway. NZTE may feel it needs
to be risk averse in order to best fulfill its obligations to Government. This may
have translated into pressure on the EDG team to be conservative in their
assessments.
186. It may be that the reason firms are being declined in increasing numbers is
because without good experience in spotting and acknowledging potential, and
perhaps with an overly strong concern of risk, the team has become too
conservative in its decisions.

32. Recommendations
187. The evaluation recommends a mechanism for more detailed feedback to the team
on the results of projects and grants, so the team may learn the results of their
decisions, and so learn about making these sorts of decisions.
188. It recommends a careful look at the data for the year 2005/2006, to see whether the
trend in application decisions continues. It also recommends that NZTE comes to
an opinion on application quality and whether standards are declining overall,
and watches the quality over the next year. This will inform NZTE on whether
scoring levels and overall positioning of the fund are appropriate.
189. If standards are not declining and the trend continues, the evaluation recommends
further resources be provided to the EDG team in terms of at least one highly
experienced case manager or former business person, perhaps temporally to assist
the team adjust their scoring, or permanently. The evaluation suggests NZTE
consider approving a random sample of applications to counter conservatism in
the assessment.
35
190. It recommends that MED and NZTE agree that given the EDG is to help young,
inexperienced firms, which are therefore a higher risk, a percentage of firms
failing in spite of EDG assistance is likely and acceptable. In turn, NZTE can
encourage its assessment team to be risk‐taking.

33. Time taken to approve grants
191. The EDG Team aims to assess a proposal within 7 days. The team has told the
evaluation that it has been successful in keeping to this timeframe this year.
192. The graph right shows the
distribution of time taken to
Days taken for a grant decision
assess applications. Durations lie
mostly between 7 and 30 days, but 30
can stretch out to 90 days and

longer. Number of applications 25
193. The team has explained to the 20
evaluation that its turnaround of

seven days is once the application 15
is complete. An application may

not be considered complete first 10
time, and so the team must send

them back. So the length of time 5
taken to assess applications is a

0
function of many firms not
101
111
10
13
16
19
1
70
77
31
34
37
52
55
58
62
66
84
90
22
25
28
7
40
43
46
49
4
sending in applications with Days
enough information for a proper

assessment.
194. The added work for the team to reassess applications and the inconvenience to the
firm in delays makes it worth considering whether the application forms are clear
enough.

34. Average time taken to approve grants over time
195. The evaluation looked to see if the assessment time changed over time. A look at
the average time taken for each decision per month shows that the average
number of days have fallen since the large averages of March, April and May 2004
(see graph below). The average is currently a month, indicated by the horizontal
line at 30 days.
196. Either the team is improving in its ability to assess applications, or firms are
submitting more complete assessments. NZTE has decided not to provide data to
the evaluation on time taken from initial lodgment to decision date post March
2005, so the evaluation does not know whether the team has improved further.
36
197. There was a very large spike in time
taken in March, April and May 2004.
It was a 500% shift in the average. Days taken to decide on grant, over time
This was due to the spike in the
numbers of applications lodged 1/12/2003 to 30/3/2005
causing very high workloads for the
180
team. This shows the level of interest
Average number of days

the team at its current size can 150
accommodate‐ around 40 120
applications a month. 90
60
35. Recommendation 30
198. NZTE is adjusting its assessment and 0
DEC 2003
DEC 2004
SEP 2004
FEB 2004
FEB 2005
JUN 2004
OCT 2004
MAR 2004
MAR 2005
JUL 2004
APR 2004
NOV 2004
JAN 2004
JAN 2005
AUG 2004
MAY 2004
approval processes to shorten their
duration; this may be sufficient to
Month Grant Allocated
reduce the time taken to revue and
decide on applications. Reducing the workload by 15% (the proportion being
devolved) may also help. However the evaluation recommends NZTE record
three dates per application‐ submission, start of assessment and decision date so
time taken can be more easily determined and tracked.
37
5. Distribution of the EDG

Note: For the sections 5,6,7,8 the data is for 18 months: October 2003 to April 2005. This reduces
the totals of applications to 723, with 517 accepted and 178 declined.

36. Summary of findings on distribution of the EDG
199. The evaluation has found the distribution is similar across regions and sectors,
and there is no significant difference in acceptance rate for any region or sector.1

37. Regional spread of firms assisted
200. Auckland has had higher numbers of applications for the EDG than other regions
(see graph below left). Wellington and Canterbury have the second largest
numbers of applications.
201. Controlling for numbers of firm per region2 shows that the rate of grant
applications per region is fairly similar, shown on the graph below right by bars of
more even height.

Number of grant applications per region Rate of grant applications per region
1/10/2003 to 7/4/2005 1/10/2003 to 7/4/2005
250 0.50
200 0.40
Application rate %
Applications
150 0.30
100 0.20
50 0.10
0 0.00
anganui/Ruapehu/Rangitikei
Chatham Islands
Nelson/Tasman
Chatham Islands
Nelson/Tasman
Northland
Northland
Rotorua
Wellington
Taranaki
Rotorua
Otago
Waikato
Wellington
Marlborough
Southland
Taupo
Taranaki
Otago
Taupo
Waikato
Marlborough
King Country
Southland
King Country
Thames Valley
Tararua
Tararua
Thames Valley
Canterbury
Wairarapa
West Coast
Canterbury
Wairarapa
Eastern Bay of Plenty

West Coast
Manawatu
Tairawhiti
Manawatu
Tairawhiti
Auckland
Kapiti/Horowhenua
Auckland
Kapiti/Horowhenua
Hawke's Bay
Hawke's Bay
Western Bay of Plenty

Region Region
1
For regional spread a chi-square test of independence gives a p-value of .282 and for sectoral spread a chi-square test
of independence gives a p-value of .229.
2
Using Statistics New Zealand 2003 data on firms per region.
38
202. Controlling further for the numbers of
firms applying per region, it can be
seen that the proportion of applications Rate of acceptances per region
1/10/2003 to 7/4/2005
accepted per region that the acceptance
rate per region is very similar. The bars
on the graph right are all of similar
100
height.
Acceptance rate %
80
203. From the above analysis it can be
concluded that the EDF is assisting 60
firms in all regions evenly. 40
20
0
Auckland
Waikato
Wairarapa
Wanganui/Ruapehu/Ra
Wellington
West Coast
Tairawhiti
Taranaki
Tararua
Taupo
Thames Valley
Southland
Canterbury
Chatham Islands
Otago
Kapiti/Horowhenua
King Country
Manawatu
Marlborough
Nelson/Tasman
Northland
Hawke's Bay
Rotorua

Region

38. Regional spread of allocations
204. Auckland has had the highest total allocations of grants from the EDF to date (see
graph below left).
205. Controlling for the number of firms in the region3 shows the amount of allocations
per region varies, but not in favour of the larger regions (see graph below right).

Allocation per region Allocation per firm in region

1/10/2003 to 7/4/2005 1/10/2003 to 7/4/2005
2500000 60
50
2000000
Allocation per firm $
Amount allocated
40
1500000
30
1000000
20
500000
10
0 0
Nelson/Tasman
Nelson/Tasman
Northland
Northland
Rotorua
Rotorua
Wellington
Wellington
Taranaki
Taranaki
Otago
Waikato
Otago
Waikato
Marlborough
Marlborough
Southland
Southland
King Country
King Country
Thames Valley
Thames Valley
Canterbury
Wairarapa
Canterbury
Wairarapa
West Coast
West Coast
Manawatu
Manawatu
Tairawhiti
Tairawhiti
Auckland
Auckland
Kapiti/Horowhenua
Kapiti/Horowhenua
Hawke's Bay
Hawke's Bay
Region Region
3
Using Statistics New Zealand 2003 data on firms per region.
39
206. Controlling for the number of
Allocation per assisted firm per region
applicants per region shows allocations 1/10/2003 to 7/4/2005
do not vary much across region (see
20,000.00
graph right).
207. The evaluation has found there is no 15,000.00
significant difference in the rate of
Allocation $
acceptance for different regions.4 10,000.00
208. The EDF is assisting firms in all regions
fairly evenly. 5,000.00

0.00
Nelson/Tasman
Northland
Rotorua
Wellington
Taranaki
Otago
Waikato
Marlborough
Southland
King Country
Thames Valley
Canterbury
Wairarapa
West Coast
Manawatu
Tairawhiti
Auckland
Kapiti/Horowhenua
Hawke's Bay

Region

39. Sectoral spread of allocations
209. NZTE divides firms into seven sectors: manufacturing; information,
communication and technology; creative and services; food and beverage, bio‐
technology; wood processing, building and interiors and education.
210. Data received from NZTE without a firm’s sector nominated have been called
‘uncoded’.
211. The manufacturing sector has made the largest number of applications, with 206.
See graph below right.
212. The next two highest are information,
communication and technology and Number of firms and applications per sector
creative and services. Both had 130
applications.
1/10/2003 to 7/4/2005
213. The manufacturing has also had the 250 No.of firms
assisted
highest number of grants allocated, Applications
200
with 160 accepted, or 31% of all grants.
The information sector had 76% of 150
Sum
applications accepted and creative and 100
services had 60% accepted.
50
214. Manufacturing has received the largest
0
total allocations: $2,095,555, but this
munication & Technology
Education
Uncoded
Food & Beverage
Manufacturing
Bio-technology
sing Building & Interiors

Creative & Services
reflects the larger number of grants

applied for (see graph bottom left and
table bottom left). sector
4
A chi-square test of independence gives a p-value of .282
40
215. The evaluation has found there is no significant difference in the rate of acceptance
for different sectors.5 The average amount allocated to a firm is also similar, no
matter the sector (graph below right).
216. From this analysis the evaluation can conclude that there is no bias toward any
sector in the approval or decline of applications.

Allocation per sector Average allocation per sector

1/10/2003 to 7/4/2005 1/10/2003 to 7/4/2005
2000000 15000.00
Amount allocated
Allocatio ns ($)
1500000
10000.00
1000000
5000.00
500000
0 0.00
Information, Communication & Technology
Wood Processing Building & Interiors

Manufact uring
Uncoded
Information, Communication & Technology
Wood Processing Building & Interiors

Manufact uring
Uncoded
Bio-technology
Creative & Services
Education
Food & Beverage

Bio-technology
Creative & Services
Education
Food & Beverage
Sector Sector

Total
Sector Allocation $
1 Manufacturing 2,095,557.51
2 Information, Communication &
Technology
1,247,433.88
3 Creative & Services 816,522.33

4 Food & Beverage 767,636.13
5 Bio-technology 657,563.62

6 Wood Processing Building & Interiors 445,792.3
7 Uncoded 281,061.84
8 Education 46,126.44

5
41
6. Grant allocation and collection by firms

40. Amounts allocated
217. 50% of grants are for $12000 and less.
Distribution of amounts allocated
20% are for $5000 and less.
218. The graph right shows a peak of 48
1/10/2003 to 7/4/2005
grants of $3000. This is caused by the
$3000 cap on two categories of activity.
150
219. The most frequent single amount

allocated is the maximum, $20,000. 120
There have been three grants above
Frequency
90
$20,000, but as of April 2005 these were
not claimed in full. 60
30

0
41. Amount claimed by closed projects $0.00 $5,000.00 $10,000.00 $15,000.00 $20,000.00

Amount Allocated
220. Firms have one year from date of

acceptance of their application to claim the approved costs from the project and
thereby their grant from NZTE. At one year the grant and project is closed. There
have been 193 projects closed to date. They have had allocated to them a total sum
of $2.165M.
221. Of this $1.79M has been claimed and paid. This is a shortfall of $366,462.43
(16.9%). The shortfall is due to not all clients claiming their full amount.
222. 15% of grants have less than 60%
claimed. See graph right.
223. The evaluation spoke with many of Distribution of proportions claimed
these firms and found that for most a
change in plans, either for the project 1/10/2003 to 7/4/2005
or for the firm as a whole, saw their
project alter or not go ahead, and so 100
they did not claim.
80
224. As this sort of thing seems likely to

Frequency
happen to firms in the future, the 60
EDG will probably have some

40
amount left over each year which was
allocated but not claimed. 20
0
0.0000 0.2000 0.4000 0.6000 0.8000 1.0000
Proportions
42
7. Analysis of applicant firms

42. Summary of findings given below.
225. EDG clients are typically small, with less than 10FTEs and less than $200,000
turnover.
226. Over half of all applicants are exporters, and firms who export show significantly
higher rates of acceptance into the fund.
227. It is not clear whether the policy criteria of ‘young start‐up’ refers to age, and not,
for example, to growth. If it applies to age, and if ‘young’ means firms less than
five years old, 40% of EDG firms are not young start‐ups; if ‘young’ means less
than 10 years old, nearly a quarter of firms funded are not young start‐ups.

43. Overall size: turnover and FTE
228. Most EDG clients are small, with less than 10 FTEs and a turnover of less than
$200,000.
229. One goal of the EDG as a foundation service is to move its clients through to more
advanced services, such as the growth services range. Almost none of the EDG
clients to date have accessed the growth services range. This may be because the
gap in capability between EDF clients and GSR clients is broader than can be
improved via the EDF. Certainly the GS Range is targeted at high growth potential
firms and this is likely to be a small percentage of the firms eligible for the EDF.
230. Looking at the graph right, the line on the y axis is at 20 FTEs, which is the
maximum allowed by the fund’s eligibility criteria, and shows that NZTE will
over‐ride that restriction on occasion.
The lack of dots in the upper right
FTE and Turnover of accepted firms
quadrangle shows that it is overridden
only if the firms are also smaller than
$5M turnover. 1/10/2003 to 7/4/2005
231. The line on the x axis is at the $5M 50
maximum, and shows that NZTE will
override that restriction on occasion 40
also, but again, it seems only if the

Full Time Staff
30
firms are smaller than 20FTEs.
20
10
$0 $2,000,000 $4,000,000 $6,000,000 $8,000,000 $10,000,000 $12,000,000

Last full year total turnover
43
232. The graph right shows that declined
firms tend to be smaller than
FTE and Turnover of declined firms
accepted firms, but firms have been
accepted who have very low
1/10/2003 to 7/4/2005
turnover.
60
44. Exporting 50
233. Based on whether they report an
Full Time Staff

40
export turnover, over half of all EDF

30
applicants are exporters, and almost
half of all accepted firms are 20
exporting. 10
234. Only about 4% of New Zealand SME 0
firms export. If firms applying to the $0 $2,000,000 $4,000,000 $6,000,000 $8,000,000 $10,000,000 $12,000,000
Last full year total turnover
EDG were typical of New Zealand
firms, only 4% would be exporting.
235. The table right shows the numbers
All applicants
exporting who apply to the EDG
(40.9%). This indicates that the EDG Frequency Percent
Not Exporting 427 59.1
firms are not a typical population of Exporting 296 40.9
firms, perhaps because it is a self‐ Total 723 100.0
selecting sample of New Zealand firms.
236. Examining the rate of acceptance and declined for exporting and non exporting
firms shows that there is a bias in favour of exporting. The evaluation has found
there is a significant difference between the rates of acceptance for firms who are
exporting and those who are not.6
237. More firms are being declined when they are not exporting than would be
expected if there were no relationship, and fewer are being accepted. Firms who
are exporting are showing somewhat higher rates of being accepted, and firms not
exporting somewhat lower rates.
238. It may be that firms who are already exporting are also firms who tend to pass all
the EDG criteria more easily.
However the EDG has been All applicants
included in NZTE’s overall Status of Project Frequency Percent
focus on exporting, and the Declined Not Exporting 120 67.4
Exporting 58 32.6
criteria ‘potential for growth’
Total 178 100.0
was defined to be national or Accepted Not Exporting 291 56.3
export. If firms failed this Exporting 226 43.7
they would fail the criteria as Total 517 100.0
a whole, and be declined the
6
44
EDG. So firms who are ‘not exporting’ had at least to be those who had the
potential for export.
239. The criteria for the EDG have been revised and the focus on exporting has been
shifted to a focus on improved business performance and net benefit to NZ of
improved business performance. The tests for the latter are improved productivity
and efficiency, limited displacement (displacement would result from funding
firms with significant domestic focus and significant domestic competition) and
presence of spillovers.

45. Age of Firms
240. Data was provided on the year firms accepted into the EDG were established.
241. 21% of firms were established
before 1995.
Year established for accepted firms
242. 79% of firms were established in
1995 and after.
1/10/2003 to 30/6/2005
243. 322 firms (62%) were established
in 2000 and after. 100
244. It is not clear whether the policy

80
criteria of ‘young start‐up’ refers
to age, and not, for example, to
Number
60
growth. If it applies to age, and if
‘young’ means firms less than five 40
years old, 40% of EDG firms are
not young start‐ups; if ‘young’ 20
means less than 10 years old,

nearly a quarter of firms funded 0
2000
2002
2004
1914
1946
1953
1964
1968
1970
1973
1975
1978
1980
1982
1984
1986
1988
1990
1992
1994
1996
1998
are not young start‐ups.
Year firm established

46. Firm Growth
245. The evaluation calculated the growth of firms in the years prior to applying to the
EDG from data included in applications forms.
246. Growth was calculated as the change in total full year turnover from 2 years prior
to applying, to 1 year prior to applying.
247. The graph below right shows the average growth rate of accepted and declined
firms for the calendar years 2003, 2004, and 2005 to April. The average growth for
accepted firms is between 0 and l.5%. If, as discussed above, ‘young’ is in terms of
growth, then most EDG clients are young.
45
248. For 2003 and 2004 accepted firms had
higher average growth than declined
Change in turnover for accepted and declined firms
firms.
249. The 2005 data is a function of low 1/10/2003 to 7/4/2005
application numbers. The data the
Status of Project
evaluation received ended in April Declined
Accepted
2005, so the data for that year is 3.00
Mean change in turnover

incomplete.
2.00
250. The graphs below of applications by
month show that for declined firms a
few had applied with high turnover, 1.00
and only a few had applied who had
low turnover. So a few applications 0.00
with high growth have skewed the

2003 2004 2005
average. With further data on Year grant awarded
turnover for declined firms for the
calendar year 2005, it may be seen whether the 2005 intake is similar to previous
years.
251. The graphs also show that most firms have growth of less than 5%.

Change in turnover of accepted firms Change in turnover of declined firms
1/10/2003 to 7/4/2005 1/10/2003 to 7/4/2005
20.00 20.00
15.00 15.00
Change in turnover
Change in turnover
10.00 10.00
5.00 5.00
0.00 0.00
-5.00 -5.00
-10.00 -10.00
Jul/03 Oct/03 Jan/04 Apr/04 Jul/04 Oct/04 Jan/05 Apr/05 Oct/03 Jan/04 Apr/04 Jul/04 Oct/04 Jan/05 Apr/05
Month grant allocated Month grant allocated
46
8. Types of assistance funded

252. Firms apply to the EDG for assistance for activities they wish to undertake. The
activities must be within specified categories and types of activities. These are
described further below.
253. Firms may undertake within one grant as many categories and types of activities
as they wish, so activity numbers given in this section do not match firm or
application numbers.

47. Advice and expertise categories
254. The types of activities which may be funded with the EDG are divided first into
advice and expertise categories. The categories are strategic business development,
feasibility studies, product development, business and operational excellence, e‐
commerce and e‐business strategies, market strategy development.
255. The largest categories are strategic business development, product development
and market development. 35% of total allocations went to product development,
27% to strategic development, 17% to market development.
256. Within each category are areas of activity. The areas are:
− strategic business development
− business/strategic plan development
− human resource
strategic plan
development Distribution of funding by activity type
− financial viability
planning 1/10/2003 to 7/4/2005
− strategic design Category
e-business
advice category
Feasibility studies
− feasibility studies Market
development
Mentoring
− product development Operational
excellence
− prototype design, Product

development categ
Strategic
development and developme
Training
testing
− intellectual property
protection
− business and operational
excellence

− international quality standards certification
− systems evaluation and development
− environmental management system
− e‐commerce and e‐business strategies
− development of a e‐strategy
− Market strategy development
− Market research or new market investigation
− Marketing plans

257. The largest activity areas are intellectual property protection, prototype
development, strategic business development, and strategic design advice (see
graph below right).
258. The graph below left shows totals allocated to date. The activities within product
development, prototype development and intellectual property protection have
absorbed a large proportion of funding to date, at around $3.5M total allocations.
Roughly one million has gone to each per annum.

Funding totals per activity type Areas of activity undertaken by accepted firms
1/10/2003 to 7/4/2005 1/10/2003 to 7/4/2005
150
3,000,000.00
Total allocated $
120
Number
2,000,000.00
90
60
1,000,000.00
30
0.00
0
Mentoring
Market development
Training
Feasibility studies
prototype devlpmt
Operational excellence
strategic design
intellectual property
strategic planning
e-business category
Tr ring
St
S y a ti
e-
C
M
M t inv
Bu t pla ati
Fi
Fe ic
Pr ility n a
In
En s e
er ual
um s/st
en
ar
ar
na eso ic p elo
te e d
bu
ai
ra l pla str
ot
st
as des g
si
vi
tif
ni
lle
ke
ke est
te
ot Stu
em n
to
ne
nc
ro
si nta
an rate d d rch
ib
ic
ng
yp
ct
ne
n
i
s
m
r
ss ma d d
o
e
va
nn on
pr me
st
ev
ur
nn
lu
in
ra age
ig
o
g
el
l
dy ice
c
a
p e n t,
g
in egi op
g
op
tio
an sea
rty de
dv
n
n
a
,r
an
la
pr sig
t
n
e
ev
ot
Category
de e
ec
c
ev
ve
pl
tio
pm
el
an
l
n
n
48
PART FIVE: Findings and Recommendations on Outcomes

9. The framework for analysis of outcomes.

48. Goals of the EDG
259. The goal of the EDG was to assist innovative firms and entrepreneurs to build
capability to test early stage business concepts and projects by enabling them to
employ expertise and advice.
260. Funding was to enable firms to:
− Engage the services of a business mentor for a period of time
investigation, trade fair participation, trade/business missions, business
exchanges, and visiting buyers.

49. Goals of the evaluation
261. The evaluation sought to learn whether the outcomes policy wished to achieve
were occurring for the firms undertaking the activities funded by the EDF.
262. Achieving this goal enables the evaluation to report on whether the intervention
designed by policy was successful. It would be successful if the issues identified
by policy were overcome by the activities firms undertook, and if the outcomes
identified by policy resulted from these activities.
263. To achieve its goals the evaluation held interviews with a sample of firms. The
population was all firms who had completed a project by April 2005, giving a total
of 191. The sample was a stratified simple random sample with a sample size
chosen to give a margin of error of 5% for estimates of proportions. This gave a
minimum sample of 129. The evaluation surveyed 132 firms.
264. The sample was stratified by the activity types the EDG is broken into by NZTE,
and those to be included were selected by MED policy. Further details are given in
section 10.

50. Specific outcomes sought by the evaluation
265. The evaluation sought to understand the objectives for firms in undertaking their
projects.
266. It wished to know whether the projects undertaken by firms had improved
business or management capability and whether firms had experienced gains in
performance as a result.
267. It wished to know the rate at which projects had gains in activity (such as
launching a product or entering a market) and performance (in terms of sales)
with and without a change in management practice.
268. It wished to determine any differences between the different project activities in
the rate at which they improved business or management capability, and the rate
at which they led to gains in performance.
269. It wished to understand the nature of the projects being undertaken in each
activity type.
270. It wished to know whether firms intended pursuing further assistance, and what
their opinion was on the time it took to fill out the applications.

51. Interpretation of the policy intent: improving management or business
capability?
271. Attempting to determine the improvements to business
and management capability by the projects undertaken Numbers of firms wanting business or management level
assistance
with funding gave rise to a definitional issue: the
difference between ‘business capability’ and
‘management capability’. 80
75
272. The statement of policy intent says the goal of the fund 70
Number
is to build: “basic business capability” but describes this 65
as the capability to handle: “day‐to‐day management of 60
the many different functions of a firm, as well as longer‐ 55
term strategic issues.” 50
Business Management
Business or management level

273. Day‐to‐day business and long‐term strategic issues may
well be separate tasks requiring different types of skills. Improving them may give
different kinds of outcomes.
274. The evaluation categorised projects into management level and business level. If
the project was on an activity of the firm considered to be a function of the firm,
such as building a prototype or the design of marketing material for a product, or
identification of distributors in a market, it was ‘business level.’ If it were strategic,
such as improving production methods or developing a marketing plan or a
branding strategy, or it was taking an overview of all firm activities and strategy,
it was considered to be ‘management level’.
50
275. The evaluation looked to see whether firms wished for help with day‐to‐day
business matters, or with strategic matters, or for help with taking a broad
overview of the fund and its overall strategy, and if there is a difference in the
outcomes for each.

52. Interpretation of the policy intent: improving management ability or
knowledge?
276. Attempting to determine how and whether external expertise gave improvements
to business and management capability gave rise to a second definitional issue: the
difference between projects delivering advice, and projects delivering facts. The
difference is also a difference in the situations and objectives of firms for their
projects.
277. The ability to manage a firm can be improved by learning ways or better ways of
performing the tasks required of managers. It can also be improved by getting the
information managers need to make management decisions and ensure firms
move ahead not with guesswork but with strategy. So via the provision of
information required to achieve a project, and via the provision of the skills
required, management’s ability can be improved.
278. However, there is quite a difference between a firm who needs advice on an issue
or situation that is, advice on how to tackle a situation; and a firm who needs only
facts to make their own decisions on what to do. The latter has abilities the former
does not; the latter knows what information it needs and knows how to act on it.
279. So the distinction between gaining knowledge and gaining ability is really about
types of firms‐ one sort will have higher capability than another because it has the
capability to implement a project just not the information it needs to do so.
280. The distinction can be seen in a situation where, for example, a firm already
knows how to enter a market, but just does not have the facts on a particular
market to know whether they should do it. This is a different type of firm from
one who does not know how to enter a market, nor what information they need to
make a decision on entering.
281. The distinction also shows in situations where a firm needs to purchase work from
externals, such as a report, because the firm needs it to be independent, or where a
firm needs external people with manufacturing machinery in order to produce a
product or prototype. Neither of these are situations where the problem for the
firm is a lack of ability in the sense of skills; rather it is ability in the sense that they
are unable to complete a project until they get the external information or technical
assistance they need.
282. The policy statement is not clear on which sort of firm it was intended that the
EDG assisted (if not both) because it described assisting firms with both
information and skills. The policy states that (italics added):
51
Foundation services aim to build basic business capability…through the
provision of generic business information, advice and training…If
entrepreneurs and managers have difficulties in accessing or developing
the necessary skills, this could lower economic efficiency and national
productivity. Small businesses and entrepreneurs often lack the financial
ability to employ all the expertise they need to get new concepts and
projects up and running…As a result, small businesses often do not
realise their full potential.
283. The issue then is whether the goal of capability building is to include firms who
can themselves implement the project or is to be restricted to those firms who need
to learn the skills to gain capability in order to implement the project. Is the fund a
channel for information to managers or is it only to build their actual capability?
284. With the possibility that the distinction does make a difference in outcomes, and in
order to aid policy resolve the question of whether the fund ought to be a channel
for information, the evaluation has looked to see whether facts and information or
advice was sought by firms, and what the difference is in outcomes.

52
10. Quantitative findings on accepted firms.

53. Guide to the findings
285. This section lays out all the findings from the research on outcomes for those
interested in the evidence itself. The evaluation’s conclusions based on the
evidence are laid out in the next section (13). Those who just wish to read the
summary of findings and conclusions may skip this section.
286. The survey was undertaken by telephone interview. The population was all firms
who had completed a project by April 2005, giving a total of 191. The sample was
a stratified simple random sample with a sample size chosen to give a margin of
error of 5% for estimates of proportions. This gave a minimum sample of 129. The
evaluation surveyed 132 firms.
287. The sample was stratified by the Stratum Population Sample size
activity types the EDG is broken Mentoring/Training 21 21
into by NZTE, and those to be Market research and
investigation 32 21
included in the evaluation were
selected by MED. The Prototype 36 22
stratification was to ensure by Systems evaluation 28 22
chance the sample did not exclude Intellectual property
protection 26 12
firms undertaking activities the Business and strategic
evaluation wished to sample. development 30 19
288. With sampling fractions which Strategic Design 18 15

were similar in each stratum, and Total 191 132
also very high, the data could be analysed as a simple random sample. No
corrections for the finite population size were made.
289. The high sampling fraction and response rate enables the sample to be generalized
to all EDG firms undertaking the same activities as the strata. The results have
only limited generalisability to the population of firms in NZ as a whole due to the
fact that the population of firms applying for grants is self selecting. If the firms
who apply for the fund in the future are different in character or if the numbers
accepted increase to a large degree, the results on outcomes found here may not
continue to apply.
290. The findings are of two sorts: proportions of firms in the EDG who saw various
outcomes subsequent to completing their project, and relationships between
certain activities or project types and outcomes. The graphs are the count of firms
reporting the outcomes.
291. The relationship is found using the statistical test: ‘chi‐square test of
independence’. The Chi‐square test is a statistical procedure to test for the
existence of an association between two categorical variables. An example of such
a comparison would be the variable ‘advice was given’ (possible answers are yes
and no) and ‘changes in management were made (possible answers are also yes or
53
no). The Chi‐square test provides a significance or p‐value for the comparison:
small values of the p‐value mean that a significant association has been found. For
the purposes of this evaluation a significance level of 5% (p=.05) is used (this
corresponds to the standard 95% confidence level), and any test with a p‐value
less than 0.05 as taken as evidence for an association.
292. Where there is evidence for an association caution should be exercised in the
interpretation of the result: the test cannot establish whether the relationship is
causal or not. Lack of evidence for an association may mean there is no
association, or that the association is too weak to detect with the given sample size.

54. Numbers of firms responding.
293. For two firms the appropriate people have not been able to be reached for
interview. One firm refused an interview. The firm is reasonably high‐profile, so
the refusal was not due to any poor performance overall. This gives a response
rate of 98%

55. Numbers of firms completing projects.
294. Of the sample 124 (94%) of firms are still in business. The 7 considered not in
business were unreachable on details provided by NZTE and are unlisted with
Telecom NZ.
295. Not all firms completed their projects. 111 (84%) had completed their projects. 13
firms cancelled their projects part‐way through either because of changes in their
plans or because the project never quite got off the ground.
296. Both numbers completing projects and those deemed exited indicate that
expecting a 100% rate of impact of the fund is unreasonable. There is likely to
always be firms whose plans change and who do not undertake the project.
297. Instead, expecting an 80% completion level seems reasonable; and this entails the
proportion of success in achieving outcomes will be less than 80%.

56. Overall satisfaction
298. 90% of firms were happy with the EDG and NZTE. Many wanted to make clear to
the evaluation how grateful they were for the assistance. These firms also
mentioned the length of the application process but felt it was justifiable given
they were seeking public funds.
299. 10% of firms had some specific issue with NZTE or the EDG process which left
them disgruntled. For the majority it was either the length of the applications or
the inflexibility of the EDG process in its position on alterations to the project mid‐
way. Some had tried to get such changes allowed and found the process
exhausting.
54
300. Being disgruntled with the process often coincided with disappointment in the
project they had undertaken. When the gains were small firms minded the time
they had spent applying.
301. The overall satisfaction level is a good result for the EDG and NZTE.

57. Proportions of EDG clients seeing changes in the way they manage their
business.
302. The evaluation looked at the proportions of firms reporting changes in the way
they manage the business.
303. 56% of firms saw changes in the way they managed the business (see graph right,
note it gives counts of respondents not overall percentage).
304. 30% of firms said projects had no effect on management. These firms typically had
undertaken projects which were not geared at
Numbers of firms with changes in the way they manage
impacting on management.
305. The changes described by those who had seen
80
changes included altering the internal practices of

the firm, learning from the project about ways to 60
Number
tackle such projects, gaining direction for the firm 40
and sometimes confirming that their approach had

been right. Firms also reported an improvement in 20
their overall ability to manage the business. 0
Yes No
306. No firms reported projects having detrimental Change
impacts on the way the firm was managed.
307. The evaluation found significant differences between the rates of effect on
management for different activity types. A chi‐square test of independence gives a
p‐value of p=.004. Mentoring, training, marketing and business planning had high
rates of effect. Prototype had especially low rates. This means that usually firms
undertaking mentoring, training, marketing and business planning projects will
see impacts from these projects on their management.

55

activity * Firm on whether it affected management Crosstabulation

Firm on whether it
affected management
Yes No Total
activity Business/strategic plan Count 12 4 16
Expected Count 10.5 5.5 16.0
% within activity_cded 75.0% 25.0% 100.0%
Intellectual property Count 5 6 11
protection Expected Count 7.2 3.8 11.0
Marketing Count 16 2 18
Mentoring/Training Count 15 2 17
Prototype Count 7 13 20
Strategic Design Count 7 4 11
Systems Evaluation Count 12 8 20
Total Count 74 39 113

58. Proportions of EDG clients undertaking projects aimed at management level
areas or business level areas
308. The evaluation categorised projects by whether they were focused on management
level activities or on business level activities and looked at the proportions
undertaking each.
Numbers of firms wanting business or management level
309. 47% of firms were undertaking projects which were at assistance
management level and 41% were undertaking

business‐level projects. 80
75
310. Of those firms who undertook management level

70
projects, 82% said they had seen changes in

Number
65
management, and 17% said they had not. Of those

60
who were undertaking business level projects, 42%

55
said they had seen changes to management practices,

50
and 51% said they had not. Business Management
Business or management level
311. The evaluation found a significant association between the proportions of firms
undertaking business‐level or management level‐projects, and activity type
undertaken by firms. A chi‐square test of independence gives a p‐value of p<.001.
Mentoring and training, business planning, systems evaluation and strategic
design had high rates of management level projects. Intellectual property and
prototype had low rates.
312. This means that one can be confident that some activities‐ intellectual property
and prototype‐ will usually not be undertaken by firms who want management
level assistance, but instead business level. The other activities will usually be
56
undertaken by firms who want management level assistance. Whether these
activities fit in the EDG depends on whether the goals of the fund are building
ability in the sense of management ability, or in the sense of business ability.
Level of assistance * activity_cded Crosstabulation
activity_cded
Business/ Intellectual
strategic property Mentoring/ Strategic Systems
plan protection Marketing Training Prototype Design Evaluation Total
Level of assistance Business Count 6 11 9 1 15 4 8 54
Expected
7.9 5.1 8.8 7.9 9.3 5.1 9.8 54.0
Count
% within
Level of 11.1% 20.4% 16.7% 1.9% 27.8% 7.4% 14.8% 100%
assistance
Management Count 11 0 10 16 5 7 13 62
Expected
9.1 5.9 10.2 9.1 10.7 5.9 11.2 62.0
Count
% within
Level of 17.7% .0% 16.1% 25.8% 8.1% 11.3% 21.0% 100%
assistance
Total Count 17 11 19 17 20 11 21 116
Expected
17.0 11.0 19.0 17.0 20.0 11.0 21.0 116.0
Count
% within
Level of 14.7% 9.5% 16.4% 14.7% 17.2% 9.5% 18.1% 100%
assistance

59. Proportions of EDG clients seeing gains in firm activity
313. The evaluation looked at the proportions of firms reporting changes or gains in
such activities as getting a product ready to launch or launching it, or
understanding the firm’s market better, or gaining a strategy, or entering a new
market.
314. 78% of firms achieved the results described above, so it is estimated that 78% of
EDG clients have used the grant to make positive effects on their firm. This is a
good result for the EDG and NZTE.
315. Furthermore, firms who report changes in management and firms who do not
both see effects on firm activity.

57

Gains in activity
Frequency Percent
Valid Developing product 7 5.3
Assists with
targeting/gaining 39 29.5
customers/investors
Basic business
2 1.5
requirement now in place
Improved performance 18 13.6
Entering Market 28 21.2
Gave firm clear strategy 9 6.8
Total 103 78.0
Total 132 100.0
60. Proportions of EDG clients seeing changes in firm performance
316. The evaluation looked at the proportions of firms reporting changes in
performance in terms of domestic or export sales.
317. 39% of firms saw changes in their firm’s
performance, with increased domestic or foreign Numbers of firms with changes in firm performance
revenue. 37% said it was too early to see any
results in revenue. 2% said the project had 80
increased productivity.
60
318. As 84% of firms undertook the project, gains in
Number
40
performance for 39% entails nearly half of firms

who make use of the grant have seen gains in 20
performance to date. 0
Increased Expanded Not yet

productivity domestic/export
319. The evaluation found no significant association sales
Change
between the proportions with effects on
performance and different types of activity. A chi‐square test of independence
gives a p‐value of p=.090
320. The table shows intellectual property protection had very high rates of firms
reporting gains were yet to eventuate, and market research and investigation were
high also. Intellectual property protection is a long and expensive process, and
does not directly lead to gains in performance but for many firms is necessary for
them. Market research and investigation also takes time to result in gains, and
many firms received the advice or information they need but were yet to enter the
market or were yet to see sales.
321. Systems evaluation and strategic design had high rates of reporting gains in terms
of sales, with business planning and mentoring being fairly high. Typically these
firms were undertaking projects which they were able to get to market more
quickly or see gains more quickly. Prototype was evenly split between reporting
gains and reporting gains were yet to eventuate, as firms were fairly evenly split
in stages of prototype development.
58

Effect on the firm performance * activity_cded Crosstabulation
activity_cded
Business/ Intellectual
strategic property Mentoring/ Strategic Systems
plan protection Marketing Training Prototype Design Evaluation Total
Effect on the firm Increased Count 0 0 0 0 1 0 2 3
performance productivity/strengthened Expected Count .4 .3 .5 .5 .6 .2 .5 3.0
position % within Effect
on the firm .0% .0% .0% .0% 33.3% .0% 66.7% 100%
performance
Expanded Count 8 2 6 9 9 5 12 51
domestic/export sales Expected Count 7.4 5.4 8.4 7.9 9.4 4.0 8.4 51.0
% within Effect
on the firm 15.7% 3.9% 11.8% 17.6% 17.6% 9.8% 23.5% 100%
performance
Not yet Count 7 9 11 7 9 3 3 49
Expected Count 7.1 5.2 8.1 7.6 9.0 3.8 8.1 49.0
% within Effect
on the firm 14.3% 18.4% 22.4% 14.3% 18.4% 6.1% 6.1% 100%
performance
Total Count 15 11 17 16 19 8 17 103
Expected Count 15.0 11.0 17.0 16.0 19.0 8.0 17.0 103.0
% within Effect
on the firm 14.6% 10.7% 16.5% 15.5% 18.4% 7.8% 16.5% 100%
performance

61. Proportions of EDG clients gaining facts or advice
322. The evaluation looked at the proportions of firms
who gained advice or assistance and at the
proportions gaining information (such as facts on a
market) or the use of specialist facilities. Numbers of firms wanting facts, advice or specialist skills
323. 50% of firms gained advice. 38% gained facts or
specialist facilities. 80
60
324. The evaluation found a significant association

Number
between the proportion of firms gaining facts or 40
advice and the activity type undertaken by the 20
firm. A chi‐square test of independence gives a p‐ 0
Advice Facts Legal Grant used Specialist

value of p<.001. Mentoring, training and strategic for internal technical
costs skills or
facilities
design had high rates for advice; intellectual Type of assistance sought
property and prototype had high rates for facts or

specialist skills such as manufacturing skills or legal skills (patent attorneys).
59
Whether firm wanted advice on what to do, or not. * activity_cded Crosstabulation
activity_cded
Intellectual
Business/str property Mentoring/ Strategic Systems
ategic plan protection Marketing Training Prototype Design Evaluation Total
Whether firm Advice Count 10 0 10 17 4 11 14 66
wanted advice on Expected Count 9.7 6.3 10.8 9.7 11.4 6.3 11.9 66.0
what to do, or not. % within Whether
firm wanted advice 15.2% .0% 15.2% 25.8% 6.1% 16.7% 21.2% 100.0%
on what to do, or not.
No Count 7 11 9 0 16 0 7 50
Expected Count 7.3 4.7 8.2 7.3 8.6 4.7 9.1 50.0
% within Whether
firm wanted advice 14.0% 22.0% 18.0% .0% 32.0% .0% 14.0% 100.0%
Total Count 17 11 19 17 20 11 21 116
Expected Count 17.0 11.0 19.0 17.0 20.0 11.0 21.0 116.0
% within Whether
firm wanted advice 14.7% 9.5% 16.4% 14.7% 17.2% 9.5% 18.1% 100.0%

62. Do facts or advice projects impact on management practice?
325. The evaluation looked at whether there was any association between having facts
or advice and seeing changes in the way firms managed their business, or whether
these two things are independent of each other.
326. The evaluation has found a significant association between the proportion of firms
with an effect on management, and having had advice. A chi‐square test of
independence gives a p‐value of p<.001.
327. An association was found between advice and changes in management practice, in
particular the rate of reporting the project affected management was higher for
groups receiving advice. This means that one can be confident that firms having
advice will usually also see changes in the way they manage their business. Firms
asking for facts will usually not see any changes.

Whether firm wanted advice on what to do, or not. * Firm on whether it affected
management Crosstabulation
Firm on whether it
affected management
Yes No Total
Whether firm Advice Count 52 14 66
wanted advice on Expected Count
what to do, or not.
43.2 22.8 66.0
% within Whether
firm wanted advice 78.8% 21.2% 100.0%
No Count 22 25 47
% within Whether
% within Whether

60

328. One can be confident that firms reporting impacts on management will usually
have also received advice. Firms reporting no impacts will usually have used the
project to purchase information or specialist skills.

63. Are facts or advice projects also management or business level projects?
329. The evaluation looked at whether there was any association between gaining facts
or advice and gaining management or business level assistance. The evaluation
has found a significant association between the proportion of firms having advice
or not and having management‐level or business‐level assistance. A chi‐square
test of independence gives a p‐value of p<.001. In particular firms receiving advice
had high rates of undertaking management level assistance; whereas firms
receiving facts had higher rates of undertaking business‐level projects.
330. This means that one can be confident that firms having advice will usually also
have management level assistance. Firms gaining facts will usually not.

Whether firm wanted advice on what to do, or not. * Level of assistance Crosstabulation

Level of assistance
Function Strategic Total
Whether firm Advice Count 8 58 66
wanted advice on Expected Count 30.7 35.3 66.0
what to do, or not. % within Whether
No Count 46 4 50
% within Whether
% within Whether

64. Do management level or business level projects impact on management practice?
331. The evaluation categorised projects by whether they were focused on management
level activities or on business level activities, and then looked at whether there
was any association between the two types of projects and seeing changes in the
way firms managed their business, or whether they are independent of each other.
332. The evaluation has found a significant association between the proportion of firms
having had management‐level or business‐level assistance and reporting impacts
on management. A chi‐square test of independence gives a p‐value of p<.001. In
particular firms receiving management‐level assistance had high rates of reporting
61
impacts on management; whereas business‐level had high rates of reporting no
impacts on management.
333. This means that one can be confident that firms undertaking management level
projects will usually also see changes in the way they manage their business.
Firms undertaking business level projects usually will not.

Level of assistance * Firm on whether it affected management Crosstabulation
Firm on whether it
affected management
Yes No Total
Level of assistance Business Count 23 28 51
% within Level of
45.1% 54.9% 100.0%
assistance
management Count 51 11 62
% within Level of
82.3% 17.7% 100.0%
assistance
% within Level of
65.5% 34.5% 100.0%
assistance

65. Do changes in management practice impact on firm performance?
334. The evaluation has found no significant association between firms who reported
management changes and those who did not and those reporting effects on
performance (in terms of domestic or export performance). A chi‐square test of
independence gives a p‐value of p=.170. Firms seeing changes in management and
firms seeing no changes had similar rates of gains in firm performance (Note that
this is rate of gain not degree of gain. The evaluation does not have data on
turnover subsequent to the project’s implementation with which to calculate
actual turnover change).
335. This means that gains in firm performance are seen both by firms who changed
management practice and by firms who did not. Projects deliver benefits to firms
in terms of gains in performance independently of whether they deliver gains in
management practice.

62

Effect on the firm performance * Firm on whether it affected management Crosstabulation
Firm on whether it
affected management
Yes No Total
Effect on the firm Increased Count 3 0 3
performance productivity/strengthened Expected Count 2.0 1.0 3.0
position % within Effect on
100.0% .0% 100.0%
the firm performance
Expanded Count 37 14 51
domestic/export sales Expected Count 34.2 16.8 51.0
% within Effect on
72.5% 27.5% 100.0%
Not yet Count 29 20 49

% within Effect on
59.2% 40.8% 100.0%

% within Effect on
67.0% 33.0% 100.0%

66. Do advice or facts projects impact on firm performance?
336. The evaluation has found an association very close to significant between projects
delivering facts or advice and those reporting effects on performance (in terms of
domestic or export performance). A chi‐square test of independence gives a p‐
value of p=.053. This means that the association is tentative, and may differ with
further evidence in time. The table shows that firms undertaking projects
delivering advice have high proportions of firms also seeing either increased
productivity or expanded sales. Projects delivering facts show lower proportions
of firms of increased productivity or expanded sales.

Whether firm wanted advice on what to do, or not. * Effect on the firm's performance Crosstabulation
Effect on the firm's performance

Increased
productivity/ Expanded
strengthene domestic/ex
d position port sales Not yet Total
Whether firm Advice Count 2 34 21 57
wanted advice on Expected Count 1.7 28.2 27.1 57.0
what to do, or not.
% within Whether
firm wanted advice 3.5% 59.6% 36.8% 100.0%
No Count 1 17 28 46
Expected Count 1.3 22.8 21.9 46.0
% within Whether
Total Count 3 51 49 103
Expected Count 3.0 51.0 49.0 103.0
% within Whether

63
67. Do management or business level projects impact on firm performance?
337. The evaluation found a significant association between the proportion of firms
where there was an effect on firm performance and projects assisting with
management or business type projects. A chi‐square test of independence gives a
p‐value of p=.023. In particular, projects assisting with management‐level activities
showed a higher rate of gains in firm performance in terms of export and domestic
revenue and improved productivity.
338. Business‐level projects had higher rates of revenue gains being yet to eventuate,
and consistent with this had a significantly higher rate of firms subsequently
launching products, and enabling firms to target market segments and attract
prospective investors. A chi‐square test of independence gives a p‐value of p<.001.
339. This means that one can be confident that firms undertaking management level
projects will usually also see gains in their performance. Firms undertaking
business level projects will not.
340. Note, however, there is a caveat to this finding. Firms undertaking business level
projects usually said they had not yet seen changes in gains as it was too early for
this given the project they undertook. Time is needed to see if the result above
holds. It may not if many of the business level project firms see gains in
performance in, say, the next six months.

Effect on the firm performance * Level of assistance Crosstabulation
Level of assistance
Business Management Total
Effect on the firm Increased Count 1 2 3
performance productivity/strengthened Expected Count 1.4 1.6 3.0
position % within Effect on
33.3% 66.7% 100.0%
Expanded Count 17 34 51
domestic/export sales Expected Count 23.8 27.2 51.0
% within Effect on
33.3% 66.7% 100.0%
Not yet Count 30 19 49
% within Effect on
61.2% 38.8% 100.0%
% within Effect on
46.6% 53.4% 100.0%
64
11. Quantitative findings: interpretation and conclusion

341. The evaluation was interested in seeing whether the EDG had resulted in changes
and gains in management practice and in activities of the firm; whether these
resulted in gains in performance, and whether there was any difference in
outcomes for activity type, and for firms undertaking projects delivering advice,
facts, management level, or business level assistance.

68. Overall
342. 84% of firms went ahead with the project; the remainder saw changes in plans
which meant they no longer wished to undertake the project.
343. 78% of firms have seen benefits from the grant, in terms of having a product ready
for launch or launching it, understanding which segments of the market to target,
improving the way the firm runs, gaining or clarifying a strategy for the future, or
entering a new market. This means that of the 84% of firms who complete a
project, 92% see positive gains from it. This is a very good result for the EDG.
344. 56% of firms saw changes in the way they managed the business; 30% of firms said
projects had no effect on management‐ but these firms typically had undertaken
projects which were not geared at impacting on management. 47% of firms gained
management level assistance and 41% gained business‐level assistance.
345. 39% of firms have seen gains in firm performance. 37% said it is too early yet to
see gains in performance. This was usually because while the grant was closed the
project was taking longer than one year. The evaluation recommends a second
look at the sample in six to twelve months, for if the 37% also see gains in
performance it will raise the success rate of the fund significantly.
346. The difference between the roughly 50% of firms who gained in terms of
management capability and the nearly 80% who have gained in terms of the
activities they used the fund for is due to ambiguities in the original policy
statement resulting in a broad interpretation by NZTE of building ‘management
capability’.
347. Firms are undertaking projects which in a broad sense improve their management
capability; but not in a narrow sense. Roughly 50% of firms had no intention when
undertaking the projects for it to have any changes to or have any effects on
management. So in terms of the fund’s capability objectives the success rate of the
EDG is 47% for self‐reported changes and 56% for changes which have been
categorised as at a management level.
348. 50% of firms use the fund to gain advice, and 38% use it to gain facts or specialist
facilities.

65
69. Activity types
349. All activity types were equal in their rates of subsequent gains in firm
performance in terms of domestic or export sales.
350. The types of activity differ in the rates at which they result in change in
management practice. Mentoring, training, marketing projects, and business
planning showed significantly higher rates of change in the way firms manage
their business. This means one can be confident that these activities will usually
have firms changing their management practice.
351. Activity type was also related to undertaking management level projects. All but
intellectual property and prototype had high rates of being used for management
level projects, with mentoring and training being especially high. This means one
can be confident that these activities will usually have firms gaining management
level assistance.
352. The activity types of intellectual property protection and prototype development
had very low rates of firms reporting changes in management, undertaking
management level projects or gaining advice. It seems these are not well fitted to
the goals of the EDG.

70. Interpreting the results: improving management capability
353. The goal of the EDG is improve firm performance via an improvement in
management and business capability.
354. The evaluation defined projects as business or management level, and looked to
see the impacts of each project type. The evaluation has found a significant
association between the proportion of firms having had management‐level or
business‐level assistance and reporting impacts on management. This means that
one can be confident that firms undertaking management level projects will
usually also see changes in the way they manage their business. Firms
undertaking business level projects usually will not.
355. The evaluation also found that firms undertaking projects defined as management
level had significantly higher rates of gain in performance. This means that one
can be confident that firms undertaking management level projects will usually
also see gains in their performance. Firms undertaking business level projects will
not.
356. Note the caveat to this finding. Firms undertaking business level projects usually
said they had not yet seen changes in gains as it was too early for this given the
project they undertook. Time is needed to see if the result above holds. It may not
if many of the business level project firms see gains in performance in, say, the
next six months.
357. Currently the proportions of firms using the EDG to undertake management level
or business level projects are fairly even. To target the fund more squarely at
management level projects the assessment of proposals and possibly the
66
description of the funds goals to the market, would need to be revised. This would
require a definition from MED for NZTE of at least the core of what ‘management
level’ is (even if the edges of the distinction with ‘business level’ are left somewhat
blurry).
358. As the fund is undergoing change with its devolution, due for piloting 1
December 2005, there is time to assess the sample again, to coordinate any changes
with final implementation of the devolved model in mid 2006.

71. Resolving the ambiguity between facts and advice.
359. The evaluation found there is an issue in whether the fund is a channel for
information to managers or whether it is to build their capability. At present, two
types of firms are accessing the fund: those who can themselves implement the
project but need information to do so, and those who need to gain the capability to
implement their project. It may be that the fund’s goal of capability building is
achieved best by including firms who can themselves implement the project or by
being restricted to those firms who need to learn the skills to gain capability in
order to implement the project.
360. The evaluation defined projects as delivering advice to build capability, or
delivering information to firms with the requisite capability, and looked to see the
impacts of each project type. The evaluation has found a significant association
between the proportion of firms having advice or not and having management‐
level or business‐level assistance and seeing changes in management practice or
not. In particular firms receiving advice had high rates of receiving management
level assistance and seeing changes in management practice; whereas firms
receiving facts had higher rates of receiving business‐level assistance and not
seeing changes in management practice. This means that one can be confident that
firms having advice will usually also gain management level assistance and
usually see changes in management practice. Firms gaining facts will usually not
see such changes.
361. As management level projects see higher rates of gain in performance, it seems
aiding management via advice is the best way to aid management to achieve gains
in performance.
362. The evaluation also found a tentative association between receiving advice and
seeing gains in performance. Firms undertaking projects delivering advice have
high proportions of firms also seeing either increased productivity or expanded
sales. Projects delivering facts show lower proportions of firms gaining increased
productivity or expanded sales.
363. Thus the fund could be at least weighted toward firms wishing to gain advice.
This can be done by adjusting the assessment to spot these sorts of firms, and
possibly the description of the fund to the market.
364. The evaluation must still make the same caveat as before, that time may eliminate
these differences. Again, with the final implementation of changes to the fund due
67
mid 2006, there is time to test these possibilities by assessing the sample again,
before any further changes are made to the fund.

72. Declined firms
365. The evaluation rang a simple random sample of declined firms. The sample size
was 58. The evaluation contacted 70% of the sample. All who were contacted
spoke to the evaluation, giving a 70% response rate. The remainder of the sample
was not able to be located.
366. 38% of the declined EDG firms went on and did the project. 33% did not.
367. It may be that the firms who were declined but undertook their projects
nonetheless did so because they had the resources to do so and this was the reason
for them being declined. The evaluation has no information on why the firms were
declined with which to confirm this supposition because many were declined
prior to those records being kept.
368. The firms who had been declined and had not undertaken the project viewed that
as a real loss.
369. Many who were declined felt they had been brushed off, could not understand it,
and were left unhappy with NZTE and the grant process. Many said they would
not reapply. This suggests that the goal of the fund to encourage firms to seek
assistance must be considered a goal for those firms who are declined as well, and
the process of declining firms should not discourage firms from seeking further
assistance. More personalized and encouraging explanations of the reasons for
being declined will help prevent the feeling of being ‘brushed off.’

73. Recommendation
370. The evaluation recommends that a process of detailing more fully the reasons for
declining firms is instigated.
371. It is recommended NZTE establishes a system of storing electronically data from
the application forms on all applicant firms’ financial and capability needs, taken
from their description in the application form of the project and their goals for the
external advice or expertise. This should be able to be compared to final outcomes
from the project funded by the grant, and would place both an evaluation in the
future and NZTE in a stronger position to assess changes in the firms post
intervention. MED should be involved in establishing this system.
68
12. Qualitative findings

74. Guide to the findings
372. The findings in this section are qualitative. They are drawn from the sample of 132
firms. They are not statistically representative of outcomes of the EDG, or the
activities occurring, that is, one cannot generalize from these statements to the
EDG as a whole. The findings are to give an understanding of the range of
situations, goals and experiences of the firms in the sample.

75. Overall
373. Firms said that the fund was a significant leg‐up to achieving their goals, and
often gained them the results they were after in a very much shorter time frame
than they felt they could have done on their own. So while it is not certain that
firms would not have done the project and achieved their goals without the grant,
the firms were sure they achieved them faster. Many also said they were able to do
the project to a much higher standard than otherwise.

76. Business/strategic plan development and feasibility studies.
374. The situations of firms undertaking this type of activity were mainly a wish to
expand into new markets, advice on the firm and its performance, or to have help
commercialising a product.
375. The projects were mostly genuine efforts at tackling skill‐gaps, that is, the firms
did not have the skills to the plans for the business themselves. Those not genuine
included a firm using the grant to fund a report it needed written by an external,
independent party, in order to secure investors. Another firm needed help in
gaining FDA approval: they needed agents in the US who were proficient at
organising and managing that process, as well as an independent chemical
analysis of their product.
376. The latter is a grey case, because without agents who know the process a firm
would be lost in the FDA bureaucracy and the firm did not have that knowledge.
They said they now know how to pull together the necessary documentation, and
who the right people in the US, and could help other companies in NZ do it.
377. Seven firms needed facts with which to decide how and whether to proceed with
their plans. Three needed assessments of markets in order to decide whether to
launch in that market, and two successfully launched as a result.
378. Firms who requested and received management capability help included a firm
who wished for a long‐term strategy after a few roller‐coaster years with major
international clients. They felt they were a small company who had struck gold
early on but didn’t know what business planning was or how they should handle
69
down‐times between clients. As a result of their project they have developed a
global plan as well as having analysed their company’s strengths and potential.
379. Other firms wished for help in improving their performance. They gained new
ways of thinking about their firm as well as specific things to work on to improve
performance. For one firm the outcome was a fundamental restructure‐ in
direction, branding, and new relationships with external researchers, as well as by
gaining a board of directors and shareholders. The former manager now operates
the business and leaves its management to his new board.

77. Intellectual property protection.
380. Protecting i.p varied in its importance for firms. I.T and bio‐tech firms view it as
essential. Software is easily copied once it is launched as a product, so software
firms must have i.p protection. Firms in the pharmaceutical industry cannot get
partners or investors without them being assured they have rights over any
product they make, for development costs so much, rights at the end of
development must be assured. In that regard firms say they don’t have a business
without i.p protection. Other firms view i.p as only as useful as they are capable of
affording to defend it. Firms said i.p was also useful for letting them know that
their idea really was original and that they were right to pursue it.
381. Two firms explained that the volumes they need to produce their products at,
either to meet the volume of product demanded by the market, or to meet the low
costs demanded by the market, means they cannot manufacture in New Zealand.
The manufacturing industry is both too small and too expensive. They said
manufacturing offshore entails they must have global patents, else they will loose
their product to rival offshore manufacturing firms.
382. Gaining global patents usually costs firms more than one hundred thousand
dollars, and the EDG does not go far toward that, although firms were grateful for
any and all assistance they had with such large costs.
383. I.p protection is not something which easily fits with the policy goals of improving
management skills or abilities. Firms already knew whether they wanted or
needed I.p For some i.p lay within a larger strategy of selling or manufacturing
offshore, but this strategy was already developed.

78. Strategic Design
384. Projects under this type of activity included projects with a narrow focus, such as
on aspects of product design, and a broader focus, on branding for the company as
a whole.
385. Product design projects included such things as developing owner’s manuals and
instructional DVDs, assistance with design, logos, wording and marketing
material for the product. Projects also included technical advice on technology
70
projects, and in these cases the difference between strategic design and prototype
development is slim.
386. Branding projects include one undertaken by a new firm still getting their product
into production but who believed their product would sell on image, and so
wanted the ‘feel’ of the company to be right. The firm had already spent money on
this but the grant gave work a leg‐up and meant the firm moved ahead more
quickly. It had a product ready for production in 12 months.
387. Another firm had been to a branding seminar and decided to re‐brand their
existing firm, and said they would never start a firm again without going through
a branding exercise first. They felt it was very important for marketing and success
in general.

79. Market research and marketing plans
388. Some of the marketing projects were done by firms who did not know how to
approach entering a new market, and needed someone who knew what
information to gather and what the firm would need to do. Some of these firms
felt they had learnt what to do from these projects and could now enter any
market on their own.
389. Other firms knew the specific information on a market they needed, but many of
these firms said they would not try to do that sort of research on their own.
390. Firms who knew what sort of information they needed said they would not feel
comfortable researching a market themselves. It is a skill that some managers said
they will continue to access externally while they remain too small to have an
employee with such skills. One firm said for a small business it is better to contract
the task out to huge experience in profiling a market, and so not be gobbled up
and spat out by big players in the U.S. To not do this is false economy.
391. Firms who did not know how to develop a plan to enter a market felt they could
learn such a skill. Many of these firms were also undertaking marketing work
under mentoring.

80. Prototype
392. Most of the firms developing prototypes used the grant to purchase facilities for
production they did not have or the technical skills needed to build the prototype
their firm did not have. Two firms used the grant to fund internal costs and did
not use external providers.
393. Only three of the firms used the grant to purchase advice on developing their
prototype.
71

81. Mentoring and training
394. This type of activity is the one most geared to building management ability, and so
has seen the greatest impact on it. Some firms have had a complete turnaround in
performance and described the results as ‘mind‐blowing.’
395. Firms have also decided to keep the mentor on, or have established a board of
directors for ongoing help.
396. There is some overlap with the marketing activities, some firms used the training
or mentoring to develop marketing plans, but the difference is that via training
and mentoring they learn both how to develop a plan and develop one for their
firm.

82. Certification and systems evaluation
397. Certification was of two different types‐ compulsory and optional certification for
a product, which required an assessment of the product, and compulsory and
option certification of the firm, which required an analysis of procedure and
documentation.
398. Some firms knew how to prepare the firm for the certification, and the grant went
to their costs or the costs of the evaluation of the firm. Others did not, and the
grant went to the costs of the consultants used.
399. Systems evaluation ranged from a review of the systems of the firm to
development of products. One firm in fact built a prototype: the firm needed
funding for the project but due to the requirement for external assistance they
hired consultants to build the prototype when they had the skills in house. The
results were not as good as they would have liked and they had to do a fair bit of
re‐development.
400. A firm who had systems reviewed had the ‘seat‐of‐the‐pants’ systems they had
built over the years reviewed and altered. The consultants documented
responsibilities and removed uncertainty on roles and procedures. As a result the
firm improved its performance, especially delivery time and saw revenue growth
as a result.

72

Appendix: Survey Questions

401. These are the questions used for the survey, and all firms were asked these
questions, but all the interviews involved asking further questions to better
understand the situation of the firm, the project and the outcomes. These varied
from firm to firm as required.

1. OK. to begin with, if you could think back to before the _______assistance was
decided upon, when you were originally thinking to seek help, what sort of help were
you after, and how did you arrive at _________?

2. So thinking about the project itself, what were your specific goals for it?

3. So you had received approval for the grant, and called in the external people you
had chosen, what happened next? what did the external people do?

4. Can you describe what sort of effects the project has had on your business, if any?

5. Have you seen any impact of this on your business’ performance

6. Can you think of any ways in which the project has influenced the way you manage
your business?

7. Do you think you will seek any more assistance for your business?

8. On balance, do you think what has resulted from this, then, was worth the time
spent on applying?

73

EDF

Загружено:

Сведения о документе

Авторское право

Доступные форматы

Поделиться этим документом

Поделиться или встроить документ

Параметры публикации

Этот документ был вам полезен?

Это неприемлемый материал?

Авторское право:

Доступные форматы

EDF

Загружено:

Авторское право:

Доступные форматы

Review of

Lack of early stage

management skills). and domestic market Improved collaboration between

Risk aversion of firms

the application forms, it may be

adjusting the project once it had 40

started. NZTE will not reimburse

application. The claims must match Worth of time taken

the tasks laid out in the original

143. Over 20 months from October 2003 to 30 $20,000.00

2004/2005 Amount claimed

146. The graph right shows firms who

so all grant inquiry by telephone is

genuine hits have increased or 500

Applications lodged Applications assessed

1/10/2003 to 7/4/2005 1/10/2003 to 30/5/2005

Declined applications over time Accepted applications over time

1/10/2003 to 30/5/2005 1/10/2003 to 30/5/2005

165. The change in the proportions Declined

166. First, demand for the fund may 0.2

be higher than the EDG can

pass. Second, it might be that

can stretch out to 90 days and

193. The team has explained to the 20

evaluation that its turnaround of

is complete. An application may

time, and so the team must send

taken to assess applications is a

sending in applications with Days

enough information for a proper

Average number of days

accommodate‐ around 40 120

Eastern Bay of Plenty

Western Bay of Plenty

Allocation per region Allocation per firm in region

Eastern Bay of Plenty

Western Bay of Plenty

significant difference in the rate of

Western Bay of Plenty

applications accepted and creative and 100

sing Building & Interiors

reflects the larger number of grants

Allocation per sector Average allocation per sector

Information, Communication & Technology

Wood Processing Building & Interiors

Wood Processing Building & Interiors

Creative & Services

Food & Beverage

Creative & Services

Food & Beverage

3 Creative & Services 816,522.33

219. The most frequent single amount

There have been three grants above

41. Amount claimed by closed projects $0.00 $5,000.00 $10,000.00 $15,000.00 $20,000.00

220. Firms have one year from date of

224. As this sort of thing seems likely to

happen to firms in the future, the 60

EDG will probably have some

also, but again, it seems only if the

$0 $2,000,000 $4,000,000 $6,000,000 $8,000,000 $10,000,000 $12,000,000

233. Based on whether they report an

Full Time Staff

export turnover, over half of all EDF

244. It is not clear whether the policy