• Home
• About MED
• Publications
• Occasional Papers
• Archive
• 2007
• 07/02
• 4. Causal Impact of Practices

• Publications

• Occasional Papers

• Archive

• 2007

4.1 Methodology

Fabling and Grimes (2006) identified three employee practices that were associated with at least one of our binary measures of firm performance, Pⁱ, at the 5% significance level.⁸ The three practices are:

• Firm measures employee satisfaction at least bi-annually (ESAT);
• Firm has performance pay for many or all staff (EPAY); and
• Firm invests in employee training related to the introduction of new or significantly improved products, services or processes (ETRN).⁹

Each of ESAT, EPAY and ETRN is a binary variable. The second and third of these practices correspond closely to practices found significant in studies reviewed in section 2. The first practice is likely to be a characteristic of general "high performance" employee practices. Reflecting this observation, the correlation coefficient between ESAT and SF_EP is 0.49.¹⁰

The choice of certain employee practices may be affected by Pⁱ, and so be endogenous. For this reason, the associative results cannot be taken to imply that adoption of these practices necessarily has a causal impact on firm success. We examine whether a causal link from each of these practices to each Pⁱ measure holds. In addition, we examine whether a causal link holds from the adoption of the suite of employee practices (SF_EP) to each Pⁱ, and we examine whether the individual practices have an impact on each Pⁱ over and above the impact of the suite of employee practices. Our approach uses a two-stage probit regression, controlling for other firm practices and characteristics, and instrumenting potentially endogenous variables. We use a range of instruments, plus a variant of the estimation technique, to test robustness of the results.

We test the impact of each of the employee practice variables after controlling for the impact of general management practices and firm characteristics. We divide the almost 200 questions in the BPS into those that we judge to be exogenous with respect to each Pⁱ, and those that may be endogenous. The practices considered exogenous are listed in the Appendix. These practices reflect underlying firm characteristics (e.g. sector) or what we judge to be properties of the underlying management capability or "idiosyncratic ability of managers" of the firm (Teece et al, 1997; Haltiwanger et al, 1999 and 2000). We consider this idiosyncratic ability is a characteristic that is not influenced by firm performance. This consideration rests on an a priori judgement that is not ultimately testable, so we are careful to test the robustness of our results across a range of instruments and also test over-identifying restrictions.

The form of the single stage equation (before instrumenting) for any employee practice variable, E^k, is shown in (1), for each Pⁱ :

Pⁱ = f_i(GF₁, …, GF_n, E^k, u_i ) (1)

where f_i reflects the probit specification corresponding to Pⁱ; GF₁, …, GF_n are n general factors formed from the variables listed in the Appendix (n is chosen according to the number of general factors estimated to be significant at the 5% level in the respective equation); and u_i is the error term with standard properties. Within this specification, for an employee practice to have a significant effect on firm performance, it must have an effect over and above any generalised impact it already has (or is correlated with) through the general management practices proxied by the GF's. Accordingly, lack of significance of an E^k does not necessarily imply that that practice is irrelevant to firm performance. Conversely, a significant result means that the impact of the practice on firm performance stands out in a test that may be biased against it, especially where the practice is correlated with general "high performance" management practices.

By assumption, each of the GF_j is independent of u_i in each equation; however, for some k, E^k may not be independent of one or more u_i. In this case, the probit estimates from (1) will be biased and inconsistent. Where a particular E^k is not considered exogenous, we instrument that employee practice with one or more variables considered exogenous.¹¹ The prospective instrument(s), Z, must fulfil the standard instrument requirements:

1. Cov (Z, u_i) = 0;
2. Cov (Z, E^k) ≠ 0.

We maintain (i) if Z is listed in the Appendix. We implement (ii) by requiring that Z be significant at the 5% level in an equation in which E^k is regressed on Z and a constant. We also require Z to be significant in the Pⁱ equation when included while E^k is excluded; these latter tests reduce the potential for weak instrument problems. Finally, we limit the number of instruments so as to minimise bias in the estimates (Angrist and Kreuger, 2001).

Four instruments meet the criteria across the three equations. They are:¹²

• firm has formal planning process (0201);
• non-sales staff visit major customers (0302);
• firm systematically measures employee satisfaction (0501);
• books, journals, shows, conferences are used as sources for innovation ideas (0934).

Intuitively, each of 0201 and 0501 (when answered in the affirmative) is a basic characteristic of "good management" practice, and hence fits well with our concept of exogenous management capability. The remaining two instruments are more idiosyncratic. Three of these instruments (0201, 0302, 0501) meet all three tests for P^f, two instruments (0302, 0501) meet all three tests for P^d and two instruments (0201, 0934) meet all three tests for P^m. At least one of 0201 and 0501 appears as an instrument for each of the Pⁱs. The fact that we have more than one instrument for each Pⁱ, means we can test over-identifying restrictions for each equation. We do so using the over-identification test of Stock and Watson (2003). One of the instruments, 0501, is also one of our E^k variables (ESAT). Where ESAT is included as an explanatory variable, we treat it as exogenous.

Our two-stage process entails first regressing the relevant E^k variable on the instrument(s), then including the explained portion of E^k (Ê^k) in (1) in place of E^k . In the cases of the two binary endogenous explanatory variables (EPAY and ETRN), Ê^k takes on the predicted probability rather than the predicted binary outcome, to retain the maximum information from the first stage regression. Angrist and Kreuger (2001) recommend a slightly different approach, estimating the first stage regression using linear regression (even in the presence of binary variables) and using the estimates from this stage as the instrumented variables in the second (probit) stage. As a robustness check, we estimate each of our preferred equations using this approach.

We obtain multiple estimates of Ê^k using alternative instruments in the first stage regression. In each case (other than for ESAT) we estimate the first stage regression using as instruments:

• each of the appropriate instruments individually;
• each of the appropriate instruments individually plus the GF's included in the equation;
• all the appropriate instruments, excluding the GF's;
• all the appropriate instruments, including the GF's.

We present the results using each approach, and check whether the results are sensitive to our instrument choice. We also present the single stage (no instrument) results.

Tables 1-3 present the results for P^f, P^d and P^m respectively. In each case, the explanatory variables are listed horizontally; instruments are listed vertically. Each column refers to tests on a specific employee practice (or suite of practices). Subsequently, we examine interactions between the employee practice variables and SF_EP.

The first line presents the results of the single stage probit regression (no instruments); the figure in each cell is the p-value corresponding to the explanatory variable (against the null hypothesis of zero effect). In subsequent lines, the figure in each cell is the p-value corresponding to the instrumented explanatory variable using the listed instruments. Each equation also includes general factors (GFs) to control for broad management practices and firm characteristics as specified in (1), but their significance is not reported for clarity. A shaded cell corresponds to the specification that has the greatest explanatory power (lowest p-value for the equation F-statistic) for that variable. We choose this equation as our preferred specification for that explanatory variable. In almost all cases the preferred specification uses all eligible instruments excluding the GF's.

We indicate where our instruments are inappropriate in terms of the requirements laid out above (denoted II),¹³ and also note where the use of a set of instruments yields a result with an a priori "wrong sign" (denoted WS). In interpreting our results, we look for consistency in results across different instrument sets and across the three performance metrics.¹⁴ We perform robustness tests on each of our preferred equations. These tests are presented in the lower block in each table. The first robustness test re-estimates the equation using the Angrist and Kreuger specification described earlier. Second, we split the sample first by size, and then by age to examine whether the results are consistent across firms of different types.¹⁵ We also discuss tests of over-identifying restrictions for each equation.

4.2 Results

In the single stage (no instruments) regression, the suite of employee practices, SF_EP, is of marginal significance in the relative profitability equation (p=5.8%) but is significant at the 1% level for each of relative productivity and market share. Once instrumented, it is significant at the 5% level in all cases. This finding is consistent with cited results concerning the importance of generalised HR practices.

Performance pay, when entered without instrumenting, is significant at 5% for each of the performance measures. When instrumented, it retains its significance in each case for the P^f and P^d measures. It loses its significance in the P^m equation when 0201 is not used as an instrument, but otherwise is significant in that equation also. Employee training is not significant at the 5% level in the single stage regression for either P^f or P^d, although it is significant for P^m. Once instrumented, it is consistently significant for P^m at the 5% level and also for P^d and P^f at 10%. Measuring employee satisfaction is significant for P^f and P^d but not for P^m.

The robustness checks indicate that the results are consistent whether we estimate the first stage by probit or by linear regression. All over-identification tests are passed at the 5% level other than one test that is significant at the 4.6% level. Together, the over-identification test results and those using the Angrist and Kreuger methodology indicate that our instruments are acceptable and that the equations are robust to different estimation approaches.

When we split the samples by size and by age, some consistent patterns emerge. Adoption of performance pay is always significant for small firms and is significant in two of the three cases for large firms. However, it is never significant (at even 10%) for medium sized firms (20-50 employees). It is never significant for young firms (less than 2 years old) but is consistently significant (in one case at the 7.9% level) for firms older than two years. Each of employee satisfaction measurement, employee training and the suite of employee practices exhibits very similar patterns.

The finding that adoption of high performance employee practices does not affect the success of younger firms is not surprising. Many of these firms are still at the start-up stage and their success most probably reflects the entrepreneur's own characteristics. The importance of employee practices for firms older than two years appears robust. However a quandary arises with the size-related results, particularly those for medium sized firms (20-50 employees). We have further broken down this category by the three age categories. When we do so, none of the practices (including SF_EP) is significant for any of the performance measures for medium sized firms in any of the age categories.

A key finding in prior papers is that individual employment practices are not as important for firm performance as adoption of a suite of high performance practices. The estimates in Tables 1-3 do not address this issue explicitly since each of the individual practices and the index of practices is entered separately.¹⁶ We examine this issue more closely by re-estimating (1), using an interaction term between the suite of practices and an individual practice, SF_EP*E^k, in place of E^k (or SF_EP). In no case does the interaction term increase the overall explanatory power for the Pⁱ relative to the better of the equations incorporating just E^k or SF_EP.

Another way that we have investigated this issue is to enter either SF_EP or SF_EP*E^k, as separate terms in the equation containing E^k. The specifications are shown as (2) and (3):

Pⁱ = f_i(GF₁, …, GF_n, E^k, SF_EP, u_i ) (2)

Pⁱ = f_i(GF₁, …, GF_n, E^k, SF_EP*E^k, u_i ) (3)

In (2), we test whether the individual practice term has a significant impact on firm performance in addition to the adoption of a suite of employee practices (and vice versa). In (3), we test whether adoption of a suite of practices amplifies the impact on firm performance of the adoption of each individual practice.

In each case, E^k and SF_EP are jointly significant at 5% in specification (2). Similarly, in each case, E^k and SF_EP*E^k are jointly significant at 5% in specification (3). Where E^k corresponds to employee training, neither the individual practice variable nor the suite (or interaction) variable is significantly different from zero at 5% in either specification for any of the performance measures. Similarly, where E^k corresponds to performance pay, neither the individual practice variable nor the suite (or interaction) variable is significantly different from zero at 5% for either profitability or productivity.¹⁷ These results reflect the moderate degree of multicollinearity between SF_EP and each of EPAY and ETRN. This collinearity makes it difficult to pinpoint whether it is the suite of practices, or the individual practice, or the interaction between the two, that is primarily determining the joint significance of the variables. Where E^k corresponds to measuring employee satisfaction, SF_EP is in each case significant in specification (2) and SF_EP*E^k is in each case significant in specification (3). The individual practice is not significantly different from zero for profitability or market share (although it is significant in each specification for productivity).

Together, the results indicate that measurement of employee satisfaction (ESAT) does not outperform the suite of employee practices, and may be best thought of as an integral component of a suite of high performance HR practices. The evidence is less clear-cut in relation to performance pay and employee training. Both appear to have explanatory power over each performance measure, and generally more so than does the suite of practices. However the data cannot distinguish whether either the suite of practices or the two individual practices have explanatory power over and above the influence of the other. At a minimum, the results suggest that each of the practices is an important component of a suite of high performance HR practices. In turn, these practices have significant ability to distinguish high performing firms from low performing firms across three different performance measures.