Key Findings
This evaluation involved:
- in-depth assessments of 8 regulatory impact analyses (RIA), considering the extent of compliance with the government's RIA requirements, the standard of the analyses, and the process followed
- an assessment of 12 regulatory impact statements (RIS), with a focus on the effectiveness of their communication function.
We concluded from the in-depth assessment that 3 out of the 8 RIAs were adequate, and that the remaining 5 were not adequate. The main areas of weakness are problem definition, showing the scale of the issue, and the analysis of cost and benefits. The risk assessment tends to be weak also.
Of the 12 regulatory impact statements (RIS) we found that 7 did not satisfactorily perform their communication function. The remaining 5 were satisfactorily, but none were particularly good. The areas of weaknesses found were similar to those described above for the RIA assessment.
Consultation is usually done well. Close attention is paid to submissions and proposals are amended accordingly. Other pluses are that the statements were all short and in relatively plain language.
The more in-depth investigation of the regulatory impact analysis (RIA) process showed that sometimes the regulatory impact statement did not do justice to the analytical substance behind the proposals.
The in-depth investigation also revealed that analysts and managers in departments are strongly in favour of improving the quality of advice and policy production process. But there were two different types of attitudes toward the regulatory impact analysis. About half of the agencies we interviewed have explicitly built in RIA responsibilities into their processes. The strong commitment to the RIA process stemmed from a belief that a good RIA would give a strong foundation to the policy advice (and thus the Cabinet paper and RIS).
The other half of departments had no special process. RIA was regarded as "reflecting good practice" and "helpful" but one that "good policy analysts" would follow "intuitively".
All RIAs we rated as adequate originated from departments with explicit responsibilities or structures for RIA processes. This raises an interesting hypothesis of a causal relationship between those structures and outcomes. It is difficult to draw any strong conclusions from this small sample. But we believe, from experience, that the co-existence of strong processes and good outcomes is likely to be linked to a third variable: a focus on the quality of processes and quality of the work.
Areas we think are worthy of attention by the Regulatory Impact Analysis Unit and departments are:
- clarifying the purpose of the RIS and how it is different from a Cabinet paper
- rationalising the guidance material
- amending the approach to adequacy certification
- raising expectations of the RIA/RIS quality
- providing support for those with no or little experience.
Back to Top