2. Approach to the Evaluation
In accordance with the Terms of Reference, this evaluation focuses on "the process followed by [agencies] to adhere to the RIA guidelines, and the extent to which that process resulted in the final RIA (including RIS) complying with the RIA requirements (as set out in Cabinet Office Circular CO(07) 3 and the Guidelines on the Regulatory Impact Analysis Requirements)."
The evaluation methodology was provided by the RIAU. It involved:
- in-depth assessments of 8 RIAs (four selected from the agencies with the greatest number of regulatory proposals over the review period, and four from agencies selected by the Regulatory Impact Analysis Unit (RIAU)). In brief, we were asked to look at:
- the extent of compliance with RIA requirements
- the overall standard of each RIA and its strengths and weaknesses
- the process followed and any constraints/facilitators
- how the process and constraints may have contributed to strengths and weaknesses identified
- a summary of our impressions from assessments of 12 RISs selected by the RIAU, with a focus on how effective each was in communicating necessary information to decision-makers and other interested parties – the analysis and communication tasks.
For the 8 in-depth assessments we assessed the RIA statements in the Cabinet papers and the RISs (a total of 7; one agency deemed that an exemption applied). We interviewed identified contacts at the authoring agencies. We also looked at discussion papers and other related documentation made available by agencies. A short 2-3 page assessment was prepared for each of the 8 RIAs. Drafts were provided to the authoring agency for comment. Section 3 summarises the main findings.
A one page assessment was prepared for each of the 12 RISs against a common framework (Table 1). This did not involve an assessment against the RIA adequacy criteria, but focused on the effectiveness of the RIS as a communication device to decision-makers and other stakeholders. Section 4 summarises the key findings.
Table 1: Framework for one page assessments
Complete – is all the required information included in the RIS?
Correct – is the information free of errors?
Convincing – are the analysis and conclusions supported by a clear logic, an appropriate assessment of costs and benefits, and supporting evidence?
Contingencies – does the statement show consideration of key risks and how they are mitigated?
Consultation – does the statement show evidence of stakeholder consultation and how any issues raised have been dealt with?
Clear and concise – is the material communicated in a plain language, and is the statement of an appropriate length
Overall – is the statement overall effective at communicating the required information to the decision-makers and stakeholders.
Back to Top