KiwiSaver joint evaluation strategy (Research and evaluation reports)

gain approval by Ministers for the high level evaluation design, research programme and budget and a formal mandate for the evaluation for a period of seven years between 2006/07 and 2012/13
provide a framework to guide evaluation activities over the lifetime of the evaluation and provide a common point of reference for the agencies involved
inform stakeholders about the proposed evaluation.

Inland Revenue, MED, Housing New Zealand, MSD, the Retirement Commission, Treasury, and Statistics New Zealand have provided input and/or have been consulted during the development of his strategy.

Focus, purpose and scope

The focus of the evaluation is KiwiSaver not the broader 'savings' policy. The evaluation will seek to establish whether and how KiwiSaver is addressing and meeting its policy objectives.

The evaluation's purpose, which will evolve over time, is twofold:

to assist early on the implementation and delivery of KiwiSaver by providing timely feedback on how delivery is progressing in relation to the design features of KiwiSaver and the initial response to and take-up of the scheme (formative)
to report on the impact of KiwiSaver to inform the assessment of the scheme's achievements, and to provide information and insight for future policy development and longer-term implementation (summative).

The scope is defined by the statement of general policy and purpose in the KiwiSaver Act 2006¹ as well as methodological and practical considerations, such as a realistic timescale. It will be difficult to determine the extent to which KiwiSaver contributes directly to a range of indirect outcomes and to extend evaluation activities beyond the foreseeable future (to look at longerterm economic or financial outcomes). However, the KiwiSaver evaluation will:

assess all process aspects of the implementation of KiwiSaver
seek to measure whether there is a change in savings behaviour as a result of KiwiSaver.

Four potential critical constraints for the KiwiSaver evaluation are:

the multi-dimensional and complex context in which KiwiSaver is being implemented
data availability, timeliness and consistency
the timeliness of evaluation reporting
determining the extent to which impacts can be attributed to KiwiSaver.

Evaluation objectives

The high-level evaluation objectives are:

Objective A: to assess the early implementation and delivery of KiwiSaver as a whole and the various components, to inform the early and on-going development and service delivery of KiwiSaver
Objective B: to assess which of the key features of KiwiSaver are generating the expected outcomes
Objective C: to assess the response to KiwiSaver in order to understand the scale and pattern of take-up
Objective D: to assess the impact KiwiSaver is having on the saving habits and asset accumulation of individuals who are not in a position to enjoy standards of living in retirement similar to those in pre-retirement
Objective E: to assess the impact of KiwiSaver on competitive superannuation markets and the financial sector.

Timeline

This evaluation will start in 2007/08 and continue until 2012/13. The six-year evaluation period takes into account the implementation of KiwiSaver and the rollout of the home ownership assistance and the expectation that longer-term trends and broader impacts may not emerge immediately but towards the end of this evaluation period.

Evaluation approach

The overall evaluation approach is to seek, where possible, to embed evaluation information collection and extraction in the design of KiwiSaver. The rationale for this is to minimise research costs, minimise the imposition of survey work on providers, employers and individuals and to establish data collection for the longer term.

The approach is based on enhancing IRD's administrative data for information on participation, using provider reports to the Government Actuary for information about the schemes, and exploring options to provide longitudinal data that will show change over time. The evaluation is dependent on decisions taken by other agencies regarding the nature and content of administrative data and identifying viable options for supply of longitudinal data.

The research tasks can be divided into six key phases of evaluation:

benchmarking
monitoring
communications evaluation
implementation studies
immediate outcomes studies
mid-term outcomes studies.

The detailed approach and focus of each research task will need to be designed as KiwiSaver unfolds and priorities and areas of interest emerge.

Communications and reporting

The nature of KiwiSaver, the range of organisations involved and the potential range of impact means that there are a large number of people and organisations with an interest in KiwiSaver.

Internal consultation within IRD, MED and Housing New Zealand will ensure that the views of the policy and delivery teams are represented in the research design and the reporting of evaluation findings, and that findings are reported to appropriate managers.

Externally, evaluation findings will be of particular interest to the Retirement Commissioner's three-yearly review of retirement income policy. Regular reporting to other interested government agencies of progress and findings will be primarily via RISC (Retirement Income Steering Committee), managed by MSD and the Treasury.

Reporting to Ministers will include regular joint evaluation reports, responses to ad hoc inquiries and reports generated by specific evaluation projects. Joint six-monthly and 12-monthly evaluation progress reports will be prepared in consultation with respective operational teams from IRD and Housing New Zealand and MED. The reports will be written in a form suitable for an external audience.

Other reports will include summary and full reports of the research activity undertaken for the evaluation, literature reviews, general management reporting (for example progress reports), Ministerial reporting, Official Information Act Requests, Parliamentary Questions, as well as specific data requests from stakeholders.

Finally, there exists considerable international interest from policy executives and research professionals in the evaluation findings of KiwiSaver.

Governance and joint working

The evaluation strategy and research will be coordinated by IRD but implemented by IRD, MED and Housing New Zealand. The Evaluation Steering Group will provide strategic management and guidance to the evaluation and agency agreement on key decisions and documents. Reporting to Ministers and the budgeting process will follow the normal routes of each agency involved in the development and implementation of the evaluation.

Quality assurance and risks

The quality of the evaluation process and output will be reviewed at planning, delivery and reporting stages via the Evaluation Steering Group, external and internal peer review, internal management processes and collaborative working by lead.

The risks associated with the joint evaluation of KiwiSaver relate to context, evaluation, data, governance and project management and communications. The evaluation teams will use the Risk Management frameworks operating with the respective agencies to manage and mitigate any risks with the evaluation programme.

¹ Ref: KiwiSaver Act 2006, Part 1 Preliminary Provisions, Section 3, para 1, p 11. 'The purpose of this Act is to encourage a long-term savings habit and asset accumulation by individuals who are not in a position to enjoy standards of living in retirement similar to those in pre-retirement. The Act aims to increase individuals' well-being and financial independence, particularly in retirement, and to provide retirement benefits.'

About us

Tax statistics

Tax data - audits and legal issues

Media

Careers centre

Newsletters and bulletins

Reports

Who we are

Our commitment to the community

Getting it right

KiwiSaver evaluation reports