Award of RESAS/001/21 – Evaluability Assessment for Scotland’s Deposit Return Scheme

Introduction1. Scotland will launch a deposit return scheme (DRS) on July 1st 2022. The design of the scheme has been led by Zero Waste Scotland on behalf of Scottish Government, and the scheme will be managed by a dedicated administrator, regulated by SEPA. To optimise both the operation of the DRS and maximise its wider benefits, as well as to inform future policy, Scottish Government are keen to understand the policy evaluation options available to them, and the likely value and costs of these options. The evaluation options should cover both an evaluation of the process of implementing and operating the new scheme, and the outcomes of its operation.Background & Context2. The DRS is being introduced in Scotland in order to increase recycling rates of drinks containers, reducing littering of these items, and increase the market for recyclate in Scotland. The DRS business case sets out the high level benefits/ outcomes expected from the scheme. The range of expected benefits include increases in the quantity and quality of recyclate collected, changes in the way recyclable material is used in Scotland, and wider economic and social benefits. In addition to the high-level benefits identified in the business case for this policy (see annex), a range of other co-benefits and indicators have been identified during design and development. An outcome evaluation of the DRS should consider whether the objectives identified in the business case are being achieved in the long-term.3. Additionally, a process evaluation is needed to consider how the rollout of the DRS is progressing in real-time and whether any lessons can be learned through this process that can be quickly implemented. In particular, policy-makers need to know whether mitigation measures proposed during the scheme design to reduce downside risks are working as expected. This process evaluation will need to be integrated into the programme and system management and will therefore be conducted separately to the long-term outcome evaluation. It is hoped that this Evaluability Assessment can scope options for both of these evaluation requirements.4. This tender is to scope the evaluation options for the DRS. The work will involve looking at the existing benefits realisation plan and talking to key policy stakeholders to understand evaluation priorities/ questions, and then identifying how required monitoring and evaluation data could be obtained to address each information requirement. The output will be an options paper for consideration by the DRS programme board. As we anticipate some baseline data may be desirable to collect in 2021, the timeline for scoping is relatively short, though we note too that 2021 may not be a “typical” baseline year, due to the Coronavirus pandemic, and would welcome suggestions on how this should be handled in any future policy evaluation.5. Evaluability Assessment (EA) is a systematic approach to planning evaluation projects. It involves structured engagement with stakeholders to consider evaluation aims and approaches. An EA seeks to determine what type of evaluation should be undertaken, what questions it should seek to answer, and how the evaluation study should be designed.6. It is important to distinguish EA from evaluation itself. The goal of an EA is to inform evaluation decisions, not to determine whether a programme or intervention is effective or cost-effective.7. The methods used in EAs vary, but there are a number of common core elements. They include:-systematic engagement with stakeholders from the outset and throughout;-elaboration, testing and refinement of an agreed theory of change .;-identification and review of existing data sources and data gaps; and-Developing conclusions and making recommendations for or against evaluation options.