Monitoring and Evaluation (M&E) Framework – Vision 2030 Jamaica–National Development Plan (NDP)

The relationship between national development results and strategy and programming is one of attribution and not direct cause and effect. This is owing to the myriad of factors, beyond the control of strategy and programme planners and implementers, that lead to the realization of national development goals and outcomes.  IERBM allows for an approach that demonstrates recognition of this reality and examines the breadth of the development space which extends beyond managing the performance of specified development programmes/actions.  Accordingly, this approach allows for systematic processes to be developed and engaged in environmental, contextual, and capacity analysis, and the utilization of the findings from this analysis to inform the framing and interpretation of lessons learned from M&E and other evidence-based processes.  Lessons learned from the experience of COVID-19 have informed a strengthening of environmental, contextual and capacity analysis to facilitate timely identification and framing of new and emerging threats and opportunities to inform strategic planning and response. Vision 2030 Jamaica was designed for the utilization of integrated evidence- and results- based management to inform continuous improvement. 

Vision 2030 Jamaica is results-based as the NDP and its implementation is geared towards achieving pre-determined long-term national development goals and outcomes. The national and sector strategies have been designed to advance the achievement of the goals and outcomes. The focus of Vision 2030 Jamaica, therefore, extends beyond the implementation of activities and their outputs to prioritizing the achievement of national outcomes and goals, which represent changes in the lives of Jamaicans, and the social, economic, environmental and governance structures and institutions that impact quality of life.  

 Vision 2030 Jamaica is evidence-based as the formulation of the national development results (national goals and outcomes) and the strategies and programmes/actions being implemented to achieve those results, have been informed by various sources of evidence. Evidence was utilized to inform the development of the NDP and supporting documents and is being utilized to inform the implementation of Vision 2030 Jamaica. The sources of evidence utilized include empirical evidence from data, statistics, and reports on the findings of scientific and systematic research covering the three dimensions of sustainability—social, economic, and environmental. The evidence utilized in Vision 2030 Jamaica has also been derived from the views and perspectives of stakeholders across various sectors and groups. Stakeholder views support policy-making, planning and implementation that are inclusive and geared towards meeting the needs of all Jamaicans. The views and perspectives of experts are also incorporated to guide evidence-based processes, including the interpretation of data.  Data, statistics, information, and lessons learned from M&E also comprise an important source of evidence. This evidence informs reporting on development performance and progress towards the national goals and strategic planning in the medium-term to advance the implementation of the long-term NDP.  It also forms a core component of accountability mechanisms.   

Accordingly, results-based M&E is a critical tenet of the integrated evidence- and results- based management framework of Vision 2030 Jamaica. It is through the use of M&E that the relevance, efficiency and effectiveness of the strategies and programmes/actions employed in the implementation of Vision 2030 Jamaica are determined. Lessons learned through M&E are utilized to inform continuous improvement in Plan implementation. The Learning Framework facilitates the translation of lessons learned into strategic action to capitalize on development gains and opportunities, and addresses gaps and threats. This also includes the formulation of relevant strategies and programmes/actions to address existing and emerging realities, including shocks such as the COVID-19 pandemic. 

The “successive 3-year Medium Term Socio-Economic Policy Framework (MTF)” serves as the medium-term strategic plan framework and forms the basis for monitoring and evaluation in the implementation of Vision 2030 Jamaica. The MTF is policy-driven and reflects policy imperatives that are aligned with the long-term National Strategy Framework. To date, the MTF has presented the national and sector strategies and priority actions that will be implemented in the medium-term to advance the achievement of the long-term goals and outcomes. The MTF presents the National Outcome Indicators being monitored and the 3-year targets for the period covered by the MTF. Accordingly, MTF 2018–2021 presents the 2021 targets for the indicators. It also includes the 3-year targets for the previous MTFs as well as the 2030 targets.  

Vision 2030 Jamaica is implemented through the alignment of the strategic/corporate and operational planning of partners in development to the MTF. These partners include the public sector, private sector, municipal corporations (MCs), academia, civil society and other non-governmental actors. International Development Partners (IDPs) align their country programmes to the MTF to support the implementation of Vision 2030 Jamaica and contribute to the achievement of the national goals. The alignment of the Results Frameworks, Strategy Frameworks, and M&E Systems of development partners with that of Vision 2030 Jamaica is important.  This ensures that among other areas, development results are similarly defined at all levels and across sectors; development actors are implementing programmes/actions geared towards advancing common strategies; and performance measurement is harmonized, including development indicators and targets.    

Since 2015, the MTF has also served as the policy framework for the implementation of the 2030 Agenda for Sustainable Development – Sustainable Development Goals (SDGs). This was made official in 2017 through Cabinet Approval of “A Road Map for SDG Implementation in Jamaica”, which identified Vision 2030 Jamaica and the MTF as the Plan framework for the implementation of the SDGs. MTF 2015–2018 showed the alignment of the SDGs with the national goals and outcomes. MTF 2018–2021 expanded on this alignment and included the alignment of SDG targets with the national and sector strategies and associated actions. 

The Vision 2030 Jamaica Secretariat, Planning Institute of Jamaica (PIOJ) is responsible for the coordination of the implementation of Vision 2030 Jamaica, and the integration of the SDGs in Vision 2030 Jamaica Plan Implementation. This includes monitoring and evaluation (M&E) and reporting on Vision 2030 Jamaica.

The objective of monitoring and evaluation (M&E) under Vision 2030 Jamaica is to advance the achievement of the national development goals and outcomes by 2030 through the utilization of evidence to promote relevance, efficiency, and effectiveness in the implementation of Vision 2030 Jamaica. This objective encompasses the monitoring of development performance and evaluation of strategy, policy, and programme effectiveness in advancing the achievement of the long-term national goals and outcomes and the global SDGs.  

Monitoring and Evaluation (M&E) under Vision 2030 Jamaica is based on a Results-Based M&E System. The M&E Framework describes and explains how the M&E System works and how the various elements interact and complement each other to meet the objectives of monitoring and evaluation.  The M&E Framework articulates the relationships between the M&E Systems and its tenets.  It also articulates the cyclical relationship consisting of strategic programming and medium term performance measures/indicator framework, implementation of the M&E Framework and Plan, Reporting, Learning towards Action that informs revisions/updates to strategic programming; and the cycle starts again.  This involves presenting a logic model of the steps to be taken and who and what will be engaged and/or undertaken to meet the objective of M&E. 

Diagram.  Key Relationships within the Vision 2030 Jamaica Results-Based M&E System 

The M&E Framework defines and explains the systemic relationships and processes for: 

• the monitoring of performance in the implementation of the NDP and progress towards achieving the national goals and outcomes
• the evaluation of the effectiveness of the strategies, policies and programmes/actions utilized in the implementation of the NDP in advancing the achievement of planned development results (outputs, outcomes, and impact)
• the utilization of lessons learned to inform strategic planning and action.

It also defines the relationships between M&E and other sources of evidence in the Vision 2030 Jamaica IERBM Framework as well as the utilization of the complementary sources of evidence to strengthen the formulation of lessons learned to inform action.  

The M&E System for Vision 2030 Jamaica comprises the frameworks, plans, tools, and institutional arrangements that collectively create the environment and approaches to monitoring and evaluation.  This is informed by and includes a National Results Framework, National Strategy Framework, Theory of Change (ToC) and an M&E Framework and Plan. It also includes a Performance Measurement Framework, Institutional Framework, Reporting Framework and Learning Framework that are defined and operationalized as part of the M&E Framework and Plan. The tenets of the M&E System are interdependent and complementary.  

The National Results Framework, National Strategy Framework and Theory of Change comprise the base of the strategic planning framework that informs monitoring and evaluation.  These determine what is to be monitored and evaluated and why.  The National Results Framework constitutes at its highest level, the national goals and outcomes. It states what is to be ultimately achieved under Vision 2030 Jamaica. The National Strategy Framework comprises the national strategies and sector strategies that form the strategic framework for programming.  It tells the nature of the changes that are required to achieve the goals and outcomes as well as what we need to do strategically at the macro-level to effect the required changes. The Theory of Change (ToC) describes and explains a theory-based causal relationship between the planned development results and the strategies and other factors that should lead to the achievement of the results.  It articulates who needs to implement the national and sector strategies and in what context. 

This involves the measurement of efficiencies in the implementation of programmes/actions, including sub-programmes and projects. It also includes the monitoring of the results of implementation demonstrated in outputs and outcomes, particularly immediate outcomes. The programmes/actions monitored are those that operationalize the national and sector strategies presented in each Medium Term Socio-Economic Policy Framework (MTF). The monitoring of programmes/actions is the remit of implementers. This information is then shared with the Vision 2030 Jamaica Secretariat, Planning Institute of Jamaica (PIOJ), for the monitoring of performance in implementing the national and sector strategies towards advancing national development results.  

The monitoring of progress towards the national outcomes/outcome monitoring involves the measurement of changes in the performance of national outcome indicators over the 21-year implementation of Vision 2030 Jamaica (2009-2030) based on a baseline year of 2007, intermediate targets set for every three years from 2009, and 2030 targets. The monitoring of progress is focused on intermediate development outcomes and impact at the national level. These intermediate outcomes and impact are the medium- to long- term development results from programmes/actions and represent the aggregate effects of the outcomes of various programmes/actions, over time. Any one programme/action cannot be causally linked to a national development outcome. However, performance related to the national outcome can be attributed to the actions that were designed to advance the achievement of the national outcome.  

The monitoring of development outcomes may show development gains or progress towards targets or the achievement/exceeding of targets. It may show no development-related change or improvement, that is, there has been no change from the baseline.  Monitoring may also show losses owing to slippages/ movement away from targets, which include slippages below the baseline.

The evaluation of programmes/actions is intended to assess the effectiveness of programme design, strategies, activities, and implementation in advancing the achievement of planned outputs and outcomes, and the extent to which actual outputs and outcomes represent what was planned. It involves the evaluation of programme processes or process evaluation and also the evaluation of programme outcomes and impact.  This informs determinations of which programme models (design, strategies, and implementation) best operationalize programme objectives towards achieving planned outputs and outcomes, and within which contexts. The scaling up of programmes/actions (sub-programmes and projects) and/or their replication should be informed by evidence from evaluation. The evaluation of programmes/actions are usually the remit of implementers.  

The evaluation of strategies is intended to assess the relevance and effectiveness of the national and sector strategies in advancing the achievement of the 15 National Outcomes of Vision 2030 Jamaica towards the achievement of the 4 National Goals. The evaluation of national and sector strategy effectiveness is primarily executed through the evaluation of the policies and sector strategy frameworks/documents that are aligned with the national and sector strategies. It is the policies and sector strategy frameworks/documents that provide the frameworks for the actions that are implemented as operationalizations of the national and sector strategies.  Policy/Sector Strategy evaluations allow for an overarching assessment of the effectiveness of programmes as operationalizations of the national and sector strategies under Vision 2030 Jamaica.  They facilitate the assessment and validation of the causal linkages between the national strategy framework (national and sector strategies) and the national outcomes.  The evaluation of strategies and policies are usually the remit of the strategy/policy owners, consisting of the strategy/policy planners and/or those responsible for coordinating the implementation of strategies/policies.

The approach to reporting on development performance under Vision 2030 Jamaica includes reporting on both development gains and gaps in the achievement of planned development outcomes and explanations for the performance. Gaps in development are usually demonstrated in performance that shows: no improvement, improvement that falls short of targets, and development losses. Performance measurement and reporting include focus on indicators of inclusion or “leaving no one behind” such as poverty prevalence and disaggregation to determine group-specific experiences of development and vulnerability, such as disaggregation by age, sex, and locale. 

Reports on National Development Results, based on monitoring of the National Outcome Indicators, are due quarterly. These comprise updates to an existing report template based on the availability of data. The majority of the indicators utilized to measure performance under Vision 2030 Jamaica are informed by annual data (calendar year). However, there is a lag in the availability of data for several indicators with some having more than a one-year lag. The data are also produced, released, and revised (for preliminary data) on different schedules and so are available at different times. Accordingly, the data updates for each indicator are based on when data become available. An Annual Progress Report is completed at the end of each Fiscal Year based on the data available up to March of that year. A 3-year Performance Report on the MTF is also completed which includes quasi-evaluation data and information and provides more in-depth details on implementation, programme performance and the factors that have contributed to performance regarding the national outcomes and goals.  

Reports on Development Results and Performance under the MTF are shared on the Vision 2030 Jamaica website and social media platforms.  Updates on the Development Results are shared at the PIOJ Quarterly Press Briefings.  

The integrated evidence- and results- based management (IERBM) framework under Vision 2030 Jamaica is geared towards continuous improvement in the strategies and systems applied in the coordination of Plan implementation and the strategic programming employed to advance the achievement of the national goals and outcomes.  The “Plan Do Check Act” is a cyclical model that informs the application of IERBM in Vision 2030 Jamaica Plan Implementation. This model is aligned with the Learning Framework, which is a key component of the results-based M&E System, as well as the overarching IERBM Framework.  The M&E Framework therefore includes focus on learning towards action. The framework for defining and situating lessons learned includes: aligning lessons learned with the strategic objectives for monitoring and evaluation; the national results and strategy frameworks; Theory of Change (ToC) and assumptions; and the framework for medium-term development results, performance measurement and strategic policy-driven programming presented in the MTF.  

The IERBM Learning Framework promotes an approach to the use of evidence that extends beyond reporting on performance to applying lessons learned to inform improvements in Plan Implementation. It includes systems and processes for transporting lessons learned into the evidence pool for strategic planning and action, including supporting the absorptive capacity of strategic planning and action frameworks to integrate such evidence. This involves the utilization of lessons learned to inform the planned intermediate national development outcomes and performance targets and strategic programming (national and sector strategies and associated actions) geared towards advancing the achievement of the development results, which are presented in each MTF.  In this scenario, lessons learned serve to inform strategic planning that builds on development gains and address gaps. It also involves the application of lessons learned in strengthening the Plan Implementation Framework, including systems for coordination, partnerships, strategic planning and integrated evidence- and results- based management (IERBM).  

The Learning Framework prioritizes the identification and documentation of lessons learned from monitoring and evaluation of development performance and its implications. Consideration is given to the implications of development performance across sectors, at different levels of development (national, parish, communities, and households) and for different groups, including the marginalized and vulnerable. 

The Learning Framework also focuses on the reasons for development performance, including examinations of the national and global context and capacities for development. This constitutes examination of context and capacity in explaining the relationship between development performance and its implications and the structure and implementation of Vision 2030 Jamaica. Hence, there is prioritization of analysis and interpretation of lessons learned from M&E based on development context and capacity. This often requires the incorporation of complementary tools such as environmental scans, contextual and capacity analyses, desk reviews and meta-analysis. The Learning Framework therefore includes methodologies and processes for contextual and capacity analyses, which are informed by the range of sources of evidence that complement M&E. Contextual analysis constitutes the assessment of the wider development context and ecosystem in which strategic planning and programming are located and development outcomes are realized. This includes internal and external factors to the sectors under consideration as well as considerations for the local, national, and global context. Opportunities, risks, and limitations are among the tenets of development context. Capacity analysis involves the assessment of the availability of required resources; and ability to define resource requirements, and engage and apply these resources in evidence- and results- based strategic planning and implementation to produce planned development results. Contextual and capacity analyses also incorporate assessments of emerging trends, forecasting and projections to inform the applications of lessons learned in policy and planning. The utilization of contextual and capacity analyses allows for the incorporation of evidence that is not limited to the processes and outcomes directly associated with strategic policy, planning, programming and facilitates comprehensive assessment of the development space that determines outcomes and impact. 

The Vision 2030 Jamaica Secretariat is currently leading a process to strengthen/develop the Results-Based M&E System for Vision 2030 Jamaica.  This includes the strengthening of the Framework and Plan and integration of the SDGs as well as the development of an Online Monitoring Platform that will be launched in 2021. Stay tuned for more…