Published on 23 July 2015
Agriculture and Development Notes on Climate Change Adaptation1
Vol. 5 No. 6 | Roberto Pedro C. Sandoval, Jr.2, Hideki Kanamaru3 and Jim Hancock4
Assessment, monitoring, and evaluation are integral parts of climate change adaptation (CCA) planning and implementation. Assessment activities contribute to defining indicators, baselines (situations at the beginning of the intervention), and baseline projections (without intervention). On the other hand, M&E activities set baselines, define indicators, measure progress, and evaluate successes and setbacks in adaptation interventions. Designing a Monitoring and Evaluation (M&E) System is therefore challenging, particularly when deciding which indicator will match with a particular goal to measure project success.
Steps in the Preparation and Appraisal of the Planning Process
- Identifying contribution to adaptation. This step considers how a certain intervention would contribute to increasing the adaptive capacity of a community. Do you wish to increase adaptive capacity through capacity building activities and field testing of good practice climate change adaptation options?
- Creating adaptation hypothesis and theory of change. In a participatory and gender-sensitive way, it outlines the possible options and their expected changes and results across activities, expected behavior changes, outcomes, and impacts. This is done at different levels—from community to national. It takes into account the various stakeholders and project beneficiaries to clearly define the adaptation needs in a particular area and link these needs with the goals of the project to come up with the adaptation hypothesis and the change expected from the program or project implemented. These adaptation hypothesis and theory of change can then be formalized into logical frameworks and indicators that would help define inputs and activities, outputs, outcomes, and impacts.
- Developing adaptation-associated indicators. Indicators are developed based on the above hypotheses and changed expectations. Developing indicators would be easy if the adaptation hypothesis and theory of change are clearly set.
- Developing results-based management. This provides a framework wherein M&E is used to drive stakeholders to focus more on results than on inputs and activities. This can be done by using a participatory M&E approach and applying “learning by doing”. For example, engaging farmer cooperators and local agricultural technicians and making them part of the M&E work allow them to focus on the results and the benefits that can be gained. Focusing on the results and benefits consequently enhances the stakeholders’ commitment to be more involved in the program.
- Carrying out of appraisal. This involves reviewing the assessment, monitoring, and evaluation (AME) design in terms of risks, technical and social feasibility, robustness, efficiency, and safeguards.
Learning as an Important and Continuous Process in M&E
Learning is enhanced at the evaluation stage when important issues are identified and lessons are drawn to improve the implementation of interventions. Learning helps build local capacities and instill a “sense of ownership” among stakeholders. This sense of ownership is important as M&E is not to be carried out only by experts. M&E should involve the participation of local stakeholders. Most importantly, as there are uncertainties in CCA interventions, the learning process significantly helps in adaptive management during implementation.
Developing Indicators: The SMART Criteria
One of the activities under assessment in a results-based framework is defining a set of indicators. The development of indicators should be guided by the SMART criteria: Simple, Measurable, Attributable, Reliable, and Time-bound. The SMART criteria was expanded to include the following parameters:
- Validity. Does the indicator measure a change in climate risk or vulnerability? To what extent?
- Precise and specific meaning. Do stakeholders agree on exactly what the indicator measures in this context? It should be remembered that definitions vary from one institution to another. Vague indicators lead to confusion.
- Practical, affordable, and simple. Is the data available at a reasonable cost and effort? Is it realistic to collect and analyze the information needed?
- Reliability. Can the indicators be consistently measured against the adaptation baseline over the short, medium, and long term?
- Sensitivity. When the respective climatic effects or adaptive behaviors change, is the indicator susceptible to these changes?
- Clear direction. Is it certain that an increase in value is good (or bad) for what particular aspect of adaptation?
- Utility. How useful and relevant are the data collected? Can they be used for other projects? Do they measure achievable results?
- Ownership. Do stakeholders agree that the indicator makes sense for testing the adaptation hypothesis?
Four Types of Indicators
Considering the abovementioned criteria, there are four types of indicators as mentioned in FAO’s Climate-Smart Agriculture Sourcebook (2013):
- Quantitative: Measurable, i.e., tons per hectare of incremental crop production; number of days a year a household has adequate meals; or number of men and women with increased income;
- Qualitative: Descriptive, i.e., beneficiary perception of satisfactory service delivery by intervention agency;
- Proxy: Gives an approximation of the desired measure and are used when one cannot directly collect data for the indicator, i.e., deposits in rural banks; and
- Indices: Composed from other indicators to provide a more simplified aggregate measure of change, i.e., percentage of population below poverty line.
Some typical indicators relating to outcomes of local and national adaptation, which will contribute to long term stability in food security and livelihoods, may include the following examples:
- Specific household or community capacities to manage key natural resources important for ecosystem resilience and services;
- Community, government and other agencies’ capacity for preparing and supporting households, including the strategy and policy, institutional and technical aspects;
- Key infrastructure, including for water and irrigation, agriculture and other rural infrastructure, which will be to standards resilient to future changes and shock, with attention given to who has access to this improved infrastructure;
- Of technology dissemination, including technologies for climate change adaptation and mitigation in each of the agricultural sectors, and measures on the uptake of technology by men and women, and specific vulnerable groups; and
- Systems for access to and sustaining key food supplies or household assets, and livelihoods. For example, a household with safe foods storage, savings, supplementary livelihoods (including remittances), assets, and social networks may be better able to access funds and recover in a crisis situation.
In determining baselines and baseline projections, one should take into account that baseline data are used to check or evaluate short-term project results. Meanwhile, baseline projection provides counterfactuals and is used for long-term impacts of adaptation interventions. There is a need to constantly update baseline projections, as climate change evolves over time. Revising or changing adaptation interventions to reflect baseline projections is thus imperative.
Challenges Particular to Climate Change Impacts and Adaptation Assessment, Monitoring, and Evaluation
- Definition and goals. Do all stakeholders agree on a particular definition? For instance, which term should be used: climate smart, climate-resilient, climate proof, or climate-informed? Even before the start of the project, during conceptualization, it is important to agree on the terms to use and on the goals of the project or program.
- Multi-sectoral issues and engaging stakeholders.
- Scales, leakages, permanency, externality, and ancillary impact. Virtually, every CCA option will produce some positive impact or negative externality and/or ancillary impact such as pollution. Whether quantifiable or not, these represent costs or benefits and should be factored in the M&E. Therefore, it is necessary to articulate clearly the uncertainties or assumptions in M&E, especially for long-term CCA options like agroforestry.
- Availability of data and information. Too much information with too little useful analysis must be avoided. Existing systems, on the other hand, should be maximized. The key is to identify the most relevant indicators and to continue to collect data for these indicators. It should also be noted that this challenge on availability of data and information has an implication on how different modelling tools are linked. Examples include using existing data from other activities for M&E and dealing with climate or weather data availability issues.
- Working with uncertainties. Managing uncertainty requires understanding the nature of the uncertainty and resulting limitations. One should take account of uncertainties and be flexible in planning adaptation strategies that withstand unpredictable futures in a robust way. Adaptation strategies must thus be complemented with robust bottom-up assessments and no-regrets options. More so, assumptions and their sensitivities should be explicitly stated and communicated. An example is the Global Climate Model (GCM) choice and downscaling methodology.
- Attribution difficulty. Up to what degree is it possible to attribute results to a project intervention rather than to other external causes? For example, is the increased adoption of CSA forestry practices a direct result of the project or intervention, or is it influenced by a large program or other external activities? Attribution difficulty may be addressed through robust sampling when setting baselines and the use of control areas. Setting baselines is important at the start of the project. Other projects undertaken in the areas such as development assistance and cash transfer programs could be determined through detailed and reliable baseline surveys. When the results of the project are evaluated, one can define more reliably attribution and identify which output or outcome can be attributed directly to the project. On the other hand, control areas or communities not covered by other projects may be included in the selection criteria of project sites or farming communities. This will enable evaluators to attribute more easily positive results to the project to be implemented.
- Inadequate capacity for assessment and monitoring and evaluation.
- Practicality of methods and tools.
The steps and indicator criteria discussed in this paper were utilized in the Analysis and Mapping of Impacts under Climate Change for Adaptation and Food Security (AMICAF) project, a comprehensive framework by the Food and Agriculture Organization (FAO) of the United Nations that aims to address climate change impacts and adaptation planning targeted at improving the food security of vulnerable household groups. The AMICAF Framework is currently being implemented as a project in the Philippines (in cooperation with the Department of Agriculture) and Peru. AMICAF has four components: I: Climate Change Impacts Assessment, II: Food Insecurity Vulnerability Analysis, III: Livelihood Adaptation to Climate Change, and IV: Awareness Raising and Institutional Mechanisms.
For more information, visit: www.fao.org/climatechange/amicaf/en
1 This learning note is based on the presentation of the first author at the Workshop on Vulnerability, Impact and Adaptation Assessment for Climate Change: Approaches, Methods, and Tools organized for the United Nations Environment Program Regional Office for Asia and the Pacific by the Asia Pacific Adaptation Network (APAN) through the Southeast Asian Regional Center for Graduate Study and Research in Agriculture (SEARCA), held on 10-12 July 2013 in Hanoi, Vietnam.
3 Dr. Hideki Kanamaru is a Natural Resources Officer (Climate Change), Climate, Energy and Tenure Division (NRC) of the Food and Agriculture Organization of the United Nations, Rome, Italy.
4 Mr. Jim Hancock is a Natural Resources Management Officer of the Food and Agriculture Organization of the United Nations, Rome, Italy.
Additional reference: FAO. 2013. Climate-smart agriculture sourcebook. Retrieved from http://www.fao.org/docrep/018/i3325e/i3325e00.htm
Packaged into an Agriculture and Development Note by Evelie Serrano and Malaya N. Montesur