Toward a methodological toolkit to mitigate the evaluator's dilemma: Assessing pastoralist rangelands management in Northern Tanzania
Impact evaluation in conservation often faces the “evaluator’s dilemma”: assessing outcomes without ideal study designs. Using a USAID‑funded rangeland intervention in Northern Tanzania, this study applies directed acyclic graphs and item‑response theory to strengthen causal inference. Perceived improvements in commons governance were positively associated with management success and remotely sensed reductions in bare land cover, aligning with the project’s theory of change. Results highlight how governance mediates intervention effectiveness and demonstrate how complex, imperfect datasets can still yield meaningful insights. The study offers methodological guidance for improving evaluation rigor when resources are limited.
Subject Tags
- Agriculture
- Data Science and Artificial Intelligence
- Land management
Abstract
Biodiversity conservation is a “crisis discipline,” with funds often directed toward urgent needs rather than evaluations. Accordingly, researchers tasked with an impact evaluation in conservation can face the evaluator's dilemma: determining the potential impacts of project activities while lacking an ideal study design. Using as an example a USAID-funded intervention to improve pastoralist communal rangeland health in Northern Tanzania, we show how methods relatively novel to the field of conservation evaluation—directed acyclic graphs and item response theory—can help to mitigate this dilemma. We find that the perceived quality of rangeland commons governance after project completion is positively associated with, and potentially causally related to, positive outcomes in the perceived successes of management and remotely sensed assessments of changes in bare land cover pre and post intervention. This is consistent with the development partner's theory of change and extends our knowledge of which factors mediate whether and how commons management interventions work. Our methodological tools demonstrate how messy and complex data can be productively leveraged, and why unequivocal causal effects cannot always be determined. More broadly, we demonstrate valuable additions to the evaluator's toolkit that can improve the validity of inferences when resources allocated to evaluation are constrained.
Citation
Borgerhoff Mulder, M., Gehrig, S., Silisyene, M., James, S., Leisher, C., Lukumay, P., ... & Robinson, N. (2025). Toward a methodological toolkit to mitigate the evaluator's dilemma: Assessing pastoralist rangelands management in Northern Tanzania. Conservation Science and Practice, 7(11), e70119. https://doi.org/10.1111/csp2.70119
TNC Authors
-
Craig Leisher
Portfolio Director, Africa
The Nature Conservancy
Email: craig.leisher@tnc.org -
Philipo Lukumay
Monitoring, Evaluation and Learning Specialist, Africa
The Nature Conservancy
Email: philipo.lukumay@tnc.org -
Alphonce B. Mallya
NTRI Program Lead, Africa
The Nature Conservancy
Email: amallya@tnc.org