Uncertainty quantification for probabilistic machine learning in earth observation using conformal prediction
Machine learning use in EO is growing, but most datasets lack reliable uncertainty info. Conformal prediction offers valid, model‑agnostic uncertainty estimates. New Earth Engine modules let these methods run directly on EO data, enabling scalable integration across tasks and improving confidence in monitoring and decision‑making.
Subject Tags
- Data Science and Artificial Intelligence
- Conservation Technology
Abstract
Machine learning is increasingly applied to Earth Observation (EO) data to obtain datasets that contribute towards international accords. However, these datasets contain inherent uncertainty that needs to be quantified reliably to avoid negative consequences. In response to the increased need to report uncertainty, we bring attention to the promise of conformal prediction within the domain of EO. Unlike previous uncertainty quantification methods, conformal prediction offers statistically valid prediction regions while concurrently supporting any machine learning model and data distribution. To support the need for conformal prediction, we reviewed EO datasets and found that only 22.5% of the datasets incorporated a degree of uncertainty information, with unreliable methods prevalent. Current open implementations require moving large amounts of EO data to the algorithms. We introduced Google Earth Engine native modules that bring conformal prediction to the data and compute, facilitating the integration of uncertainty quantification into existing traditional and deep learning modelling workflows. To demonstrate the versatility and scalability of these tools we apply them to valued EO applications spanning local to global extents, regression, and classification tasks. Subsequently, we discuss the opportunities arising from the use of conformal prediction in EO. We anticipate that accessible and easy-to-use tools, such as those provided here, will drive wider adoption of rigorous uncertainty quantification in EO, thereby enhancing the reliability of downstream uses such as operational monitoring and decision-making.
Citation
Singh, G., Moncrieff, G., Venter, Z., Cawse-Nicholson, K., Slingsby, J. and Robinson, T.B., 2024. Uncertainty quantification for probabilistic machine learning in earth observation using conformal prediction. Scientific Reports, 14(1), p.16166. https://doi.org/10.1038/s41598-024-65954-w
TNC Authors
-
Glenn Moncrieff
Applied Special Data Scientist
Email: glenn.moncrieff@tnc.org