Markus Walker, Philipp S. Bien, Uwe D. Hanebeck

16th Symposium Sensor Data Fusion: Trends, Solutions, and Applications, Bonn, Germany, Voronoi Trust Regions for Local Calibration Testing in Supervised Machine Learning Models

Abstract:

The assessment of prediction quality in machine learning models is crucial, particularly within specific regions of the input space, as black-box models do not perform equally well in every region. Common methods, such as mean squared error and calibration measures, are unable to assess local quality. To address this issue, we propose a novel approach based on Voronoi tessellation, which provides a visual and intuitive method for analyzing two-dimensional input spaces. Our method identifies regions in the input space, assesses the calibration of predictions within these regions, and is implemented for regression tasks in multi-input systems. The effectiveness of our approach is exemplified using Bayesian neural networks (BNNs) and shows that our proposed method provides a clearer understanding of the quality of predictions in different input space regions.