Classifier Calibration

A survey on how to assess and improve predicted class probabilities

Peter Flach, University of Bristol, UK, Peter.Flach@bristol.ac.uk , www.cs.bris.ac.uk/~flach/

Miquel Perello-Nieto, University of Bristol, UK, miquel.perellonieto@bristol.ac.uk, https://www.perellonieto.com/

Hao Song, University of Bristol, UK, hao.song@bristol.ac.uk

Meelis Kull, University of Tartu, Estonia, meelis.kull@ut.ee

Telmo Silva Filho, Federal University of Paraiba, Brazil, telmo@de.ufpb.br

Additional material

Additional material to generate some of the figures from the survey can be found in the GitHub repository classifier-calibration/additional_material.

Tools

We are developing a Python library with tools to evaluate the calibration of models. PyCalib has its own documentation page, and can be installed from the Python Package Index Pypi pip install pycalib.

Citation

This work has been published in the Machine Learning journal. You may want to use the following citation if you want to reference this work.

@Article{SilvaFilho2023,
author={Silva Filho, Telmo
and Song, Hao
and Perello-Nieto, Miquel
and Santos-Rodriguez, Raul
and Kull, Meelis
and Flach, Peter},
title={Classifier calibration: a survey on how to assess and improve predicted class probabilities},
journal={Machine Learning},
year={2023},
month={May},
day={16},
issn={1573-0565},
doi={10.1007/s10994-023-06336-7},
url={https://doi.org/10.1007/s10994-023-06336-7}
}