Cookies on this website

We use cookies to ensure that we give you the best experience on our website. If you click 'Accept all cookies' we'll assume that you are happy to receive all cookies and you won't see this message again. If you click 'Reject all non-essential cookies' only necessary cookies providing core functionality such as security, network management, and accessibility will be enabled. Click 'Find out more' for information on how to change your cookie settings.

OBJECTIVE: Convolutional neural networks (CNNs) have demonstrated promise in automated cardiac magnetic resonance image segmentation. However, when using CNNs in a large real-world dataset, it is important to quantify segmentation uncertainty and identify segmentations which could be problematic. In this work, we performed a systematic study of Bayesian and non-Bayesian methods for estimating uncertainty in segmentation neural networks. METHODS: We evaluated Bayes by Backprop, Monte Carlo Dropout, Deep Ensembles, and Stochastic Segmentation Networks in terms of segmentation accuracy, probability calibration, uncertainty on out-of-distribution images, and segmentation quality control. RESULTS: We observed that Deep Ensembles outperformed the other methods except for images with heavy noise and blurring distortions. We showed that Bayes by Backprop is more robust to noise distortions while Stochastic Segmentation Networks are more resistant to blurring distortions. For segmentation quality control, we showed that segmentation uncertainty is correlated with segmentation accuracy for all the methods. With the incorporation of uncertainty estimates, we were able to reduce the percentage of poor segmentation to 5% by flagging 31-48% of the most uncertain segmentations for manual review, substantially lower than random review without using neural network uncertainty (reviewing 75-78% of all images). CONCLUSION: This work provides a comprehensive evaluation of uncertainty estimation methods and showed that Deep Ensembles outperformed other methods in most cases. SIGNIFICANCE: Neural network uncertainty measures can help identify potentially inaccurate segmentations and alert users for manual review.

Original publication

DOI

10.1109/TBME.2022.3232730

Type

Journal article

Journal

IEEE Trans Biomed Eng

Publication Date

06/2023

Volume

70

Pages

1955 - 1966

Keywords

Uncertainty, Benchmarking, Neural Networks, Computer, Magnetic Resonance Imaging, Radiography, Image Processing, Computer-Assisted