A randomised controlled trial led by Professor Paul Leeson has shown that doctors who use an AI algorithm to assess echocardiograms are more accurate and more confident in their decision making. The study, published in the journal JACC Cardiovascular Imaging, is the first to show that an algorithm can automatically categorise severity of coronary artery disease from a simple echo scan.
Stress echocardiography is one of the most widely used imaging tools to assess coronary artery disease, with clinicians comparing images of how the heart’s wall moves before and after exercise or a pharmacological stressor. Severe coronary artery disease affects the heart muscle’s ability to contract, and this is visible as changes in how the heart wall moves on an echo scan.
However, these changes need to be spotted ‘by eye’, by a trained and experienced assessor. Different observers for the same echo scan can arrive at different conclusions.
Professor Leeson’s group have therefore been exploring whether automated image assessments using AI can overcome these limitations. But using this in actual clinical practice has been slow, because any tools needs to be very robust and generalizable systems.
The research team therefore developed a bespoke sequence of AI algorithms that could automatically process and extract novel features in a stress echocardiogram. Then, they incorporated these features into a machine learning model, which was trained to distinguish between patients with significant coronary heart disease and those without disease.
To make sure that this machine learning model was working, the researchers tested it on a separate set of echocardiograms, and found that it correctly picked out echocardiograms showing signs of severe coronary heart disease 86% of the time.
Finally, to assess how using the AI algorithm would work in actual clinical practice, the researchers asked four experienced echocardiographers to look through the same echocardiograms that the algorithm had classified. For half of the scans the echocardiographers were provided with a copy of the AI report to help them make a decision. Then, a few months later, they were asked to review the scans again but this time provided with the AI information on the other half of the scans Each time they reviewed a scan they also indicated how confident they were of their judgement.
The study found that when the echocardiographers had the AI-generated report, they identified 95% of the scans showing coronary artery disease, compared to 85% when they didn’t have AI assistance. Having the AI classifier information also resulted in a 10% increase in self-rated confidence in their judgement, and a 29% decrease in non-confident judgements.
What really distinguishes this study is that we have tested how an algorithm might operate in actual clinical practice, and we find that it fundamentally changes how doctors make their decisions
- Professor Paul Leeson
The research team have now recruited the first participants for a follow-up randomised trial to run across 15 UK sites over the next 18 months, funded by a multi-million pound grant from the NIHR AI in Healthcare Awards. This follow-up trial is testing the tool prospectively in the clinic and will mean NHS cardiology patients in the trial will, for the first time, receive care guided by the AI.
Read the full study.