Through developing a deep learning model to automatically detect clinically significant intracranial aneurysms on computed tomography (CT) angiography, clinicians’ sensitivity, accuracy, and interrater reliability were found to improve when compared to detection on original CT angiography scans. The corresponding study, published in JAMA, details the application of a convolutional neural network used to augment CT angiography aneurysm interpretation—whereby clinicians were provided with the models’ predictions in the form of regions of interest segmentations directly overlaid on top of CT angiography examinations.
Although CT angiography augmentation elicited no statistically significant change in mean specificity and time to diagnosis compared to unaugmented CT angiography, the authors, Allison Park and colleagues from Stanford University (Stanford, USA), write: “If integrated into the standard workflow, this diagnostic tool could substantially decrease both cost and time to diagnosis, potentially leading to more efficient treatment and more favourable outcomes.”
In their paper, Park et al allude to the importance of an automated detection tool, given the potential catastrophic outcome of a missed aneurysm at risk of rupture, alongside the significant variability that remains among clinicians when diagnosing aneurysms. The latter, they say, is typically attributed to a lack of experience, subspecialty neuroradiology training, complex neurovascular anatomy, or the labour-intensive nature of identifying aneurysms.
Furthermore, the authors posit that the tools that are currently used to improve clinician aneurysm detection on CT angiography are labour- and time-intensive. Acknowledging the recent success of deep learning applications in clinical image-based recognition tasks, the authors point to studies that have displayed the superiority of a 2D convolutional neural network in detecting intracranial haemorrhage and other acute brain findings. Yet Park et al put forward, “Prior to this study, deep learning had not been applied to CT angiography, which is the first-line imaging modality for detecting cerebral aneurysms.”
In light of this, the study investigators aimed to develop and apply a neural network segmentation model, known as the HeadXNet model, which was able to generate precise voxel-by-voxel predictions of intracranial aneurysms on head CT angiography imaging to augment clinicians’ intracranial aneurysm diagnostic performance. Park and team developed the 3D convolutional neural network architecture using a training set of 611 head CT angiography examinations to generate aneurysms segmentations.
Subsequently, a test set of 115 examinations from these segmentation outcomes were given to the participating clinicians. Through a crossover design that used a randomised order and a 14-day washout period, eight clinicians diagnosed the presence of aneurysm on the test set, both with and without model augmentation.
The authors note that head and neck examinations that were performed between January 2003 and May 2017 (at a single academic medical centre) were used to train, validate, and test the model. Examinations positive for aneurysm had at least one clinically significant, nonruptured intracranial aneurysm. In contrast, those with haemorrhage, ruptured aneurysm, posttraumatic or infectious pseudoaneurysm, arteriovenous malformation, surgical clips, coils, catheters, or other surgical hardware were excluded, while all other CT angiography examinations were considered controls.
In total, the dataset included 818 examinations from 662 unique patients, with 328 CT angiography examinations (40.1%) containing at least one intracranial aneurysm and 490 examinations (59.9%) without intracranial aneurysms. The group of clinicians reading the test set were composed of six board-certified practicing radiologists, one practicing neurosurgeon, and one radiology resident, with their experience ranging from two to 12 years.
In the current study, augmenting CT angiography with artificial intelligence-produced segmentation predictions elicited an increase of 0.059 (95% CI: 0.028, 0.091; adjusted p=0.01) in clinicians’ mean sensitivity, while mean accuracy increased by 0.038 (95% CI: 0.014, 0.062; adjusted p=0.02) and mean interrater agreement increased by 0.060, from 0.799 to 0.859 (adjusted p=0.05).
“The [data] suggests that integration of an artificial intelligence-assisted diagnostic model may augment clinicians’ performance with dependable and accurate predictions and thereby optimise care,” write Park and colleagues. Moreover, terms of moving forward, the team postulate, “Future work should investigate the performance of this model prospectively and in application of data from other institutions and hospitals.”