Article Text

Download PDFPDF

Original research
Automated interpretation of the coronary angioscopy with deep convolutional neural networks
  1. Toru Miyoshi1,2,
  2. Akinori Higaki2,3,
  3. Hideo Kawakami1 and
  4. Osamu Yamaguchi2
  1. 1Department of Cardiology, Ehime Prefectural Imabari Hospital, Imabari, Japan
  2. 2Department of Cardiology, Pulmonology, Hypertension and Nephrology, Ehime University Graduate School of Medicine, Toon, Japan
  3. 3Hypertension and Vascular Research Unit, Lady Davis Institute for Medical Research, Montreal, Quebec, Canada
  1. Correspondence to Dr Akinori Higaki; akinori.higaki{at}


Background Coronary angioscopy (CAS) is a useful modality to assess atherosclerotic changes, but interpretation of the images requires expert knowledge. Deep convolutional neural networks (DCNN) can be used for diagnostic prediction and image synthesis.

Methods 107 images from 47 patients, who underwent CAS in our hospital between 2014 and 2017, and 864 images, selected from 142 MEDLINE-indexed articles published between 2000 and 2019, were analysed. First, we developed a prediction model for the angioscopic findings. Next, we made a generative adversarial networks (GAN) model to simulate the CAS images. Finally, we tried to control the output images according to the angioscopic findings with conditional GAN architecture.

Results For both yellow colour (YC) grade and neointimal coverage (NC) grade, we could observe strong correlations between the true grades and the predicted values (YC grade, average r=0.80±0.02, p<0.001; NC grade, average r=0.73±0.02, p<0.001). The binary classification model for the red thrombus yielded 0.71±0.03 F1-score and the area under the receiver operator characteristic curve was 0.91±0.02. The standard GAN model could generate realistic CAS images (average Inception score=3.57±0.06). GAN-based data augmentation improved the performance of the prediction models. In the conditional GAN model, there were significant correlations between given values and the expert’s diagnosis in YC grade but not in NC grade.

Conclusion DCNN is useful in both predictive and generative modelling that can help develop the diagnostic support system for CAS.

  • coronary artery disease
  • imaging and diagnostics
  • coronary angioscopy

This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See:

View Full Text

Statistics from


  • Contributors TM contributed to collecting the patient data, data annotation and writing the original draft. AH contributed to conceptualisation, development of methodology, implementing computer programmes and reviewing the manuscript. HK has contributed to the supervision of the research activity. OY has contributed to the project administration.

  • Funding The authors have not declared a specific grant for this research from any funding agency in the public, commercial or not-for-profit sectors.

  • Competing interests None declared.

  • Patient consent for publication Not required.

  • Ethics approval This retrospective observatory study was performed following the principles of the Declaration of Helsinki and the Japanese ethical guidelines for clinical research. The study protocol was approved by the institutional review boards and the ethics committees of Ehime Prefectural Imabari Hospital.

  • Provenance and peer review Not commissioned; externally peer reviewed.

  • Data availability statement Data are available in a public, open access repository. Data are available on reasonable request. Source codes used in this study are available from the online repositories: Mendeley,, DOI: 10.17632/9dx23j5d64.2. Anonymised image data which is used in this study can be obtained from the corresponding author ( on reasonable request such as reproducibility assessment.

Request Permissions

If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.