…
comment: The repository administrator has not yet configured an RDF license.
primary topic: Translating medical image to radiological report: Adaptive multilevel multi-attention approach

Translating medical image to radiological report: Adaptive multilevel multi-attention approach
type: AcademicArticle
type: Article
abstract: Background and Objective: Medical imaging techniques are widely employed in disease diagnosis and treatment. A readily available medical report can be a useful tool in assisting an expert for investigating the patient's health. A radiologist can benefit from an automatic medical image to radiological report translation system while preparing a final report. Previous attempts on automatic medical report generation task includes image captioning algorithms without taking domain-specific visual and textual contents into account, thus arises the question about credibility of generated report. Methods: In this work, a novel Adaptive Multilevel Multi-Attention (AMLMA) approach is proposed by offering domain-specific visual-textual knowledge to generate a thorough and believable radiological report for any view of a human chest X-ray image. The proposed approach leverages the encoder-decoder framework incorporated with multiple adaptive attention mechanisms. The potential of a convolutional neural network (CNN) with residual attention module (RAM) is demonstrated as a strong visual encoder for multi-label abnormality detection. The multilevel visual features (local and global) are extracted from proposed visual encoder to retrieve regional-level and abstract-level radiology-based semantic information. The Word2Vec and FastText word embeddings are trained on medical reports to acquire radiological knowledge and further used as textual encoders, feeding as input to Bi-directional Long Short Term Memory (Bi-LSTM) network to learn the co-relationship between medical terminologies in radiological reports. The AMLMA employs a weighted multilevel association of adaptive visual-semantic attention and visual-based linguistic attention mechanisms. This association of adaptive attention is exploited as a decoder and produces significant improvements in the report generation task. Results: The proposed approach is evaluated on a publicly available Indiana University chest X-ray (IU-CX…
date: 2022
volume: 221
is part of: Computer Methods and Programs in Biomedicine
status: peerReviewed
status: published
author list: authors
type: EPrint
type: ArticleEPrint
is part of: repository

Elsevier Ireland Ltd
type: Organization

Computer Methods and Programs in Biomedicine
type: Collection
issn: 01692607

A.V. Nandedkar
type: Person
given name: A.V.
family name: Nandedkar

I. Faye
type: Person
given name: I.
family name: Faye

G.O. Gajbhiye
type: Person
given name: G.O.
family name: Gajbhiye

HTML Summary of #16700 Translating medical image to radiological report: Adaptive multilevel multi-attention approach
format: text/html
primary topic: Translating medical image to radiological report: Adaptive multilevel multi-attention approach