查看文章

General Purpose Vision System (GPVS) is a task-agnostic vision-language system that inputs an image and a question from which the system recognizes the tasks to be performed and outputs bounding boxes, confidence scores, and text outputs to answer the question. While much attention to GPVS has been recently given in the computer vision field, its medical field applications are still in their infancy. This paper presents MED-GPVS, a customized deep learning-based GPVS on biomedical images to perform various vision tasks, such as object detection and visual question answering, on medical images to facilitate precision medicine/e-health services. Our envisioned MED-GPVS takes an image and a natural language text as inputs, and then outputs bounding boxes, confidence scores, and generates a caption (i.e., the answer to the posed query). For example, if a medical image of a patient’s abdomen is …