Volumetric Image Segmentation on Multimodal Medical Images using Deep Learning

Information

• Publication Type: Master Thesis
• Workgroup(s)/Project(s): not specified
• Date: July 2020
• Date (Start): 10. December 2019
• Date (End): 25. July 2020
• Second Supervisor: Katja Bühler
• Diploma Examination: 5. August 2020
• Open Access: yes
• First Supervisor: Eduard Gröller
• Pages: 118
• Keywords: tumor segmentation, deep learning

Abstract

The automatic segmentation of tumors on different imaging modalities supports medical experts in patient diagnosis and treatment. Magnetic resonance imaging (MRl), Computed Tomography (CT), or Positron Emission Tomography (PET) show the tumor in a different anatomical. functional, or molecular context. The fusion of this multimodal information leads to more profound knowledge and enabler more precise diagnoses. So far, the potential of multimodal data is only used by a few established segmentation methods. Moreover, much less is known about multimodal methods that provide several multimodal-specific tumor segmentations instead of single segmentations for a specific modality. This thesis aims to develop a segmentation method that uses multimodal context to improve t the modality-specific segmentation results. For the implementation, an artificial neural network is used, which is based on a fully convolution neural network. The network architecture has been designed to learn complex multimodal features to predict multiple tumor segmentations on different modalities efficiently. The evaluation is based on a dataset consisting of MRl aid PET /CT scans of soft soft tissue tumors. The experiment investigated how different network architectures, multimodal fusion strategies, and input modalities affect the segmentation results. Tbc investigation showed that multimodal rondels lead to significantly better results than models for single modalities. Promising results have been achieved with multimodal models that segment several modality-specific tumor contours simultaneously.

Additional Files and Images

Weblinks

• Entry in reposiTUm (TU Wien Publication Database)
• DOI: 10.34726/hss.2020.73220

BibTeX

@mastersthesis{Neubauer2020,
  title =      "Volumetric Image Segmentation on Multimodal Medical Images
               using Deep Learning",
  author =     "Theresa Neubauer",
  year =       "2020",
  abstract =   "The automatic segmentation of tumors on different imaging
               modalities supports medical experts in patient diagnosis and
               treatment. Magnetic resonance imaging (MRl), Computed
               Tomography (CT), or Positron Emission Tomography (PET) show
               the tumor in a different anatomical. functional, or
               molecular context. The fusion of this multimodal information
               leads to more profound knowledge and enabler more precise
               diagnoses. So far, the potential of multimodal data is only
               used by a few established segmentation methods. Moreover,
               much less is known about multimodal methods that provide
               several multimodal-specific tumor segmentations instead of
               single segmentations for a specific modality.  This thesis
               aims to develop a segmentation method that uses multimodal
               context to improve t the modality-specific segmentation
               results. For the implementation, an artificial neural
               network is used, which is based on a fully convolution
               neural network. The network architecture  has been designed
               to learn complex multimodal features to predict multiple
               tumor segmentations on different modalities efficiently. 
               The evaluation is based on a dataset consisting of MRl aid
               PET /CT scans of soft soft tissue tumors. The experiment
               investigated how different network architectures, multimodal
               fusion strategies, and input modalities affect the
               segmentation results. Tbc investigation showed that
               multimodal rondels lead to significantly better results than
               models for single modalities. Promising results have been
               achieved with multimodal models that segment several
               modality-specific tumor contours simultaneously. ",
  month =      jul,
  pages =      "118",
  address =    "Favoritenstrasse 9-11/E193-02, A-1040 Vienna, Austria",
  school =     "Research Unit of Computer Graphics, Institute of Visual
               Computing and Human-Centered Technology, Faculty of
               Informatics, TU Wien",
  keywords =   "tumor segmentation, deep learning",
  URL =        "https://www.cg.tuwien.ac.at/research/publications/2020/Neubauer2020/",
}