Automatic Contour Refinement for Deep Learning Auto-segmentation of Complex Organs in MRI-guided Adaptive Radiation Therapy

Open AccessPublished:April 20, 2022DOI:



      Fast and accurate auto-segmentation on daily images is essential for magnetic resonance imaging (MRI)–guided adaptive radiation therapy (ART). However, the state-of-the-art auto-segmentation based on deep learning still has limited success, particularly for complex structures in the abdomen. This study aimed to develop an automatic contour refinement (ACR) process to quickly correct for unacceptable auto-segmented contours.

      Methods and Materials

      An improved level set–based active contour model (ACM) was implemented for the ACR process and was tested on the deep learning–based auto-segmentation of 80 abdominal MRI sets along with their ground truth contours. The performance of the ACR process was evaluated using 4 contour accuracy metrics: the Dice similarity coefficient (DSC), mean distance to agreement (MDA), surface DSC, and added path length (APL) on the auto-segmented contours of the small bowel, large bowel, combined bowels, pancreas, duodenum, and stomach.


      A portion (3%-39%) of the corrected contours became practically acceptable per the American Association of Physicists in Medicine Task Group 132 (TG-132) recommendation (DSC >0.8 and MDA <3 mm). The best correction performance was seen in the combined bowels, where for the contours with major errors (initial DSC <0.5 or MDA >8 mm), the mean DSC increased from 0.34 to 0.59, the mean MDA decreased from 7.02 mm to 5.23 mm, and the APL reduced by almost 20 mm, whereas for the contours with minor errors, the mean DSC increased from 0.72 to 0.79, the mean MDA decreased from 3.35 mm to 3.29 mm, and more than one-third (39%) of the ACR contours became clinically acceptable. The execution time for the ACR process on one subregion was less than 2 seconds using an NVIDIA GTX 1060 GPU.


      The ACR process implemented based on the ACM was able to quickly correct for some inaccurate contours produced from MRI-based deep learning auto-segmentation of complex abdominal anatomy. The ACR method may be integrated into the auto-segmentation step to accelerate the process of MRI-guided ART.
      To read this article in full you will need to make a payment
      ASTRO Member Login
      ASTRO Members, full access to the journal is a member benefit. Use your society credentials to access all journal content and features.
      One-time access price info
      • For academic or personal research use, select 'Academic and Personal'
      • For corporate R&D use, select 'Corporate R&D Professionals'
      Already a print subscriber? Claim online access
      Already an online subscriber? Sign in
      Institutional Access: Sign in to ScienceDirect


        • Mutic S
        • Dempsey JF.
        The ViewRay system: Magnetic resonance–guided and controlled radiotherapy.
        Semin Radiat Oncol. 2014; : 196-199
        • Lagendijk JJ
        • Raaymakers BW
        • Van Vulpen M.
        The magnetic resonance imaging–linac system.
        Semin Radiat Oncol. 2014; : 207-209
        • Paulson ES
        • Ahunbay E
        • Chen X
        • et al.
        4D-MRI driven MR-guided online adaptive radiotherapy for abdominal stereotactic body radiation therapy on a high field MR-Linac: Implementation and initial clinical experience.
        Clinical and translational radiation oncology. 2020; 23: 72-79
        • Lamb J
        • Cao M
        • Kishan A
        • et al.
        Online adaptive radiation therapy: Implementation of a new process of care.
        Cureus. 2017; 9: e1618
        • Güngör G
        • Serbez İ
        • Temur B
        • et al.
        Time analysis of online adaptive magnetic resonance–guided radiation therapy workflow according to anatomical sites.
        Pract Radiat Oncol. 2021; 11: e11-e21
        • Fu Y
        • Mazur TR
        • Wu X
        • et al.
        A novel MRI segmentation method using CNN-based correction network for MRI-guided adaptive radiation therapy.
        Med Phys. 2018; 45: 5129-5137
        • Bobo MF
        • Bao S
        • Huo Y
        • et al.
        Fully convolutional neural networks improve abdominal organ segmentation.
        Proc SPIE Int Soc Opt Eng. 2018; 10574105742V
        • Chen Y
        • Ruan D
        • Xiao J
        • et al.
        Fully automated multi-organ segmentation in abdominal magnetic resonance imaging with deep neural networks.
        Med Phys. 2020; 47: 4971
        • Amjad A
        • Xu J
        • Thill D
        • et al.
        Deep learning-based auto-segmentation on CT and MRI for abdominal structures.
        Int J Radiat Oncol Biol Phys. 2020; 108: S100-S101
        • Savenije MH
        • Maspero M
        • Sikkes GG
        • et al.
        Clinical implementation of MRI-based organs-at-risk auto-segmentation with convolutional networks for prostate radiation therapy.
        Radiat Oncol. 2020; 15: 1-12
        • Elguindi S
        • Zelefsky MJ
        • Jiang J
        • et al.
        Deep learning-based auto-segmentation of targets and organs-at-risk for magnetic resonance imaging only planning of prostate radiation therapy.
        Phys Imaging Radiat Oncol. 2019; 12: 80-86
        • Krähenbühl P
        • Koltun V.
        Efficient inference in fully connected crfs with gaussian edge potentials.
        Adv Neural Inf Process Syst. 2011; 24: 109-117
        • Christ PF
        • Elshaer MEA
        • Ettlinger F
        • et al.
        Automatic liver and lesion segmentation in CT using cascaded fully convolutional neural networks and 3D conditional random fields.
        Med Image Comput Comput Assist Interv. 2016; : 415-423
        • Kamnitsas K
        • Ledig C
        • Newcombe VF
        • et al.
        Efficient multi-scale 3D CNN with fully connected CRF for accurate brain lesion segmentation.
        Med Image Anal. 2017; 36: 61-78
        • Zhang K
        • Zhang L
        • Song H
        • Zhou W.
        Active contours with selective local or global segmentation: A new formulation and level set method.
        Image and Vis Comput. 2010; 28: 668-676
        • Hoang Ngan Le T
        • Luu K
        • Duong CN
        • et al.
        Active contour model in deep learning era: A revise and review.
        Applications of Hybrid Metaheuristic Algorithms for Image Processing. 2020; : 231-260
        • Semelka RC
        • Kelekis NL
        • Thomasson D
        • Brown MA
        • Laub GA.
        HASTE MR imaging: Description of technique and preliminary results in the abdomen.
        J Magn Res Imaging. 1996; 6: 698-699
        • Zhang Y
        • Paulson E
        • Lim S
        • et al.
        A Patient-Specific Autosegmentation Strategy Using Multi-Input Deformable Image Registration for Magnetic Resonance Imaging–Guided Online Adaptive Radiation Therapy: A Feasibility Study.
        Advances in radiation oncology. 2020; 5: 1350-1358
        • Tustison NJ
        • Avants BB
        • Cook PA
        • et al.
        N4ITK: Improved N3 bias correction.
        IEEE Trans Med Imaging. 2010; 29: 1310-1320
        • Perona P
        • Malik J.
        Scale-space and edge detection using anisotropic diffusion.
        IEEE Trans Pattern Anal Machine Intelligence. 1990; 12: 629-639
        • Zuiderveld K.
        Contrast limited adaptive histogram equalization.
        Graphics Gems. 1994; : 474-485
        • Amjad A
        • Xu J
        • Thill D
        • et al.
        General and custom deep learning autosegmentation models for organs in head and neck, abdomen, and male pelvis.
        Medical Physics. 2022; 49: 1686-1700
        • Yu L
        • Yang X
        • Chen H
        • Qin J
        • Heng PA.
        Volumetric ConvNets with mixed residual connections for automated prostate segmentation from 3D MR images.
        Proceedings of the Thirty-First AAAI Conference on Artificial Intelligence (AAAI-17). 2017; : 66-72
        • Brock KK
        • Mutic S
        • McNutt TR
        • Li H
        • Kessler ML.
        Use of image registration and fusion algorithms and techniques in radiation therapy: Report of the AAPM Radiation Therapy Committee Task Group No. 132.
        Med Phys. 2017; 44: e43-e76
        • Hatamizadeh A
        • Hoogi A
        • Sengupta D
        • et al.
        Deep active lesion segmentation.
        International Workshop on Machine Learning in Medical Imaging. 2019; : 98-105
        • Chan TF
        • Vese LA.
        Active contours without edges.
        IEEE Trans Image Process. 2001; 10: 266-277
        • Pan Y
        • Birdwell JD
        • Djouadi SM.
        Efficient implementation of the Chan-Vese models without solving PDEs.
        2006 IEEE Workshop on Multimedia Signal Processing. 2006; : 350-354
      1. Nikolov S, Blackwell S, Zverovitch A, et al. Deep learning to achieve clinically applicable segmentation of head and neck anatomy for radiation therapy. arXiv preprint arXiv:1809.04430.2018.

        • Vaassen F
        • Hazelaar C
        • Vaniqui A
        • et al.
        Evaluation of measures for assessing time-saving of automatic organ-at-risk segmentation in radiation therapy.
        Phys Imaging Radiat Oncol. 2020; 13: 1-6
      2. Amjad A, Xu J, Thill D, et al. Improving Deep Learning Auto-Segmentation Using an Adaptive Spatial Resolution Approach. MEDICAL PHYSICS. 482021.

        • Amjad A
        • Xu J
        • Zhu X
        • et al.
        Deep Learning Auto-Segmentation on Multi-Sequence MRI for MR-Guided Adaptive Radiation Therapy.
        American Society for Radiation Oncology (ASTRO) Annual Meeting. 2021;
        • Zhang Y
        • Plautz TE
        • Hao Y
        • Kinchen C
        • Li XA
        Texture‐based, automatic contour validation for online adaptive replanning: a feasibility study on abdominal organs.
        Medical physics. 2019; 46: 4010-4020