If you don't remember your password, you can reset it by entering your email address and clicking the Reset Password button. You will then receive an email that contains a secure link for resetting your password
If the address matches a valid account an email will be sent to __email__ with instructions for resetting your password
Leiden University Medical Center, Ophthalmology, Leiden, NetherlandsLeiden University Medical Center, Radiology, Leiden, NetherlandsLeiden University Medical Center, Radiation Oncology, Leiden, Netherlands
Several efforts are being undertaken toward MRI-based treatment planning for ocular proton therapy for uveal melanoma (UM). The interobserver variability of the gross target volume (GTV) on magnetic resonance imaging (MRI) is one of the important parameters to design safety margins for a reliable treatment. Therefore, this study assessed the interobserver variation in GTV delineation of UM on MRI.
Methods and Materials
Six observers delineated the GTV in 10 different patients using the Big Brother contouring software. Patients were scanned at 3T MRI with a surface coil, and tumors were delineated separately on contrast enhanced 3DT1 (T1gd) and 3DT2-weighted scans with an isotropic acquisition resolution of 0.8 mm. Volume difference and overall local variation (median standard deviation of the distance between the delineated contours and the median contour) were analyzed for each GTV. Additionally, the local variation was analyzed for 4 interfaces: sclera, vitreous, retinal detachment, and tumor-choroid interface.
Results
The average GTV was significantly larger on T1gd (0.57cm3) compared with T2 (0.51cm3, P = .01). A not significant higher interobserver variation was found on T1gd (0.41 mm) compared with T2 (0.35 mm). The largest variations were found at the tumor-choroid interface due to peritumoral enhancement (T1gd, 0.62 mm; T2, 0.52 mm). As a result, a larger part of this tumor-choroid interface appeared to be included on T1gd-based GTVs compared with T2, explaining the smaller volumes on T2.
Conclusions
The interobserver variation of 0.4 mm on MRI are low with respect to the voxel size of 0.8 mm, enabling small treatment margins. We recommend delineation based on the T1gd-weighted scans, as choroidal tumor extensions might be missed.
Introduction
Uveal melanoma (UM) arises from melanocytes and is the most common primary intraocular tumor, occurring at a rate of approximately 14 cases per million person-years.
Currently, gross target volume (GTV) definition in PT planning for UM is based on a generic model of the eye and tumor, constructed using marker positions and 2-dimensional imaging such as fundus photographs and ocular ultrasound.
This distinct difference compared with other tumor sites, where computed tomography (CT) and magnetic resonance imaging (MRI) are commonly fused for target and organ-at-risk (OAR) delineation,
is primarily the result of the poor image quality that could be obtained with conventional ocular MRI techniques. As a result, currently a generic model is used, which has a limited possibility to account for variations in tumor and globe shape. However, over the last decade, eye-specific MRI protocols have resolved the historically poor image quality of ocular MRIs, resulting in increased use of MRI in ocular oncology.
Jaarsma-Coes MG, Ferreira TA, Marinkovic M, et al. Comparison of MRI-based and conventional measurements for proton beam therapy of uveal melanoma. [e-pub ahead of print]. Ophthalmol Retin.https://doi.org/10.1016/j.oret.2022.06.019, accessed 09 January 2023.
full 3-dimensional (3D) imaging-based GTV definition is currently not commonly used in ocular radiation therapy.
MRI-based tumor and OAR definition could be valuable for ocular PT as its excellent soft tissue contrast and a 3D representation of the tumor and organs at risk
Therefore, several efforts are being undertaken to enable a fully MRI-based treatment planning for ocular PT as it would allow for a more patient-specific geometric description of the tumor and OAR than the currently used model-based approach.
As delineation variability is an important source of uncertainty in radiation therapy, it contributes to a significant portion of the treatment margins.
For ocular MRI, however, this variation is currently unknown and, because of the eye's small size and eye-specific imaging challenges such as eye-blink artefacts,
Measuring the human retinal oxygenation response to a hyperoxic challenge using MRI: Eliminating blinking artifacts and demonstrating proof of concept.
results from other anatomies cannot be translated to the eyes. Therefore, the aim of this study is to assess the interobserver variation on GTV delineation of UM on MRI.
Methods and Materials
To determine the interobserver variability on T2 and contrast enhanced T1 (T1gd) MRIs, 6 observers delineated the GTV in 10 different UM patients. These patients were retrospectively selected to represent the wide spectrum of UM in terms of lesion size, location, and presence of retinal detachment. Three patients were scanned as part of a prospective study, which has been approved by the local ethics committee (Medical Ethics Committee Lieden The Hague Delft, P16.186) and subjects were scanned after written informed consent. Seven patients received an MRI as part of clinical care, and their data was included retrospectively with approval of the local ethics committee.
Image acquisition and registration
All patients were scanned on a 3T MRI scanner (Ingenia, Philips Healthcare, the Netherlands) using a 47-mm diameter surface coil as described by Ferreira et al.
In this study, the 3D T2- and 3D T1- weighted scan before (T1 and T2) and a 3D T1 weighted scan after contrast administration (T1gd, 0.1 mmol/kg gadoterate megliumine; DOTAREM, Guebert, Roissy CdG Cedex, France) were used for delineation. Both T1 and T1gd scans were acquired in 2 minutes with a echo time (TE) of 26 ms and repetition time (TR) of 400 ms. The T2 scan was acquired in 3 minutes with a TE of 305 ms and a TR of 2500 ms. Both T2 and T1gd were acquired with spectral presaturation with inversion recovery fat suppression. All scans were acquired with a 0.8-mm isotropic resolution and reconstructed on the MRI scanner with a resolution of 0.4 mm × 0.4 mm × 0.4 mm for T1 and 0.3 mm × 0.3 mm × 0.4 mm for T2 using 0 filling.
was used to register the T1 and T2 to the T1gd images. Before registration all scans were resampled using nearest neighbor interpolation to a resolution of 0.3 mm isotropic. A mask, created by segmenting the sclera on the T1gd, was used to restrict the registration region of interest to the globe.
Delineation of GTV
After a pilot study with 4 patients to get acquainted with the software and test the delineation guidelines, 2 radiation oncologists, 2 head and neck radiologists, and 2 ophthalmologists delineated the GTV of 10 patients on T1gd and T2 after instruction on the use of the program and delineation guidelines. For contouring the Big Brother
training contouring software, developed at The Netherlands Cancer Institute and University of Manchester (England), was used. The GTV was delineated independently on the T2 and T1gd images. The other sequences (T1 and T1gd or T2) were visible in a side window with linked cursor as a reference to differentiate hemorrhage and tumor. Observers were masked to other imaging data, such as fundus photographs.
A combined evaluation of all 3 sequences is needed to differentiate UM from adjacent tissues. UM are hyperintense on T1 and hypointense on T2 compared with the vitreous and can be either hyperintense, isointense, or hypointense compared with the choroid on T1 depending on the amount of pigmentation. Retinal detachment is hyperintense on T1 and hypo or isointense compared with the vitreous on T2 and can be distinguished from tumor because of the lack of enhancement.
Therefore, areas with a hypointense signal compared with the vitreous on T2 and enhancing on T1gd were compatible with tumor (Fig. 1) and included in the GTV. Nonenhancing regions were considered retinal detachment (RD, Fig. 1) and not included in the GTV. The sclera, a hypointense structure adjacent to the vitreous and tumor, was not included in the GTV.
Figure 1Delineation instructions areas inside the sclera (arrow) with a hypointense signal compared with the vitreous (*) on T2 (C) and enhancing on T1gd (B) compared with the T1w (A) were compatible with tumor and were included in the gross target volume (†). Areas with a hypointense signal compared with the vitreous on T2 but not enhancing on T1gd were considered retinal detachment and not included in the gross target volume (‡).
We assessed the delineation variation between observers in terms of a standard deviation (SD), such that this data can in future be used for margin definition. Each GTV distances was calculated with respect to a median surface (which acts as an arbitrary reference), defined as the surface encompassing the voxels designated by at least 50% of the observers as part of the GTV (Fig. 2A). For each point of this median surface, the perpendicular distance was measured to the GTV of each observer, and the SD of these distances (local SD, Fig. 2B) was used as a measure of local observer variation.
This local SD quantifies the spatial variation between contours, for example, a larger local SD corresponds to a larger variation.
Figure 2Data analysis. For each patient, the median contour (A, red) of the delineated gross tumor volumes (GTVs,†) represents a 50% coverage of all GTVs of the individual observers (blue). For each point on this surface the local standard deviation (SD; variation in perpendicular distance to each individual delineation [B]) was determined. These distances were compared between patients to determine the interobserver variability and were also analyzed for different interfaces (C). Four interfaces were analyzed: tumor/vitreous, tumor/sclera, tumor-choroid interface, tumor/retinal detachment. Retinal detachment (‡) can be isointense as the uveal melanoma; however, it does not enhance as can be seen in Fig. 4Q.
On each median contour, points adjacent to the sclera, vitreous, the tumor-choroid interface (edge), and/or retinal detachment were labeled to allow separate analysis for these regions. Areas which were adjacent to the lens or close to 2 regions remained unclassified (Fig. 2C).
Statistics
The distance distributions per patient and per segment were not normally distributed, therefore, the median and 75th percentile were reported. A paired t test was performed to compare the average tumor volume and the median and 75th percentile observer variations per patient between delineations based on T1gd and T2 images. A t test was used to compare the median observer variation between the different regions. Statistics were performed in Python version 3.6.6 using SciPy version 1.5.4.
Results
Patient characteristics
Five patients underwent PT: 4 ruthenium brachytherapy, and in 1 patient the affected eye was enucleated. The average tumor prominence on ultrasound was 7.6 mm (range, 5.1-13.0 mm) and the average largest basal diameter was 14.8 mm (range, 11.7-18.0 mm). In 6 patients, retinal detachment was described by the radiologist during the pretreatment evaluation of the MRIs. Six tumors were hyperintense on T1-weighted images, 2 isointense, and 1 hypointense compared with the choroid, whereas 1 bipartite tumor consisted of both a hyper- and hypointense part. Eight tumors were hypointense compared with the eye muscles on T226 whereas 2 were isointense. Two tumors were located juxtapupillary, and in 4 patients the tumor involved the ciliary body. None of the patients had extrascleral extension. The tumor characteristics are described in Table 1.
The RD was only adjacent to the gross target volume at the most inferior edge (out of plane) and therefore not included in the region analysis. Prominence and LBD as measured on ultrasound. SI as described by the radiologist. RD as described by the radiologist.
Bilobar
Abbreviation: LBD = largest basal diameter; PT = proton beam therapy; Pt = patient; RD = retinal detachment; SI = signal intensity.
The RD was only adjacent to the gross target volume at the most inferior edge (out of plane) and therefore not included in the region analysis.Prominence and LBD as measured on ultrasound. SI as described by the radiologist. RD as described by the radiologist.
A large variation in average tumor volume was observed between patients ranging from 0.16 to 1.79 cm3 on both T1gd and T2, which was expected based on the different sizes of the lesions. The average delineated tumor volume was significantly higher on T1gd (0.57 cm3) compared with T2 (0.51 cm3, P = .01; Fig. 3ABC). Generally, tumors were delineated larger on T1gd compared with T2 except for patient 8, where the T2 volume was larger than the T1gd volume. Especially the posterior part of this tumor appeared to be delineated larger on T2 compared with T1gd. A visual inspection of all acquired images, especially the multislice images (not used for delineation), showed that this difference in tumor volume is likely caused by a small retinal detachment (Fig. 3C, arrow) incorrectly included in the GTV on T2 weighted images.
Figure 3Gross target volume (GTV). A, The average and standard deviation (SD) of the GTV for both T1gd-based (blue) and T2-based (orange) delineations. The delineated GTV was significantly larger on T1gd compared with T2 (P < .01). B, The relative volume (mean ± SD) with respect to the median volume on T1gd shows that, for most the patients, the tumor was delineated larger on T1gd than on T2. C, In most patients, the GTV based on T1gd (blue) is larger compared with T2 (orange). This difference seems to originate from the tumor-choroid interface. The left insert shows all individual delineations. Upper insert shows T1gd scan without delineations. Lower insert shows T2. D, Patient 8 is the only patient where a larger tumor volume was delineated on T2 compared with T1gd. Comparison with multislice images, which have a higher in-plane resolution than the 3-dimensional images used for contouring, show a small, nonenhancing, retinal detachment (arrow). This retinal detachment appears to be included in the T2 based GTV, explaining the larger volume.
Combining all local SDs showed a slightly higher median SD on T1gd (0.41 mm) compared with T2 (0.35 mm), and a similar relation was observed for the 75th percentile (T1gd, 0.60 mm vs T2, 0.54 mm) and 95th percentile (T1gd, 1.0 mm vs T2, 1.1 mm). Pairwise comparison per patient showed no significant difference between median local SD and 75th percentile (P = .12 and P = .15, respectively). Overall, observers had similar distances from the median contour, except for one of the radiologists who consistently had a lower distance from the median contour compared with the other observers (P = .004, Table E1).
Visual inspection of the local SDs for all individual patients showed mostly gradual changes in the local SD although hotspots with a higher observer variation were detected at the tumor-choroid interface (Fig. 4, open arrows) and in proximity of retinal detachment (Fig. 4, broad arrow).
Figure 4Local standard deviation (SD) of all patients (pt). Representative slice showing the local SD per patient (pt) on T1gd and T2 with histogram of all local SDs at the bottom of the image (x-axis is between 0 and 2 mm). Generally, the highest local SDs were located at the edge of the tumor base (open arrows; A, G, I, J, L, R) and at the tumor-retinal detachment border (broad arrow; R). The gross tumor volume on T2 at the level of the choroid tended to be more circular shaped in contrast to the gross tumor volume on T1gd which had a sharper tumor-choroid interface (dotted arrow; patients 2, 5, 6, and 8). In pt 10 (S), the flat tumor extension was missed by most observers (arrowhead).
The individual delineations showed that higher local SDs at the tumor-choroid interface corresponded with choroidal enhancement, which was included in the GTV by some but not all observers (Fig. 5C). In patient 9 (Fig. 4R), however, the high local SD in proximity of the retinal detachment was a result of partial inclusion of retinal detachment by 1 observer (ophthalmologist, Fig. 5D insert). Another area with a high local SD was observed in patient 4 due to wide tumor margin and partial inclusion of the levator palpebrae by an observer and omission of part of the tumor base in the GTV by another observer.
Figure 5Observer variation. A, B, Distribution of the local observer variation per region for T1gd (blue) and T2 (orange) and entire gross target volume (GTV, †). A significant difference in the observer variation between T1gd and T2 was only found for the vitreous (P < .001). C, D, Higher local standard deviations (SDs) were found at the tumor-choroid interface and in proximity of retinal detachment (‡). Higher variation at the tumor-choroid interface was most likely a result of disagreement between observers about inclusion of (enhancing) choroid. Some observers included the enhancing choroid in the GVT (left) and others did not (right). Retinal detachment was not always clearly visible. An example of a patient (pt) where it can be challenging to differentiate retinal detachment from tumor based on T2 alone is shown in panel D. Abbreviation: RD = retinal detachment.
As the local SD can depend on the type of tissue adjacent to the tumor, for example, vitreous or retinal detachment, the observer variation was compared for 4 separate regions (Fig. 5). A paired t test was used to compare the sequence dependent median local SD per patient for the 4 separate regions (Fig. 5). A significant lower local SD was found at the vitreous interface on T2 (T1gd, 0.39 mm vs T2, 0.24 mm; P < .001) and a similar trend for the tumor-choroid interface (T1gd, 0.62 mm vs T2, 0.52 mm; P = .08). This was in line with the 75th percentile of the local SD distribution for vitreous (T1gd, 0.49 mm vs T2, 0.34 mm; P = .005) and tumor-choroid interface (T1gd, 0.90 mm vs T2, 0.71 mm; P = .15). In line with the observations in Fig. 4, significant higher variations were found at the tumor-choroid interface compared with sclera, vitreous, and retinal detachment (sclera-tumor-choroid interface: T1gd, P = .008 and T2, P = .004; vitreous-tumor-choroid interface: T1gd, P = <0.001 and T2, P = .002; vitreous-retinal detachment: T1gd, P = .03 and T2, P = .001) (Fig. 5 A, C).
Discussion
We found in general an interobserver variation of 0.4 mm, which is essential information for the determination of the margin for MRI-based radiation therapy planning of intraocular tumors. This variation in significantly smaller than for other malignancies, such as pancreatic, prostate, and recurrent gynecological cancer, where observer variations in the order of 2 to 10 mm are commonly found.
An inter-observer study to determine radiotherapy planning target volumes for recurrent gynaecological cancer comparing magnetic resonance imaging only with computed tomography-magnetic resonance imaging.
The higher agreement between observers found in this study is likely the result of relatively high resolution of the MRIs, which is also needed for a small target organ, such as the eye.
Potentially this variation can be reduced even further, for instance by improving the delineation instructions based on the results of this study.
The observer variation of 0.4 mm was approximately half of the acquisition voxel size (0.8 mm isotropic) and in line with previous eye segmentation studies showing a segmentation reproducibility of less than 1 voxel.
In addition to the uncertainties in the model based tumor definition, this margin is used to account for the variation in patient set up between fractions, movement of the eye during treatment, and beam characteristics.
As no prior publications were found on the observer variation of the GTV delineation in UM on high resolution MRI scans, the observer variation was compared with uncertainties in conventional measurements for ocular PT planning. Currently GTV definition of conventional ocular PT planning is based on the distance between tumor and tantalum markers in combination with tumor prominence and largest basal diameter measurements on ultrasound and fundoscopy. Studies demonstrated an observer variation (SD) with B-scan ultrasonography for prominence measurement of 0.6 mm
Even though these uncertainties and variations are not directly comparable as they involve different types of measures, it indicates that the observer variation observed on MRI might be similar or smaller than the current standard. Comparison with CT, which is commonly used for ocular stereotactic radiosurgery,
was not possible as no literature was found on the observer variation of intraocular GTV delineation with CT.
We showed that the observer variation depends on the type of tissue adjacent to the tumor, with the lowest variation at the tumor-vitreous interface. The highest variation was observed at the tumor-choroid interface, especially on T1gd as there was no agreement on whether choroidal enhancement should be included in the GTV. Although more clear instructions on whether or not to consider this choroidal enhancement as tumor will likely reduce the observer variation, it is more important to know if this enhancement contains tumor cells. Unfortunately, no histopathologic validation was found in the literature. On T2, the choroidal enhancing area is generally hyperintense compared with the tumor and isointense compared with vitreous and was therefore not included in the GTV. This is the primary source of the volume differences between T1gd and T2 segmented GTVs. Out of concern for choroidal microinvasion, we recommend including choroidal enhancement in the GTV until proven otherwise by histopathology. Clarity about the etiology of enhancement and reduction of the observer variability around the tumor edge is most important for tumors located in close proximity of OARs such as macula and optic nerve. This is also relevant for application of MRI-based tumor models outside ocular PT such as treatment decision making, brachytherapy planning, or follow-up after treatment. Especially for brachytherapy, where the base of the tumor determines the size of the brachytherapy applicator, it is important to have an accurate and reliable determination of the tumor base. Additionally, fundus imaging can be used for verification of the tumor base. However, it should be noted that optical aberrations and mismatches in the fusion are potential additional sources of errors.
Jaarsma-Coes MG, Ferreira TA, Marinkovic M, et al. Comparison of MRI-based and conventional measurements for proton beam therapy of uveal melanoma. [e-pub ahead of print]. Ophthalmol Retin.https://doi.org/10.1016/j.oret.2022.06.019, accessed 09 January 2023.
Unexpectedly, no difference in interobserver variability was found between T1gd and T2 at the retinal detachment interface. This might be due to the limited sample size, especially of the patients with retinal detachment. However, inclusion of a small retinal detachment on T2-weighted images in patient 8 most likely led to overestimation of tumor volume. Additionally, in rare cases, a hemorrhagic retinal detachment can be difficult to distinguish from tumor when only using T1gd weighted images.
This underlines the importance of the use of high-resolution scans and comparison of different MRI sequences for tissue characterization, as it allows for an accurate discrimination between both components.
Diagnostic methods and therapeutic options of uveal melanoma with emphasis on MR imaging—Part I: MR imaging with pathologic correlation and technical considerations.
Compared with the tumor-choroid interface and retinal detachment, the base of the tumor had a relatively low observer variation. Nonetheless, enhancing muscle insertions might result in increased variation locally when mistaken for tumor. Finally, extra care should also be taken in case of flat UM or tumors with flat extensions as these can be missed on MRI.
Even though T2 has a slightly, not significant, lower observer variation compared with T1gd, we recommend delineation on T1gd as enhancement might represent tumor invasion which might be missed on T2. Moreover, differentiation between tumor and retinal detachment might be more difficult on T2. However, to achieve the most accurate GTV delineation, it is important to use the multiple scan sequences for tissue differentiation and to choose the least affected sequence in case of motion artifacts. The determined observer variation aids in establishing the margin for MRI-based ocular PT treatment planning. Furthermore, the regional SD differences might lead to different treatment beam design strategies by, for example, preferring the major axis of the tumor to be in the anterior-posterior direction. Outside PT, MRI-based tumor models might be used for treatment decision making, brachytherapy planning, and follow-up after treatment.
Although this study assessed the observer variation for MRI alone, in clinical practice, the strengths of different modalities will be combined to determine the GTV. Therefore, the final variation will depend on the variation of each of these modalities and on how accurately these can be combined. To prevent the confounding effect of errors in the registration of different modalities, for example, fundus photography and MRI, we based these results on MRI-data alone and used the median GTV as a ground truth, similar to other studies.
An inter-observer study to determine radiotherapy planning target volumes for recurrent gynaecological cancer comparing magnetic resonance imaging only with computed tomography-magnetic resonance imaging.
However, to use this information in clinical practice, it would be valuable to also determine the uncertainties in the other steps in ocular proton therapy, for example, variation in patient setup, so a more modern margin recipe for ocular PT can be developed instead of the currently used gross margin of 2.5 mm.
In this context, a study on the use of MRI to define the OAR would also be relevant, as their location and extend are currently approximated. Finally, although the results of this study contribute to a marker less GTV definition in ocular PT, in the current praxis, markers will still be required for the accurate positioning the tumor with respect to the proton beam.
Conclusions
The interobserver variation of 0.4 mm on MRI is low with respect to the voxel size and currently used treatment margins in ocular PT. Higher interobserver variation was found at the tumor-choroid interface due to unspecified enhancement. This localized increase in variation might be reduced by additional guidelines and training. We prefer to delineate based on the T1gd because of clearer tissue margins and assume peritumoral enhancement might represent tumor invasion.
Jaarsma-Coes MG, Ferreira TA, Marinkovic M, et al. Comparison of MRI-based and conventional measurements for proton beam therapy of uveal melanoma. [e-pub ahead of print]. Ophthalmol Retin.https://doi.org/10.1016/j.oret.2022.06.019, accessed 09 January 2023.
Measuring the human retinal oxygenation response to a hyperoxic challenge using MRI: Eliminating blinking artifacts and demonstrating proof of concept.
An inter-observer study to determine radiotherapy planning target volumes for recurrent gynaecological cancer comparing magnetic resonance imaging only with computed tomography-magnetic resonance imaging.
Diagnostic methods and therapeutic options of uveal melanoma with emphasis on MR imaging—Part I: MR imaging with pathologic correlation and technical considerations.
Sources of support: This work was supported by the Netherlands Organization for Scientific Research (NWO) [protons4vision 14654].
Disclosures: We received research support from Philips Health care. They had no role in the design of the study; in the collection, analyses, nor interpretation of data; in the writing of the manuscript; nor in the decision to publish the results.
The data sets generated during the study are not publicly available as they contains potentially identifying information. The data are available from the corresponding author on reasonable request and after the signing of a data transfer agreement.
The RD was only adjacent to the gross target volume at the most inferior edge (out of plane) and therefore not included in the region analysis.Prominence and LBD as measured on ultrasound. SI as described by the radiologist. RD as described by the radiologist.
00255-X&id=gr1.jpg){kind=link}
00255-X&id=gr2.jpg){kind=link}
00255-X&id=gr3.jpg){kind=link}
00255-X&id=gr4.jpg){kind=link}
00255-X&id=gr5.jpg){kind=link}