If you don't remember your password, you can reset it by entering your email address and clicking the Reset Password button. You will then receive an email that contains a secure link for resetting your password
If the address matches a valid account an email will be sent to __email__ with instructions for resetting your password
While a rising share of scientific research articles are being published in open access (OA) journals, their impact on resident research in radiation oncology is unknown. Thus, we sought to determine the number, content, and costs of first-author, PubMed-searchable articles radiation oncology residents in the United States (US) published in OA journals in recent years.
Methods and Materials
We built a database of first-author, PubMed-searchable articles published by US radiation oncology residents who graduated between 2015 and 2019. We then classified each journal in which these articles appeared as either OA or non-OA and obtained the current article-processing charge (APC) for each publication that appeared in an OA journal.
Results
The residents in this study published 2637 first-author, PubMed-searchable articles, 555 of which (21.0%) appeared in 138 OA journals. The number of publications in OA journals per resident increased from 0.47 for the class of 2015 to 0.79 for the class of 2019. Publications in OA journals garnered fewer citations than those in non-OA journals (8.9 vs 14.9, P < .01). Furthermore, 90.6% of OA journals levy an APC for original research reports (median, $1896), which is positively correlated with their 2019 impact factor (r = 0.63, P < .01). Aggregate APCs totaled $900,319.21 and appeared to increase over the study period.
Conclusions
The number of first-author, PubMed-searchable articles published by graduating US radiation oncology residents in OA journals rose significantly between 2015 and 2019. To maximize the benefits of OA publishing in the future, US radiation oncology residents will need to ensure that they use vetted OA journals to publish their research findings and avoid predatory journals.
Introduction
A rising share of scientific research articles have been published open access (OA) in recent years, with potentially transformative effects on academic institutions, publishers, the scientific community, and society-at-large. Within biomedicine, a significant proportion of the increase in OA publications appears to be due to the growth of OA journals. In fact, in a 2013 analysis, Kurata et al
estimated that the number of PubMed-searchable biomedical articles available via OA journals increased by over 90% between 2006 and 2010.
This shift has likely been driven by a number of factors. Some authors who choose to publish in OA journals may wish to have their content freely accessible to all readers, while others may reside at institutions with specific OA policies. At the same time, while the intentions of many OA publishers are noble, some commentators have voiced concern that such pay-to-publish practices may provide an economic incentive to publish lower impact research with less rigorous peer review.
In the publish-or-perish culture inherent in much of academic medicine, some researchers may be attracted to the more streamlined publication processes such journals offer.
Scholarly activity is an essential component of radiation oncology residency training in the United States (US).
However, the influence that the emergence of OA journals has had on resident research in radiation oncology is unclear. Thus, in this study, we aimed to characterize the work US radiation oncology residents have published in OA journals, determine its impact, and estimate its costs.
Methods and Materials
We used multiple publicly available sources of information, including the Association of Residents in Radiation Oncology Directory, residency training program and hospital websites, the National Plan and Provider Enumeration System National Provider Identifier Registry, and Medicare Provider Utilization and Payment Data, to build a nearly comprehensive database of US radiation oncology residents who graduated from Accreditation Council for Graduate Medical Education–accredited residency programs between 2015 and 2019. We then searched PubMed to identify the first-author articles for every resident in our database that were published between the start of radiation oncology residency (July 1 of the second postgraduate year) and 3 months after the completion of residency (September 30 of the graduating year). Only articles in which the author's affiliation matched their residency program were included in this analysis. These eligibility criteria have been employed in prior studies to quantify the research productivity of US radiation oncology residents.
Publications were classified as either original research, reviews, case reports, or commentaries. The number of citations each publication received were downloaded from Scopus on October 15, 2020.
If an article was not indexed in Scopus, the number of citations it received was obtained from PubMed.
We extracted journal metadata (including the country of publication, International standard serial number, journal website, publication start year, and publisher) from the National Library of Medicine Catalog. The 2019 impact factor was obtained from the Journal of Citation Reports. We then dichotomously classified each journal in which US radiation oncology residents published as either OA or non-OA by using journal websites and comparing our results with The Directory of Open Access Journals. For the purposes of this study, journals were classified as OA if all of their articles were freely available to the public, without restriction, at the time of publication. To establish an upper bound for the total article-processing charges (APCs) paid by US radiation oncology residency training programs to fund the publication of first-author resident articles in OA journals, we obtained the current APC for each publication in our database via journal websites. APCs listed in currencies other than US dollars were converted into US dollars by using international exchange rates as of August 31, 2020. This study was classified as exempt human research by our institutional review board.
We also evaluated whether several demographic factors, including having completed medical school and residency at the same institution, male sex, PhD status, and the size of the residency training program attended (defined as the total number of residents who graduated between 2015 and 2019), were associated with the likelihood of publishing an article in an OA journal during residency. First, via a multivariate logistic regression, we tested whether these factors predicted whether individuals who published a first-author article during residency published in an OA journal. Second, using a multivariate Poisson model, we tested whether these factors were associated with the proportion of first-author articles individuals published in OA journals. These analyses were performed in R version 4.0.2, and the null hypothesis was rejected for P < .05.
Results
First-author publications in OA journals
We identified 909 US radiation oncology residents from 89 residency training programs who graduated from 2015 to 2019. An additional 6 residency programs did not graduate any residents during this period. This sample represented 99.2% (909/916) of all postgraduate year 5 residents and 100% (95/95) of residency training programs over the study period.
Of the 909 residents identified, 697 (76.7%) published at least 1 first-author, PubMed-searchable article during residency. Three hundred fifty residents (38.5%) published at least 1 first-author, PubMed-searchable article in an OA journal.
Of the 2637 first-author articles published by the US radiation oncology residents in this study, 555 (21.0%) were published in OA journals (Table 1). The number of first-author publications per resident in OA journals increased from 0.47 articles for the graduating class of 2015 to 0.79 articles for the graduating class of 2019 (P < .01). Additionally, the percentage of graduating residents who published a first-author article in an OA journal increased from 32.2% for the graduating class of 2015 to 46.6% for the graduating class of 2019 (P < .01). The proportion of publications in OA journals, however, did not change over the study period (21.8% for the class of 2015 vs 25.2% for the class of 2019, P = .22).
Table 1Graduating US radiation oncology resident demographics, 2015 to 2019
Demographic
All residents
2019 grads
2018 grads
2017 grads
2016 grads
2015 grads
Number of graduating residents
909
191
185
177
182
174
Number of graduating residents with FA publication (%)
697 (76.7%)
150 (78.5%)
147 (79.5%)
140 (79.1%)
134 (73.6%)
126 (72.4%)
Total FA publications
2637
596
645
589
430
377
Total FA publications in OA journals
555
150
130
120
73
82
FA publications in OA journals/resident
0.61
0.79
0.70
0.68
0.40
0.47
Percentage of total FA publications in OA journals
21.0%
25.2%
20.2%
20.4%
17.0%
21.8%
Total article-processing charges
$900,319.21
$233,213.28
$209,990.00
$205,874.44
$123,543.79
$127,697.70
Article-processing charges/resident
$990.45
$1,221.01
$1,135.08
$1,163.13
$678.81
$733.89
Abbreviations: FA = first author; grads = graduates; OA = open access; US = United States.
Publications in OA and non-OA journals differed significantly by type. Publications in OA journals were more likely to consist of reviews (19.1% vs 12.2%; P < .01) and case reports (16.4% vs 3.8%; P < .01) and less likely to consist of original research (57.8% vs 72.8%; P < .01) and commentaries (6.7% vs 11.2%; P < .01). There was a statistically significant correlation between the number of first-author articles published by a given resident and the proportion of those articles that represented original research (r = 0.33; P < .01) and commentaries (r = 0.14; P < .01). The correlation between the number of first-author articles published by a given resident and the proportion of those articles that represented case reports (r = 0.02; P = .57) and reviews (r = 0.06; P = .06), however, was not statistically significant. Publications in OA journals were also significantly less likely to be grant supported than publications in non-OA journals (10.1% vs 22.1%; P < .01).
On average, publications in OA journals garnered 1.5 fewer citations per year after publication (mean, 2.0 vs 3.5; P < .01) and 6 fewer citations overall (mean, 8.9 vs 14.9; P < .01) than their counterparts in non-OA journals (Tables 2 and 3). The difference was primarily driven by original research reports, which garnered 47.4% and 47.6% fewer citations in OA journals per year after publication and overall, respectively, compared with non-OA journals (2.0 vs 3.8, P < .01 and 8.7 vs 16.6, P < .01, respectively). There was no difference (P > .05) in the number of citations, both per year after publication and overall, among case reports, commentaries, and reviews in OA and non-OA journals.
Table 2Mean number of citations garnered by resident first-author publications per year after publication
Among US radiation oncology residents who published a first-author, PubMed-searchable article, none of the demographic variables tested were associated with the likelihood of having published in an OA journal on logistic regression (Table 4). On Poisson regression, residency size was negatively associated with the proportion of publications that residents published in OA journals (incident ratio, 0.97; P < .01).
Table 4Multivariable regression analysis of predictors of publishing a first-author article in an open access journal by graduating US radiation oncology residents, 2015 to 2019
Overall, the first-author articles in this study were published in 392 peer-reviewed journals, 138 of which (35.2%) were OA. Only 99 of the 138 (71.7%) OA journals were listed in The Directory of Open Access Journals. OA journals were significantly more likely to be based outside of the US than their non-OA counterparts (61.6% vs 45.3%; P < .01). The most common countries in which OA journals were based were the US (53 journals), the United Kingdom (38 journals), Switzerland (11 journals), New Zealand (8 journals), and China (7 journals).
The vast majority of OA journals identified in this study began publishing within the last 15 years (Fig. 1). In fact, 119 of the 138 OA journals (86.2%), accounting for 470 of the 555 first-author publications (84.7%) in OA journals, began publishing on or after January 1, 2005. In contrast, 51 of the 254 non-OA journals (20.1%), accounting for 470 of the 2082 first-author publications (22.6%) in non-OA journals, began publishing on or after January 1, 2005. Overall, 50% of all first-author articles (470 of 940) in journals that began publishing on or after January 1, 2005, were published in OA journals. Six journals in this study have ceased publication per the National Library of Medicine Catalog, 5 of which were OA (3.6% of OA journals). All 6 of these journals began publishing on or after January 1, 2005.
Figure 1The number of journals in which first-author, PubMed-searchable articles by radiation oncology residents in the United States appeared by the year they first began publishing.
The most common OA journals in which US radiation oncology residents published were Advances in Radiation Oncology (72 publications, 13.0% of OA publications, publication start year 2015), Frontiers in Oncology (52 publications, 9.4% of OA publications, publication start year 2011), Cureus (40 publications, 7.2% of OA publications, publication start year 2009), Radiation Oncology (29 publications, 5.2% of OA publications, publication start year 2006), and the Journal of Gastrointestinal Oncology (28 publications, 5.0% of OA publications, publication start year 2010). The median 2019 impact factor of the OA journals in which US radiation oncology residents published was 1.58 (range, 0.00-12.12; interquartile range, 0.00-3.30); 61 OA journals (44.2% of all OA journals) lacked a 2019 impact factor. The APCs OA journals assessed for original research reports were highly and positively correlated (r = 0.63; P < .01) with their 2019 impact factor (Fig. 2). In other words, OA journals that had higher impact factors tended to charge more to publish original research reports.
Figure 2Relationship between article-processing charges for original research reports and the 2019 impact factor of the open access journals in which radiation oncology residents in the United States published.
Of the 138 OA journals in which US radiation oncology residents published, 125 (90.6%) assessed an APC for original research reports at the time of this analysis. The median APC for original research reports in these journals was $1896 (mean, $1885.41; range, $238.61-$6000.00; interquartile range, $1190.00-$2400.00). The journals with the highest APCs for original research reports were Dermatology and Therapy ($6000), Nature Communications ($5380), The Journal of Clinical Investigation ($4650), Genome Medicine ($4290), and the Journal of Gynecologic Oncology ($3900).
APCs for resident research in OA journals appear to have risen significantly over the study period (Table 1). Estimated APCs for members of the class of 2015 totaled $127,697.70, while APCs for the members of the class of 2019 totaled $233,213.28, an increase of 82.6%. Likewise, the APCs per resident increased from $733.89 for members of the class of 2015 to $1221.01 for members of the class of 2019, an increase of 66.4%. Aggregate APCs totaled $900,319.21 for all members of the US radiation oncology residency classes of 2015 to 2019.
Discussion
Over the past 2 decades, the research productivity of US radiation oncology residents has been increasing. In fact, the average number of first-author, PubMed-searchable articles published by US radiation oncology residents has risen from 1.01 articles per resident from 2002 to 2007 to 2.90 articles per resident from 2015 to 2019, an increase of 187%. Even between 2015 and 2019, the average number of first-author, PubMed-searchable articles published by US radiation oncology residents rose from 2.17 articles per resident to 3.12 articles per resident, an increase of 44%.
Several factors may be contributing to this increase in research productivity among US radiation oncology residents. The quality of incoming radiation oncology residents, for example, as measured by their academic and research achievements in medical school, increased over the past decade.
Additionally, most radiation oncology residency programs now provide their residents with several months of dedicated research time, which may result in increased numbers of first-author publications.
More broadly, an increasing number of medical journals, partly driven by the growth of the OA publishing model, may be providing US radiation oncology residents with more opportunities to submit and publish their work.
In fact, in this study, we found that the number of first-author, PubMed-searchable articles per resident in OA journals rose from 0.47 articles for the graduating class of 2015 to 0.79 articles for the graduating class of 2019, an increase of 68.1%. This increase (0.32 articles per resident) accounted for 33.7% of the overall increase in first-author, PubMed-searchable articles published per resident (0.95 articles per resident) between the US radiation oncology residency classes of 2015 and 2019.
To the best of our knowledge, this is the largest analysis of the resident research publications of members of an entire medical specialty in OA journals to date. We found that 21.0% of the first-author, PubMed-searchable articles published by US radiation oncology residents who graduated between 2015 and 2019 appeared in OA journals. The vast majority of OA journals identified in this study began publishing on or after January 1, 2005, and levy an APC for original research reports. Additionally, APCs for the articles in our database increased by 82.6% between 2015 and 2019, amounting to $900,319.21 over the study period.
While we focused on radiation oncology residents, there are reasons to believe that our results are at least partially generalizable to medical residents as a whole. In fact, the emergence of OA journals in radiation oncology is not an isolated phenomenon. Over the past 2 decades, OA journals have become increasingly common throughout biomedicine.
Their emergence, however, has not been without controversy, as some have expressed concern about the rising number of predatory journals and the increasing share of research budgets spent on OA publications. At the same time, others have argued that the rise of OA journals may assist in improving the visibility of scholarly research, disseminating scientific knowledge, and fostering global scientific collaboration by increasing the accessibility of scientific research.
These features can yield real benefits. For example, as COVID-19 first took hold in the US, the OA journal Advances in Radiation Oncology rapidly shared the experiences of academic medical centers in viral hotspots and disseminated best practices to the international radiation oncology community.
Moreover, for radiation oncology residents, the proliferation of OA journals in recent years has had a democratizing effect, allowing for greater opportunities to publish research that in the past may not have found a place in the literature. To maintain these benefits, while minimizing harms, in the future, residents will need to ensure that they use vetted OA journals to publish their findings and avoid predatory journals.
There are several limitations to this study. We did not include articles that were published OA in hybrid journals, thus precluding a broader examination of the impact of the OA publishing model in radiation oncology. Additionally, some journals may have switched their publishing models over the course of the study period. Given our methodology, we would have been unable to accurately detect such transitions.
Moreover, the cost figures in this study are not meant to be exact estimates. APCs, current as of August 31, 2020, may have changed over the course of the study period. Additionally, we were unable to account for articles whose APCs were waived by the publishing journal. Thus, the APCs used to calculate costs in this study may not necessarily reflect the precise costs incurred by residency training programs to publish the scholarly work of their residents.
Lastly, the criteria for inclusion of first-author publications in this study, which required publications to have been published within 3 months of the completion of residency, may have failed to capture all research completed during residency due to delays in the publication process. Moreover, the number of first-author, PubMed-searchable publications does not necessarily represent the totality of a resident's research output during residency.
The strength of this study lies in the near complete accounting of US radiation oncology residents, and their publications, that we were able to assemble in our database. As noted, we were able to identify the first-author, PubMed-searchable publications of nearly all (99.2%) US radiation oncology residents who graduated between 2015 and 2019. As a result, we believe that this study represents the most comprehensive examination of the influence OA journals have had on resident research in an entire medical specialty to date.
Conclusion
The number of first-author, PubMed-searchable articles published by graduating US radiation oncology residents in OA journals rose significantly between 2015 and 2019. To maximize the benefits of OA publishing in the future, US radiation oncology residents will need to ensure that they use vetted OA journals to publish their research findings and avoid predatory journals.
References
Kurata K
Morioka T
Yokoi K
Matsubayashi M.
Remarkable growth of open access in the biomedical field: Analysis of PubMed articles from 2006 to 2010.
Some of the data reported in this article was presented in abstract form at the 2021 American Society for Radiation Oncology Annual Meeting in Chicago, Illinois (October 24-27, 2021).
Sources of support: This work had no specific funding.
Disclosures: The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.
Research data are stored in an institutional repository and will be shared upon reasonable request to the corresponding author.
Logistic regression for the likelihood of publishing at least 1 open access article.
00227-5&id=gr1.jpg){kind=link}
00227-5&id=gr2.jpg){kind=link}