Protecting scientific integrity in the era of AI
The John Duckett Memorial Lecture examines the growing risks of AI-generated research and the urgent need for human oversight in medical publishing.
The pace of medical publication is accelerating, and the doubling time of medical knowledge is shrinking. In 1950, the doubling time of medical knowledge was 50 years. The pace in 2020 was just 73 days.
“There is an explosion of publication of manuscripts,” said Anthony Herndon, MD, professor of surgery/urology and chief of pediatric urology at Virginia Commonwealth University, and editor-in-chief of the Journal of Pediatric Urology.
“This preceded ChatGPT in 2022 by the emergence of predatory journals, but after ChatGPT, there was a further explosion,” he said. “In 2025, there were close to four million manuscripts published. Artificial intelligence (AI) has leveled the playing field, created ghost citations and manufactured data. Language barriers have been removed, peer review has been published under stress and there are concerns about intellectual property.”
Dr. Herndon delivered the 2026 John Duckett Memorial Lecture, “The Role of AI in Scientific Publication: Believe It or Bot,” during the Sunday morning Plenary. Scientific publication may not be on the brink of collapse, but there is a pressing need for human intervention to take back control of scientific research, writing, review and publication.
A recent survey found that researchers themselves recognize the risks posed by AI. Nearly half of 5,000 researchers surveyed (47%) were reluctant to use AI without clear guidelines, and 15% were opposed to using AI. The remaining 38% supported the use of AI with guidelines.
Ethical standards in research have not changed, Dr. Hendron said. Transparent, descriptive research remains the gold standard. Use of AI for manuscript editing with disclosure is broadly accepted, but there is a strong consensus against the use of AI for peer review.
The publishing world has adopted broad guidelines governing the use of AI. The World Association of Medical Editors (WAME), International Committee of Medical Journal Editors (ICMJE) and the Committee on Publication Ethics (COPE) agree that AI cannot be listed as an author because it fails the basic obligations of authorship. Nor can AI give final approval to a manuscript or review a manuscript. Human oversight is necessary, and all stakeholders, authors, editors and reviewers must be disclosed and described in the methods section.
COPE holds authors, editors and reviewers accountable for AI errors. And because confidentiality is non-negotiable in scientific publishing, COPE explicitly forbids the uploading of manuscripts to AI for peer review.
“We have threats to scientific integrity,” Dr. Hendron said. “LLMs (large language models) and the algorithms they create keep referencing themselves. You get scope contraction, a vicious cycle of algorithms referencing themselves and creating ghost citations. AI-adapted research tends to keep citing itself, so you have a contraction of research. You have AI hallucinations that generate content that’s plausible but may not be factually correct. And you have the opportunity for intentional misuse.”
Still, AI is alluring. A recent analysis of over 42 million publications across three decades, segmented by AI-adapted research and human-only research, revealed that AI-adapted research generated 100% more subsequent citations and threefold more published papers. AI-adapted research groups were larger than human-only research, with a 5% contraction in the scope of knowledge cited. Most citations (80%) were to the same top 22% of papers. Researchers who adopted AI advanced their career tracks by 1.3 years.
“We have to establish security guidelines within the journals,” Dr. Herndon said. “Royal Society Open Science recently published the “Stockholm Declaration,” a call to action for all researchers to take back the scientific publication space. We have to maintain transparent disclosure, and we must require AI training for reviewers and authors alike. We have the guidelines; we have a working framework. We have to integrate this in a mainstream fashion, maintaining this delicate balance in favor of human oversight.”
