The incorporation of generative artificial intelligence (AI) and large language models like ChatGPT into research methodologies has elicited both enthusiasm and trepidation. Although the potential for simplifying tasks and boosting efficiency is appealing, the risk of inaccuracies and “hallucinations” presents a substantial hurdle.
In a new study published in Pharmacoeconomics and Policy , a team from Tulane University Celia Scott Weatherhead School of Public Health and Tropical Medicine in New Orleans assessed ChatGPT’s effectiveness in conducting literature reviews.
Employing established criteria from systematic reviews on chronic illnesses, the authors instructed ChatGPT to locate pertinent articles. Notably, the outcomes were troubling. Despite generating a list of article titles, a substantial fraction (42%) was either fictitious or misattributed, underscoring the problem of “hallucinations” in AI-generated content. Only 18% of the citations corresponded to the actual articles included in systematic reviews.
“ChatGPT can be a valuable asset for researchers, but it's not without flaws,” asserts Debra Winberg, lead author of the study. “Our results highlight the constraints of current AI algorithms, and the necessity for researchers to critically assess and validate information produced by AI tools.”
“The swift advancement of AI tools is promising, but we must ensure their responsible and effective use,” adds co-author Dennis Xuan. “Currently, generated information is heavily influenced by the phrasing of the question, the specified criteria and publicly available resources. Hence, healthcare decision and policy makers cannot yet rely on pure generative AI output without knowing whether humans were involved in the entire research process.”
In summary, there appears to exist an ability ceiling above which the current ChatGPT algorithms cannot reach.
###
Contact the author: Dennis Xuan, Tulane University, New Orleans, USA , dxuan@tulane.edu
The publisher KeAi was established by Elsevier and China Science Publishing & Media Ltd to unfold quality research globally. In 2013, our focus shifted to open access publishing. We now proudly publish more than 200 world-class, open access, English language journals, spanning all scientific disciplines. Many of these are titles we publish in partnership with prestigious societies and academic institutions, such as the National Natural Science Foundation of China (NSFC).
Pharmacoeconomics and Policy
Literature review
Not applicable
Assessing the suitability of generative AI in the execution of literature retrieval within literature reviews
The authors declare that there are no conflicts of interest.