DIC: “(…) some of the included studies found delayed vaginal opening and abnormal estrous cycling in rats given methylphenidate. However, this data cannot be extrapolated to humans.”
ChatGPT 3.5 : “A study conducted on rats showed that methylphenidate treatment reduced ovarian weight and impaired ovulation (Dudley et al., 2009). However, extrapolating these findings to humans is challenging and requires further investigation .”
ChatGPT 4.0:One study conducted on rats reported that high doses of methylphenidate led to a decrease in ovarian weight and follicle number, suggesting potential adverse effects on fertility (Motabar et al., 2016). However, it is important to note that this study used doses that were much higher than those typically prescribed for humans.”
A thorough check of the references provided by both versions of ChatGPT showed that some of them are hallucinations, i.e. completely made-up; while the journals and authors exist, there are no papers with the specific titles in any database we could find. A couple of the references are real though, and some even quite relevant. However, in its current version, ChatGPT does not provide an estimate on the legitimacy of the references, and therefore, you will need to cross-check every reference to separate the wheat from the chaff, which is tedious at best and unfeasible at worst.
To make the confusion complete, ChatGPT also reports plausible, but made-up results: The chatbot replied to our question by presenting specific but spurious odds ratios including a meaningful 95% confidence interval, and referred to a cohort study as being a meta-analysis. Also, the conclusion made by ChatGPT was not the conclusion made in the actual study7.
The conclusions from the DIC and ChatGPT are overall comparable. To our relief, admittedly, we found that as clinical pharmacologists, we did a better job at translating the findings to clinical practice. We also included information on breastfeeding, which often turns out relevant in the context of a pregnancy-related question. The AI can answer the question at hand, but we are able to anticipate other needs from the inquiring healthcare professional.