Quick Buzz Feed
AI is making science easier to access but harder to trust. Scientific literacy must catch up, writes Giulia Stefenelli
AI creates two points of vulnerability when it comes to trust. Firstly, it can be used to create or manipulate research components that may later enter the scientific record. Secondly, it can distort genuine research when findings are selectively presented, summarised or explained to readers. The first problem concerns the integrity of the scientific record, the second, whether existing research is represented accurately and in context. Recent studies show both the scale of the challenge and the possibility of addressing it. A study published by Asai et al. (2026) found that GPT-4o, tested without external retrieval, fabricated citations in 78–90% of cases when asked to cite recent scientific literature. Yet OpenScholar, a specialized AI system that retrieves information from millions of open-access papers, achieved citation accuracy comparable to human experts. Another study published by Topaz et al. (2026) investigated 2.5 million biomedical papers in PubMed Central’s Open Access collection. It reported a rise in the rate of suspected fabricated references between 2023 and early 2026, demonstrating that misleading citations can pass existing editorial checks and enter the scientific record. Together, these findings point to a crucial distinction; AI can widen access to scientific knowledge, but its value depends on whether the information it provides can be traced to authentic sources and checked for accuracy.
Readers and editorial systems often rely on familiar signals of credibility. This includes manuscript structure, methodological detail, institutional affiliations, citations, polished figures and consistency between sections. AI tools can reproduce these features because they are trained on extensive collections of scientific and academic text. However, reproducing the patterns of scientific writing does not demonstrate that the underlying research occurred. AI can also generate or manipulate several research components together, including text, images, graphs, datasets, statistical results and citations. A fabricated dataset may appear to support a fabricated figure, which in turn appears to support a fabricated conclusion. This internal consistency can make misleading material harder to detect, but it is not evidence of authenticity. Reliable research requires authentic data, appropriate methods and analyses, transparent reporting, critical scrutiny and identifiable people who remain accountable for the work. Fabrication is not the only risk. AI-generated content may cite genuine studies while misrepresenting what they show. Imagine a reader asking whether a supplement prevents dementia. An AI summary might cite an observational study linking supplement use to lower dementia rates, but present that association as evidence of prevention. The reference is real, yet the explanation turns an association into a causal claim. Simply checking that the citation exists would not catch the distortion. AI may also overlook methodological weaknesses, present preliminary findings as established knowledge or combine conflicting evidence into a false impression of consensus. Clear language and genuine references can still give an inaccurate account of the evidence. Scientific literacy must therefore include questions such as: Where did this claim originate? What kind of evidence supports it? Has it been peer reviewed? What uncertainties or limitations remain? Has the research been corrected or retracted? Has AI transformed or selectively presented the original information? Peer review, reproducible analysis and transparent reporting strengthen credibility, but none guarantees that every conclusion is correct. This does not mean every reader must become a methodological expert. It means the origin, status and limitations of research should be visible and understandable, including when scientific findings are presented through AI.
Detecting whether a passage was probably generated by AI is not the same as verifying whether the research is reliable. A manuscript may contain AI-assisted language yet report sound research. By contrast, a text written entirely by a person may contain fabricated data, inappropriate methods or references that do not support its claims. AI detection is probabilistic and can generate false positives. Verification focuses on whether the study took place, the data are authentic, the methods are appropriate and the conclusions follow from the results. Editorial checks should therefore not be built primarily around identifying AI-generated text. They should strengthen the verification of the research components that matter, regardless of how the text was produced. AI can assist with targeted integrity checks, but people must remain accountable for decisions affecting publication and correction of the record.
To expect readers to be able to validate content independently is unrealistic. Verification requires access to sources, methodological knowledge, time and often specialist tools and knowledge. As AI makes scientific-sounding content faster and cheaper to produce, the cost of verification does not fall at the same rate. This creates a growing burden for editors, reviewers, researchers and readers. Review by subject-matter experts remains essential, but expertise is scarce and reviewer capacity is already limited. Expecting reviewers to identify every fabricated citation, manipulated image, unreliable dataset or invalid analysis without appropriate support is not feasible. Equally, more intensive screening at the submission stage is also not straightforward. Stronger editorial checks can help prevent unreliable research from being published, but poorly designed or overly burdensome checks may also block legitimate research. Furthermore, automated tools may generate false positives, while additional requirements can place excessive burdens on authors and research institutions. Verification also has an operational cost. It requires investment in specialist staff, infrastructure and editorial expertise as submission volumes and manuscript complexity increase. Higher publication volumes do not, in themselves, indicate weaker standards, but they increase the importance of demonstrating that editorial capacity and safeguards are developing accordingly. Publishers must balance efficiency and timely decisions with the scrutiny needed to protect the published record. Misconduct prevention cannot rely on detection tools alone. As manipulation techniques become more sophisticated, individual checks may quickly become outdated or easier to evade. These checks should be supported by clear information about the provenance, status and history of research, and its underlying data.
Academic publishers are key to maintaining the scientific record. They are responsible for establishing robust editorial and peer-review processes and for supporting post-publication scrutiny. They must also maintain a transparent record by setting clear expectations for AI use. This includes AI-disclosure and providing accurate information on article type, peer-review status, data availability, funding, conflicts of interest, corrections, retractions and version history. Information should be easy for readers to find and machine-readable to support accurate interpretation by AI systems. Publishers must also determine which checks are needed and when. Checks should reflect the risks involved, with closer examination of references, images or data, or specialist integrity review where needed. Open access can widen participation in post-publication scrutiny, and open peer review can make evaluation more visible through published review reports and, where appropriate and with consent, reviewer identities. However, transparency does not always require disclosing identities. Confidentiality and anonymity may remain important for protecting independent judgment. Ultimately, publishers cannot achieve trust in science alone. Responsibility is shared, but it is not interchangeable. Researchers remain responsible and accountable for the authenticity of their work and any use of AI within it. Research institutions must support them through oversight, data management, training, clear AI-use policies and effective procedures for investigating concerns. AI providers are responsible for how their systems retrieve, transform and represent research. They should link claims to reliable, identifiable sources, distinguish evidence types and publication status, communicate uncertainty and reflect available corrections or retractions. They should also distinguish research findings from AI-added interpretation. Scientific literacy requires informed judgment, but readers cannot exercise it well when the evidence behind a claim is hidden or misrepresented. Publishers and AI providers should make that evidence easier to inspect, its limitations easier to understand and corrections harder to miss. The goal is not to make scientific information sound trustworthy. It is to give people a sound basis for deciding how much confidence it deserves. Dr Giulia Stefenelli is Scientific Communications Lead at MDPI AG