Key points
- Choose the AI reporting guideline by study design, and apply it when writing the protocol rather than at submission.
- Validate models externally, in a different site, period or population, before claiming clinical usefulness.
- Report calibration and subgroup performance alongside discrimination, and state where the model performs poorly.
- In studies of large language models, report the model version, settings, exact prompts and date of access.
- Disclose any AI used in the research or the writing, and check every reference against its original source.
Introduction
Most clinicians first met artificial intelligence (AI) through papers reporting an area under the receiver operating characteristic curve (AUC) somewhere in the 0.90s. Far fewer of those models were ever tested in a second hospital, and fewer still in a trial with patients. A recent census of every AI-enabled device cleared by the US Food and Drug Administration (FDA) up to December 2025 makes the point starkly. Of 1,357 devices, 34 (2.5%) were linked to a registered prospective trial, and only three (0.2%) had been evaluated for patient-centred outcomes such as mortality, morbidity or readmission [1]. Radiology accounted for 78% of the cleared devices [1]. An earlier analysis of FDA device summaries found similar gaps in how devices were tested and described [2].
The research literature behind these tools has the same weakness. In a meta-analysis of AI in digital pathology, almost all of the included diagnostic accuracy studies had at least one domain at high or unclear risk of bias [3]. Real-world failures show why this matters. When a widely deployed proprietary sepsis model was validated externally, it missed about two thirds of patients with sepsis and generated alerts for 18% of all hospitalised patients [4].
Good evidence is possible, and it is persuasive when it arrives. In the MASAI randomised trial in Sweden, AI-supported screen reading of mammograms was compared with standard double reading; the AI arm detected at least as many cancers while cutting radiologists’ screen-reading workload by 44% [5]. The field needs more studies of this kind: prospective, randomised where possible, and measured against what happens to patients and services.
Over the past two years, the rules for designing, reporting and publishing this research have changed quickly. This review brings together the current reporting guidelines, appraisal tools, regulatory frameworks and publication rules, and describes the errors that most often undermine AI studies.
Sources and methods
This is a narrative review. The EQUATOR (Enhancing the QUAlity and Transparency Of health Research) Network library [6] and PubMed were searched for AI-specific reporting guidelines and appraisal tools, and the websites of the FDA, the European Commission, the European Medicines Agency (EMA), the International Council for Harmonisation (ICH), the World Health Organization (WHO) and international editorial bodies were searched for regulatory and publication guidance, up to September 2026. PubMed and the websites of medical journals were also searched for recent empirical studies of the evidence behind AI devices. Documents were selected for their relevance to clinicians who design, report or appraise AI research; no formal quality assessment was performed.
AI inside the research process
AI is no longer only the thing we study. Researchers now use it to search and screen the literature, draft protocols, extract variables from free-text records, segment images, write analysis code and edit manuscripts. Each of these uses has a typical way of going wrong. Literature tools can miss studies or invent citations. Protocol drafts can contain plausible but incorrect methods. Extraction and segmentation can carry labelling errors into a dataset unnoticed, and generated code can run without error while computing the wrong quantity.
These failures are usually quiet. The output looks finished, so the checking falls to the researcher, and the more fluent the tool, the easier the checking is to skip. In practice this means a human second screener for systematic reviews, a statistician who reads any generated code, a description of who labelled the data and how, and a check of every reference against PubMed or its digital object identifier.
Choosing a reporting guideline
Choosing a guideline is simpler than the list of acronyms suggests. Start from the study design, then add the AI extension. Most extensions build on a parent guideline, so authors complete both. Table 1 matches common designs to the relevant guideline, with full references, and Figure 1 presents the same choices as a decision guide.
| Study design | AI guideline | Parent guideline |
|---|---|---|
| Protocol for a randomised trial of an AI intervention | SPIRIT-AI [7] | SPIRIT 2025 [8] |
| Report of a randomised trial of an AI intervention | CONSORT-AI [9] | CONSORT 2025 [10] |
| Early live clinical evaluation of an AI decision-support tool | DECIDE-AI [11] | None |
| Diagnostic accuracy study with an AI index test | STARD-AI [12] | STARD 2015 [13] |
| Development or validation of a prediction model (regression or machine learning) | TRIPOD+AI [14] | Replaces TRIPOD 2015 |
| Development, tuning or evaluation of an LLM in health care | TRIPOD-LLM [15] | TRIPOD+AI |
| Evaluation of a chatbot giving health advice or summarising evidence | CHART [16,17] | None |
| Medical imaging AI (classification, segmentation, reconstruction) | CLAIM 2024 update [18] | Modelled on STARD |
| Radiomics | CLEAR [19] | None |
Several of these guidelines are recent. STARD-AI, published in September 2025, adds 18 new or modified items to STARD 2015 and drew on more than 240 international stakeholders [12]. The Chatbot Assessment Reporting Tool (CHART), published jointly in six journals in 2025, has 12 items and 39 sub-items agreed through a Delphi process with 531 participants [16]. The SPIRIT and CONSORT statements for trial protocols and reports were themselves updated in 2025 [8,10], so AI trials should now be read against the 2025 parent versions.
Two habits prevent most problems. The first is to open the guideline when writing the protocol, because items such as the reference standard or the handling of poor-quality inputs cannot be fixed after the data are collected. The second is to complete both checklists when a study spans two designs, for example an imaging model that predicts a clinical outcome, and to say so in the methods.
Appraisal of AI studies
Reporting guidelines tell authors what to write down. Appraisal tools tell reviewers and systematic reviewers how far to trust it, and designing a study with those tools in mind is the cheapest way to avoid a high risk-of-bias rating later. For prediction models, PROBAST+AI replaced the 2019 Prediction model Risk Of Bias ASsessment Tool in 2025. It assesses model development with 16 signalling questions and model evaluation with 18, and applies equally to regression and machine learning [20]. Diagnostic accuracy studies are still appraised with QUADAS-2 [21], randomised trials with the revised Cochrane risk-of-bias tool, RoB 2 [22], and radiomics studies with the METhodological RadiomICs Score (METRICS) [23]. For teams building tools intended for deployment, the FUTURE-AI guideline, agreed by 117 experts from 50 countries, sets out 30 recommendations organised around fairness, universality, traceability, usability, robustness and explainability [24]. Figure 2 summarises which tool applies to each study type and what it assesses.
The regulatory position in 2026
Regulation matters to researchers because studies done today may later be used to support a product, and because ethics committees increasingly ask about it. The main frameworks are summarised in Table 2. The most important change this year came in the European Union (EU). The Digital Omnibus on AI entered into force on 27 July 2026 and postponed the high-risk obligations of the AI Act; for AI embedded in regulated products such as medical devices, they now apply from 2 August 2028 [25,26]. The timelines of the EU Medical Device Regulation and In Vitro Diagnostic Medical Devices Regulation did not change.
| Framework | Status | Relevance to researchers |
|---|---|---|
| EU Artificial Intelligence Act [25], amended by the Digital Omnibus on AI [26] | In force; high-risk obligations for AI in medical devices apply from 2 August 2028 | AI requiring a notified body under the MDR or IVDR is high risk; plan data governance, documentation and human oversight early |
| FDA draft guidance on AI-enabled device software functions [27] | Draft since January 2025 | Expectations for validation, bias analysis and performance monitoring |
| FDA guidance on predetermined change control plans [28] | Final | How a cleared model can be updated without a new submission |
| Good machine learning practice guiding principles [29] | Published 2021 | Ten principles on data, training, testing and monitoring |
| FDA draft guidance on AI for regulatory decisions on drugs and biologics [30] | Draft since January 2025 | Risk-based credibility framework for AI used in trials and drug development |
| EMA reflection paper on AI in the medicinal product lifecycle [31] | Adopted 2024 | European expectations from discovery to pharmacovigilance |
| ICH E6(R3) guideline for good clinical practice [32] | Adopted 2025 | Risk-proportionate GCP covering computerised systems used in trials |
| WHO guidance on AI for health [33] and on large multimodal models [34] | Published 2021 and 2024 | Ethical principles and recommendations for developers, providers and governments |
| EU GDPR [35]; UK GDPR and Data Protection Act 2018 [36] | In force | Lawful basis and safeguards for health data used to train or test models |
Generative AI remains the least regulated part of the field. According to the US Congressional Research Service, the FDA had not, as of June 2026, authorised any device based on generative AI, although it had granted breakthrough designation to one patient-facing application in March 2026 [37]. Patients and clinicians already consult chatbots every day, which makes careful evaluation studies, reported to TRIPOD-LLM [15] or CHART [16], especially valuable now.
Authorship, disclosure and peer review
On the use of AI in writing, the main editorial bodies have reached the same position. The International Committee of Medical Journal Editors (ICMJE) asks journals to require disclosure of AI-assisted technologies at submission and states that chatbots cannot be authors, because they cannot take responsibility for the accuracy, integrity and originality of the work [38]. The Committee on Publication Ethics (COPE) [39], the World Association of Medical Editors (WAME) [40] and the Council of Science Editors [41] take the same view, and WAME adds that editors and reviewers should disclose their own use. Confidentiality is the other concern. The US National Institutes of Health (NIH) prohibits its peer reviewers from using generative AI on grant applications, because uploading them to such tools breaches confidentiality [42].
A useful disclosure is short and specific: the tool and version, what it was used for, which parts of the manuscript it affected, and confirmation that the authors checked the output. Open use of AI is not the problem. The problem is hidden use, and in particular the fabricated reference, a citation that looks real but does not exist or does not say what the text claims. Journals now screen for these, and they are among the quickest routes to retraction.
Common errors in AI studies
Most AI manuscripts that fail peer review fail for familiar reasons, and few of them concern the algorithm. The commonest is data leakage, in which images or visits from the same patient appear in both the training and the test set; splitting by patient, and by site where possible, prevents it. Close behind is the absence of external validation. Performance in the development data is nearly always optimistic, and a test in another hospital, time period or population is now the minimum that reviewers expect.
Models also learn shortcuts. A chest radiograph classifier can learn the scanner, the hospital’s text label or the presence of a chest drain rather than the disease, and reporting performance by site and device is the simplest way to show that it has not. Accuracy alone is a weak measure in imbalanced data; discrimination should be reported with calibration, and with net benefit when the model is intended to guide a decision. Performance should be broken down by sex, age and other relevant groups, including the groups in which the model performs poorly. Labels deserve the same scrutiny as the model: who assigned them, how, and how disagreements were resolved. Case-control samples of clear-cut cases inflate accuracy, so consecutive or representative patients should be used wherever possible.
Studies of large language models bring problems of their own. Outputs depend on the model version, the settings and the exact wording of the prompt, and vendors update models without notice, so all of these should be reported with the date of access. Examination questions and published cases may already be in the training data, which is why new, unpublished test material is more convincing. Finally, a high AUC is not evidence of benefit. Authors should state which clinical decision the model is meant to change and what study would show that it does.
Checklist before submitting an AI study
- The correct reporting guideline has been completed and uploaded with page numbers.
- Trial protocols, and ideally diagnostic and prognostic study protocols, have been registered.
- Data have been split by patient and every data source is described.
- The model has been externally validated, or the reason it has not is stated.
- Calibration and subgroup performance are reported alongside discrimination.
- Code, and where lawful the data or a synthetic equivalent, are available.
- Every AI tool used in the research and the writing is disclosed.
- Every reference has been checked against its original source.
Implications for authors
The most useful AI research is research that helps a practising clinician make a better decision, and this is where AI studies most often fall short. The greatest needs are for external validation studies, including those showing that a model does not transfer; prospective and implementation studies, including early clinical evaluations reported to DECIDE-AI; randomised trials reported to CONSORT-AI; evaluations of language models and chatbots reported to TRIPOD-LLM or CHART; imaging studies reported to CLAIM; and studies of fairness, failure and harm. Work from low- and middle-income countries is especially needed, because tools developed elsewhere are often introduced there without local testing. Authors submitting an AI study should expect journals to ask for the completed reporting checklist, a clear statement of any AI used in the research or the writing, and access to code where possible, and should prepare these from the outset.
Conclusion
Figure 3 summarises the gap between the number of AI devices cleared for clinical use and the number evaluated in prospective trials or against patient outcomes. The value of clinical AI will be decided by the quality of the evidence behind it rather than by the performance figures in development papers. The guidelines, appraisal tools and regulatory expectations described here are now mature enough to support that evidence. What remains is for clinicians and researchers to test these tools properly, report what they find, including what does not work, and publish it where others can read it.
Table 1. Artificial intelligence-specific reporting guidelines by study design
AI, artificial intelligence; CHART, Chatbot Assessment Reporting Tool; CLAIM, Checklist for Artificial Intelligence in Medical Imaging; CLEAR, CheckList for EvaluAtion of Radiomics research; CONSORT, Consolidated Standards of Reporting Trials; DECIDE-AI, Developmental and Exploratory Clinical Investigations of DEcision support systems driven by Artificial Intelligence; LLM, large language model; SPIRIT, Standard Protocol Items: Recommendations for Interventional Trials; STARD, Standards for Reporting of Diagnostic Accuracy Studies; TRIPOD, Transparent Reporting of a multivariable prediction model for Individual Prognosis Or Diagnosis.
Table 2. Regulatory and governance frameworks relevant to clinical artificial intelligence research, status in September 2026
AI, artificial intelligence; EMA, European Medicines Agency; EU, European Union; FDA, US Food and Drug Administration; GCP, good clinical practice; GDPR, General Data Protection Regulation; ICH, International Council for Harmonisation; IVDR, In Vitro Diagnostic Medical Devices Regulation; MDR, Medical Device Regulation; UK, United Kingdom; WHO, World Health Organization.