Wikipedia:Identifying reliable sources (medicine)

Source: Wikipedia, the free encyclopedia.

identifying reliable sources

Ideal sources for biomedical information include: review articles (especially systematic reviews) published in reputable medical journals; academic and professional books written by experts in the relevant fields and from respected publishers; and guidelines or position statements from national or international expert bodies. Primary sources should generally not be used for medical content, as such sources often include unreliable or preliminary information; for example, early lab results which don't hold in later clinical trials.

See the reliable sources noticeboard for questions about reliability of specific sources, and feel free to ask at WikiProjects such as WikiProject Medicine and WikiProject Pharmacology.


Types of sources

In the biomedical literature:

  • A primary source is one in which the authors directly participated in the research and documented their personal experiences. They examined the patients, injected the rats, ran the experiments, or supervised those who did. Many papers published in medical journals are primary sources for facts about the research and discoveries made.
  • A secondary source summarizes one or more primary or secondary sources to provide an overview of current understanding of the topic, to make recommendations, or to combine results of several studies. Examples include literature reviews or systematic reviews found in medical journals, specialist academic or professional books, and medical guidelines or position statements published by major health organizations.
  • A tertiary source summarizes a range of secondary sources. Undergraduate or graduate level textbooks, edited scientific books, lay scientific books, and encyclopedias are tertiary sources.

Biomedical v. general information

Biomedical information requires sourcing that complies with this guideline, whereas general information in the same article may not.

For example, an article on Dr Foster's Magic Purple Pills could contain both biomedical and non-biomedical claims:

  • Dr Foster's pills cure everything. A biomedical claim! Strong MEDRS sourcing is definitely required here (see
  • The pills were invented by Dr Archibald Foster and released onto the market in 2015. This is
  • They are purple and triangular, packaged one to a box,[citation needed] as no-one ever manages to swallow a second one.[medical citation needed]

Basic advice

Avoid primary sources

Per the Wikipedia policies of


Primary sources should NOT normally be used as a basis for biomedical content. This is because primary biomedical literature is exploratory and often not reliable (any given primary source may be contradicted by another). Any text that relies on primary sources should usually have minimal


Respect secondary sources

Primary sources should not be cited with intent of "debunking", contradicting, or countering conclusions made by secondary sources.

original research, and Wikipedia is not a venue for open research
. Controversies or uncertainties in medicine should be supported by reliable secondary sources describing the varying viewpoints. Primary sources should not be aggregated or presented without context in order to undermine proportionate representation of opinion in a field. If material can be supported by either primary or secondary sources – the secondary sources should be used. Primary sources may be presented together with secondary sources.

Findings are often touted in the popular press as soon as primary research is reported, before the scientific community has analyzed and commented on the results. Therefore, such sources should generally be omitted (see

randomized clinical trials
with surprising results), they should be described appropriately as from a single study:

"A large, NIH-funded study published in 2010 found that selenium and Vitamin E supplements, separately as well as together, did not decrease the risk of getting prostate cancer and that vitamin E may increase the risk; they were previously thought to prevent prostate cancer." (citing


Given time a review will be published, and the primary sources should preferably be replaced with the review. Using secondary sources then allows facts to be stated with greater reliability:

"Neither vitamin E nor selenium decreases the risk of prostate cancer and vitamin E may increase it." (citing


If no reviews on the subject are published in a reasonable amount of time, then the content and primary source should be removed.

A reason to avoid primary sources in the biomedical field – especially papers reporting results of in vitro experiments – is that they are often not replicable[1][2][3] (see also replication crisis) and are therefore unsuitable for use in generating encyclopedic, reliable biomedical content. Scientists at Bayer reported in 2011 that they were able to replicate results in only ~20 to 25% of prominent studies they examined;[4] scientists from Amgen followed with a 2012 publication showing that they were only able to replicate 6 (11%) of 53 high-impact publications and called for higher standards in scientific publishing.[5] Further, the fact that a claim is published in a refereed journal need not make it true. Even well-designed randomized experiments will occasionally produce spurious results. Experiments and studies can produce flawed results or even fall victim to deliberate fraud (e.g. the Retracted article on dopaminergic neurotoxicity of MDMA and the Schön scandal.)

Summarize scientific consensus

Martin Rimm.) Be careful of material published in disreputable journals or disreputable fields. (See: Sokal affair

Wikipedia policies on the

WHO), in textbooks, or in scholarly monographs
. Although significant-minority views are welcome in Wikipedia, such views must be presented in the context of their acceptance by experts in the field. Additionally, the views of tiny minorities need not be reported.

Finally, make readers aware of controversies that are stated in reliable sources. A well-referenced article will point to specific journal articles or specific theories proposed by specific researchers.

Assess evidence quality

When writing about any


The best evidence for efficacy of treatments and other health interventions is mainly from

meta-analyses of randomized controlled trials (RCTs).[10] Systematic reviews of literature that include non-randomized studies are less reliable.[11]
Narrative reviews can help establish the context of evidence quality.

Lower levels of evidence in medical research come from primary studies (see

case control studies; cross-sectional studies (surveys), and other correlation studies such as ecological studies; other retrospective analyses (including retrospective cohort studies); and non-evidence-based expert opinion or clinical experience. Case reports and series are especially avoided, as they are uncontrolled

Speculative proposals and early-stage research should not be cited to imply wide acceptance. For example, results of an early-stage clinical trial
would not be appropriate in the 'Treatment' section of a disease article because future treatments have little bearing on current practice. The results might – in some cases – be appropriate for inclusion in an article specifically dedicated to the treatment in question or to the researchers or businesses involved in it. Such information, particularly when citing secondary sources, may be appropriate in research sections of disease articles. To prevent misunderstanding, the text should clearly identify the level of research cited (e.g., "first-in-human safety testing").

Several formal systems exist for assessing the quality of available evidence on medical subjects.[12][13] Here, "assess evidence quality" essentially means editors should determine the appropriate type of source and quality of publication. Respect the levels of evidence: Do not reject a higher-level source (e.g., a meta-analysis) in favor of a lower one (e.g., any primary source) because of personal objections to the inclusion criteria, references, funding sources, or conclusions in the higher-level source. Editors should not perform detailed academic peer review.

Avoid over-emphasizing single studies, particularly in vitro or animal studies

In vitro studies and animal models serve a central role in research, and are invaluable in determining mechanistic pathways and generating hypotheses. However, in vitro and animal-model findings do not translate consistently into clinical effects in human beings. Where in vitro and animal-model data are cited on Wikipedia, it should be clear to the reader that the data are pre-clinical, and the article text should avoid stating or implying that reported findings hold true in humans. The level of support for a hypothesis should be evident to a reader.

Using small-scale, single studies makes for weak evidence, and allows for

cherry picking
of data. Studies cited or mentioned in Wikipedia should be put in context by using high-quality secondary sources rather than by using the primary sources.

Use up-to-date evidence

Keeping an article up-to-date while maintaining the more-important goal of reliability is important. These instructions are appropriate for actively researched areas with many primary sources and several reviews, and may need to be relaxed in areas where little progress is being made or where few reviews are published.

  • In many topics, a review that was conducted more than five or so years ago will have been superseded by more up-to-date ones, and editors should try to find those newer sources, to determine whether the expert opinion has changed since the older sources were written. The range of reviews you examine should be wide enough to catch at least one full review cycle, containing newer reviews written and published in the light of older ones and of more-recent primary studies.
  • Assessing reviews may be difficult. While the most-recent reviews include later research results, this does not automatically give more weight to the most recent review (see recentism).
  • Prefer recent reviews to older primary sources on the same topic. If recent reviews do not mention an older primary source, the older source is dubious. Conversely, an older primary source that is seminal, replicated, and often-cited may be mentioned in the main text in a context established by reviews. For instance, the article Genetics could mention Darwin's 1859 book On the Origin of Species as part of a discussion supported by recent reviews.

There are exceptions to these rules of thumb:

  • History sections often cite older work.
  • Cochrane Library reviews and NICE guidelines are generally of high quality and are periodically re-examined even if their initial publication dates fall outside the 5-year window.
  • A newer source which is of lower quality does not supersede an older source of higher quality.

Use independent sources

Many treatments or proposed treatments lack good research into their efficacy and safety. In such cases, reliable sources may be difficult to find, while unreliable sources are readily available. When writing about medical claims not supported by mainstream research, it is vital that

enough to have its own article or relevant for mention in other articles.

Symposia and supplements to academic journals are commonly sponsored by industry groups with a financial interest in the outcome of the research reported. They may lack independent editorial oversight and peer review with no supervision of content by the parent journal.[14] Such articles do not share the reliability of their parent journal.[15] Indications that an article was published in a supplement may be fairly subtle; for instance, a letter "s" added to a page number,[16] or "Suppl." in a reference.


Bias caused by conflicts of interest is an important issue in medical research. It arises in part due to financial interests that compete within medicine. Disclosure of conflicts of interest is mandated, but isn't always done – and, even when it is, it may not be helpful. A source can also simply be bad, where biases in criteria make it less than ideal. Claims of bias should not be made lightly – if you simply call out results as biased, you may introduce your own bias. Claims of bias should be sourced to reliable secondary sources, and are not reason to omit sources without consensus – instead, qualify sources with information of why a source may be biased, and who is calling it biased.

Obvious or overt bias in a source is a difficult problem for Wikipedia. If there is consensus on an article that a certain source should be omitted for bias, it may be excluded. It may be simpler to find a "better" source – either a higher quality study type or a more specific source instead (see

). If no high-quality source exists for a controversial statement it is best to leave it out; this is not bias.

Personal conflicts of interest

Use your best judgement when writing about topics where you may have a conflict of interest: citing yourself on Wikipedia is problematic. Citing your own organization, such as a governmental health agency or an NGO producing high-quality systematic reviews is generally acceptable – if it is done to improve coverage of a topic, and not with the sole purpose of driving traffic to your site. All edits should improve neutral encyclopedic coverage; anything else, such as promoting an organization is

not allowed

According to the

conflict of interest guideline
 – conflicts of interest (COI) must be disclosed. Editing on topics where one is involved or closely related, especially when there is potential financial gain, is discouraged. Medicine is not an exception. One way to contribute with a COI is to post on talk-pages, suggesting edits. Another alternative is the
articles for creation
pathway. These methods are often best when writing about oneself, one's organization or company – but may be less so when there is a potential conflict of interest in a research field. For example, one may legitimately be an authority on a certain topic – a volunteer who reads the talk-page will not always have the knowledge to assess the sources properly. Then it is better to follow ordinary editing protocol, disclosing any COI and to be careful not to overemphasize your own sources.

Choosing sources

open access journals can be useful as sources for images in Wikipedia articles. Because the above image was published under the terms of a Creative Commons license, it can be uploaded to Wikimedia Commons and used
on Wikipedia. Click on the above image to find its source.

Non-free content

A Wikipedia article should cite high-quality reliable sources

PLoS Medicine, publish only freely readable sources. Also, a few sources are in the public domain; these include many U.S. government publications, such as the Morbidity and Mortality Weekly Report of the Centers for Disease Control and Prevention

Don't just cite the abstract


WikiProject Medicine's talk page to either provide an electronic copy or read the source and summarize what it says; if neither is possible, the editor may need to settle for using a lower-impact

Biomedical journals

Peer-reviewed medical journals are a natural choice as a source for up-to-date medical information in Wikipedia articles. Journal articles come in many different types, and are a mixture of primary and secondary sources. Primary publications describe new research, while review articles summarize and integrate a topic of research into an overall view. In medicine, primary sources include clinical trials, which test new treatments. In addition to experiments, primary sources normally contain introductory, background, or review sections that place their research in the context of previous work; these sections may be cited in Wikipedia with care: they are often incomplete[18] and typically less reliable than reviews or other sources, such as textbooks, which are intended to be reasonably comprehensive. If challenged by another editor in good faith, the primary source should be supplemented with a more appropriate source.

Broadly speaking, reviews may be

reproducible methodology to select primary (or sometimes secondary) studies meeting explicit criteria to address a specific question. Such reviews should be more reliable and accurate and less prone to bias than a narrative review.[12] Systematic reviews and meta-analyses of randomized controlled trials can provide strong evidence of the clinical efficacy of particular treatments in given scenarios, which may in turn be incorporated into medical guidelines
or institutional position papers (ideal sources for clinical recommendations). It is normally best to use reviews and meta-analyses where possible. Reviews give a balanced and general perspective of a topic, and are usually easier to understand. However, whereas a narrative review may give a panorama of current knowledge on a particular topic, a systematic review tends to have a narrower focus.

Journals may specialize in particular article types. A few, such as Evidence-based Dentistry (

Citing medical sources for details). Others, such as Journal of Medical Biography, publish historical material that can be valuable for History sections, but is rarely useful for current medicine. Still others, such as Medical Hypotheses
, publish speculative proposals that are not reliable sources for biomedical topics.

List of core journals


Journal of the American Medical Association (JAMA), the Annals of Internal Medicine, The BMJ (British Medical Journal), and the Canadian Medical Association Journal. Core basic science and biology journals include Nature, Science and Cell

Predatory journals

Avoid articles from journals with

WP:RS#Predatory journals and the #References section below for examples of such publishers.[21][22]) Other indications that a journal article may not be reliable are its publication in a journal that is not indexed in the bibliographic database MEDLINE,[23] or its content being outside the journal's normal scope (for instance, an article on the efficacy of a new cancer treatment in a psychiatric journal or the surgical techniques for hip replacement in a urology
journal). Determining the reliability of any individual journal article may also take into account whether the article has garnered significant positive citations in sources of undisputed reliability, suggesting wider acceptance in the medical literature despite any red flags suggested here.

An archive of

Unreliable/Predatory Source Detector
script can be leveraged to facilitate the detection of predatory journals.

Some baseline methods to identify questionable journals have reached consensus in the academic community.[24]

Symposia and supplements to academic journals are often (but far from always) unacceptable sources. They are commonly sponsored by industry groups with a financial interest in the outcome of the research reported. They may lack independent editorial oversight and peer review, with no supervision of content by the parent journal.[14] Such shill articles do not share the reliability of their parent journal,[15] being essentially paid ads disguised as academic articles. Such supplements, and those that do not clearly declare their editorial policy and conflicts of interest, should not be cited.

Indications that an article was published in a supplement may be fairly subtle; for instance, a letter "S" added to a page number,

The Times Higher Education Supplement
. A sponsored supplement need not necessarily have a COI with its medical content; for instance, public health agencies may also sponsor supplements. However, groups that do have a COI may hide behind layers of front organizations with innocuous names, so the ultimate funding sources should always be ascertained.


High-quality textbooks can be a good source to start an article, and often include general overviews of a field or subject. However, books generally move slower than journal sources, and are often several years behind the current state of evidence. This makes using up-to-date books even more important. Medical textbooks published by academic publishers are often excellent secondary sources. If a textbook is intended for students, it may not be as thorough as a monograph or chapter in a textbook intended for professionals or postgraduates. Ensure that the book is up to date, unless a historical perspective is required. Doody's maintains a list of core health sciences books, which is available only to subscribers.

Springer Verlag, Wolters Kluwer, and Informa
) publish specialized medical book series with good editorial oversight; volumes in these series summarize the latest research in narrow areas, usually in a more extensive format than journal reviews. Specialized biomedical encyclopaedias published by these established publishers are often of good quality, but as a tertiary source, the information may be too terse for detailed articles.

Additionally, popular science books are useful sources, but generally should not be referenced on Wikipedia to support medical statements (see

self-published books or books published by vanity presses undergo no independent fact-checking or peer review and, consequently, are not reliable sources. However, books published by university presses or the National Academy of Sciences
tend to be well-researched and useful for most purposes.

Medical and scientific organizations

Guidelines and position statements provided by major medical and scientific organizations are important on Wikipedia because they present recommendations and opinions that many caregivers rely upon (or may even be legally obliged to follow).

Statements and information from reputable major medical and scientific bodies may be valuable encyclopedic sources. These bodies include the

. The reliability of these sources ranges from formal scientific reports, which can be the equal of the best reviews published in medical journals, through public guides and service announcements, which have the advantage of being freely readable, but are generally less authoritative than the underlying medical literature.

Guidelines by major medical and scientific organizations sometimes clash with one another (for example, the World Health Organization and American Heart Association on salt intake), which should be resolved in accordance with

avoid weasel words and phrases
by tying together separate statements with "however", "this is not supported by", etc. The image below attempts to clarify some internal ranking of statements from different organizations in the weight they are given on Wikipedia.

Health technology assessments
or HTAs are the gold standard when it comes to assessing evidence quality. They take into account various aspects such as effect, risks, economic costs, and ethical concerns of a treatment. They seldom make recommendations, but instead explain most effective treatments, potential hazards and discuss gaps in knowledge. Their name is somewhat of a misnomer as they do not need to concern "technology" as perceived by the public – but rather any intervention intended to improve health.

Popular press

The popular press is generally not a reliable source for scientific and medical information in articles. Most

generally considered independent, primary sources
. A news article should therefore not be used as a sole source for a medical fact or figure. Editors are encouraged to seek out the scholarly research behind the news story. One possibility is to cite a higher-quality source along with a more-accessible popular source.

Conversely, the high-quality popular press can be a good source for social, biographical, current-affairs, financial, and historical information in a medical article. For example, popular science magazines such as

general reliable sources guidelines. Sources for evaluating health-care media coverage include the review website Health News Review along with specialized academic journals, such as the Journal of Health Communication; reviews can also appear in the American Journal of Public Health, the Columbia Journalism Review and others. Health News Review's criteria for rating news stories[32]
can help to get a general idea of the quality of a medical news article.

Other sources

Press releases, newsletters, advocacy and self-help publications, blogs and other websites, and other sources contain a wide range of biomedical information ranging from factual to fraudulent, with a high percentage being of low quality. Conference abstracts present incomplete and unpublished data and undergo varying levels of review; they are often unreviewed and their initial conclusions may have changed dramatically if and when the data are finally ready for publication.

surprising claims, and carefully identified in the text as preliminary work. Medical information resources such as WebMD and eMedicine are usually acceptable sources for uncontroversial information; however, as much as possible Wikipedia articles should cite the more established literature directly. UpToDate
is less preferred as it is not possible to reference specific versions of their articles, archives do not exist, and it can be difficult to access.

Searching for sources

Search engines are commonly used to find biomedical sources. Each engine has quirks, advantages, and disadvantages, and may not return the results that the editor needs unless used carefully. It typically takes experience and practice to recognize when a search has not been effective; even if an editor finds useful sources, they may have missed other sources that would have been more useful or they may generate pages and pages of less-than-useful material. A good strategy for avoiding sole reliance on search engines is to find a few recent high-quality sources and follow their citations to see what the search engine missed. It can also be helpful to perform a plain web search rather than one of scholarly articles only.

practice guidelines, and/or to freely readable sources. Although PubMed is a comprehensive database, many of its indexed journals restrict online access. Another website, PubMed Central, provides free access to full texts. While it is often not the official published version, it is a peer-reviewed manuscript that is substantially the same, but lacks minor copy-editing by the publisher.[37]

When looking at an individual abstract on the PubMed website, an editor can consult "Publication Types", "MeSH Terms", etc. at the bottom of the page to see how the document has been classified in PubMed. For example, a page that is tagged as "Comment" or "Letter" is a

DOAJ and other databases as well as the status and publishing track of authors if they make extraordinary claims. There is no magic number
, but it is useful to compare the authors to others' in the same field of study.


See also


  1. .
  2. ^ Naik G (2 December 2011). "Scientists' Elusive Goal: Reproducing Study Results". Wall Street Journal.
  3. ^ Nature's Challenges in Reproducibility initiative
  4. S2CID 16180896
  5. .
  6. .
  7. ^ "Evidence-Based Decision Making: Introduction and Formulating Good Clinical Questions | Continuing Education Course | Course Pages |". Retrieved 3 September 2015.
  8. ^ "SUNY Downstate EBM Tutorial". Retrieved 3 September 2015.
  9. ^ "The Journey of Research - Levels of Evidence | CAPhO". Archived from the original on 21 February 2016. Retrieved 3 September 2015.
  10. .
  11. .
  12. ^ .
  13. from the original on 14 May 2021. Retrieved 14 May 2021.
  14. ^ a b Fees F (2016), Recommendations for the conduct, reporting, editing, and publication of scholarly work in medical journals (PDF), archived (PDF) from the original on 5 March 2014, retrieved 12 January 2019 Conflicts-of-interest section Archived 2018-12-30 at the Wayback Machine, [Last update on 2015 Dec].
  15. ^
    PMID 8015117
  16. .
  17. .
  18. .
  19. ^ "Abridged Index Medicus (AIM or "Core Clinical") Journal Titles". NLM. Retrieved 17 November 2012.
  20. ^ Hill DR, Stickell H, Crow SJ (2003). "Brandon/Hill selected list of print books for the small medical library" (PDF). Mt. Sinai School of Medicine. Archived from the original (PDF) on 7 August 2011. Retrieved 16 September 2008.
  21. ^ Jakaria Rahman A, Engels TC (25 February 2015). Predatory open access journals in a performance-based funding model: Common journals in Beall's list and in version V of the VABB-SHW (PDF) (Report). University of Antwerp, Gezaghebbende Panel.
  22. ^ Beall J (31 December 2016). "Potential, possible, or probable predatory scholarly open-access publishers". Archived from the original on 17 January 2017.
  23. ^ To determine if a journal is MEDLINE indexed, go to this website, and search for the name of the journal. On the journal page, under the heading "Current Indexing Status", you can see whether or not the journal is currently indexed. Note that journals that have changed names or ceased publication will not be "currently" indexed on MEDLINE, but their indexing status, when they were being published, can be viewed under other headings on that same page.
  24. .
  25. (PDF) from the original on 17 November 2018. Retrieved 12 January 2019.
  26. ^ See this discussion of how to identify shill academic articles cited in Wikipedia.
  27. PMID 16404471
  28. . The Guardian.
  29. PMID 18507496.
    Goldacre, Ben (20 June 2008). "Why reading should not be believing"
    . Guardian.
  30. .
  31. .
  32. ^ "How we rate stories". Health News Review. 2008. Archived from the original on 23 July 2012. Retrieved 26 March 2009.
  33. S2CID 29281534
  34. .
  35. ^ "PubMed User Guide". PubMed. Retrieved 14 May 2021.
  36. ^ "PubMed tutorial: filters". NLM. Retrieved 17 November 2012.
  37. S2CID 44572906
    . Retrieved 24 October 2008.
  38. ^ "PubMed: Publication Types". NLM. Retrieved 29 January 2020.

Further reading