Wikipedia:Why MEDRS?

The result of all of this is that the world is awash with content about health. All kinds of media holler at us every day, about "new THIS" and "shocking THAT". Very often that content is dead wrong, or dramatically overstates what we can confidently say, based on the science. And many people have strong ideas that are not based on science at all.

But as an encyclopedia, Wikipedia is committed to providing reliable information to the public. We have nothing to do with hype or eyeballs or the 24-hour news cycle. We go slow, and say what is certain (which includes saying "we don't know" or "there is insufficient evidence to say X"). All of Wikipedia stands on, and is based on, the consensus of whatever field a given article falls within. We always have to think carefully about what sources we use to generate content, and this is especially true for health-related content. For health-related content, the field is

Wikipedia is an encyclopedia. It is

Our mission is to express the sum of human knowledge – "accepted knowledge", in the words of

• Sometimes they are scientists, who treat Wikipedia articles like they themselves are
  synthesize
  a story from primary sources. But articles here are encyclopedia articles, which is a different genre. Each article is meant to be "a summary of accepted knowledge regarding its subject". Secondary sources are where conclusions stated in primary sources are "accepted" or not.
• Sometimes they are everyday people, who don't understand that the scientific literature is where science happens – it is where scientists talk to each other. The scientific literature is not really intended for the general public. The Internet has made it more available to the public, as has the open access movement. Both are a mixed blessing. The downside is that everyday people may take research papers out of context from the ongoing and always-developing discussion among scientists, and take individual results as some kind of gospel truth, when each paper is really just a stepping stone (sometimes a false one) as we (humanity) apply the scientific method to understanding the world around us. Non-scientists may not know that many research articles in biology turn out to be dead ends, or unreplicable, or even withdrawn. When a research paper is published, we cannot know if it will eventually turn out to be replicable and/or accepted and built on by the relevant field, and if it will not. Reviews tell us that.[d]
• Sometimes editors wanting to use primary sources are agenda-driven — there is something in the real world that is very important to them, and they want that idea expressed in WP and given strong
  WP:WEIGHT
  . In the very act of doing that — in selecting a given primary source and giving it a lot of weight (or any weight at all, actually) — they are performing original research. It is sometimes hard to get people to see this.

Wikipedia is not about what you think is important, right now, nor even about what the media is hyping today. It is about what we know, as expressed in reliable, secondary sources. (Independent ones!) It is so hard for people to differentiate what they see and what they "know" from what humanity — as expressed by experts in a given field — knows.

It is hard for people to think like scholars, with discipline, and actually listen to and be taught by reliable, independent, secondary sources instead of acting like barroom philosophers who shoot from the hip or letting media hype drive them.

NPOV depends mightily upon editors' grasp of secondary sources. We have to find good ones – recent, independent ones – and absorb them, and see what the mainstream positions are in the field, what are "significant minority opinions", and what views are just plain

What makes this even more challenging is that because this is a volunteer project, Wikipedia editors often come here and stay here due to some passion. This passion is a double-edged sword. It drives engagement and the creation of content, but too often brings with it

• While
  WP:OR
  allows primary sources to be used, it is "only with care, because it is easy to misuse them."
• WP:NPOV
  says "Neutrality assigns weight to viewpoints in proportion to their prominence. However, when reputable sources contradict one another and are relatively equal in prominence, describe both approaches and work for balance. This involves describing the opposing views clearly, drawing on secondary or tertiary sources that describe the disagreement from a disinterested viewpoint."
• Misuse of primary sources
  section of the BLP policy."

The call to use independent secondary sources is deep in the guts of Wikipedia. This is a meta-issue — a question of what it means to be an editor on Wikipedia.

Biology is difficult

Biology is difficult. It is still a young science, and our knowledge of even basic things is fragmentary, and even our big-picture ideas are changing all the time. Human biology — our understanding of what is going on inside healthy people and inside sick people — is even harder, and there are serious barriers to furthering our understanding. People in the physical sciences or technology seem to have an especially hard time understanding this.

The physical sciences have given us deep insight into material reality, and because the science there has progressed so far, we can do amazing things. For example, Moore's law is a direct result of our advances in physics and materials science and our ability to apply science — to create technology to serve us, to the point where we now have amazing things like smartphones — computers we can hold in our hands and interact with in intuitive ways, capabilities that just a couple decades ago would have taken an entire room full of equipment to provide and that only cutting-edge scientists could operate.

Physics deals with dead matter. We can poke and prod without doing harm, and what we are looking at is what we are looking at. Life (made of physical matter, of course) is way more complicated. In comparison, "dead" is easy; life is hard.

Biology remains primarily an observational science. Don't get me wrong — biologists do experiments — they poke and prod living things in various ways, to help them try to flesh out the pictures we are still forming about what is going on in living things. But we are not in possession of a set of "laws of nature" such as those that govern physics. Even what we once called the central dogma of molecular biology — that DNA "makes" RNA which "makes" proteins — has turned out to be far more complicated than biologists originally thought. We still don't fully understand what something as basic as aspirin does in the human body, much less what it does in a particular person's body. We understand a lot, but our knowledge is far from perfect. Medicine like aspirin is technology — we are doing our best to apply the findings of biological science to solve problems. We understand what aspirin itself is, very well (the chemistry, not the biology), but what happens when you put it into an average human body, or a particular person's body, is another question altogether. The science is too weak in biology, especially human biology, to apply and evolve technology with anywhere near the speed of information technology.

These fields are different worlds, scientifically speaking. (I am not even getting into structural differences that make the markets so different — innovators in medicine have to deal with regulators and whether insurance companies will pay for things, with serious ethical issues involved in experimenting on animals and humans, and with the huge amounts of money and time and risk in bringing new products to market. All these make medicine a different universe from information technology.)

Going a little deeper into the science...

Most everybody has heard of "DNA", but what is it? It is a polymer — it is a chemical made up of many subunits all connected in a chain. Each of those subunits is a chemical called a nucleotide. In simple terms, there are four different nucleotides: adenine, guanine, thymine, and cytosine, and we often describe the chain they make when they link together by using the first letters of their names: A, G, T, and C. So DNA is a long chain of As, Gs, Ts, and Cs. We can describe a given instance of a DNA molecule as a chain of letters: AAGTCTTGACT, etc.

A "genome" is, basically, all the DNA in a cell. (A given species will have a pretty consistent genome on a high level, but every instance of that species will be slightly different — there will be many small variants — some of them a single nucleotide change, some of them being whole deletions or rearrangements of DNA segments. But genomes remain consistent enough from organism to organism within a species that we can indeed meaningfully talk about "species"). The simplest bacteria (which are some of the simplest living organisms) have DNA that is a chain 139,000 nucleotides long (ATCTG, etc., times ~139,000). Microscopic, mind you!

But ... who cares? Why does that matter? Well, DNA is kind of the "blueprint" of the cell.

(We need to be careful here — people use a lot of metaphors in biology, and they are starting to slip into thinking about "genomes" as pure information—as literally some kind of code, like software. But in the real world, any genome is DNA, which is not abstract information. DNA is an actual, physical thing in every cell in every living thing. It interacts physically with other actual chemicals, which in turn interact with other chemicals, and so on and so on. The sum of those actual interactions is what we call "life", and even "consciousness".)

Within the long, long chemical chains of DNA, certain segments function as a kind of code (we call these segments "genes"). The cell has machinery (yes! — actual mini-machinery that is amazing to behold and consider — see this youtube video for an animation) that creates a different polymer,

So, trying to understand normal basic human biology is hard. Trying to figure out what is going on in a disease is really hard, too. For example, everybody knows that Alzheimer's is a terrible disease, and we have spent gobs of money trying to figure out what causes it. One of the bad actors is a piece of a protein. The piece is called "A beta" and the whole protein is called "APP". Well, with all the money we have spent, we still do not know what APP does in normal brains, and we still don't understand why the A beta piece gets cut out of it. We don't understand why neurons die in the brains of Alzheimer's patients, nor how to stop them from dying. That is crazy, right? It starts to make some sense when you realize that we have no way (really!) of looking inside a living human being's skull and seeing in detail — way down at the cellular level — what is going on. It's a serious problem! Anyway, we are scrabbling around in the dark. Humans are really, really complicated biological things. There's so much going on.

Since we can't chop up living human beings or do crazy experiments on them, how do we try to figure out human biology? We use models — mostly other organisms on which we can do experiments, and based on the results, we can then try to make guesses about human biology. You might have heard the joke about searching for keys where the light is better. This is what biology is like. People do research in mice, or in cells in petri dishes, or they cut up dead people. We do controlled experiments that make sense, and we can start to put stories together about what is going on. And while we are making progress, our answers are still pretty crappy, pretty fragmentary. (This is why we do experiments on animals. A lot of people, including scientists, struggle ethically with whether it is acceptable to do experiments on animals, and if so, how. It is not an easy question. How will we learn about biology if we cannot do experiments with living beings, especially ones that are similar to us? How do we actually see what is going inside a living being if we do not cut it open and look? We do not have any technology that allows us to non-invasively look deep inside a living thing on a microscopic level in real time. That technology just doesn't exist in the real world—we have no

Another thing scientists do are "epidemiological studies". These are studies of a lot of living people where you measure a bunch of things and try to find correlations. But correlations are dangerous. For example, say a study found that college kids who sleep in their clothes tend to wake up with headaches—that's a correlation. But what does this really mean? Does wearing clothes while you sleep make you sleep poorly, or maybe cut off blood to your head or something? Well...the study didn't measure how much beer people drank the night before! Right? Now it all makes sense. In this case, the beer drinking is what we call a "

Scientists also conduct

Biologists are working like crazy to understand "life" and are under all kinds of pressure to get grants and publish papers. They publish boatloads and boatloads of papers.

These articles are not written for the general public. Scientists do experiments using their model systems, as discussed above, and publish the results in order to talk to other scientists. This is the raw stuff of science. It is messy, and scientists know that they are groping their way toward the truth, together. These papers are very important to science, but they are of almost no value to the general public.

In addition to this, there are some problems with academic science and publishing even in the most reputable journals. Academic scientists are on a kind of hamster wheel. Their research is funded by grants that last for a few years at most. They need to string together grant after grant, in order to keep their labs going. Scientists who run labs spend a huge amount of time seeking out funding opportunities and writing research proposals to try to win them. Generally, in order to win the next grant, you publish high-profile, important papers using the grant you have now. So there is a huge force pushing academic scientists to move from one experiment to the next and to draw conclusions from their research that are important. "Publish or perish" is real — if a scientist cannot win grant funding to keep a lab going, the lab will be closed down and dispersed. It is as harsh as being in sales — you eat what you kill. You can see the potential for problems here.

But how exactly does this hamster wheel affect science?

When you do an experiment, you try very hard to execute it perfectly, so that you actually do what you intended to do and get a valid result. But how do you know if the result you got is true or is just some random answer? This comes down to statistics. If you flip a coin three times, you might get heads three times in a row. Should you stop there and decide that when you flip a coin, you always get heads? Is that "true"? (We all know it is not!) Maybe you should repeat that experiment, and again flip a coin three times. But you could still get heads (or tails) every time. However, if you flip a coin a hundred times, you will likely get about 50 heads and 50 tails. And if you flip a coin a thousand times, you will very likely get about 500 heads and a similar number of tails. The number of "flips" is called the "N" in experimental design. If the N is too small, it doesn't matter how many times you repeat the experiments — none of the experiments are valid. You need a big enough N to get a result you can trust.

Increasing N costs money and time. And repeating a high N experiment costs a LOT of money and time. So scientists often use the minimum N they can that will enable them to publish. And many journals allow scientists to publish results — and conclusions drawn from them — with small Ns. As a result, there are many, many papers published in the scientific literature that turn out to have conclusions that cannot be considered true because the N is too small.

This is starting to become a matter of concern in the scientific community.^[12] Drug discovery scientists at Bayer reported in 2011 that they were able to replicate results in only ~20–25% of the prominent studies they examined;^[3] scientists from Amgen followed with a publication in 2012 showing that they were only able to replicate six (11%) of fifty-three high-impact publications and called for higher standards in scientific publishing.^[13] The journal Nature announced in April 2013 that in response to these and other articles showing a widespread problem with reproducibility, it was taking measures to raise its standards.^[14]

So when you pick up any given published paper, we don't know what is going on well enough to judge whether the conclusions will "stick" or not. Even scientists don't know.

In Wikipedia, these research papers are "primary sources." Hopefully, you now have a good understanding of why these papers are not reliable descriptions of reality. They shouldn't be used by the general public for anything, much less creating encyclopedic content.

Now from time to time, scientists sit down and read a bunch of research papers. They think about them and write what we call "reviews", where they try to fit all the primary research together in a way that makes sense. The scientist doing the review will generally cite the primary studies that are part of this description. Generally, reviews do not say things like "that paper is bunk. We are going to ignore it." Instead, they just ignore papers that turn out to be false leads. This is really important. Only egregiously bad papers are actually retracted; there are loads and loads of papers that draw conclusions that turned out not to be true, but that remain in the literature. People who are not experts in the field have no way of knowing which research papers have been left in the dust by the scientific community. These papers are not retracted, nor are they labelled in any way. They just sit there, ignored.

The reviews are written primarily for other scientists in the field — reviews are one of the key tools that the scientific community uses to map itself, to step back and see where things stand. For Wikipedia, these reviews are "secondary sources", and they are dramatically more reliable than primary sources. They give us the consensus (or, if not consensus, the emerging consensus, or a clear picture of what the main contending theories are) in any given field about what is true and what is not, and what is still unknown or uncertain.

Part of this is pretty obvious, but other parts are more complex and deserve some discussion.

Humans are

And people have all kinds of strong ideas about health – it is a fundamental thing we all care about, both for ourselves and our loved ones. Something goes wrong with your child and your doctor tells you that they are autistic. How did this happen?? What happened to my child! It's easy to fall prey to people pushing theories like the now-completely-discredited vaccine theory. (a drastic "correlation is not causation" mistake). Worse, when people fall prey to baloney like that, they start to advocate for society to take action. This is really, really problematic—especially in the case of vaccines, where there are serious risks not only to your child but to other people and their children, in not vaccinating your child. Or say your child is prone to earaches and your doctor keeps prescribing antibiotics, yet your child still gets earaches and now has an upset stomach, and so you turn to alternative medicine to try to mitigate things... and then after you make the switch your child stops suffering. You might become convinced that it was the switch that made the difference and that "modern chemical medicine" is bad and "natural medicine" is good. (But again, see the "correlation is not causation" fallacy). This goes on and on. People have experiences, and want to generalize from them. When scientists hear about this, they say, "Oy".

The popular press and health news

Knowing that people are keenly interested in health-related matters, the media loves to grab science news and pump it up — this sells newspapers and pulls eyes to TV shows and websites. This is something that has been happening more and more over the past thirty years or so, and is driven in part by the 24-hour news cycle and its hunger for stories. But the popular press is really, really unreliable for health news. For example, the BBC—very respected! —reported in 2011 that some Swedish surgeons had "carried out the world's first synthetic organ transplant". They put that in bold print at the top of their article. The problem is that this was dead wrong. Another team published an article in 2006 on their work with artificial bladders—work they started in 2001.

Or let's take stories about food. Let's see, should I drink coffee or not? Maybe I will live longer and drive safer, and hey, if I am woman maybe I will be less depressed, but oh, no! it alters my estrogen levels and maybe it will screw up my baby. Every one of those links is from the New York Times, and is just from the past couple of years. I think it is terrible to jerk the public around like this. A newspaper has an excuse, but Wikipedia does not — we need to provide reliable information to the public.

Why does this happen? Newspapers want to tell stories that sell. And maybe these science/health topics are considered "soft" stories—public interest more than hard news, and are not subject to the same strict editorial and fact checking that real hard news stories are. They generally run with press releases.

Also, hospitals and medical colleges love to put out hype-y press releases when their scientists and doctors do things. Making a splash draws great attention, enhances reputation, and attracts donors, great faculty, and great students. There is a whole world of conflict of interest there. Likewise, individual scientists compete with each other for grant funding (it is really, really hard to win research grants today!) and for publication in high-profile journals. Splashy press releases help raise an investigator's profile. A lot of "science journalism" —too much of it—is unreliable.

Finally, there is a lot of money involved in health-related matters. Conflict of interest is a serious issue in publishing, and almost every journal requires its authors to report any possible conflicts of interest so that reviewers and readers can consider those conflicts when they judge the conclusions that authors draw.

As mentioned above, scientists write reviews from time to time, which are dramatically more reliable than primary sources.

There are different kinds of reviews—some of these are kind of impressionistic, where a senior scientist in a field sits back and reflects. Others are more serious and more detailed, and actually do statistics and analyze and criticize the papers they are pulling together. That latter kind—systematic, critical reviews—are by far the most valuable, both for us at Wikipedia, and for anybody trying to understand what the hell is going on. Some of these systematic critical reviews are written especially for doctors to help them understand where things stand. The

Also, from time to time, major scientific or medical organizations come out with statements on important issues—the

This is what

• user script
  which highlights potentially unreliable citations.