Survivorship bias
Survivorship bias or survival bias is the
Survivorship bias is a form of selection bias that can lead to overly optimistic beliefs because multiple failures are overlooked, such as when companies that no longer exist are excluded from analyses of financial performance. It can also lead to the false belief that the successes in a group have some special property, rather than just coincidence as in correlation "proves" causality.
Another kind of survivorship bias would involve thinking that an incident happened in a particular way because the only people who were involved in the incident who can speak about it are those who survived it. Even if one knew that some people are dead, they would not have their voice to add to the conversation, making it biased.
As a general experimental flaw
The
Writing about the Rhine case in
If sufficiently many
Survivorship bias is one of the research issues brought up in the provocative 2005 paper "Why Most Published Research Findings Are False", which shows that a large number of published medical research papers contain results that cannot be replicated.[3]
One famous example of immortal time bias was discovered in a study by Redelmeier and Singh in the
Examples
Business, finance, and economics
In finance, survivorship bias is the tendency for failed companies to be excluded from performance studies because they no longer exist. It often causes the results of studies to skew higher because only companies that were successful enough to survive until the end of the period are included. For example, a mutual fund company's selection of funds today will include only those that are successful now. Many losing funds are closed and merged into other funds to hide poor performance. In theory, 70% of extant funds could truthfully claim to have performance in the first quartile of their peers, if the peer group includes funds that have closed.[7]
In 1996, Elton, Gruber, and Blake showed that survivorship bias is larger in the small-fund sector than in large mutual funds (presumably because small funds have a high probability of folding).[8] They estimate the size of the bias across the U.S. mutual fund industry as 0.9% per annum, where the bias is defined and measured as:
"Bias is defined as average α for surviving funds minus average α for all funds"
(Whereα is the risk-adjusted return over the S&P 500. This is the standard measure of mutual fund out-performance).
Additionally, in quantitative backtesting of market performance or other characteristics, survivorship bias is the use of a current index membership set rather than using the actual constituent changes over time. Consider a backtest to 1990 to find the average performance (total return) of S&P 500 members who have paid dividends within the previous year. To use the current 500 members only and create a historical equity line of the total return of the companies that met the criteria would be adding survivorship bias to the results. S&P maintains an index of healthy companies, removing companies that no longer meet their criteria as a representative of the large-cap U.S. stock market. Companies that had healthy growth on their way to inclusion in the S&P 500 would be counted as if they were in the index during that growth period, which they were not. Instead there may have been another company in the index that was losing market capitalization and was destined for the S&P 600 Small-cap Index that was later removed and would not be counted in the results. Using the actual membership of the index and applying entry and exit dates to gain the appropriate return during inclusion in the index would allow for a bias-free output.
Michael Shermer in Scientific American[9] and Larry Smith of the University of Waterloo[10] have described how advice about commercial success distorts perceptions of it by ignoring all of the businesses and college dropouts that failed.[11] Journalist and author David McRaney observes that the "advice business is a monopoly run by survivors. When something becomes a non-survivor, it is either completely eliminated, or whatever voice it has is muted to zero".[12] Alec Liu wrote in Vice that "for every Mark Zuckerberg, there's thousands of also-rans, who had parties no one ever attended, obsolete before we ever knew they existed."[13]
In his book
History
Diagoras of Melos was asked concerning paintings of those who had escaped shipwreck: "Look, you who think the gods have no care of human things, what do you say to so many persons preserved from death by their especial favour?", to which Diagoras replied: "Why, I say that their pictures are not here who were cast away, who are by much the greater number."[15]
Susan Mumm has described how survival bias leads historians to study organisations that are still in existence more than those that have closed. This means large, successful organisations such as the Women's Institute, which were well organised and still have accessible archives for historians to work from, are studied more than smaller charitable organisations, even though these may have done a great deal of work.[16]
Architecture and construction
Just as new buildings are being built every day and older structures are constantly torn down, the story of most civil and urban architecture involves a process of constant renewal, renovation, and revolution. Only the most beautiful, useful, and structurally sound buildings survive from one generation to the next. This creates a selection effect where the ugliest and weakest buildings of history have been eradicated (disappearing from public view, leaving the visible impression that all earlier buildings were more beautiful and better built).
Highly competitive career
Whether it be movie stars, athletes, musicians, or CEOs of multibillion-dollar corporations who dropped out of school, popular media often tells the story of the determined individual who pursues their dreams and beats the odds. There is much less focus on the many people that may be similarly skilled and determined, but fail to ever find success because of factors beyond their control or other (seemingly) random events. There is also a tendency to overlook resources and events that helped enable such success, that those who failed didn't have.[17]
This creates a false public perception that anyone can achieve great things if they have the ability and make the effort. The overwhelming majority of failures are not visible to the public eye, and only those who survive the selective pressures of their competitive environment are seen regularly.
Military
During
Cats
In a study performed in 1987, it was reported that cats who fall from less than six stories, and are still alive, have greater injuries than cats who fall from higher than six stories.[23][24] It has been proposed that this might happen because cats reach terminal velocity after righting themselves at about five stories, and after this point they relax, leading to less severe injuries in cats who have fallen from six or more stories.[25] In 1996, The Straight Dope newspaper column proposed that another possible explanation for this phenomenon would be survivorship bias. Cats that die in falls are less likely to be brought to a veterinarian than injured cats, and thus many of the cats killed in falls from higher buildings are not reported in studies of the subject.[26]
Tropical trees
Tropical vines and
Studies of evolution
Large groups of organisms called
Business law
Survivorship bias can raise truth-in-advertising issues when the success rate advertised for a product or service is measured by reference to a population whose makeup differs from that of the target audience for the advertisement. This is especially important when
- the advertisement either fails to disclose the relevant differences between the two populations, or describes them in insufficient detail; and
- these differences result from the company's deliberate "pre-screening" of prospective customers to ensure that only customers with traits increasing their likelihood of success are allowed to purchase the product or service, especially when the company's selection procedures or evaluation standards are kept secret; and
- the company offering the product or service charges a fee, especially one that is non-refundable or not disclosed in the advertisement, for the privilege of attempting to become a customer.
For example, the advertisements of online dating service eHarmony.com pass this test because they fail the first two prongs but not the third:
- they claim a success rate significantly higher than that of competing services while generally not disclosing that the rate is calculated with respect to a viewership subset of individuals who possess traits that increase their likelihood of finding and maintaining relationships and lack traits that pose obstacles to their doing so, and
- the company deliberately selects for these traits by administering a lengthy pre-screening process designed to reject prospective customers who lack the former traits or possess the latter ones, but
- the company does not charge a fee for administration of its pre-screening test; thus its prospective customers face no "downside risk" other than wasting their time, expending the effort involved in completing the pre-screening process,[30] and suffering disappointment.
See also
- Anthropic principle – Hypothesis about sapient life and the universe[31]
- Availability heuristic – Bias towards recently acquired information
- Cherry picking – Fallacy of incomplete evidence
- Fooled by Randomness – Book by Nassim Nicholas Taleb
- Inspiration porn – Ableist portrayal of disabled people
- Lindy effect – Theorized increase of longevity with age
- Malmquist bias – Sampling bias in astronomy
- Meta-analysis – Statistical method that summarizes data from multiple sources
- Multiple comparisons problem – Statistical interpretation with many tests
- Myth of meritocracy – Sociological concept
- Post hoc ergo propter hoc – Fallacy of assumption of causality based on sequence of events
- Selection principle – Rule in mathematics
- Texas sharpshooter fallacy – Statistical fallacy
References
- ^ Shermer, Michael (3 August 2011). "Deviations: A Skeptical Investigation of Edgar Cayce's Association for Research and Enlightenment". Skeptic.com. Retrieved 21 May 2018.
- ^ Fads and Fallacies in the Name of Science, Martin Gardner, p. 303, 1957, Dover Publications Inc.
- ^ PMID 16060722.
- ^ Goodchild van Hilten, Lucy. "Why it's time to publish research "failures"". Elsevier Connect. Elsevier. Archived from the original on 2 June 2021. Retrieved 31 May 2021.
- PMID 11352696.
- S2CID 13724567.
- ^ "How to think about diversification - Page 3 - Bogleheads.org". www.bogleheads.org. Archived from the original on 2020-10-29. Retrieved 2020-10-26.
- S2CID 154097782. In this paper the researchers eliminate survivorship bias by following the returns on all funds extant at the end of 1976. They show that other researchers have drawn spurious conclusions by failing to include the bias in regressions on fund performance.
- ^ Michael Shermer (2014-08-19). "How the Survivor Bias Distorts Reality". Scientific American. Archived from the original on 2019-04-29. Retrieved 2015-07-20.
- ^ Carmine Gallo (2012-12-07). "High-Tech Dropouts Misinterpret Steve Jobs' Advice". Forbes. Archived from the original on 2019-04-02. Retrieved 2017-09-15.
- ^ Robert J Zimmer (2013-03-01). "The Myth of the Successful College Dropout: Why It Could Make Millions of Young Americans Poorer". The Atlantic. Archived from the original on 2019-04-09. Retrieved 2017-03-11.
- ^ Karen E. Klein. "How Survivorship Bias Tricks Entrepreneurs". Bloomberg. Archived from the original on 2016-03-07. Retrieved 2017-03-11.
- ^ Alec Liu (2012-10-03). "What Happened to the Facebook Killer? It's Complicated". Motherboard - Tech by Vice.
- ISBN 9780679604181.
- ^ Cicero, De Natura Deor., iii. 37.
- ^ Mumm, Susan (2010). Women and Philanthropic Cultures, in Women, Gender and Religious Cultures in Britain, 1800-1940, Eds Sue Morgan and Jacqueline deVries . London: Routledge.
- ^ Karen E. Klein (2014-08-11). "How Survivorship Bias Tricks Entrepreneurs". Bloomberg Business. Archived from the original on 2017-05-23. Retrieved 2017-04-11.
- ^ a b Wald, Abraham. (1943). A Method of Estimating Plane Vulnerability Based on Damage of Survivors. Statistical Research Group, Columbia University. CRC 432 — reprint from July 1980 Archived 2019-07-13 at the Wayback Machine. Center for Naval Analyses.
- JSTOR 2287454.
- ^ "Bullet Holes & Bias: The Story of Abraham Wald". mcdreeamie-musings. Archived from the original on 2020-04-16. Retrieved 2020-05-29.
- ^ "AMS :: Feature Column :: The Legend of Abraham Wald". American Mathematical Society. Archived from the original on 2020-05-27. Retrieved 2020-05-29.
- PMID 3692980.
- ^ "Highrise Syndrome in Cats". Archived from the original on 2015-02-21. Retrieved 2014-06-13.
- ^ Falling Cats Archived 2007-07-12 at the Wayback Machine
- ^ "Do cats always land unharmed on their feet, no matter how far they fall?". The Straight Dope. July 19, 1996. Archived from the original on 2012-04-16. Retrieved 2008-03-13.
- from the original on 2020-05-06. Retrieved 2020-08-29.
- ISSN 0022-0477.
- (PDF) from the original on 2019-04-28. Retrieved 2018-12-12.
- Washington Post. Archivedfrom the original on May 16, 2017. Retrieved September 15, 2017.
- ^ "Archived copy" (PDF). Archived (PDF) from the original on 2019-07-05. Retrieved 2019-07-05.
{{cite web}}
: CS1 maint: archived copy as title (link)