Link rot

Source: Wikipedia, the free encyclopedia.
Page Not Found
A rotten link usually leads to an error message

Link rot (also called link death, link breaking, or reference rot) is the phenomenon of hyperlinks tending over time to cease to point to their originally targeted file, web page, or server due to that resource being relocated to a new address or becoming permanently unavailable. A link that no longer points to its target, often called a broken, dead, or orphaned link, is a specific form of dangling pointer.

The rate of link rot is a subject of study and research due to its significance to the internet's ability to preserve information. Estimates of that rate vary dramatically between studies. Information professionals have warned that link rot could make important archival data disappear, potentially impacting the legal system and scholarship.

Prevalence

A number of studies have examined the prevalence of link rot within the World Wide Web, in academic literature that uses URLs to cite web content, and within digital libraries.

A 2002 study suggested that link rot within digital libraries is considerably slower than on the web, finding that about 3% of the objects were no longer accessible after one year[1] (equating to a half-life of nearly 23 years).

A 2003 study found that on the Web, about one link out of every 200 broke each week,[2] suggesting a half-life of 138 weeks. This rate was largely confirmed by a 2016–2017 study of links in Yahoo! Directory (which had stopped updating in 2014 after 21 years of development) that found the half-life of the directory's links to be two years.[3]

A 2004 study showed that subsets of Web links (such as those targeting specific file types or those hosted by academic institutions) could have dramatically different half-lives.

New York Times articles published between 1996 and 2019 found a half-life of about 15 years (with significant variance among content topics) but noted that 13% of functional links no longer lead to the original content—a phenomenon called content drift.[10]

A 2013 study found that 49% of links in U.S. Supreme court opinions are dead.[11]

A 2023 study looking at United States COVID-19 dashboards found that 23% of the state dashboards available in February of 2021 were no longer available at the previous URLs in April of 2023.[12]

Causes

Link rot can result from several occurrences. A target web page may be removed. The server that hosts the target page could fail, be removed from service, or relocate to a new domain name. As far back as 1999, it was noted that with the amount of material that can be stored on a hard drive, "a single disk failure could be like the burning of the library at Alexandria."[13] A domain name's registration may lapse or be transferred to another party. Some causes will result in the link failing to find any target and returning an error such as HTTP 404. Other causes will cause a link to target content other than what was intended by the link's author.

Other reasons for broken links include:

  • the restructuring of websites that causes changes in URLs (e.g. domain.net/pine_tree might be moved to domain.net/tree/pine)
  • relocation of formerly free content to behind a paywall[12]
  • a change in server architecture that results in code such as PHP functioning differently
  • dynamic page content such as search results that changes by design
  • deletion of the target page and/or its content
  • the presence of user-specific information (such as a login name) within the link
  • deliberate blocking by
    firewalls
  • the expiration of a
    domain name registration

Prevention and detection

Strategies for preventing link rot can focus on placing content where its likelihood of persisting is higher, authoring links that are less likely to be broken, taking steps to preserve existing links, or repairing links whose targets have been relocated or removed.[citation needed]

The creation of URLs that will not change with time is the fundamental method of preventing link rot. Preventive planning has been championed by Tim Berners-Lee and other web pioneers.[14]

Strategies pertaining to the authorship of links include:

Strategies pertaining to the protection of existing links include:

The detection of broken links may be done manually or automatically. Automated methods include

200 OK response but point to content that has changed.[24]

See also

Further reading

References

  1. from the original on 2020-07-19. Retrieved 2019-09-24.
  2. ^ Fetterly, Dennis; Manasse, Mark; Najork, Marc; Wiener, Janet (2003). "A large-scale study of the evolution of web pages". Proceedings of the 12th international conference on World Wide Web. Archived from the original on 9 July 2011. Retrieved 14 September 2010.
  3. ^ van der Graaf, Hans. "The half-life of a link is two year". ZOMDir's blog. Archived from the original on 2017-10-17. Retrieved 2019-01-31.
  4. ^ a b Koehler, Wallace (2004). "A longitudinal study of web pages continued: a consideration of document persistence". Information Research. 9 (2). Archived from the original on 2017-09-11. Retrieved 2019-01-31.
  5. ^ "All-Time Weblock Report". August 2015. Archived from the original on 4 March 2016. Retrieved 12 January 2016.
  6. ^ a b McCown, Frank; Chan, Sheffan; Nelson, Michael L.; Bollen, Johan (2005). "The Availability and Persistence of Web References in D-Lib Magazine" (PDF). Proceedings of the 5th International Web Archiving Workshop and Digital Preservation (IWAW'05). Archived from the original (PDF) on 2012-07-17. Retrieved 2005-10-12.
  7. from the original on 2020-07-23. Retrieved 2007-09-29.
  8. .
  9. .
  10. ^ "What the ephemerality of the Web means for your hyperlinks". Columbia Journalism Review. Archived from the original on 2021-08-02. Retrieved 2021-08-02.
  11. ^ Garber, Megan (2013-09-23). "49% of the Links Cited in Supreme Court Decisions Are Broken". The Atlantic. Retrieved 2024-01-10.
  12. ^ .
  13. .
  14. ^ Berners-Lee, Tim (1998). "Cool URIs Don't Change". Archived from the original on 2000-03-02. Retrieved 2019-01-31.
  15. ^ a b Kille, Leighton Walter (8 November 2014). "The Growing Problem of Internet "Link Rot" and Best Practices for Media and Online Publishers". Journalist's Resource, Harvard Kennedy School. Archived from the original on 12 January 2015. Retrieved 16 January 2015.
  16. ^ Sicilia, Miguel-Angel, et al. "Decentralized Persistent Identifiers: a basic model for immutable handlers Archived 2023-05-10 at the Wayback Machine." Procedia computer science 146 (2019): 123-130.
  17. ^ "Internet Archive: Digital Library of Free Books, Movies, Music & Wayback Machine". 2001-03-10. Archived from the original on 26 January 1997. Retrieved 7 October 2013.
  18. PMID 16403724
    .
  19. (PDF) from the original on 1 November 2020. Retrieved 10 June 2020.
  20. ^ "Harvard University's Berkman Center Releases Amber, a "Mutual Aid" Tool for Bloggers & Website Owners to Help Keep the Web Available | Berkman Center". cyber.law.harvard.edu. Archived from the original on 2016-02-02. Retrieved 2016-01-28.
  21. ^ "Arweave - A community-driven ecosystem". arweave.org. Archived from the original on 2023-03-15. Retrieved 2023-03-15.
  22. ^ Rønn-Jensen, Jesper (2007-10-05). "Software Eliminates User Errors And Linkrot". Justaddwater.dk. Archived from the original on 11 October 2007. Retrieved 5 October 2007.
  23. ^ Mueller, John (2007-12-14). "FYI on Google Toolbar's Latest Features". Google Webmaster Central Blog. Archived from the original on 13 September 2008. Retrieved 9 July 2008.
  24. .

External links