Wikipedia:Wikipedia Signpost/2015-04-29/Recent research
Military history, cricket, and Australia targeted in Wikipedia articles' popularity vs. quality; how copyright damages economy
A monthly overview of recent academic research about Wikipedia and other Wikimedia projects, also published as the Wikimedia Research Newsletter.
Popularity does not breed quality (and vice versa)
This paper[1] provides evidence that quality of an article is not a simple function of its popularity, or, in the words of the authors, that there is "extensive misalignment between production and consumption" in peer communities such as Wikipedia. As the author notes, reader demand for some topics (e.g. LGBT topics or pages about countries) is poorly satisfied, whereas there is over-abundance of quality on topics of comparatively little interest, such as military history.
Rank | Popular and underdeveloped topics | High-quality, not popular topics |
---|---|---|
1 | Countries | Cricket |
2 | Pop music | Tropical cyclones |
3 | Internet | Middle Ages |
4 | Comedy | Politics |
5 | Technology | Fungi |
6 | Religion | Birds |
7 | Science fiction | Military history |
8 | Rock music | Ships |
9 | Psychology | England |
10 | LGBT studies | Australia |
The authors arrived at this conclusion by comparing data on page views to articles on English, French, Russian, and Portuguese Wikipedias to their respective
Excessive copyright terms proven to be a cost for society, via English Wikipedia images
Paul J. Heald and his coauthors at the
Context
In recent years, authorities have started acknowledging possible errors in copyright legislation of the past, which would have been prevented by an evidence-based approach. Heald mentions the Hargreaves Report (2011), endorsed by the UK's IP office, but other examples can be found in World Intellectual Property Organization reports. This awakening corresponds to the work by researchers and think tanks to prove the importance of public domain and certain damages of copyright.[supp 1]
The importance of evidence-based legislation can't be overstated, especially in the current process of EU copyright revision.
As Heald notes, past copyright policy has relied on a number of incorrect assumptions, in short:
- that private value equates with social welfare; that is, any payment associated to copyright makes society richer;
- that only private value is generated by sales under copyright monopoly;
- that absence of copyright reduces both distribution and associated payments.
Recent studies, some of which are mentioned in this paper (Pollock, Waldfogel, Heald), have instead found strong indicators that:
- consumer surplus(i.e. amounts saved by consumers) can be higher and hence contribute more to social welfare;
- absence of copyright may produce higher private value as well;
- works under traditional copyright, especially given the phenomenon of orphan works, don't manage to cover the entire market, resulting in a loss of knowledge distribution as well as of potential sales.
In short, it seems that "the public is better off when a work becomes freely available", insofar as copyright has been "robust enough to stimulate the creation of the work in the first place" and that a work "must remain available to the public after it falls into the public domain".
Findings
However, it is impossible to measure the value of knowledge acquired by society and, even considering the mere monetary value, it is impossible to measure transactions which did not happen. The English Wikipedia is used by the authors as dataset because its history is open to inspection and its content is unencumbered by copyright payments, so every "transaction" is public.
In particular, the study measures what would be the cost of gratis images not being available for use on English Wikipedia articles, as a
The calculation is done in three passages.
- 362 authors of New York Timesbestsellers of 1895–1969 are considered. Their English Wikipedia articles are checked for inclusion of portraits and copyright status thereof; the increase in page views caused by the presence of the image is calculated. To depurate other factors, authors are compared in "matched pairs" of similar popularity as suggested by Amazon review or pageviews in mid 2009. Only the lowest scoring months are considered, the general increase in pageviews is discounted, etc.
- The first proxy considered is how much it would cost to buy the images from traditional image sellers, in the hypothetical (and absurd) case that article authors were allowed to. Such an image typically costs around US$100 even if it is in the public domain or identical to the one used by our articles.
- The second proxy is how much the added pageviews are worth in terms of potential advertising revenue ($0.53 cents/view, according to [1]).
- The values are then validated on a different dataset, some hundreds composers and lyricists.
- The amounts are then expanded proportionally to all English Wikipedia articles by considering images and pageviews of a sample of 300 articles.
Clearly, the number of inferences is great, but the authors believe the findings to be robust. The pageview increase, depending on the method, was 6%, 17% or 19%, and at any rate positive. Authors with most images were those died before 1880, an outcome which has no possible technological reason nor any welfare justification: it's clearly a distortion produced by copyright.
For those fond of price tags, the English Wikipedia images were esteemed to be worth about $30,000/year for those 362 writers, or about $30m in hypothetical advertising revenue for English Wikipedia, or $200m–230m in hypothetical costs of image purchase.
At any rate, this reviewer thinks that the positive impact of the lack of copyright royalties is proven and confirms the authors' thesis. It is quite challenging to extend the finding to the whole English Wikipedia, all Wikimedia projects, the entire free knowledge landscape and finally the overall cultural works market; and even more fragile to put a price tag on it. However, this kind of one-number communication device is widely used to explain the impact of legislation and numbers traditionally used by legislators are way more fragile than this. Moreover, the study makes it possible to prove a positive impact on important literature authors and their life, i.e. their reputation, which is supposed to be the aim of copyright laws, while financial transactions are only means.
Methodological nitpicks
There are several possible observations to be made about details of the study.
- Only few hundreds articles were considered, and only on the English Wikipedia. Measuring pageviews is not explained in detail, but it clearly relied on stats.grok.se, on whose limitations see the stats.grok.se FAQ and m:Research:Page view.
- ElasticSearch, was briefly enabled but then disabled for performance reasons.)
- The author uses an artificial definition of "public domain" to match the cases which the study was able to measure, i.e. gratis images. Only 67% of the images were in the public domain while 13% were in fair use and 19% released in some way by the author. As for the releases by the authors, all cases are confusingly conflated: in particular "a Creative Commons" and "unprotected" are two incorrect terms used, which fail to recognise that CC images are copyrighted works and that not all CC images are free cultural works. This mix makes it hard to extend the results to the public domain proper, i.e. the works without any copyright protection, as well as to Wikimedia projects other than the English Wikipedia where fair use is less common. This may not affect the result on the welfare impact for the English Wikipedia but has a higher impact on the dates: namely, the fact that people who died before 2000 have less images may just mean that the English Wikipedia rules allowed fair use more for them because Wikipedia photographers would not be able to shoot photos themselves.
- Again on terminology, it is disappointing that Wikipedia's article authors are called "page builders", as if they were mechanical workers (with all due respect for mechanical workers). There is no reason to reserve the term "authors" to the professional writers who are the subjects of those articles. An artificial restriction of the pool of people who can assert to be "authors" is one of the main propaganda tools of the "pro-copyright" lobby.
Briefly
- "Automatic text summarization of Wikipedia articles":neural networksusing different features to pick sentences to summarize (English?) Wikipedia articles. They compared their results to Microsoft Word 2007 and found out results are very different.
- Relationship between Google searches and Wikipedia edits:[4] A student course paper developed a model to find a correlation between the number of searches on Google resulting from an increased public interest in a subject, and the number of edits made to that subject’s corresponding Wikipedia page. Google Trends data from 2012 for “Barack Obama”, “Google“ and “Mathematics” was compared with Wikipedia page revisions of the corresponding articles within the same period. Instead of the actual data, which was unavailable, the paper applied approximation techniques to estimate the number of Google searches and the number of Wikipedia edits during a given period. Except for a few instances of spikes matching up, no clear correlation between Google searches and Wikipedia edits was found. Similar results were observed when more graphs were generated for different topics. The model made no provision for disproving the existence of a correlation. These limitations render the results of the study still inconclusive.
- How much of the Amazon rainforest would it take to print out Wikipedia?:List of law clerks of the Supreme Court of the United States, perhaps more accurate estimates could have been made.
- Perceptions of bot services: This study[6] looked at how Wikipedians perceive bots, to enhance our understanding of the relationship between human and bot editors. The authors find that the bots are perceived as either "servants" or "policemen". Overall, the bots are well accepted by the community, a factor the authors attribute to the fact that most bots are clearly labelled as and seen as extensions of human actors (tools used by advanced Wikipedians). The authors nonetheless observe that where bots make large number of minor edits, they are most likely to attract criticism. Still, the necessity for such labor, maintaining categories, templates and such, is, according to actors, a widely recognized and accepted element of Wikipedia's life.
Other recent publications
A list of other recent publications that could not be covered in time for this issue – contributions are always welcome for reviewing or summarizing newly published research.
- "P2Pedia: a peer-to-peer wiki for decentralized collaboration"[7] (screencast demo; see also User:HaeB/Timeline_of_distributed_Wikipedia_proposals)
- '"Distributed wikis: a survey"[8] From the abstract: "We identify three classes of distributed wiki systems, each using a different collaboration model and distribution scheme for its pages: highly available wikis, decentralized social wikis and federated wikis."
- "Detection speculations using active learning" ("Deteccion de Especulaciones utilizando Active Learning")weasel wordson the English Wikipedia)
References
- ^ Morten Warncke-Wang; Vivek Ranjan; Loren Terveen & Brent Hecht (2015). "Misalignment Between Supply and Demand of Quality Content in Peer Production Communities" (PDF). Pre-print PDF, to appear in the Proceedings of the The 9th International AAAI Conference on Web and Social Media (ICWSM).
- ^ Heald, Paul J. and Erickson, Kris and Kretschmer, Martin, "The Valuation of Unprotected Works: A Case Study of Public Domain Photographs on Wikipedia" (February 4, 2015). Available at SSRN: http://ssrn.com/abstract=2560572 or http://dx.doi.org/10.2139/ssrn.2560572
- ^ Claire, Charron (2014). "Analysing Trends Between US Google Searches and English Wikipedia Page Edits" (PDF). Retrieved 26 April 2015.
{{cite journal}}
: Cite journal requires|journal=
(help) - ^ Harwood, George; Walker, Evangeline (2015). "How Much of the Amazon Would it Take to Print the Internet?". Journal of Interdisciplinary Science Topics. 4. Centre for Interdisciplinary Science, University of Leicester.
- ^
Clément, Maxime; Guitton, Matthieu J. (September 2015). "Interacting with bots online: Users' reactions to actions of automated programs in Wikipedia". Computers in Human Behavior. 50: 66–75. ISSN 0747-5632.
- S2CID 35114840.
- S2CID 45142475.
- ^ Benjamín Machíın Serna: "Deteccion de Especulaciones utilizando Active Learning". Student thesis, Universidad de la República – Uruguay, 2013 PDF)
- Supplementary references and notes:
- ^ The most important of these initiatives is probably the 2009 Public Domain Manifesto. Some examples in the context of orphan works: m:Italian cultural heritage on the Wikimedia projects#Advocating for the public domain bibliography commented in an Italian paper by this reviewer. http://arxiv.org/abs/1411.6675
Discuss this story
I haven't read the first paper yet, but I think two factors might explain some of it. Perhaps editors feel that it is easier, more manageable, and less intimidating to tackle a smaller-scale subject, such as theUS Navy. Also, perhaps editors mistakenly assume that these broad topics are already well covered by the encyclopedia. Gamaliel (talk) 02:47, 1 May 2015 (UTC)[reply ]
So, Comedy and Science Fiction topics are underdeveloped, while Politics and Birds are High-quality ... and this is a problem? Curly Turkey ¡gobble! 04:36, 1 May 2015 (UTC)[reply]
One of those cricket FAs in the topic mentioned is
Hi everyone, and apologies for being late to the party! In case you don't know, I'm the first author on the paper about popularity/quality. Thank you all for a very interesting discussion, I have jotted down notes from it once already, and will re-read it and write down more notes. The links to previous discussions along these lines are also very helpful, although I haven't yet had the time to read all of them (some of them are quite large). Let me comment on a few specific things, before I go dish out actual thanks to everyone. I'll be adding this talk page to my watchlist in case there are follow-up questions, and I welcome questions or comments on my talk page as well, of course, and I can be emailed if you want to reach me off-wiki.
Gamaliel brings up an important point with regards to why these general subjects don't have FAs (size of the topic), and Karanacs' work on Texas Revolution is a good example (massive kudos for that effort!) We think along the same lines in the paper, although perhaps not at clearly. Figuring out why something occurs was outside the scope of this paper (it's analytical, we try to describe what the world looks like, so to speak), but as I continue my research I am interested in building tools to support contributors who are interested in working on these types of articles, and then those types of issues are of course very important.
Maury Markowitz and Curly Turkey mentioned the long tail, and Jack mentioned contributors choosing from self-interest. The latter is part of our motivation for studying this and something we point to several times in the paper, we wanted to know more about how that type of work selection affects systems like Wikipedia. When it comes to the long tail, it's typically not a "problem" in the popularity context. In all four languages we studied the majority of articles are stub/start quality and they do not get a lot of views, so there is no issue there. It's also clear that because Wikipedia's contributors are volunteers, they're free to leave, and therefore a central decision process on what to work on us unlikely to happen (we discuss this in the paper). Yet, I'm thinking that it would be great if we could figure out a way to serve high-quality content to a larger portion of Wikipedia's audience, which as Karanacs pointed out doesn't mean I'd want to decrease other parts.
Lastly, a technical detail: cricket is, as Dweller and Jack point out, not an unpopular topic. In our paper we were interested in understanding what topics are in the two extremes: highly-popular non-FAs, and FAs that aren't particularly popular. In the latter group, the relative risk of encountering an article from WikiProject Cricket is very high, which is why that project made our list. In other words, we didn't try to define the entirety of topics as popular/not-popular, we instead looked at specific subsets of articles to understand more about them.
Thanks again for the comments, everyone, and please do ask if you have questions! Regards, Nettrom (talk) 22:33, 5 May 2015 (UTC)[reply]