Corpus linguistics
This article's use of external links may not follow Wikipedia's policies or guidelines. (June 2022) |
Corpus linguistics is the
The text-corpus method uses the body of texts written in any natural language to derive the set of abstract rules which govern that language. Those results can be used to explore the relationships between that subject language and other languages which have undergone a similar analysis. The first such corpora were manually derived from source texts, but now that work is automated.
Corpora have not only been used for linguistics research, they have also been used to compile
Experts in the field have differing views about the annotation of a corpus. These views range from
History
Part of a series on |
Linguistics |
---|
Portal |
Some of the earliest efforts at grammatical description were based at least in part on corpora of particular religious or cultural significance. For example,
, andEnglish corpora
A landmark in modern corpus linguistics was the publication of Computational Analysis of Present-Day American English in 1967. Written by Henry Kučera and W. Nelson Francis, the work was based on an analysis of the Brown Corpus, which was a contemporary compilation of about a million American English words, carefully selected from a wide variety of sources.[4] Brown's corpus was the first computerized corpus designed for linguistic research.[5] Kučera and Francis subjected the Brown Corpus to a variety of computational analyses and then combined elements of linguistics, language teaching, psychology, statistics, and sociology to create a rich and variegated opus. A further key publication was Randolph Quirk's "Towards a description of English Usage" in 1960[6] in which he introduced the Survey of English Usage. Quirk's corpus was the first modern corpus to be built with the purpose of representing the whole language.[7]
Shortly thereafter, Boston publisher
Other publishers followed suit. The British publisher Collins'
The
The first computerized corpus of transcribed spoken language was constructed in 1971 by the Montreal French Project,[9] containing one million words, which inspired Shana Poplack's much larger corpus of spoken French in the Ottawa-Hull area.[10]
Multilingual Corpora
In the 1990s, many of the notable early successes on statistical methods in natural-language programming (NLP) occurred in the field of machine translation, due especially to work at IBM Research. These systems were able to take advantage of existing multilingual textual corpora that had been produced by the Parliament of Canada and the European Union as a result of laws calling for the translation of all governmental proceedings into all official languages of the corresponding systems of government.
There are corpora in non-European languages as well. For example, the National Institute for Japanese Language and Linguistics in Japan has built a number of corpora of spoken and written Japanese. Sign language corpora have also been created using video data.[11]
Ancient languages corpora
Besides these corpora of living languages, computerized corpora have also been made of collections of texts in ancient languages. An example is the Andersen-Forbes database of the Hebrew Bible, developed since the 1970s, in which every clause is parsed using graphs representing up to seven levels of syntax, and every segment tagged with seven fields of information.[12][13] The Quranic Arabic Corpus is an annotated corpus for the Classical Arabic language of the Quran. This is a recent project with multiple layers of annotation including morphological segmentation, part-of-speech tagging, and syntactic analysis using dependency grammar.[14] The Digital Corpus of Sanskrit (DCS) is a "Sandhi-split corpus of Sanskrit texts with full morphological and lexical analysis... designed for text-historical research in Sanskrit linguistics and philology."[15]
Corpora from specific fields
Besides pure linguistic inquiry, researchers had begun to apply corpus linguistics to other academic and professional fields, such as the emerging sub-discipline of
Methods
Corpus linguistics has generated a number of research methods, which attempt to trace a path from data to theory. Wallis and Nelson (2001)[20] first introduced what they called the 3A perspective: Annotation, Abstraction and Analysis.
- Annotation consists of the application of a scheme to texts. Annotations may include structural markup, part-of-speech tagging, parsing, and numerous other representations.
- Abstraction consists of the translation (mapping) of terms in the scheme to terms in a theoretically motivated model or dataset. Abstraction typically includes linguist-directed search but may include e.g., rule-learning for parsers.
- Analysis consists of statistically probing, manipulating and generalising from the dataset. Analysis might include statistical evaluations, optimisation of rule-bases or knowledge discovery methods.
Most lexical corpora today are part-of-speech-tagged (POS-tagged). However even corpus linguists who work with 'unannotated plain text' inevitably apply some method to isolate salient terms. In such situations annotation and abstraction are combined in a lexical search.
The advantage of publishing an annotated corpus is that other users can then perform experiments on the corpus (through corpus managers). Linguists with other interests and differing perspectives than the originators' can exploit this work. By sharing data, corpus linguists are able to treat the corpus as a locus of linguistic debate and further study.[21]
See also
- A Linguistic Atlas of Early Middle English
- Collocation
- Collostructional analysis
- Concordance (KWIC)
- Keyword (linguistics)
- Linguistic Data Consortium
- List of text corpora
- Machine translation
- Natural Language Toolkit
- Pattern grammar
- Search engines: they access the "web corpus"
- Semantic prosody
- Speech corpus
- Text corpus
- Translation memory
- Treebank
- Word list
Notes and references
- ISBN 978-0-08-044854-1, retrieved 31 October 2023
- ^ Sinclair, J. 'The automatic analysis of corpora', in Svartvik, J. (ed.) Directions in Corpus Linguistics (Proceedings of Nobel Symposium 82). Berlin: Mouton de Gruyter. 1992.
- ^ Wallis, S. 'Annotation, Retrieval and Experimentation', in Meurman-Solin, A. & Nurmi, A.A. (ed.) Annotating Variation and Change. Helsinki: Varieng, [University of Helsinki]. 2007. e-Published
- ISBN 978-0870571053.
- ISBN 978-0-08-043076-8, retrieved 31 October 2023
- .
- ISBN 978-0-08-043076-8, retrieved 31 October 2023
- ISBN 978-0582517349.
- ^ Sankoff, David; Sankoff, Gillian (1973). Darnell, R. (ed.). "Sample survey methods and computer-assisted analysis in the study of grammatical variation". Canadian Languages in Their Social Context. Edmonton: Linguistic Research Incorporated: 7–63.
- ISBN 978-90-272-3546-6.
- ^ "National Center for Sign Language and Gesture Resources at B.U." www.bu.edu. Retrieved 31 October 2023.
- ^ Andersen, Francis I.; Forbes, A. Dean (2003), "Hebrew Grammar Visualized: I. Syntax", Ancient Near Eastern Studies, vol. 40, pp. 43–61 [45]
- ISBN 0-931464-26-9
- ^ Dukes, K., Atwell, E. and Habash, N. 'Supervised Collaboration for Syntactic Annotation of Quranic Arabic'. Language Resources and Evaluation Journal. 2011.
- ^ "Digital Corpus of Sanskrit (DCS)". Retrieved 28 June 2022.
- arXiv:2204.13384.
- ISBN 979-10-95546-34-4.
- ISBN 978-0-08-044854-1, retrieved 31 October 2023
- ^ Mainz, Johannes Gutenberg-Universität. "Corpus Linguistics | ENGLISH LINGUISTICS". Johannes Gutenberg-Universität Mainz (in German). Retrieved 31 October 2023.
- ^ Wallis, S. and Nelson G. Knowledge discovery in grammatically analysed corpora. Data Mining and Knowledge Discovery, 5: 307–340. 2001.
- ^ Baker, Paul; Egbert, Jesse, eds. (2016). Triangulating Methodological Approaches in Corpus-Linguistic Research. New York: Routledge.
Further reading
Books
- Biber, D., Conrad, S., Reppen R. Corpus Linguistics, Investigating Language Structure and Use, Cambridge: Cambridge UP, 1998. ISBN 0-521-49957-7
- McCarthy, D., and Sampson G. Corpus Linguistics: Readings in a Widening Discipline, Continuum, 2005. ISBN 0-8264-8803-X
- Facchinetti, R. Theoretical Description and Practical Applications of Linguistic Corpora. Verona: QuiEdit, 2007 ISBN 978-88-89480-37-3
- Facchinetti, R. (ed.) Corpus Linguistics 25 Years on. New York/Amsterdam: Rodopi, 2007 ISBN 978-90-420-2195-2
- Facchinetti, R. and Rissanen M. (eds.) Corpus-based Studies of Diachronic English. Bern: Peter Lang, 2006 ISBN 3-03910-851-4
- Lenders, W. Computational lexicography and corpus linguistics until ca. 1970/1980, in: Gouws, R. H., Heid, U., Schweickard, W., Wiegand, H. E. (eds.) Dictionaries – An International Encyclopedia of Lexicography. Supplementary Volume: Recent Developments with Focus on Electronic and Computational Lexicography. Berlin: De Gruyter Mouton, 2013 ISBN 978-3112146651
- Fuß, Eric et al. (Eds.): Grammar and Corpora 2016, Heidelberg: Heidelberg University Publishing, 2018. ).
- Stefanowitsch A. 2020. Corpus linguistics: A guide to the methodology. Berlin: Language Science Press. doi:10.5281/zenodo.3735822 Open Access https://langsci-press.org/catalog/book/148.
Book series
Book series in this field include:
- Language and Computers (Brill)
- Studies in Corpus Linguistics (John Benjamins)
- English Corpus Linguistics (Peter Lang)
- Corpus and Discourse (Bloomsbury)
Journals
There are several international peer-reviewed journals dedicated to corpus linguistics, for example:
- Corpora
- Corpus Linguistics and Linguistic Theory
- ICAME Journal
- International Journal of Corpus Linguistics
- Language Resources and Evaluation Journal, supported by the European Language Resources Association
- Research in Corpus Linguistics, supported by the Spanish Association for Corpus Linguistics (AELINCO)
External links
- Bookmarks for Corpus-based Linguists – very comprehensive site with categorized and annotated links to language corpora, software, references, etc.
- Corpora discussion list
- Freely-available, web-based corpora (100 million – 400 million words each): American (COCA, COHA), British (BNC), Time, Spanish, Portuguese
- Manuel Barbera's overview site
- AskOxford.com the composition and use of the Oxford Corpus
- DMCBC.com
- Datum Multilanguage Corpora Based on chinese free sample download
- Corpus4u Community a Chinese online forum for corpus linguistics
- McEnery and Wilson's Corpus Linguistics Page
- Corpus Linguistics with R mailing list
- Research and Development Unit for English Studies Archived 29 October 2010 at the Wayback Machine
- Survey of English Usage
- The Centre for Corpus Linguistics at Birmingham University Archived 9 April 2003 at the Wayback Machine
- Tools for Corpus Linguistics (annotated list)
- Gateway to Corpus Linguistics on the Internet: an annotated guide to corpus resources on the web
- Biomedical corpora
- Linguistic Data Consortium, a major distributor of corpora
- Penn Parsed Corpora of Historical English
- Corsis: (formerly Tenka Text) an GPLed) corpus analysis tool written in C#
- ICECUP and Fuzzy Tree Fragments
- Discussion group text mining
- A corpus linguistics related conference MAG 2017: You can find some information and events related to Metadiscourse Across Genres by visiting MAG 2017 website.
- Corpus of Political Speeches, Free access to political speeches by American and Chinese politicians, developed by Hong Kong Baptist University Library
- LightTag -Text Annotation Tool, A text annotation tool for machine learning corpus focused on team management
- LIVAC Synchronous Corpus