Significance testing of word frequencies in corpora

Visa fullständig post



Permalänk

http://hdl.handle.net/10138/223820

Citation

Lijffijt , J , Nevalainen , T , Säily , T , Papapetrou , P , Puolamäki , K & Mannila , H 2016 , ' Significance testing of word frequencies in corpora ' , Digital scholarship in the humanities , vol. 31 , no. 2 , pp. 374-397 . https://doi.org/10.1093/llc/fqu064

Titel: Significance testing of word frequencies in corpora
Författare: Lijffijt, Jefrey; Nevalainen, Terttu; Säily, Tanja; Papapetrou, Panagiotis; Puolamäki, Kai; Mannila, Heikki
Upphovmannens organisation: Department of Modern Languages 2010-2017
Datum: 2016
Språk: eng
Sidantal: 24
Tillhör serie: Digital scholarship in the humanities
ISSN: 2055-7671
DOI: https://doi.org/10.1093/llc/fqu064
Permanenta länken (URI): http://hdl.handle.net/10138/223820
Abstrakt: Finding out whether a word occurs significantly more often in one text or corpus than in another is an important question in analysing corpora. As noted by Kilgarriff (Language is never, ever, ever, random, Corpus Linguistics and Linguistic Theory, 2005; 1(2): 263–76.), the use of the X2 and log-likelihood ratio tests is problematic in this context, as they are based on the assumption that all samples are statistically independent of each other. However, words within a text are not independent. As pointed out in Kilgarriff (Comparing corpora, International Journal of Corpus Linguistics, 2001; 6(1): 1–37) and Paquot and Bestgen (Distinctive words in academic writing: a comparison of three statistical tests for keyword extraction. In Jucker, A., Schreier, D., and Hundt, M. (eds), Corpora: Pragmatics and Discourse. Amsterdam: Rodopi, 2009, pp. 247–69), it is possible to represent the data differently and employ other tests, such that we assume independence at the level of texts rather than individual words. This allows us to account for the distribution of words within a corpus. In this article we compare the significance estimates of various statistical tests in a controlled resampling experiment and in a practical setting, studying differences between texts produced by male and female fiction writers in the British National Corpus. We find that the choice of the test, and hence data representation, matters. We conclude that significance testing can be used to find consequential differences between corpora, but that assuming independence between all words may lead to overestimating the significance of the observed differences, especially for poorly dispersed words. We recommend the use of the t-test, Wilcoxon rank sum test, or bootstrap test for comparing word frequencies across corpora.
Subject: 113 Computer and information sciences
significance testing
bootstrap
chi-square test
log-likelihood ratio test
keywords
6121 Languages
corpus linguistics
text corpora
British National Corpus
Referentgranskad: Ja
Användningsbegränsning: openAccess
Parallelpublicerad version: acceptedVersion
Finansierad av: Unknown funder
Finansierings ID:


Filer under denna titel

Totalt antal nerladdningar: Laddar...

Filer Storlek Format Granska
llc_postprint.pdf 1.793Mb PDF Granska/Öppna

Detta dokument registreras i samling:

Visa fullständig post