Automatic Language Identification in Texts : A Survey

Show full item record



Permalink

http://hdl.handle.net/10138/305331

Citation

Jauhiainen , T , Lui , M , Zampieri , M , Baldwin , T & Lindén , K 2019 , ' Automatic Language Identification in Texts : A Survey ' , Journal of Artificial Intelligence Research , vol. 65 , pp. 675-782 . https://doi.org/10.1613/jair.1.11675

Title: Automatic Language Identification in Texts : A Survey
Author: Jauhiainen, Tommi; Lui, Marco; Zampieri, Marcos; Baldwin, Timothy; Lindén, Krister
Contributor organization: Krister Linden / Research Group
Language Technology
Centre of Excellence in Ancient Near Eastern Empires (ANEE)
Department of Digital Humanities
Date: 2019-08-25
Language: eng
Number of pages: 108
Belongs to series: Journal of Artificial Intelligence Research
ISSN: 1076-9757
DOI: https://doi.org/10.1613/jair.1.11675
URI: http://hdl.handle.net/10138/305331
Abstract: Language identification (LI) is the problem of determining the natural language that a document or part thereof is written in. Automatic LI has been extensively researched for over fifty years. Today, LI is a key part of many text processing pipelines, as text processing techniques generally assume that the language of the input text is known. Research in this area has recently been especially active. This article provides a brief history of LI research, and an extensive survey of the features and methods used so far in the LI literature. For describing the features and methods we introduce a unified notation. We discuss evaluation methods, applications of LI, as well as off-the-shelf LI systems that do not require training by the end user. Finally, we identify open issues, survey the work to date on each issue, and propose future directions for research in LI.
Subject: 6121 Languages
213 Electronic, automation and communications engineering, electronics
Peer reviewed: Yes
Rights: unspecified
Usage restriction: openAccess
Self-archived version: publishedVersion


Files in this item

Total number of downloads: Loading...

Files Size Format View
AutomaticLanguageIdent.pdf 553.2Kb PDF View/Open

This item appears in the following Collection(s)

Show full item record