Longest–commonest Match

Authors: Adam Kilgarriff, Vít Baisa, Pavel Rychlý, Miloš Jakubíček

Abstract:
Finding two-word collocations is a well-studied task within natural language processing. The result of this task for a given headword is usually a list of collocations sorted by a salience score. In corpus manager Sketch Engine, these pairs are extracted from data using a word sketch grammar relation rules and log-dice statistics resulting in a sorted list of triples <headword, grammar-relation, collocate>. The longest–commonest match is a straightforward extension  of these two-word collocations into multiword expressions. The resulting expressions are also very useful for representing the most common realisation of the collocational pair and to facilitate the interpretation of the raw triplet because sometimes, for such a triple, it is not clear from what texts it comes. We present here an algorithm behind the longest–commonest match together with a simple evaluation. The longest–commonest match is already implemented in Sketch Engine.

Keywords: multiword expresion; collocation; word sketch; Sketch Engine

Reference: In Kosem, I., Jakubíček, M., Kallas, J., Krek, S. (eds.) Electronic lexicography in the 21st century: linking lexical data in the digital age. Proceedings of the eLex 2015 conference, 11-13 August 2015, Herstmonceux Castle, United Kingdom. Ljubljana/Brighton: Trojina, Institute for Applied Slovene Studies/Lexical Computing Ltd., pp. 397-404.

URL: https://elex.link/elex2015/proceedings/eLex_2015_26_Kilgarriff+etal.pdf

Published: 2015