Automatic Keyword Extraction: Combining Machine Learning and Natural Language Processing

In order to make further progress in the field of automatic text retrieval, approaches extending current standard full text indexing methods need to be investigated. This book advocates keyword indexing as one such feasible approach. More specifically it discusses the development of an algorithm for automatic keyword extraction and presents anumber of experiments in which the performance of the algorithm is incrementally improved. Automatic keyword extraction is the task of automatically selecting a small set of terms describing the content of a single document. That a keyword is extracted means that it is present verbatim in the document to which it is assigned. The approach taken is that of supervised machine learning, that is, prediction models are constructed from documents with known keywords. The work presented is linguistically oriented in the sense that the output from natural language processing tools is a considerable factor both for the pre-processing of the data, as well as for the performance of the prediction models. This is a book for anybody in the field of language technology who is interested in the applicable but challenging area of automatic keyword indexing.