Language Models and Smoothing Methods for Collections with Large Variation in Document Length

  • Citation-Key:
    Abdulmutalib/Fuhr:08
  • Title:
    Language Models and Smoothing Methods for Collections with Large Variation in Document Length
  • Author(s):
    Najeeb Abdulmutalib
    Norbert Fuhr
  • In:
    • Citation-Key:
      Tjoa/Wagner:08
    • Title:
      19th International Workshop on Database and Expert Systems Applications (DEXA 2008), 1-5 September 2008, Turin, Italy
    • Editor(s):
      A M. Tjoa
      R. R. Wagner
    • Publisher:
      IEEE Computer Society
    • In:
      DEXA Workshops
    • Year:
      2008
    BibTeX entry
  • Page(s):
    9-14

Abstract:

In this paper we present a new language model based on an odds formula, which explicitly incorporates document length as a parameter. Furthermore, a new smoothing method called exponential smoothing is introduced, which can be combined with most language models. We present experimental results for various language models and smoothing methods on a collection with large document length variation, and show that our new methods compare favorably with the best approaches known so far.
BibTeX entry

Fulltext as PDF