Language Models and Smoothing Methods for Collections with Large Variation in Document Length
-
- Citation-Key:
- Abdulmutalib/Fuhr:08
-
- Title:
- Language Models and Smoothing Methods for Collections with Large Variation in Document Length
-
- Author(s):
- Najeeb Abdulmutalib
- Norbert Fuhr
-
- In:
-
- Citation-Key:
- Tjoa/Wagner:08
-
- Title:
- 19th International Workshop on Database and Expert Systems Applications (DEXA 2008), 1-5 September 2008, Turin, Italy
-
- Editor(s):
- A M. Tjoa
- R. R. Wagner
-
- Publisher:
- IEEE Computer Society
-
- In:
- DEXA Workshops
-
- Year:
- 2008
-
- Page(s):
- 9-14
Abstract:
In this paper we present a new language model based on an odds formula, which explicitly incorporates document length as a parameter. Furthermore, a new smoothing method called exponential smoothing is introduced, which can be combined with most language models. We present experimental results for various language models and smoothing methods on a collection with large document length variation, and show that our new methods compare favorably with the best approaches known so far.
Fulltext as PDF