Evaluating Efficiency vs. Effectiveness for Vague Queries and Similarity Search in Digital Libraries

  • Citation-Key:
    Fuhr:02
  • Title:
    Evaluating Efficiency vs. Effectiveness for Vague Queries and Similarity Search in Digital Libraries
  • Author(s):
    N. Fuhr
  • Publisher:
    ERCIM
  • In:
    Fourth DELOS/NSF Workshop on Evaluation of Digital Libraries: Testbeds, Measurements, and Metrics
  • Year:
    2002

Abstract:

In this paper, we discuss the relationship between result quality and effectiveness, thus motivating approximation methods for computation-intensive searches. For evaluating the quality of approximations, we first propose a new paradigm for measuring the quality of content-based searches. For standard evaluations, this new paradigm yields metrics similar to known measures. However, if the user is only interested in a small number of documents, we get different metrics. Based on our paradigm, we derive a new measure for evaluating the quality of approximation methods for similarity searches. Finally, we discuss the relationship between similarity-based and relevance-based evaluations, and we justify the need for the latter.
BibTeX entry

Fulltext as PDF