Evaluating Efficiency vs. Effectiveness for Vague Queries and Similarity Search in Digital Libraries
- Evaluating Efficiency vs. Effectiveness for Vague Queries and Similarity Search in Digital Libraries
- N. Fuhr
- Fourth DELOS/NSF Workshop on Evaluation of Digital Libraries: Testbeds, Measurements, and Metrics
In this paper, we discuss the relationship between result quality and effectiveness, thus motivating approximation methods for computation-intensive searches. For evaluating the quality of approximations, we first propose a new paradigm for measuring the quality of content-based searches. For standard evaluations, this new paradigm yields metrics similar to known measures. However, if the user is only interested in a small number of documents, we get different metrics. Based on our paradigm, we derive a new measure for evaluating the quality of approximation methods for similarity searches. Finally, we discuss the relationship between similarity-based and relevance-based evaluations, and we justify the need for the latter.
Fulltext as PDF