Evaluating different methods of estimating retrieval quality for resource selection

  • Zitationsschlüssel:
    Nottelmann/Fuhr:03a
  • Titel:
    Evaluating different methods of estimating retrieval quality for resource selection
  • Autor(en):
    H. Nottelmann
    N. Fuhr
  • In:
    • Zitationsschlüssel:
      SIGIR:03
    • Titel:
      Proceedings of the 26th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval
    • Herausgeber:
      Jamie Callan
      Gordon Cormack
      Charles Clarke
      David Hawking
      Alan Smeaton
    • Verlag:
      ACM
    • In:
      Proceedings of the 26th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval
    • Jahr:
      2003
  • Jahr:
    2003

Zusammenfassung:


In a federated digital library system, it is too expensive to query every accessible library. Resource selection is the task to decide to which libraries a query should be routed. Most existing resource selection algorithms compute a library ranking in a heuristic way. In contrast, the decision-theoretic framework (DTF) follows a different approach on a better theoretic foundation: It computes a selection which minimises the overall costs (e.g. retrieval quality, time, money) of the distributed retrieval. For estimating retrieval quality the recall-precision function is proposed. In this paper, we introduce two new methods: The first one computes the empirical distribution of the probabilities of relevance from a small library sample, and assumes it to be representative for the whole library. The second method assumes that the indexing weights follow a normal distribution, leading to a normal distribution for the document scores. Furthermore, we present the first evaluation of DTF by comparing this theoretical approach with the heuristical state-of-the-art system CORI; here we find that DTF outperforms CORI in most cases.

Volltext als PDF