ivory.core.eval

Class Qrels

java.lang.Object

ivory.core.eval.Qrels

Direct Known Subclasses:

GradedQrels

public class Qrels
extends Object

Representation of relevance judgments. In TREC parlance, qrels are judgments made by humans as to whether a document is relevant to an information need (i.e., topic). Typically, qrels are created by a process known as "pooling" in large-scale system evaluations such as those at TREC.

Author:

Jimmy Lin

Constructor Summary

Constructors

Constructor and Description
Qrels(String file) Creates a Qrels object from a file

Method Summary

Methods

Modifier and Type	Method and Description
Set<String>	getQids() Returns a set containing the topic ids.
Set<String>	getReldocsForQid(String qid) Returns the set of relevant documents for a topic.
float	helperHDFSTrainWSDTopics() Used with RunQueryHDFSTrainWSD class
boolean	isRelevant(String qid, String docid) Determines if a document is relevant for a topic.

Methods inherited from class java.lang.Object

equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

Qrels

public Qrels(String file)

Creates a Qrels object from a file

Parameters:

file - file containing qrels

Method Detail

getQids

public Set<String> getQids()

Returns a set containing the topic ids.

Returns:

a set containing the topic ids

getReldocsForQid

public Set<String> getReldocsForQid(String qid)

Returns the set of relevant documents for a topic.

Parameters:

qid - topic id

Returns:

the set of relevant documents

helperHDFSTrainWSDTopics

public float helperHDFSTrainWSDTopics()

Used with RunQueryHDFSTrainWSD class

Returns:

number of topics needed by RunQueryHDFSTrainWSD to compute effectiveness scores

isRelevant

public boolean isRelevant(String qid,
                 String docid)

Determines if a document is relevant for a topic.

Parameters:

qid - topic id

docid - id of the document to test

Returns:

true if the document is relevant