org.openimaj.experiment.evaluation.retrieval.analysers

Class IREvalAnalyser<QUERY,DOCUMENT extends org.openimaj.data.identity.Identifiable>

java.lang.Object

org.openimaj.experiment.evaluation.retrieval.analysers.IREvalAnalyser<QUERY,DOCUMENT>

Type Parameters:

QUERY - Type of query

DOCUMENT - Type of document

All Implemented Interfaces:

RetrievalAnalyser<IREvalResult,QUERY,DOCUMENT>

public class IREvalAnalyser<QUERY,DOCUMENT extends org.openimaj.data.identity.Identifiable>
extends Object
implements RetrievalAnalyser<IREvalResult,QUERY,DOCUMENT>

An evaluator suitable for any kind of result document from the RetrievalEngine. Retrieval statistics are calculated using IREval.

If the retrieved documents are instances of Ranked, then the rank is determined through Ranked.getRank(), otherwise the rank of each result document is automatically determined from its position in the results list.

Similarly, if the retrieved documents are instances of Scored, then the score is determined through Scored.getScore(), otherwise the score of each result document is set as 1.0/rank.

If the queries are Identifiable, then the query ids in the outputted SetRetrievalEvaluator are the ID of the query; if the queries are not Identifiable, then the Object.toString() method is used instead.

Author:

Jonathon Hare (jsh2@ecs.soton.ac.uk)

Constructor Summary

Constructors

Constructor and Description
IREvalAnalyser()

Method Summary

Modifier and Type	Method and Description
IREvalResult	analyse(Map<QUERY,List<DOCUMENT>> results, Map<QUERY,Set<DOCUMENT>> relevant) Analyse ranked results from a RetrievalEngine against a ground-truth set of relevant results and produce an AnalysisResult which can be read by a human.
protected static <Q,D extends org.openimaj.data.identity.Identifiable> TreeMap<String,ArrayList<RetrievalEvaluator.Judgment>>	convertRelevant(Map<Q,Set<D>> relevant)
protected static <Q,D extends org.openimaj.data.identity.Identifiable> TreeMap<String,ArrayList<RetrievalEvaluator.Document>>	convertResults(Map<Q,List<D>> results)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

IREvalAnalyser

public IREvalAnalyser()

Method Detail

convertResults

protected static <Q,D extends org.openimaj.data.identity.Identifiable> TreeMap<String,ArrayList<RetrievalEvaluator.Document>> convertResults(Map<Q,List<D>> results)

convertRelevant

protected static <Q,D extends org.openimaj.data.identity.Identifiable> TreeMap<String,ArrayList<RetrievalEvaluator.Judgment>> convertRelevant(Map<Q,Set<D>> relevant)

analyse

public IREvalResult analyse(Map<QUERY,List<DOCUMENT>> results,
                            Map<QUERY,Set<DOCUMENT>> relevant)

Description copied from interface: RetrievalAnalyser

Analyse ranked results from a RetrievalEngine against a ground-truth set of relevant results and produce an AnalysisResult which can be read by a human.

Specified by:

analyse in interface RetrievalAnalyser<IREvalResult,QUERY,DOCUMENT extends org.openimaj.data.identity.Identifiable>

Parameters:

results - the results per query from the RetrievalEngine.

relevant - the ground-truth relevant documents per query.

Returns:

the results of the analysis