+++ /dev/null
-<!doctype html public "-//w3c//dtd html 4.0 transitional//en">
-<!--
- Licensed to the Apache Software Foundation (ASF) under one or more
- contributor license agreements. See the NOTICE file distributed with
- this work for additional information regarding copyright ownership.
- The ASF licenses this file to You under the Apache License, Version 2.0
- (the "License"); you may not use this file except in compliance with
- the License. You may obtain a copy of the License at
-
- http://www.apache.org/licenses/LICENSE-2.0
-
- Unless required by applicable law or agreed to in writing, software
- distributed under the License is distributed on an "AS IS" BASIS,
- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- See the License for the specific language governing permissions and
- limitations under the License.
--->
-<html>
-<body>
-<h2>Search Quality Benchmarking.</h2>
-<p>
-This package allows to benchmark search quality of a Lucene application.
-<p>
-In order to use this package you should provide:
-<ul>
- <li>A <a href="../../search/Searcher.html">searcher</a>.</li>
- <li><a href="QualityQuery.html">Quality queries</a>.</li>
- <li><a href="Judge.html">Judging object</a>.</li>
- <li><a href="utils/SubmissionReport.html">Reporting object</a>.</li>
-</ul>
-<p>
-For benchmarking TREC collections with TREC QRels, take a look at the
-<a href="trec/package-summary.html">trec package</a>.
-<p>
-Here is a sample code used to run the TREC 2006 queries 701-850 on the .Gov2 collection:
-
-<pre>
- File topicsFile = new File("topics-701-850.txt");
- File qrelsFile = new File("qrels-701-850.txt");
- Searcher searcher = new IndexSearcher("index");
-
- int maxResults = 1000;
- String docNameField = "docname";
-
- PrintWriter logger = new PrintWriter(System.out,true);
-
- // use trec utilities to read trec topics into quality queries
- TrecTopicsReader qReader = new TrecTopicsReader();
- QualityQuery qqs[] = qReader.readQueries(new BufferedReader(new FileReader(topicsFile)));
-
- // prepare judge, with trec utilities that read from a QRels file
- Judge judge = new TrecJudge(new BufferedReader(new FileReader(qrelsFile)));
-
- // validate topics & judgments match each other
- judge.validateData(qqs, logger);
-
- // set the parsing of quality queries into Lucene queries.
- QualityQueryParser qqParser = new SimpleQQParser("title", "body");
-
- // run the benchmark
- QualityBenchmark qrun = new QualityBenchmark(qqs, qqParser, searcher, docNameField);
- SubmissionReport submitLog = null;
- QualityStats stats[] = qrun.execute(maxResults, judge, submitLog, logger);
-
- // print an avarage sum of the results
- QualityStats avg = QualityStats.average(stats);
- avg.log("SUMMARY",2,logger, " ");
-</pre>
-
-<p>
-Some immediate ways to modify this program to your needs are:
-<ul>
- <li>To run on different formats of queries and judgements provide your own
- <a href="Judge.html">Judge</a> and
- <a href="QualityQuery.html">Quality queries</a>.</li>
- <li>Create sophisticated Lucene queries by supplying a different
- <a href="QualityQueryParser.html">Quality query parser</a>.</li>
-</ul>
-
-</body>
-
-</html>