[TREC-9] The Ninth Text REtrieval Conference
2000-11-05
org.kosen.entty.User@2334eb5e
행사&학회소개
보고서작성신청
TREC-9는 11월13일부터 16일까지 미국 MD에 있는 NIST에서 개최.
TREC은 미국의 NIST와 DARPA의 지원을 받아, 1992년부터 TIPSTER text program의 하나로 시작된 것으로, 정보검색 분야의 새로운 기술의 창출구라고도 할 수 있다.
정보검색의 새로운 응용에 대해서 시스템의 평가를 같은 환경에서 제한된 시간내에 하고, 각 연구방법들을 비교 평가하는 자리이다.
*** 정보검색분야의 대부분의 연구는 TREC에서 제시하고 있는 방식에 따르고 있다고 할 수 있을 정도이다.
*** TREC 학회의 참여는 아무나 가능한 것이 아니라, TREC의 시스템 평가에 참여한 제한된 사람들만이 참석가능하다.
올해 KAIST팀에서도 TREC-9의 평가에 참가하여, 회의에 참석하게 되었다.
TREC-9에서 다루고 있는 분야는 다음과 같다:
Filtering Track
A task in which the user's information need is stable (and some relevant documents are known) but there is a stream of new documents. For each document, the system must make a binary decision as to whether the document should be retrieved (as opposed to forming a ranked list).
Cross-Language Track
A track that investigates the ability of retrieval systems to find documents that pertain to a topic regardless of the language in which the document is written. In TREC-9, the cross-language track will use Mandarin documents and English topics.
Interactive Track
A track studying user interaction with text retrieval systems. This year's track will use the Web document collection and a task similar (but not identical) to the Question Answering Track. All participating groups follow a common experimental protocol that provides insights into user searching.
Query Track
A track designed to foster research on the effects of query variability and analysis on retrieval performance. Each participant constructs several different versions of existing TREC topics. All groups then run all versions of the topics
Question Answering Track
A track designed to take a step closer to *information* retrieval rather than *document* retrieval. For each of a set of 500 questions, systems produce a text extract that answers the question. Different runs will have different limits on the maximum length of the extract, including a short phrase (a few words), 50 bytes, and 250 bytes.
Spoken Document Retrieval Track
A track that investigates the effects of speech recognition errors on retrieval performance. The task to be performed in TREC-9 is still to be determined. Please contact the track coordinator as soon as possible if you are interested in this track.
Web Track
A track featuring ad hoc search tasks on a document set that is a snapshot of the World Wide Web. The main focus of the track will be to form a Web test collection using pooled relevance judgments. The document set will be a 10GB subsample of the existing VLC2 document set. Topics will be created at NIST by taking queries from search engine logs and retro-fitting topic statements around them. (Thus, the true web query will be there, but there will also be a narrative explaining how it will be judged.) Relevance judgments will then be made using the traditional TREC pooling methodology, with NIST assessors doing the judging.
TREC은 미국의 NIST와 DARPA의 지원을 받아, 1992년부터 TIPSTER text program의 하나로 시작된 것으로, 정보검색 분야의 새로운 기술의 창출구라고도 할 수 있다.
정보검색의 새로운 응용에 대해서 시스템의 평가를 같은 환경에서 제한된 시간내에 하고, 각 연구방법들을 비교 평가하는 자리이다.
*** 정보검색분야의 대부분의 연구는 TREC에서 제시하고 있는 방식에 따르고 있다고 할 수 있을 정도이다.
*** TREC 학회의 참여는 아무나 가능한 것이 아니라, TREC의 시스템 평가에 참여한 제한된 사람들만이 참석가능하다.
올해 KAIST팀에서도 TREC-9의 평가에 참가하여, 회의에 참석하게 되었다.
TREC-9에서 다루고 있는 분야는 다음과 같다:
Filtering Track
A task in which the user's information need is stable (and some relevant documents are known) but there is a stream of new documents. For each document, the system must make a binary decision as to whether the document should be retrieved (as opposed to forming a ranked list).
Cross-Language Track
A track that investigates the ability of retrieval systems to find documents that pertain to a topic regardless of the language in which the document is written. In TREC-9, the cross-language track will use Mandarin documents and English topics.
Interactive Track
A track studying user interaction with text retrieval systems. This year's track will use the Web document collection and a task similar (but not identical) to the Question Answering Track. All participating groups follow a common experimental protocol that provides insights into user searching.
Query Track
A track designed to foster research on the effects of query variability and analysis on retrieval performance. Each participant constructs several different versions of existing TREC topics. All groups then run all versions of the topics
Question Answering Track
A track designed to take a step closer to *information* retrieval rather than *document* retrieval. For each of a set of 500 questions, systems produce a text extract that answers the question. Different runs will have different limits on the maximum length of the extract, including a short phrase (a few words), 50 bytes, and 250 bytes.
Spoken Document Retrieval Track
A track that investigates the effects of speech recognition errors on retrieval performance. The task to be performed in TREC-9 is still to be determined. Please contact the track coordinator as soon as possible if you are interested in this track.
Web Track
A track featuring ad hoc search tasks on a document set that is a snapshot of the World Wide Web. The main focus of the track will be to form a Web test collection using pooled relevance judgments. The document set will be a 10GB subsample of the existing VLC2 document set. Topics will be created at NIST by taking queries from search engine logs and retro-fitting topic statements around them. (Thus, the true web query will be there, but there will also be a narrative explaining how it will be judged.) Relevance judgments will then be made using the traditional TREC pooling methodology, with NIST assessors doing the judging.