cff-version: 1.2.0
title: "TREC Competition: Information Retrieval on AP 88-90"
authors:
- family-names: Loyer
given-names: Dominique S.
orcid: "https://orcid.org/0009-0003-9713-7109"
version: 1.0.0
date-released: 2025-01-06
url: "https://github.com/DominiqueLoyer/TREC_COMPETITION_88-90_AP"
repository-code: "https://github.com/DominiqueLoyer/TREC_COMPETITION_88-90_AP"
license: MIT
keywords:
- "Information Retrieval"
- "TREC"
- "BM25"Complete Python implementation of information retrieval models evaluated on TREC AP 88-90 collections.
This project implements and evaluates multiple information retrieval approaches including BM25, TF-IDF, and Vector Space Models on the TREC AP (Associated Press) 88-90 collection containing 165,000 documents.
- BM25 ranking model with parameter tuning
- TF-IDF vector space model
- Query expansion techniques (RM3, pseudo-relevance feedback)
- Comprehensive evaluation metrics (MAP, NDCG, Precision@K)
- Integration with Pyserini for efficient indexing and retrieval
- Comparative analysis of stemming strategies
pip install numpy pandas scikit-learn pyserini
This TREC repository has been successfully integrated into the systemFactChecking project (v2.3.0) as the core Information Retrieval engine for evidence gathering in fact-checking and credibility verification.
Key Integration Components:
trec_retriever.py: Evidence retrieval module based on this repositorytrec_dataset.py: TREC AP88-90 dataset loaderir_engine.py: BM25, TF-IDF, and Query Likelihood modelseval_metrics.py: MAP, NDCG, Precision@K evaluation metrics
For detailed analysis of the integration, see TREC_INTEGRATION_ANALYSIS.md
• Baseline (BM25, long queries): MAP=0.2205 • With Query Expansion (RM3): MAP=0.2948 (+34%) • Best configuration: Long query terms + BM25 + RM3 expansion
• Collection: TREC AP 88-90 (165,000 documents) • Queries: TREC topics • Format: Standard TREC format
• Evaluation of Information Retrieval Models and Query Expansion on the TREC AP 88-90 Collection • Evaluation de Modeles de Recherche d’Information • Evaluation de modeles de ponderations pour la recherche d’information sur TREC AP 88-90
Training data • Corpus: AP 1988–1990 newswire subset from the TREC collections. • Preprocessing: • Standard TREC tokenization and cleaning (e.g., SGML tag removal, lower‑casing if applicable). • Train/dev/test splits following the official TREC topics and relevance judgments. • For full details and statistics, see the associated dataset: DomLoyer/trec-ap88-90-corpus . Training procedure • Objective: • Point‑wise, pair‑wise or list‑wise ranking loss (e.g., cross‑entropy over relevant vs. non‑relevant documents). • Hyperparameters (example, à adapter à ta config réelle): • Encoder: t5-base (or the actual base model you used). • Batch size, learning rate, number of epochs, max sequence length, optimizer. • Hardware: • Specify GPU(s), training time and important implementation details. Evaluation • Benchmarks: • TREC ad‑hoc tasks on AP 88–90 topics. • Metrics: • Mean Average Precision (MAP). • Normalized Discounted Cumulative Gain (nDCG) at standard cut‑offs. • Report: • Provide a small table summarizing your main scores (e.g., MAP@100, nDCG@20) compared to baselines. Ethical considerations • The dataset contains news articles from a specific historical period and geographical context. • Users should be aware of potential biases and outdated terminology present in the corpus. • Do not deploy this model directly in user‑facing search engines without additional filtering, monitoring and evaluation. Citation If you use this model or the underlying corpus in your research, please cite: • Your paper / preprint describing the model and experiments. • The original TREC collection or any official references required by the dataset license.