8000 [codex] Separate topics and qrels data by lintool · Pull Request #118 · castorini/eval · GitHub
[go: up one dir, main page]

Skip to content

[codex] Separate topics and qrels data - #118

Merged
lintool merged 2 commits into
masterfrom
agent/separate-topics-and-qrels
Aug 9, 2026
Merged

lintool merged 2 commits into
masterfrom
agent/separate-topics-and-qrels

Conversation

@lintool
@lintool lintool commented Aug 9, 2026
Copy link
Copy Markdown
Member

Summary

  • separate evaluation topics and qrels into dedicated directories
  • move topic and qrels metadata and aliases to the repository root
  • update the README with the new repository purpose, layout, and branding

Validation

  • git diff --check
  • python -m json.tool topics.json >/dev/null
  • python -m json.tool topics-aliases.json >/dev/null
  • python -m json.tool qrels.json >/dev/null
  • python -m json.tool qrels-aliases.json >/dev/null

@lintool
lintool merged commit 43add83 into master Aug 9, 2026
@lintool
lintool deleted the agent/separate-topics-and-qrels branch August 9, 2026 13:57
@lintool
lintool commented Aug 9, 2026
Copy link
Copy Markdown
Member Author

Also, I've renamed repo from anserini-tool to eval to better reflect its contents.

@jvfd3
jvfd3 commented Aug 17, 2026
Copy link
Copy Markdown

This broke the Pyserini's 2cr for BEIR for pyserini version 1.4.0 as reported in this issue.

@lintool
lintool commented Aug 23, 2026
Copy link
Copy Markdown
Member Author

Unfortunately, the most obvious solution to add a symlink won't work, as a raw.githubusercontent.com URL does not redirect. We'll have to fix downstream consumers...

@lintool lintool mentioned this pull request Aug 23, 2026
@lintool
lintool commented Aug 23, 2026
Copy link
Copy Markdown
Member Author

I added note about breakage in README: #127

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants

0