Groundtruth Dynamic Benchmarking (Geology Edition) v1.0.0 Is Live

Eigenform dynamic benchmark: Ground Truth: Dynamic Benchmarking in Highly Specific Fields

Release notes: Release notes

The first runnable release our Groundtruth Dynamic Benchmarking System is live on GitHub. The core concept is simple: rather than offering a suite a benchmarking problems, we offer a pipeline for generating reliable benchmark problems and scoring rubrics for whatever problem you’re working to solve.

What you get

  • A working example, ready to run. The sample rubric is grounded in a real corpus: 34 mineral deposit records from the Yudnamutana Copper district, South Australia, openly licensed under CC BY 4.0 AU. You don’t need to track down source material yourself. Clone the repo, fill in your API keys, and ./start_eval.sh runs end to end.
  • A rubric format built to resist grade inflation. A hard gate per question, non-overlapping graduated components, required concepts, and do-not-credit lists. The mechanics, and the problems that shaped them, are covered in Dynamic Benchmarking: How We Did It.
  • Structured, auditable grading. The judge returns a full breakdown of which components it credited and why, instead of a single number. It flags anything it can’t verify from the rubric alone rather than guessing.
  • The authoring skill itself. The same corpus-to-rubric pipeline we used to build the geology edition ships in the repo, so you can point it at your own corpus and build your own edition.

Try it now

bash 
git clone https://github.com/EigenformAI/groundtruth-dynamic-benchmarking
cd groundtruth-dynamic-benchmarking
cp .env.example .env   # fill in the keys you need
uv sync
./start_eval.sh

Read more