Should I write my eval criteria or rubric before I look at model outputs?

Draft some, but expect to revise them once you read outputs. In our EvalGen study, we named this criteria drift: people need criteria to grade outputs, but grading outputs changes the criteria they want, and some criteria depended on the specific outputs people saw. In Data Flywheels for LLM Applications, we argue you cannot find good criteria by theorizing about failure modes alone.

Sources: Who Validates the Validators? Aligning LLM-Assisted Evaluation of LLM Outputs with Human Preferences (UIST 2024); Data Flywheels for LLM Applications; Do I need a reference answer or rubric before annotating data? (AI Evals FAQ, Hamel Husain and Shreya Shankar)

Guide: What Are LLM Evals?

More questions on this topic

All questions about using AI for data and knowledge work