What are LLM evals?

LLM evals are checks that tell you whether an LLM application’s outputs are good enough for the people who use it. Each eval is either a code check, such as “the output parses as JSON”, or an LLM judge, a prompt that asks a model whether an output meets a criterion, such as “the reply answers the customer’s question”.

Start with the AI Evals FAQ

Hamel Husain and Shreya Shankar answer the questions people ask most about evals in AI Evals: Everything You Need to Know. Good first reads:

Our research

Questions