AI Feature Evaluation Framework

Design an evaluation framework for an AI feature.

Feature: {{describe}}
Inputs and outputs: {{describe_2}}
What 'good' looks like: {{how_would_an_expert_judge_a_great_output}}
What 'bad' looks like: {{examples_of_failures}}
Scale: {{outputs_per_day}}

Please design an eval framework with:
1. Automated metrics (no human review needed)
2. Human eval criteria: rubric with scoring dimensions
3. Golden set: how to build reference input/output pairs
4. Sample size per evaluation cycle
5. Decision rule: at what quality threshold do you ship, watch, or roll back?

Copied to clipboard.