Develop new tasks, metrics or experimental methods for capabilities and user experience issues that existing evaluations miss, and test their validity, bias and reproducibility.
Build reproducible agent evaluations, regression tests and end-to-end observability, using real tasks and production failures to guide system improvements.