Should you rely on public benchmark scores when choosing a model?
Should you rely on public benchmark scores when choosing a model? @VikParuchuri argues "no" in this talk. To illustrate this, he shows specific problems found in popular OCR benchmarks: ParseBench: • A test expects 310 gigawatts of solar-panel capacity when th
Hamel Husain says he has known Ajay for a while and that Ajay is focused on tools for viewing data for evals, calling it a promising new entrant in the eval tools space.