Can decision models be trusted for evals? Benchmarks put to test
A researcher and Hamel Husain tested top decision models like Jev, d1-ultra, clef, and pplx-decider on their dataset, examining failure detection, probability calibration, and threshold tradeoffs.