Can we forecast benchmark accuracy from models that score zero?
Can we forecast benchmark accuracy from models that score zero? We (co-leads @LukeBailey181 @rckpudi ) turn a benchmark into a series of easier hinted versions. Jointly modeling scale C and hint level H enables forecasting GSM8K acc (H=0) within 3 pts from 560