I've seen a lot of pushback to this, with the following complaints: 1. "The benchmark owners already know about the flaws, and are working on it." 2. "Why are they spending time critiquing and not contributing to the benchmark" OR "critiquing known issues are not helpful". 3.
Benchmark Criticism Sparks Pushback
Hamel Husain summarized pushback to benchmark critiques: owners allegedly know flaws, critics should contribute instead, and pointing out known issues is unhelpful.