language models

Long-context models struggle to find corpus contradictions

sewon__min says long-context LMs do well on RULER-like tasks but suffer at finding every contradiction as corpora scale, even with in-domain fine-tuning.