TAG COLLECTION

long context

All AI Pulse updates tagged "long context".

3 signals

Long-context models struggle to find corpus contradictions

sewon__min says long-context LMs do well on RULER-like tasks but suffer at finding every contradiction as corpora scale, even with in-domain fine-tuning.

Signal score 100Source

X.com09/23, 20:19Model Release

MiMo-V3 adopts HySparse2 for cheaper long context

MiMo-V3 is getting a new architecture: core HySparse2 cuts prefill and KV cache and improves long-context retrieval versus MiMo-V2.6 Hybrid SWA, with 5.02× lower prefill FLOPs at 1M tokens.

MiMo-V3 HySparse2 KV cache long context

Signal score 100Source

X.com06/12, 23:48Infrastructure

AgentPerf treats agentic AI as a workload

AgentPerf treats agentic AI as a long-running workload, considering memory and networking.

rohanpaul_ai AI MoE long context

Signal score 13Source