You can't solve real-world physical tasks through autoregressive sampling alone. The physical world is entirely unforgiving to hallucination or approximate pattern matching. If we want actual autonomous robotics we have to engineer causality and discrete reasoning back into the
Chollet: Autoregressive Sampling Alone Can't Solve Physical Tasks
François Chollet argues the physical world is unforgiving to hallucination and approximate pattern matching; true autonomous robotics requires engineering causality and discrete reasoning back in.