What would pretraining with zero real data look like? Can a randomly initialized model learn to generate all of its training data entirely through self-play? Find out below! :)
Pretraining with zero real data via self-play
Stanford AI Lab asks: can a randomly initialized model generate all its training data through self-play, with zero real data pretraining?