Jerusalem, 5 November, 2026 (TPS-IL) — Researchers from Tel Aviv University and Stanford University have demonstrated that artificial intelligence (AI) models can develop general learning abilities without being trained on real-world data, potentially opening a new path for AI development as high-quality training material becomes increasingly scarce.
The findings could help address one of the industry’s biggest constraints: the supply of human-generated training data.
“We show that a model can get steadily better at predicting real text, images, audio, music, DNA and code without ever training on any of it,” Dr. Yoav Levine told The Press Service of Israel. The study, published as a preprint on the online research repository arXiv, has not yet undergone peer review.
The research was supervised by Levine of Tel Aviv University’s School of Computer Science and AI and conducted as part of the postdoctoral work of Dr. Kfir Dolev, in collaboration with Stanford University researchers.
Unlike conventional AI systems, which learn from enormous collections of books, websites, images and recordings, the new approach uses two networks that start from scratch and learn together. One generates short computer programs that produce sequences of symbols, while the other learns to predict the next symbol. The first network receives feedback on how much its exercises help the second improve, gradually learning to generate more useful challenges.
“The usual assumption is that AI needs compute and data. We ask how far can you get without data, based on compute alone,” Levine told TPS-IL.
He compared the approach to AlphaZero, which learned chess and Go through self-play rather than studying human games. The researchers found that the learning model’s own progress could serve as a measure of success, guiding the other network to generate increasingly useful exercises.
To test whether the skills transferred beyond artificial training, the researchers evaluated the model on real-world material it had never encountered before. Its predictive abilities improved across several domains as more computing resources were added. It could also infer rules from examples presented during testing without additional training.
The researchers also found that the system independently generated mathematical structures, including geometric sequences, Fibonacci-like patterns, and quadratic and cubic sequences. Levine said none appeared in 164 million randomly sampled programs from the same space, suggesting the system discovered them because they helped its learning partner make better predictions.
Beyond the Data Bottleneck
The method could provide a new source of training material in fields where data is scarce or expensive to obtain.
“This matters when high-quality data becomes the bottleneck, as many expect to be the case in human-generated text or code, and even more in fields where data is scarce or expensive,” Levine told TPS-IL.
It could also serve as a preliminary training stage before conventional AI training. In their experiments, the researchers found that models exposed to self-generated exercises learned faster when given real-world data.
There are significant limitations, however. The experiments involved models with fewer than 25 million parameters, more than 10,000 times smaller than today’s largest AI systems. Whether the results hold at that scale remains unknown. Levine said the researchers are now investigating whether the approach can be extended to substantially larger models.
Self-directed learning also cannot replace factual knowledge about the world. A system can discover mathematical patterns through computation, but it cannot independently establish historical facts or learn about physical reality without observing it.
“Self-play teaches general structure, the patterns shared across many kinds of data, but not facts about our world, like who wrote Hamlet,” Levine told TPS-IL. “Those still have to come from real data, so in these aspects it complements human data rather than replacing it.”
The researchers also addressed the risk of AI systems repeating and amplifying their own mistakes.
“Our learner trains on what programs output when they are actually run. The data is exactly what each program computes, so there are no hallucinations to reinforce,” Levine explained to TPS-IL. “The risks we do face are the program-writer getting stuck in a narrow niche, forgetting what it found earlier, or gaming its reward. We guard against these by rewarding real learning progress, keeping an archive of diverse programs, replaying earlier programs so they aren’t forgotten, keeping the generator close to a simple starting distribution over programs.”
The next major milestone, Levine said, would be demonstrating that the approach works with models containing billions of parameters and that self-generated training can replace a measurable amount of conventional training data. Ultimately, researchers want to determine whether an AI model can continue improving through self-play even after exhausting its available real-world training material.