A paper published in the Journal of Cosmology and Astroparticle Physics on June 10 asks a deceptively simple question: if you train a neural network on the standard cosmological model and then ask it to look for physics beyond that model, does the pretraining help or hurt?
The answer, from a Princeton and Flatiron Institute team, is both. Transfer learning — pretraining on cheap ΛCDM simulations, then fine-tuning on expensive beyond-ΛCDM runs — cut the number of simulations needed by more than a factor of 10 in some scenarios. But it also introduced a failure mode the authors call negative transfer: the network learned the standard model’s patterns so thoroughly that it confused genuinely new physics with familiar old parameters.
What transfer learning actually does here
Cosmological simulations are expensive. Running a single N-body simulation of the universe under modified gravity or massive neutrinos can take thousands of CPU-hours. If you want to train a neural network to infer cosmological parameters from survey data, you need hundreds or thousands of those simulations to build a training set. For the standard ΛCDM model, those training sets already exist. For beyond-ΛCDM theories — the models that might contain new physics — they don’t, because nobody can afford to run that many.
Transfer learning offers a workaround. You pretrain the network on ΛCDM simulations, which are plentiful and cheaper to generate. The network learns the basic relationships between cosmological parameters and the large-scale structure of the universe — how matter clusters, how voids form, how the power spectrum changes with different values of Ωm or σ8. Then you take that pretrained network and fine-tune it on a much smaller set of beyond-ΛCDM simulations.
Adrian Bayer, a co-author at the Flatiron Institute and Princeton, described it as “basically a shortcut — rather than training the AI directly on the most computationally expensive simulations, we first use simpler and less expensive ΛCDM simulations to give the AI an idea of what’s happening, and only afterward move to the more complex models.”
The Quijote test bed
The team tested transfer learning on the Quijote simulation suite, a publicly available library of over 44,000 N-body simulations that map the same cosmic volume under different cosmological assumptions. Quijote includes ΛCDM runs as well as simulations with massive neutrinos, modified gravity, and primordial non-Gaussianities — three extensions of the standard model that current and future surveys hope to constrain.
For each beyond-ΛCDM scenario, the team compared two approaches:
- Train a network from scratch on beyond-ΛCDM simulations only
- Pretrain on ΛCDM, then fine-tune on the same beyond-ΛCDM data
In the best cases, the transfer approach matched the from-scratch accuracy using roughly one-tenth the number of expensive simulations. For survey teams planning to analyze data from Rubin Observatory’s LSST or ESA’s Euclid mission, that’s a meaningful saving — it means you can explore more theoretical models without proportionally scaling your compute budget.
Where it breaks: negative transfer
The team found that in specific scenarios, pretraining on ΛCDM actually made the network worse at detecting new physics than training from scratch. They call this negative transfer, and it’s driven by real physical degeneracies between the standard model and its extensions.
The clearest example involves neutrino mass. Massive neutrinos suppress the growth of structure on small scales — they free-stream out of overdense regions because they move too fast for gravity to trap them. But in the standard ΛCDM model, a parameter called σ8 (which measures the overall amplitude of matter clustering) can produce a very similar suppression pattern. The pretrained network had already learned to associate that pattern with σ8 variations. When fine-tuned on simulations where the suppression came from neutrino mass instead, the network struggled to tell the difference.
As first author Veena Krishnaraj put it: “The negative transfer is not random. It is driven by underlying physical degeneracies in the model.”
And that’s the core tension with transfer learning in this context: it works by assuming that early-layer features are reusable across tasks. In computer vision, that’s mostly true — edges and textures don’t change between cats and cars. But in cosmology, the “features” are statistical summaries of the matter distribution, and some of those summaries look identical whether they’re produced by σ8 or by neutrino mass. The network doesn’t just carry useful knowledge forward; it carries biases.
Architecture matters
One detail I found particularly interesting: the team reported that bottleneck architectures — networks with a compressed intermediate representation — performed better on transfer learning than wider architectures. The bottleneck forces the network to learn a compact summary of the simulation, and that compact summary apparently generalizes better across cosmological models.
This makes sense if you think about it in information-theoretic terms. A wider network can memorize specific parameter-to-output mappings from the ΛCDM pretraining set. A bottleneck network can’t afford to memorize — it has to extract the underlying structure. When you then fine-tune on beyond-ΛCDM data, the bottleneck representation is more adaptable precisely because it’s less specific.
It’s a familiar pattern from other domains (this is how variational autoencoders work, more or less), but seeing it validated in a cosmological context is useful. It suggests that the architecture choice isn’t just a hyperparameter to tune — it determines whether transfer learning helps or hurts.
What this means for the next generation of surveys
Rubin Observatory will start its 10-year Legacy Survey of Space and Time in the coming years, imaging the entire southern sky every few nights. Euclid is already in orbit, mapping the geometry of the universe across 10 billion years of cosmic history. Both missions will generate data volumes that demand automated inference — no human team can fit cosmological parameters by hand to millions of galaxy shapes and redshifts.
Transfer learning is one of the most promising tools for scaling that inference to beyond-ΛCDM models. But this paper is a clear warning: if the new physics you’re searching for happens to mimic an existing parameter in your pretraining set, the network will miss it. And you won’t necessarily know it’s missing it, because the loss function looks fine — the network is confidently predicting something, just the wrong thing.
The practical implication: teams using transfer learning for cosmological inference need to specifically test for negative transfer in their parameter space. They can’t just pretrain on ΛCDM and assume the features will carry over. The σ8–neutrino degeneracy is a known problem in cosmology, but there may be other degeneracies that only emerge when you extend the model in directions nobody has tested yet.
An ML problem disguised as a cosmology problem
I’ve spent enough time on both sides of the ML-and-science divide to recognize this pattern. Transfer learning assumes that tasks share low-level structure. In most practical applications — NLP, vision, speech — that’s a safe bet, because the underlying data-generating processes really do share features. Language has grammar. Images have edges.
But cosmological models are designed to be distinguishable. The whole point of running beyond-ΛCDM simulations is that they make different predictions from the standard model. When those predictions overlap with an existing ΛCDM parameter’s effect, you’ve hit a regime where transfer learning’s core assumption breaks down. The low-level features aren’t shared — they’re degenerate.
It’s a useful reminder that ML techniques don’t port between domains for free. The physics of the problem shapes which ML tools work and where they fail. That’s not a reason to avoid transfer learning in cosmology — the 10× simulation savings are real and valuable. But it’s a reason to treat the pretrained model as a starting point with known blind spots, not as a foundation you can build on without checking the ground underneath.
The paper is open access on JCAP, and the preprint is on arXiv. If you’re interested in the Quijote simulation suite that made this kind of cross-model comparison possible, the documentation and data access are public.