AI is about to make scientific ideas cheap. Knowing which ones are true will stay expensive. That gap is where the next generation of scientists will build their careers.
Over the last few months, off the back of reading about Andrej Karpathy’s GPT2 speedrun work and the pretty astonishing consequences of open competition, I’ve started writing detailed evaluations and letting AI loops run against them, continuously, sometimes for weeks at a time. I designed the objectives and tests (both with help), and then let the loops do the rest: propose, try, keep or discard, repeat.
Also, after reading about work on harness evolution, I started wrapping these inner loops with outer optimizers. Essentially, the same loop approach, but instead of optimizing the scientific method directly, the outer loop optimizes the harness that runs the inner loop. I’ve watched the inner loops become more effective as a result.
I've never quite lost the itch that drew me into the lab: taking a difficult problem apart and finding a better way. These days I spend more time building tools for scientists than doing the science myself, but I still enjoy getting my hands dirty. Lately, that's meant a side project exploring more efficient ways to turn raw mass spectrometry data (DIA) into reliable evidence for quantitative biological analysis. One thing that has really struck me is how capable frontier LLMs have become at drawing connections between seemingly unrelated fields. Given the right harness and the freedom to explore, they can recognize when an idea from one discipline might solve a problem in another.
Some of science’s greatest advances have come from exactly this kind of cross-pollination. One of the biggest advances in MRI, for example, came not from medicine, but from mathematics: the realization that an image could be reconstructed from far fewer measurements. What’s exciting is that AI can now explore these connections systematically, across a breadth of knowledge that no individual researcher could realistically possess, and then test whether those borrowed ideas actually work.
But something else happened that I found even more interesting. In one experiment, the AI proposed (drawn from a relatively obscure signaling concept) a different way of combining signals from peptide fragments to improve identification in complex biological samples. It implemented the change, ran the tests, and the results looked promising. A meaningful improvement over the previous approach.
Except that wasn’t the end of the experiment. My loop was also required to validate promising changes against an entirely separate dataset it hadn’t optimized against. And when it did, the improvement didn’t hold up. Performance actually got worse.
What surprised me wasn’t that the idea failed, that’s just science. It was that the system had caught the problem itself. It had proposed an idea, implemented it, measured an improvement, challenged that result with independent evidence, and recognized that the change wasn’t sufficiently generalizable. It could then discard or revise the approach and continue exploring, without needing me to intervene in any way.
This was still a computational optimization problem, not an autonomous biological discovery, and I’d deliberately built the skepticism into the evaluation. But the system could propose ideas, implement them, discover their limitations and reject them without my intervention. It was beginning to automate something fundamental about how we do science.
And it’s this process of seeing how effective properly orchestrated intelligence can be that I now find difficult to unsee. The loop was genuinely inventive. But more importantly, it had a way to test its own inventions, discover their limitations, and use that evidence to guide an iteration that is essentially only limited by my wallet.
I’ve been thinking about what happens when we can do this across more of science. Not just generate hypotheses faster, but close the entire loop between ideas, experiments, evidence, learning and the next idea. I think it’s where the next decade of science is rapidly heading.
There is a useful counterargument to all this excitement. Arvind Narayanan, the Princeton computer scientist behind AI as Normal Technology, uses the example of Amtrak’s new Acela trains. The trains themselves can travel at well over 150 miles an hour, but average speeds across the US have barely changed. The tracks are too curved, the signaling is old, and the surrounding infrastructure was never designed for those speeds.
His point is that AI is often the train. We can make one part of a process extraordinarily fast without doing much to improve the overall outcome.
Having spent thirteen years at WEHI building a proteomics capacity, and working with the Royal Melbourne Hospital, I can clearly see this problem. The pace of scientific research was rarely determined by how quickly someone could come up with an idea. It was determined by ethics approvals, sample availability, instrument time, assay development, and sometimes the weeks it takes to cultivate the right cells to tell you whether you were right.
It's much the same reason bringing a new drug to market often takes a decade or more. Discovering a promising molecule is only the beginning. Establishing whether it actually works, whether it's safe, and whether those results translate from experimental models into people takes years of painstaking investigation. Making the scientist think ten times faster doesn't necessarily make the science happen ten times faster.
But I think we’re now starting to see the early signs of change. We’re not just building faster trains anymore. We’re beginning to build systems that can identify where the tracks are broken, propose how to fix them, and test whether the repairs actually work. And I believe it is this distinction that matters enormously.
Mathematics gives us perhaps the clearest glimpse of what happens when these systems become effective. A mathematical proof has a remarkable property: it can be checked. The process of finding the answer might be extraordinarily difficult, but establishing whether a formal proof is valid can be comparatively straightforward. It’s almost an ideal environment for autoresearch.
On 6 October, OpenAI released an initial collection of 722 AI-generated mathematical manuscripts, covering 372 families of results across a remarkable breadth of mathematics. Some address longstanding open problems, and some include computer-verifiable proofs. The results are still being checked and revised, but the sheer scale of what was produced would have seemed implausible only a few years ago.
But something equally interesting is happening within the mathematical community itself. Terence Tao recently reflected on what he calls “Math 2.0”. Mathematics has traditionally placed enormous value on being the first to solve a problem. AI is beginning to make that particular achievement much less scarce.
His concern is that simply harvesting solutions doesn’t necessarily produce understanding. Mathematical progress has always involved much more than finding answers. It involves explaining why something works, developing new ways to think about a problem, and opening up directions that other mathematicians can pursue. I find this a fascinating shift. If solving problems becomes relatively cheap, perhaps the most valuable contribution moves towards knowing which problems are worth solving, understanding what the answers mean, and recognizing the new questions they open up.
I recently listened to physicist Jonathan Gorard make a distinction that strongly resonated with me: science as a product, and science as a process. Science as a product is relatively easy to explain. We want to cure diseases, develop better medicines, produce clean energy, and solve problems that matter to society. These are outcomes we value, and understandably want science to deliver.
But science as a process is something different. Gorard doesn’t simply mean the sequence of experiments and analyses. He means the experience of trying to understand. The curiosity that makes someone spend years thinking about why proteins fold the way they do, what drives turbulence, or how the universe works, even when there is no obvious application in sight.
For generations, we’ve justified much of the second by appealing to the first. We fund curiosity-driven research partly because, eventually, it might produce something useful. That relationship has helped sustain careers, institutions, and entire communities of scientists. What happens to that arrangement when AI can increasingly produce the results of scientific reasoning without people having to undertake so much of the process?
I don’t think we should dismiss that concern. There’s something genuinely unsettling about the possibility that an activity people have built their lives around could change beyond recognition. But if curiosity and understanding are valuable in their own right, perhaps we shouldn’t measure the future of science solely by the products it generates. Perhaps the more interesting question is what becomes possible for the people who still want to understand.
A similar conversation is happening in biology. My old colleague and friend Henning Walczak, together with Ivan Đikić, recently published a thoughtful Comment in Nature Cell Biology on scientific culture in the age of AI. They make the case for preserving what they call “Natural Intelligence”: the curiosity, creativity, experimental insight and scientific judgment that underpin scientific discovery. Their concern is that replacing experienced scientists with AI risks losing the intellectual environments where unexpected observations and unconventional thinking emerge. I understand that concern.
Henning and Ivan refer to Arthur Koestler’s concept of “bisociation”, where creativity comes from bringing together ideas from seemingly unrelated fields. That’s interesting, because it’s precisely one of the things I’ve been watching these AI loops do: recognizing potentially useful connections, implementing them, and testing whether they work. I’m not sure we can continue to assume this kind of creativity is exclusively human. But perhaps that isn’t the most interesting question.
For most of scientific history, even the most curious scientist could follow only so many threads. Imagine being able to explore an unconventional hypothesis that would otherwise sit in a notebook for years, or investigate an unexpected observation rather than setting it aside. Imagine doing that continuously, with each experiment informing the next, including the ones that fail. I wonder how much we’ve missed simply because we couldn’t pursue it all.
This is why I find myself more optimistic about the future of scientists than some of the recent commentary suggests. We may be approaching a point where an individual scientist’s curiosity can drive an amount of investigation that previously required an entire laboratory. The profound change may not be that AI can think like a scientist, but that scientists can explore science in ways that were never previously possible.
Of course, biology has a much harder problem than mathematics. We can’t simply check the answer against a formal proof. Our experiments are slow, measurements are imperfect, and living systems have an extraordinary ability to surprise us. That is where the next challenge lies: not just generating ideas, but finding ways to test them and learn from the evidence.
I increasingly think about autoresearch in biology as operating on two different clocks. The first is fast. It’s the computational loop I’ve been experimenting with. Given existing data, it can propose a change, implement it, evaluate the result, and try again, often in minutes or hours. The second clock is much slower. It’s the experimental loop, where we have to generate new evidence to establish whether something is actually true.
Consider a relatively simple example. An AI system identifies a compound that appears to reverse a disease-associated protein signature in cells. It tests the compound, measures the proteins, and finds that the signature has moved towards a healthy state.
A success? Possibly. But there are several explanations. The compound might have corrected the underlying disease mechanism. It might have suppressed the measurement without changing the biology. It might have killed the most affected cells, leaving behind a healthier-looking population. Or it might have triggered a general stress response that happened to resemble recovery.
The initial experiment may not distinguish between any of these possibilities. Rather than simply reporting a promising result, imagine the system recognizing these competing explanations and designing the next experiment specifically to distinguish between them: a time course, a second cell model, an orthogonal measurement, or an experiment that selectively disrupts the suspected mechanism. It then runs the experiment, evaluates the evidence, updates its understanding, and decides what to do next.
Closing the physical experimental loop is harder, and potentially much more valuable. In drug discovery, the opportunity isn’t simply to generate more targets or compounds. It’s to reach better decisions about which ones deserve further investment. An experiment that convincingly rules out a promising but incorrect mechanism may ultimately be more valuable than another experiment that produces an encouraging result.
Cells, animals and people don’t run at the speed of GPUs. But there’s enormous room to improve which experiments we choose to run, how we interpret them, and what we learn from the ones that fail.
In my own autoresearch experiments, the evaluator is deliberately held fixed while the loop is optimizing. Otherwise, the system could simply change the rules to make itself look better. This isn’t a theoretical concern. Sakana AI’s Darwin Gödel Machine demonstrated both impressive improvements and some revealing failures: it sometimes produced fake test logs, and in other cases altered the markers intended to detect unreliable tool use. A sufficiently capable optimizer will find weaknesses in whatever you ask it to optimize.
But science presents an interesting complication. Sometimes the experiment doesn’t just tell us that our proposed answer was wrong. It tells us that we were asking the wrong question, or measuring the wrong thing. If compound variations repeatedly improve a disease signature without improving the underlying biology, perhaps the signature itself is the problem.
The evaluator needs to change.
And there’s a deeper reason this is difficult, beyond the risk of an AI gaming a score. Gorard makes an important point about the physical sciences. Unlike a mathematical proof, an experimental observation is never entirely separate from the assumptions and models we use to interpret it. We don’t have a universal proof checker for biology.
I find this particularly relevant to proteomics. When I say that a protein has increased in abundance, I haven’t directly counted the proteins in the sample. I’ve measured signals from ions, inferred the peptides they came from, estimated protein abundance, and made decisions about interference, normalization and statistical uncertainty along the way. These can be extraordinarily reliable measurements. But they are still built on a chain of inference.
An AI might optimize a method against a perfectly reproducible evaluator and still end up further from the biological truth if the evaluator is asking the wrong question. The challenge isn’t simply to determine whether a hypothesis is true. It’s to establish what the evidence supports, under which conditions, with what uncertainty, and what observation would change our minds. Evaluators need to do more than return a score: they need to make assumptions visible, preserve the evidence behind a result, and help us recognize when independent observations challenge what we thought we knew.
This suggests two connected but distinct processes. One explores possible solutions against a defined standard of evidence. The other asks whether that standard remains appropriate as new evidence accumulates. The first can run rapidly and autonomously. The second needs stronger safeguards: independent measurements, recorded changes, comparisons against earlier results, and, where appropriate, human scientific judgment.
What interests me is that a failed experiment can improve both the next experiment and the system that designs and evaluates future experiments. As we begin accumulating more than results, we accumulate better ways of deciding which results to trust. If those lessons can be retained and reused across experiments and teams, an organization doesn’t just generate more data; it becomes better at learning from it.
I’ve been wondering what happens when the cost of generating and exploring scientific hypotheses falls by several orders of magnitude. Biology has an almost unimaginable amount of latent demand.
At the Colonial Foundation Healthy Ageing Centre, we generated urine and plasma proteomes from around 12,000 volunteers, sampled at two time points, with a decade of associated clinical data. Think about the number of questions contained within a dataset like that. Which proteins predict disease years before symptoms emerge? Which changes are consequences of aging, and which might actually contribute to it? What interactions exist between diseases, medications, genetics and the molecular changes we measure?
We have investigated only a small fraction of the possibilities. And that’s a single dataset, from a single program. Across biology, we have spent decades generating extraordinary quantities of data, while the hypotheses we can seriously investigate have remained constrained by the time and attention of human researchers.
I don’t think AI simply makes that existing work cheaper. I think it dramatically expands the amount of science we can attempt. And as hypotheses become cheap, something else becomes disproportionately valuable: the ability to determine which ones are true.
Reliable measurement. Independent validation. Experiments that distinguish competing explanations. Automation that shortens the time between asking a question and obtaining new evidence. These are not peripheral tasks. They become central to the rate of scientific discovery.
What happens when a single scientist can direct hundreds of computational experiments, with systems proposing the next steps, challenging their own findings, and accumulating evidence continuously?
Scientific careers won’t emerge untouched. Some of the activities we train people for, and reward them for doing, may become far less scarce. But science as a profession and science as a human pursuit aren’t quite the same thing.
There’s also a difficult question about training the next generation. If AI does much of the exploratory work, how does a young scientist develop the intuition to recognize a flawed experiment, challenge an attractive interpretation, or design an evaluator worth trusting? Much of that judgment comes from doing the work, including getting it wrong.
I don’t think the most interesting answer is that scientists become managers of AI agents. Spending our days reviewing logs and approving trivial decisions would be a fairly miserable outcome. Scientists could instead spend more of their time asking important questions, designing ways to distinguish competing explanations, deciding which evidence matters, and connecting discoveries that would otherwise remain isolated.
There’s another distinction that Gorard and Tao both touch on, which I find fascinating. A result can be correct without necessarily giving us much understanding. An AI might produce a valid proof that few people can follow, or identify a biological mechanism that survives independent validation without explaining it in a way that changes how we think about the system.
That doesn’t make the discovery worthless. Scientific progress has always involved more than accumulating correct answers. It also involves building understanding that others can use, challenge and extend. AI may become extraordinarily good at that too. I see no reason to assume otherwise. But it’s a different objective from simply producing more results, and one I suspect we’ll need to value more explicitly.
And perhaps we need to reconsider what scientific achievement looks like. We’ve long rewarded the first person to make a discovery. But what about the person who designs an experiment that makes an entire class of discoveries possible? Or the team that develops a measurement method that thousands of autonomous experiments can rely on? What about a reliable evaluator that rejects a thousand plausible but incorrect explanations before finding one that survives?
These contributions have always mattered, but I suspect they’re about to matter much more. Perhaps the scientist’s most valuable contribution will increasingly be the quality of the questions and standards of evidence they bring to the process.
For anyone early in their scientific career, I can’t think of a more interesting time to be entering the field. Much of the technology needed to start experimenting with these ideas is already available. You don’t need to build a frontier model or have access to a huge computing cluster.
Start with a problem you understand well. Something with a measurable outcome and, ideally, independent data against which you can test your conclusions. Define what success looks like. Give an AI agent the tools to propose and implement changes. Let it run, evaluate its own proposals, reject the ones that fail, and continue. Keep a record of everything, including the failures. And importantly, don’t just ask whether the score improved. Ask whether the evidence is sufficient to believe the improvement is real.
I suspect even a few weeks of doing this will change how you think about your own research. It certainly has for me. At Mass Dynamics, this is increasingly the future we’re thinking about: connecting biological questions, experimental evidence, reliable measurements and the evaluations that allow both humans and AI to learn from them. Building towards a system where the fast computational loops and slower experimental loops can inform each other, and where knowledge gained through one experiment improves the next.
We’re not there yet. Nobody is. But after watching these systems work, fail, validate themselves and continue (albeit in orthogonal but related fields), I find it increasingly difficult to imagine science not moving in this direction.
I can’t promise that AI will cure disease, or that biological discovery will suddenly run at the speed of software development. But I can imagine a world where far fewer good ideas are abandoned simply because nobody has the time to explore them. Where experiments become progressively more informative and failed hypotheses contribute directly to better ones.
And perhaps a world where more people can experience the pleasure of discovery, not just consume its products. The interesting question isn’t whether AI can do the work of a scientist, but how much further a scientist’s curiosity can now reach.
The trains are getting faster. Perhaps the most exciting part is that we’re finally learning how to build the track.
This blog post was produced by Assoc. Prof. Andrew Webb using a combination of original notes from discussions and insights. All statistics and quotes referenced are drawn from internal research and published sources. Final compilation was completed with assistance from ChatGPT. Any errors or omissions are unintentional, and the content is provided for informational purposes only. The views, thoughts, and opinions expressed in this text belong solely to the author, and not necessarily to the author's employers, organization, committees or other group or individual