30-second version: What AI really lowers isn’t the cost of writing code — it’s the search cost across the entire hypothesis space. An idea that used to have a 20% success rate and take two days to test wasn’t worth trying before; now it is. So the bottleneck moves from execution to selection: when you can run two hundred experiments a day, the hard part is which two are worth looking at. And alpha discovery has gotten cheap — alpha validation hasn’t. As the search space grows, multiple testing, overfitting, and selection bias all get worse; you can always fish a pretty equity curve out of pure noise. So the researcher’s job shifts from “finding one good result” to “ranking a pile of results by strength of evidence.” What’s scarce is no longer the ability to search, but the prior knowledge of where to search, and the discipline to actually kill a signal.
This is the follow-up to When Writing Code Is No Longer the Bottleneck in Quant Research: that piece was about the feeling of it; this one tries to push it into an outline-style framework for how AI agents change the methodology of quant research itself. Most of the sections below could each grow into a standalone article — for now, this is just the skeleton.
1. Starting Point: What’s Really Changing May Not Be Coding, but Experiment Cost Link to heading
The old research workflow looked roughly like this:
Propose an idea
↓
Decide whether it's worth doing
↓
Write code
↓
Prepare data
↓
Run the experiment
↓
Analyze results
↓
Decide whether to continue
There’s an implicit constraint baked in here:
Experiments cost something.
So the researcher has to do a lot of filtering before the experiment even happens.
An idea with even a 20% chance of success might never get tried if implementing it takes two or three days.
AI agents are changing that constraint.
If a research idea can be completed in a few hours, or overnight:
- data processing
- feature construction
- backtesting
- statistical testing
- robustness checks
- visualization
- cross-validation across multiple implementations
then the researcher is no longer facing:
“What do I have time left to research?”
but instead:
“Since I can try anything, what should I actually try?”
This might be the single most important starting point of the whole shift.
2. From a Handful of Experiments to Large-Scale Hypothesis Search Link to heading
The old pattern looked more like:
Idea A → dig deep
Idea B → abandon
Idea C → dig deep
The future pattern might look like:
Idea 1
Idea 2
Idea 3
...
Idea 1000
↓
cheap experiments
↓
rapid elimination
↓
a handful of directions worth pursuing
This means research might shift from:
depth-first
toward:
breadth-first exploration + depth-first validation
The researcher no longer needs to accurately judge, right at the outset, whether an idea deserves a large time investment.
You can filter first, at extremely low cost.
Hence:
What AI lowers may not just be the execution cost of a single experiment, but the search cost across the entire hypothesis space.
3. The New Bottleneck: Selection, Not Execution Link to heading
If a researcher can only complete two experiments a day, improving coding efficiency matters a great deal.
But if two hundred experiments a day become possible, things change.
The question becomes:
Which two are worth looking at?
Or even:
Which ten are worth continuing with?
This produces new bottlenecks:
- hypothesis selection
- experiment selection
- result selection
- attention allocation
In the past, a researcher’s scarce resources were mainly:
time + coding capacity
Going forward, it might become:
attention + judgment
So the definition of “research efficiency” also shifts.
It’s no longer:
how many experiments you can run in a day.
But rather:
how much of your limited human attention ends up focused on questions that genuinely matter.
4. Falling Experiment Cost Doesn’t Mean Falling Cost of “Discovering Alpha” Link to heading
This is a genuinely counterintuitive point worth digging into.
Suppose in the past you could only test 20 strategies.
Now you can test 20,000.
Then even if every single strategy is pure noise, it becomes trivially easy to find:
- high IC
- high Sharpe
- high t-stat
- a beautiful-looking equity curve
because some noise will always happen to perform well.
So:
The cost of finding a pretty result may approach zero.
But:
The cost of proving that result isn’t a fluke hasn’t dropped at all.
It might even rise.
Because as the search space expands:
- multiple testing gets worse
- data mining gets worse
- selection bias gets worse
- backtest overfitting gets worse
- researcher degrees of freedom multiply
So an interesting shift may emerge:
Alpha discovery keeps getting cheaper, while alpha validation keeps getting more important.
5. Exploration and Validation May Fully Separate Link to heading
In traditional research:
discover a phenomenon
↓
tune parameters
↓
discover it again
↓
adjust the rules
↓
keep backtesting
Exploration and validation are usually tangled together.
But if experiment cost is low enough, the two can be cleanly split into two phases.
Exploration Link to heading
Goal:
Search as broadly as possible for interesting phenomena.
Allowed:
- lots of attempts
- lots of parameters
- lots of data slices
- multiple hypotheses
- fast failure
This phase doesn’t demand rigorous proof.
Validation Link to heading
Once a direction is settled on:
- freeze the hypothesis
- freeze the data processing
- freeze the parameters
- clearly define out-of-sample data
- validate on independent data
- validate across multiple markets
- validate across different regimes
- cross-implementation checks
- check alternative explanations
This is the phase that actually answers:
Is this thing believable?
AI may make exploration extremely cheap, while the value of research discipline increasingly concentrates in validation.
6. The Researcher May Shift from “Finding Results” to “Managing Evidence” Link to heading
Going forward, you might face something like this:
1000 experiments
↓
100 significant results
↓
30 hold up out-of-sample
↓
10 have an economic rationale
↓
5 hold across markets
↓
2 worth betting on
At that point, the researcher’s job is no longer just:
finding one pretty result.
It’s:
building strength-of-evidence ratings across different results.
You can build an evidence hierarchy like this:
Weaker evidence Link to heading
- single market
- single sample period
- single parameter set
- single data source
- significant in-sample
Medium evidence Link to heading
- out-of-sample
- stable across parameters
- stable across stock universes
- consistent across multiple implementations
Stronger evidence Link to heading
- multiple markets
- multiple regimes
- independent data sources
- an economic mechanism holds up
- alternative explanations ruled out
- reproducible via an independent research process
What the researcher ultimately needs to do is:
judge which results deserve to be believed.
7. Negative Results May, for the First Time, Truly Become an Asset Link to heading
In the past:
“This factor doesn’t work.”
Rarely became a formal output.
Because validating a negative result takes time, and researchers would rather spend that time on the next potentially successful idea.
But once experiment cost approaches zero, you can accumulate:
Hypothesis
↓
Experiment
↓
Failure
↓
Reason for failure
Which could give rise to:
a Failed Hypothesis Database
For example:
- a certain class of technical indicator doesn’t work in a given universe
- a certain volatility signal is just a liquidity proxy
- a certain cross-sectional signal vanishes out-of-sample
- a certain event effect only exists in a specific regime
These negative results are knowledge in their own right.
And they can keep future agents from re-exploring a patch of ground that’s already been proven barren.
8. Research May Shift from a Linear Process to a Feedback System Link to heading
Traditional research:
Hypothesis
↓
Experiment
↓
Result
↓
Report
Once agent-driven, this might become:
Hypothesis
↓
Experiment
↓
Result
↓
Interpretation
↓
New Hypothesis
↓
Experiment
↓
Result
↓
...
You could even have the agent automatically generate the next batch of experiments based on the results.
So research shifts from:
a one-off project
to:
a continuously running feedback loop
The question that actually matters becomes:
at which nodes in this loop should a human step in?
Not every experiment necessarily needs a human involved.
Humans are better suited to stepping in for:
- defining the hypothesis space
- choosing a research direction
- designing key experiments
- judging anomalous results
- making the final call on the evidence
9. The Most Dangerous Scenario: Not That AI Can’t Find Anything, but That It Finds Too Much Link to heading
In the past:
“How do I find alpha?”
Going forward, this might become:
“How do I avoid being buried under a mountain of fake alpha?”
If you can freely combine:
- features
- universes
- holding periods
- rebalancing rules
- transformations
- parameter ranges
- time periods
then the search space is nearly unbounded.
A machine will absolutely find a large number of things that look effective.
So:
The stronger the search capability, the more important research discipline becomes.
A powerful research system can’t just be responsible for:
finding signals.
It also has to be responsible for:
killing signals.
And even:
proving why a given signal should be killed.
10. What’s Genuinely Scarce May Become “Priors” Link to heading
Once everyone can cheaply search a massive hypothesis space:
Who searches more?
may no longer be the core advantage.
The real question becomes:
Who knows where to search?
This re-elevates:
- market intuition
- domain knowledge
- economic mechanism
- prior beliefs
- sensitivity to anomalies
- understanding of the data-generating process
So AI doesn’t necessarily eliminate human experience.
It may instead make experience more important than ever:
not because experience can replace AI, but because experience determines where AI should even be searching.
11. The Researcher May Gradually Become “the Manager of a Research Lab” Link to heading
If a single researcher can simultaneously manage:
Agent 1 → Hypothesis A
Agent 2 → Hypothesis B
Agent 3 → Hypothesis C
...
Agent 100 → Hypothesis Z
then one person’s actual research capacity could shift by orders of magnitude.
The researcher is no longer necessarily:
the person doing every experiment by hand.
But instead more like:
the person managing hypothesis generation, experiment generation, and evidence accumulation.
This is also quite different from the traditional notion of a “research assistant.”
Because AI isn’t just executing already well-defined tasks — it may also take part in:
- generating hypotheses
- designing experiments
- implementing experiments
- analyzing results
- generating the next round of hypotheses
12. A New Research Production Function May Emerge Link to heading
You could roughly express the old research capacity as:
Research Output
≈
Researcher Time
×
Coding Efficiency
×
Research Skill
AI has primarily boosted:
Coding Efficiency
But once coding efficiency is already high enough, further gains from pushing it higher yield diminishing returns.
Going forward, it might look closer to:
Research Output
≈
Hypothesis Quality
×
Experiment Capacity
×
Validation Quality
×
Researcher Judgment
Once Experiment Capacity can be scaled up massively via agents:
the relative importance of Hypothesis Quality and Validation Quality rises.
This may be the more precise way of expressing “where the researcher’s value shifts to.”
13. A Question Worth Continuing to Ask Link to heading
If:
- coding is nearly free
- data processing is nearly free
- backtesting is nearly free
- experimentation is nearly free
- agents can iterate on their own
then:
what still remains expensive?
What’s likely left:
- human attention
- high-quality data
- independent validation
- real trading validation
- research priors
- causal explanation
- judgment about anomalies
- ultimately owning the decision
This suggests the scarcest resource of the future may not be computing power.
It might be:
credible evidence, and the people capable of judging it.
14. A Few Core Directions Worth Developing Further Link to heading
Each of the following could later be developed into a standalone article.
A. What happens to alpha discovery once experiment cost approaches zero? Link to heading
Core question:
Search cost ↓
→ hypothesis space ↑
→ experiments ↑
→ false discoveries ↑
Key topics:
- hypothesis search
- multiple testing
- backtest overfitting
- signal discovery
- exploration vs. exploitation
B. Why might alpha validation matter more than alpha discovery? Link to heading
Core question:
Finding a significant result keeps getting cheaper — but proving it’s real hasn’t gotten any cheaper.
Key topics:
- out-of-sample testing
- multiple testing
- independent replication
- robustness
- alternative explanations
- evidence hierarchy
C. If AI can propose its own next experiment, what’s left for the researcher to do? Link to heading
Core question:
Where should the human stay in the loop?
Key topics:
- hypothesis generation
- experiment design
- agent feedback loops
- autonomous research
- human judgment
D. Might the researcher’s edge shift from “execution ability” to “priors”? Link to heading
Core question:
When the search space is large enough, knowing where to search may matter more than the searching itself.
Key topics:
- domain knowledge
- market intuition
- economic mechanism
- priors
- research taste
E. Why might failed research become a new kind of knowledge asset? Link to heading
Core question:
Once experiments are cheap enough, should failed hypotheses be systematically preserved?
Key topics:
- negative results
- failed hypothesis database
- research memory
- avoiding repeated exploration
- agent institutional memory
15. A Provisional Core Judgment Link to heading
For now, the whole idea can be condensed into one sentence:
What AI is really changing may not be “how fast a researcher can run one experiment,” but “how many ideas a researcher is willing to try.”
If that judgment holds, then the next question worth researching is no longer just:
Can AI write code faster?
But rather:
Once the cost of hypothesis search drops dramatically, will research methodology, the researcher’s role, and the very standard for what counts as evidence all change together?
This might be a more worthwhile question to discuss than “will AI replace quant researchers.”