Epistasis Explained: Why the Best Protein Variant Is Not Always the Best Experiment

A protein language model can rank thousands of variants before anyone enters the lab. That sounds like the hard part is solved: score every candidate, select the predicted winners, and test them.

But this only answers:

Which variants does the model expect to work best?

A scientist may need to answer a different question:

Which variants should we measure if we want to understand how mutations interact?

The best predicted variant and the best experiment are not necessarily the same. This distinction is the idea behind epibudget, an open-source project I built to explore protein experimental design under a fixed measurement budget.

1. The experiment is much smaller than the search space

Imagine that a protein study varies four positions. At each position, the original amino acid can be replaced by any of the other 19 standard amino acids.

If we consider all single, double and triple mutants, the candidate set contains:

• 76 single mutants;
• 2,166 double mutants;
• 27,436 triple mutants;
• 29,678 candidates in total.

Now suppose the laboratory can measure only 96 variants.

That plate covers about 0.3% of the candidate set. The main problem is no longer generating candidates. It is deciding which 96 measurements deserve to exist.

This is an experimental-design problem: how should a scarce measurement budget be allocated to answer a scientific question?

2. Mutation effects do not always add up

Suppose mutation A improves a measured protein property by +0.2, while mutation B improves it by +0.1.

If the mutations act independently, we expect the double mutant AB to improve the property by +0.2 + 0.1 = +0.3.

But imagine that the experiment measures AB at +0.8.

The extra +0.5 means that the effect of A depends on whether B is present, and vice versa. This non-additive interaction is called epistasis.

For two mutations, relative to the reference protein:

In our example, ε(A,B) = 0.8 − 0.2 − 0.1 = 0.5.

Epistasis matters because a mutation cannot always be assigned one fixed effect. Its effect can change with its genetic background.

3. Measuring AB alone does not reveal the interaction

If we measure only the double mutant AB, we observe one number. That number may contain:

• the effect of A;
• the effect of B;
• the interaction between A and B;
• experimental noise.

One observation cannot tell us how much came from each component.

To isolate the pairwise interaction directly, we need the related family of measurements: the reference protein, A, B and AB.

The same principle becomes more demanding for three mutations. Isolating an irreducible three-way interaction requires accounting for the reference, all three singles, all three doubles and the triple.

Protein variants are therefore not just independent candidates in a list. They form a measurement network. One measured variant can contribute to several interaction calculations, while another may answer a much narrower question.

4. A good prediction is not automatically a good experiment

Consider two ways to fill a 96-well plate.

Strategy A: test the predicted winners

A protein model ranks the variants by predicted fitness, and the experiment measures the top 96.

This is a sensible exploitation strategy when the goal is to find a strong candidate quickly. But the selected variants may cluster in one small region of sequence space or omit the lower-order variants needed to interpret combinations.

Strategy B: choose measurements for what they can teach us

The experiment instead prioritizes variants according to how they contribute to the interaction structure we want to study.

Some of these variants may not look like likely winners. Their value comes from constraining or connecting multiple relationships in the landscape.

Neither strategy is universally better. They optimize different objectives:

The scientific objective must be chosen before the plate is designed.

5. What epibudget actually does

epibudget is a Python command-line tool that ranks protein variants under a fixed budget.

Its evaluated baseline, called structural, assigns each candidate a score based on the number of interaction terms in which it appears. Here, “structural” refers to the structure of the interaction graph, not the protein's three-dimensional structure.

On the symmetric four-site benchmarks used in the project, this score gives the same value to every variant of the same mutation order. It therefore reduces to:

1. single mutants first;
2. double mutants second;
3. triple mutants third;
4. a declared random seed to break ties within each group.

This is a useful baseline because lower-order measurements are reused across many interaction terms. But an important limitation emerged during the audit: the score rewards participation in interaction terms additively. It does not explicitly reward completing a measurement family, and it does not by itself guarantee that any interaction becomes identifiable.

That distinction is fundamental:

Appearing in many equations is not the same as providing enough independent measurements to solve those equations.

epibudget therefore selects measurements; it is not an epistasis-inference package, and its baseline should not be described as automatically reconstructing an interaction map.

6. Where ESM-2 enters the project

ESM-2 is a protein language model. Like a text model that learns which words are plausible in a sentence, it learns which amino acids are plausible in a protein sequence.

epibudget explored two ESM-2 signals.

Conjoint variant scoring

If A and B are each scored on the unchanged reference sequence and their scores are simply added, the prediction is additive by construction. It cannot express predicted pairwise epistasis.

epibudget instead applies all mutations in a candidate before reading their conditional scores. In AB, A is scored while B is already present, and B is scored while A is present. This preserves the possibility of context-dependent, non-additive model signal.

Masking dispersion

The project also repeatedly masked parts of the surrounding sequence and rescored each candidate. If a candidate's score changed substantially across perturbations, it received a high dispersion value, written as τ²(v).

The experimental hypothesis was that this sensitivity might help prioritize informative measurements:

where n(v) is the interaction-count score.

But masking dispersion is not calibrated predictive uncertainty. A model can be sensitive to artificial masking and still be correct, or remain stable and still be wrong. The relationship had to be tested rather than assumed.

7. How to compare selection strategies fairly

The benchmark separates two stages.

First, each strategy selects its plate without access to experimental fitness labels. This prevents a method from using the answers it is supposed to predict.

Only after selection are the measured fitness values revealed. The same downstream learner is then trained from each plate and evaluated on held-out variants.

For a fair comparison, the following stay fixed:

• the measurement budget;
• the candidate universe;
• the downstream model;
• the train/test partitions;
• the metrics.

Only the plate-selection strategy changes.

This design asks a clean question: at equal cost and with the same learner, which selected measurements lead to better downstream predictions?

8. What the project found — and what it did not

The historical downstream v1 artifacts favored the particular loop-count plates over fitness-greedy plates on GB1 and TrpB. The masking-dispersion weighting did not pass its incremental gate on either landscape.

These observations are encouraging for the value of a simple measurement-structure baseline, but they are not promoted estimates of the acquisition method as a whole. The loop-count rule contains many ties, and the tracked runs did not evaluate performance across a distribution of tie-breaking seeds.

The project also produced an important correction. An earlier analysis interpreted a correlation between predicted and measured epistasis contrasts as map recovery. A later audit showed that both sides of the comparison shared measured lower-order terms. Correlation could therefore improve partly because the same experimental values appeared on both sides, even if prediction of the unmeasured terms did not improve.

That interpretation is now withdrawn. The old correlation remains a diagnostic, not evidence that epibudget reconstructed an epistasis map.

The defensible conclusions are narrower:

1. predicted fitness and experimental value are different objectives;
2. conjoint scoring can retain context-dependent model signal;
3. interaction-count allocation is a useful and reproducible baseline;
4. masking dispersion did not demonstrate added value in the tested v1 procedure;
5. identifiability and metric design are as important as model sophistication.

Reporting the correction is part of the result. In scientific software, discovering that an appealing metric does not support the claimed interpretation is useful evidence, not a reason to hide the experiment.

9. The limits define the next experiment

epibudget is an early, static design study. Its evidence is limited by:

• two four-site protein landscapes;
• one fixed downstream learner;
• plates selected in a single batch;
• uncalibrated masking dispersion;
• unresolved selection variability across tie seeds;
• no tracked corrected map-recovery artifact.

A stronger future system would select experiments sequentially:

1. choose a first batch;
2. measure it;
3. update the model and its uncertainty;
4. estimate which remaining measurement would reduce uncertainty most;
5. choose the next batch.

This would turn a static ranking into a true closed loop between prediction and experiment.

10. The larger lesson for AI for Science

AI makes candidate generation and prediction cheaper. Laboratory measurements remain expensive.

That shifts the bottleneck. The question is no longer only whether a model can score a large search space. It is whether a system can decide which evidence is worth producing.

Three quantities should remain separate:

1. Predicted effect — how well the model expects a variant to perform.
2. Predictive uncertainty — how wrong the model expects its prediction might be.
3. Experimental value — how much measuring the variant would help answer the current scientific question.

epibudget does not solve that full problem. It makes the problem explicit, implements testable baselines, and shows how easily an attractive scientific claim can outrun what an evaluation truly identifies.

The best next experiment is not always the candidate the model likes most. It is the measurement that changes what we can justifiably know.

Go deeper