Measure for Information, Not for Fitness: Designing Protein Experiments to Reveal Epistasis

Terminology

• Variant: One version of a protein sequence containing one or more amino-acid changes.
• Site: A position in that sequence.
• Experimental plate: A container with wells in which variants are measured. The budget B is the number of variants available for measurement.
• Label: The experimental outcome measured for a specific variant.
• Protein landscape: A mapping from variants to their corresponding measured outcomes.

Introduction

A protein language model can score thousands of mutations before anyone touches a pipette. That suggests an obvious recipe for protein engineering: rank the variants, keep the top predictions, and test them.

The recipe answers one question:

Which variants does the model expect to perform well?

It stays silent on a second one:

Which variants should we measure if we want to learn how mutations interact?

The two questions need not have the same answer. A plate concentrated on predicted winners serves exploitation but may be poorly suited to mapping interactions. If most wells sample the same small neighbourhood of sequence space, the experiment may reveal little about the landscape's shape or the behaviour of untested combinations.

I built epibudget to take the second question seriously: given a fixed budget of wells, it selects variants meant to expose epistatic structure instead of simply maximizing predicted fitness.

Epistasis makes prediction a relational problem

Epistasis is the non-additive interaction between mutations: the effect of one depends on the genetic background of the others.^[1]

Here, A and B are abstract labels for mutations at two selected positions in the protein sequence; they are not amino-acid names. Suppose mutation A shifts a protein's fitness by +0.2 and mutation B shifts it by +0.1. An additive model predicts +0.3 for the double mutant AB. If the measured effect is +0.8, the surplus +0.5 is pairwise epistasis:

Here, Δ(v) is the measured effect of variant v relative to the reference construct: the starting protein sequence from which the variants were generated.

A's effect now depends on whether B is present, and the reverse holds too. For three mutations, the same bookkeeping extends through inclusion–exclusion:

In plain language, this formula removes the effects already explained by the singles and doubles, leaving the third-order interaction term.

This is why a combination measured on its own is not enough. Pinning down an interaction also requires the surrounding members of its inclusion–exclusion loop. For a pair, that family contains A, B, and AB relative to the reference construct. For a triple, it contains seven mutant measurements. Measure the complete family and the interaction is determined. Leave members out and it must be inferred.

Here, a measurement loop is not a biological feedback loop. It is the complete family of related measurements needed to isolate one interaction coefficient. The candidate set is therefore better understood as a measurement network^[2] than as a collection of independent points. Some variants contribute to many interaction loops.^[3] Measuring one of them helps determine several interaction coefficients at once.

Geodetic triangulation is a useful analogy. A survey network becomes informative when its measurements form constraints that have to close consistently. A protein experiment behaves the same way: its chosen singles, doubles, and triples become informative when they jointly constrain the epistasis map.

A plate budget changes the objective

The GB1^[5] and TrpB^[6] benchmarks each vary four selected sites in the protein sequence while holding every other position fixed. Each site can contain any of the 20 standard amino acids.

Because one amino acid is already present at each site in the reference construct, each position has 19 possible non-reference substitutions. A single mutant changes one site, a double changes two, and a triple changes three. The resulting candidate universe—the complete set of variants allowed by these rules—contains 29,678 variants: 76 singles, 2,166 doubles, and 27,436 triples.

The registered budgets are B = 48, B = 96, and B = 192 variants. Even the largest simulated plate covers less than 1% of the candidate universe.

Scarcity forces a choice. Spend those wells on the variants most likely to be fit? On a random sample? Or on variants that jointly constrain as many interactions as possible? Fitness-greedy selection is exploitation: it asks the model for its favourite sequences. Epistasis mapping is an information problem: it asks which measurements would make hidden interaction structure identifiable.

A two-part ESM-2 hypothesis

The registered profile used the esm2_t33_650M checkpoint of ESM-2^[4]. ESM-2 is a protein language model: much as a text model learns which words are plausible from the surrounding sentence, ESM-2 learns which amino acids are plausible from the surrounding protein sequence.

ESM-2 plays two roles. The first is to score every candidate without seeing experimental labels. This introduces a technical trap: if mutations A and B are scored independently on the unchanged reference sequence and their scores are simply added, the resulting prediction is additive by construction. The model's predicted pairwise epistasis is then exactly zero.

epibudget therefore scores variants conjointly. Conjoint scoring means that all mutations in a candidate are inserted into the sequence before any of them is scored. For AB, A is evaluated while B is present, and B while A is present. Their context-dependent scores are then combined, allowing the prediction for AB to differ from the sum of two mutations scored separately.

The second role is more speculative. Masking temporarily hides a subset of non-mutated sequence positions before ESM-2 scores the same candidate again. Each candidate is rescored under16 masking perturbations. Greater variation produces a larger τ²(v). This statistic measures sensitivity to artificial context perturbation; it is not calibrated predictive uncertainty.

The intuition was to prefer variants that contribute to many interactions and, among those, variants whose ESM scores are more sensitive to context. An ablation tests the contribution of one component by removing it while leaving everything else unchanged.

Assigning every candidate the same dispersion removes the weighting term and yields the loop-count baseline, named structural in the tracked artifacts. Here, structural refers to interaction structure, not protein 3D structure.

The five tracked strategies are random; fitness (fitness-greedy); practice;structural (loop count alone); and info (dispersion-weighted loop count).

In v1, every candidate receives one fixed score before selection begins. Selecting one variant does not lower the score of related variants, even when they may provide partly redundant information. The method is therefore a static ranking, not a sequential Bayesian design that updates after observing experimental results.

Evaluation without label leakage

An ablation estimates a component's contribution by comparing methods that differ only in that component. In v1, the two ablation methods differ only in the ESM masking-dispersion term. Both rank candidates before any measured fitness is revealed.

The loop-count baseline scores a candidate v by n(v), the number of interaction terms containing v. The dispersion-weighted method uses n(v) × τ²(v), whereτ²(v) is the ESM masking dispersion across 16 perturbations.

Both methods prioritize variants appearing in many interaction terms. The ablation tests whether masking dispersion improves selection beyond loop count.

In the symmetric four-site candidate universe, all candidates of the same mutation order have identical loop counts: 1,140 for singles, 39 for doubles, and 1 for triples.

Consequently, loop count cannot rank candidates within a mutation order. Tied candidates may therefore be selected according to enumeration order unless ties are broken explicitly.

A seeded tie reanalysis randomizes tied candidates while recording the random seed. This separates the effect of the scoring rule from enumeration order.

Evidence status

On TrpB, info outperforms fitness and random in pairwise map recovery. However, structural, the simpler loop-count baseline, performs better at B = 96 andB = 192, but not at B = 48. These results do not support an added benefit from masking dispersion.

On GB1, a map-recovery re-analysis using cached model scores, without an additional GPU run, remains inconclusive. Pairwise correlation improves at all registered budgets, but the relative squared-error criterion is not met. No method is therefore declared the GB1 map-recovery winner.

For downstream prediction, the registered structural-minus-fitnesslearning-curve AUC is positive in 20/20 partitions on both GB1 and TrpB. Theinfo-minus-structural gate at B = 192 is positive in15/20 GB1 partitions and 0/20 TrpB partitions, below the registered threshold. Thus, the analysis supports structural allocation, not an incremental contribution from masking dispersion.

A registered gate is a success criterion specified before inspection of the final results.^[7] All findings remain provisional and are limited to the evaluated benchmarks, learner, and experimental protocol. They are not presented as general conclusions.

The TrpB source paper reports 871 imputed fitness values, but the public mirror does not identify them row by row.

What v1 established at the implementation level

1. Variant selection can be posed as measurement design. Predicted fitness and expected experimental value are distinct objectives.
2. Interaction-loop counts define a reproducible baseline. The implementation exposes exactly which interaction terms contain each candidate.
3. Conjoint scoring preserves context-dependent model signal. Applying every mutation before reading conditional scores avoids the additive shortcut that would erase interaction signal by construction.
4. The masking-dispersion term can be isolated. The code supports a direct score-level ablation without dressing τ² up as calibrated uncertainty.
5. The implementation enforces the label boundary. The confirmatory benchmark fixes the plate before any measured fitness is revealed.

The registered downstream benchmark adds comparative evidence for the allocation objective: under the fixed learner, structural selection is supported over fitness-greedy selection on both landscapes. This does not establish generality beyond these datasets and this learner.

Why masking dispersion remains an open question

Sensitivity to an artificially masked sequence context is not the same as calibrated predictive uncertainty. A prediction can vary under masking and still be accurate; it can also remain stable and be wrong. Current calibration evidence does not establish τ² as a posterior uncertainty estimate.

The v1 variance model also assumes errors are independent across variants. Members of an epistatic loop share sequence context and pass through the same model, so their errors may be correlated. Ignoring those covariances keeps the score simple but may drop information a meaningful information-gain calculation would need.

The conclusion remains narrow: current provisional results do not support an added contribution from masking dispersion. This evidence concerns one static heuristic, two four-site landscapes, and one downstream learner; it does not show that contextual dispersion can never be useful.

Next validation steps

The current version is a single-shot allocator: it ranks the complete candidate pool once and returns one plate of B variants.

A future extension could be sequential. It would measure a first plate, update its beliefs and uncertainty estimates for the remaining variants, and then select the next plate based on what the experiment resolved.

That would make the geodetic analogy literal. Once one part of the interaction network has been measured, related measurements may become less valuable because they provide redundant information. Measurements in unresolved parts of the network would become more valuable.

A future version should also keep three quantities separate:

1. Predicted effect: how well the model expects a variant to perform.
2. Calibrated predictive error: how uncertain that prediction is expected to be.
3. Experimental value: how informative measuring the variant would be for the current scientific question, given what has already been measured.

The first helps predict, the second quantifies predictive uncertainty, and the third guides experimental design. One score should not stand in for all three.

Implications for experimental design

Protein language models make predictions cheap.

Experiments stay scarce.

That asymmetry is what makes allocation matter more over time: as candidate generation scales, the bottleneck moves from proposing sequences to deciding which evidence is worth buying.

The current results do not support the original masking-dispersion hypothesis. They support a narrower design result: under the registered learner, structural loop-count selection yields better downstream ranking than fitness-greedy selection on GB1 and TrpB. This comparative result remains provisional and does not establish broader generality.

The useful question is not “Which variants does the model like?” but “Which measurements make the interaction structure identifiable?”

That is the criterion that should shape the next experimental round.

References

1. Phillips, P. C. Epistasis — the essential role of gene interactions in the structure and evolution of genetic systems. Nature Reviews Genetics (2008).
2. Poelwijk, F. J. et al. The context-dependence of mutations: a linkage of formalisms. PLOS Computational Biology (2016).
3. Faure, A. J. et al. An extension of the Walsh-Hadamard transform to calculate and model epistasis in genetic landscapes of arbitrary shape and complexity. PLOS Computational Biology (2024).
4. Lin, Z. et al. Evolutionary-scale prediction of atomic-level protein structure with a language model. Science (2023).
5. Wu, N. C. et al. Adaptation in protein fitness landscapes is facilitated by indirect paths. eLife (2016).
6. Johnston, K. E. et al. A combinatorially complete epistatic fitness landscape in an enzyme active site. PNAS (2024).
7. Frozen validation protocol, limitations, and tracked evidence artifact.