Measure for Information, Not for Fitness: Designing Protein Experiments to Reveal Epistasis
A protein language model can rank thousands of variants, but ranking does not identify which measurements are most useful for learning mutation interactions. This study compares label-blind allocation strategies under fixed experimental budgets. Historical downstream artifacts favored particular loop-count plates over fitness-greedy plates on GB1 and TrpB, while masking dispersion did not pass its incremental gate. A later audit withdrew the original epistasis-map-recovery interpretation because the metric shared measured lower-order terms between prediction and truth.
Figure n°1: From target protein to experimental plate
Description: epibudget scores complete variants, maps their interaction structure, and converts a fixed experimental budget into a ranked measurement plate.
Terminology
- Variant: One version of a protein sequence containing one or more amino-acid changes.
- Site: A position in that sequence.
- Experimental plate: A container with wells in which variants are measured. The budget
Bis the number of variants available for measurement. - Label: The experimental outcome measured for a specific variant.
- Protein landscape: A mapping from variants to their corresponding measured outcomes.
Introduction
A protein language model can score thousands of mutations before anyone touches a pipette. That suggests an obvious recipe for protein engineering: rank the variants, keep the top predictions, and test them.
The recipe answers one question:
Which variants does the model expect to perform well?
It stays silent on a second one:
Which variants should we measure if we want to learn how mutations interact?
The two questions need not have the same answer. A plate concentrated on predicted winners serves exploitation but may be poorly suited to mapping interactions. If most wells sample the same small neighbourhood of sequence space, the experiment may reveal little about the landscape's shape or the behaviour of untested combinations.
I built epibudget to take the second question seriously: given a fixed budget of wells, it selects variants meant to expose epistatic structure instead of simply maximizing predicted fitness.
Epistasis makes prediction a relational problem
Epistasis is the non-additive interaction between mutations: the effect of one depends on the genetic background of the others.[1]
Here, A and B are abstract labels for mutations at two selected positions in the protein sequence; they are not amino-acid names. Suppose mutation A shifts a protein's fitness by +0.2 and mutation B shifts it by +0.1. An additive model predicts +0.3 for the double mutant AB. If the measured effect is +0.8, the surplus +0.5 is pairwise epistasis:
Here, Δ(v) is the measured effect of variant v relative to the reference construct: the starting protein sequence from which the variants were generated.
A's effect now depends on whether B is present, and the reverse holds too. For three mutations, the same bookkeeping extends through inclusion–exclusion:
In plain language, this formula removes the effects already explained by the singles and doubles, leaving the third-order interaction term.
This is why a combination measured on its own is not enough. Pinning down an interaction also requires the surrounding members of its inclusion–exclusion loop. For a pair, that family contains A, B, and AB relative to the reference construct. For a triple, it contains seven mutant measurements. Measure the complete family and the interaction is determined. Leave members out and it must be inferred.
Here, a measurement loop is not a biological feedback loop. It is the complete family of related measurements needed to isolate one interaction coefficient. The candidate set is therefore better understood as a measurement network[2] than as a collection of independent points. Some variants contribute to many interaction loops.[3] Measuring one of them helps determine several interaction coefficients at once.
Geodetic triangulation is a useful analogy. A survey network becomes informative when its measurements form constraints that have to close consistently. A protein experiment behaves the same way: its chosen singles, doubles, and triples become informative when they jointly constrain the epistasis map.
Figure n°2: Interaction coefficients are defined by closed measurement loops
Description: Pairwise and third-order coefficients are calculated from complete families of measurements centered on the reference construct. An isolated combination cannot determine its own epistatic effect.
A plate budget changes the objective
The GB1[5] and TrpB[6] benchmarks each vary four selected sites in the protein sequence while holding every other position fixed. Each site can contain any of the 20 standard amino acids.
Because one amino acid is already present at each site in the reference construct, each position has 19 possible non-reference substitutions. A single mutant changes one site, a double changes two, and a triple changes three. The resulting candidate universe—the complete set of variants allowed by these rules—contains 29,678 variants: 76 singles, 2,166 doubles, and 27,436 triples.
The registered budgets are B = 48, B = 96, and B = 192 variants. Even the largest simulated plate covers less than 1% of the candidate universe.
Scarcity forces a choice. Spend those wells on the variants most likely to be fit? On a random sample? Or on variants that jointly constrain as many interactions as possible? Fitness-greedy selection is exploitation: it asks the model for its favourite sequences. Epistasis mapping is an information problem: it asks which measurements would make hidden interaction structure identifiable.
A two-part ESM-2 hypothesis
The registered profile used the esm2_t33_650M checkpoint of ESM-2[4]. ESM-2 is a protein language model: much as a text model learns which words are plausible from the surrounding sentence, ESM-2 learns which amino acids are plausible from the surrounding protein sequence.
ESM-2 plays two roles. The first is to score every candidate without seeing experimental labels. This introduces a technical trap: if mutations A and B are scored independently on the unchanged reference sequence and their scores are simply added, the resulting prediction is additive by construction. The model's predicted pairwise epistasis is then exactly zero.
epibudget therefore scores variants conjointly. Conjoint scoring means that all mutations in a candidate are inserted into the sequence before any of them is scored. For AB, A is evaluated while B is present, and B while A is present. Their context-dependent scores are then combined, allowing the prediction for AB to differ from the sum of two mutations scored separately.
The second role is more speculative. Masking temporarily hides a subset of non-mutated sequence positions before ESM-2 scores the same candidate again. Each candidate is rescored under 16 masking perturbations. Greater variation produces a larger τ²(v). This statistic measures sensitivity to artificial context perturbation; it is not calibrated predictive uncertainty.
The intuition was to prefer variants that contribute to many interactions and, among those, variants whose ESM scores are more sensitive to context. An ablation tests the contribution of one component by removing it while leaving everything else unchanged.
Assigning every candidate the same dispersion removes the weighting term and yields the loop-count baseline, recorded as structural in the tracked artifacts. This label identifies allocation by interaction-loop coverage rather than a representation of tertiary protein structure.
The score rewards participation in interaction terms additively. It does not explicitly reward completing a measurement family, and it does not by itself guarantee that any interaction coefficient becomes identifiable.
The five tracked strategies are random; fitness (fitness-greedy); practice; structural (loop count alone); and info (dispersion-weighted loop count).
In v1, every candidate receives one fixed score before selection begins. Selecting one variant does not lower the score of related variants, even when they may provide partly redundant information. The method is therefore a static ranking, not a sequential Bayesian design that updates after observing experimental results.
Criterion interaction coverage · Plate A, B, C
Ranks by n(v), the number of interaction terms containing v. A single mutant appears in the greatest number of interaction terms, so it enters the plate first.
Figure n°3: Same candidates, different experimental plates
Description: Five static strategies select the same number of variants from a shared candidate universe. Only the selection criterion changes which variants enter the plate.
Evaluation without label leakage
An ablation estimates a component's contribution by comparing methods that differ only in that component. In v1, the two ablation methods differ only in the ESM masking-dispersion term. Both rank candidates before any measured fitness is revealed.
The loop-count baseline scores a candidate v by n(v), the number of interaction terms containing v. The dispersion-weighted method uses n(v) × τ²(v), where τ²(v) is the ESM masking dispersion across 16 perturbations.
Both methods prioritize variants appearing in many interaction terms. The ablation tests whether masking dispersion improves selection beyond loop count.
In the symmetric four-site candidate universe, all candidates of the same mutation order have identical loop counts: 1,140 for singles, 39 for doubles, and 1 for triples.
Consequently, loop count cannot rank candidates within a mutation order. Tied candidates may therefore be selected according to enumeration order unless ties are broken explicitly.
A seeded tie reanalysis randomizes tied candidates while recording the random seed. This separates the effect of the scoring rule from enumeration order.
Figure n°4: What masking dispersion adds to the v1 score
Description: The ablation compares loop count alone with loop count weighted by ESM masking dispersion. Current evidence does not establish an additional benefit from the dispersion term.
Evidence status
The evidence boundary changed after a mathematical audit of the recovery metric.
The original analysis correlated predicted and measured epistasis contrasts. Both quantities contained the same signed sum of purchased lower-order measurements. Their correlation could therefore increase through this shared skeleton, even if predictions for the unmeasured terms did not improve. The former map-recovery interpretation is withdrawn; those correlations are retained only as diagnostics.
The audit did not only remove a claim: it specified the replacement. Recovery is now posed directly on the Fourier coefficients of the unmeasured interaction terms, which cannot inherit the purchased lower-order skeleton.
No tracked corrected-recovery artifact currently demonstrates positive error reduction under that test. The project therefore makes no public claim that any method reconstructed an epistasis map.
The tracked downstream v1 artifacts remain historical observations on particular selected plates. Under one tie realization, the learning-curve AUC favored structural over fitness-greedy on GB1 and TrpB. The incremental info versus structural gate did not pass on either landscape. Because the loop-count score is constant within mutation order and the tracked artifacts do not sample tie seeds, these results do not estimate the acquisition method over its selection distribution.
The supported interpretation is narrower: the project distinguishes fitness optimization from measurement design, implements reproducible label-blind baselines, preserves context-dependent ESM-2 scoring, and exposes why identifiability, selection variability and leakage-resistant metrics are central to scientific evaluation.
A registered gate is a success criterion specified before inspection of the final results.[7] All findings remain provisional and are limited to the evaluated benchmarks, learner, and experimental protocol. They are not presented as general conclusions.
The TrpB source paper reports 871 imputed fitness values, but the public mirror does not identify them row by row.
Show the withdrawn diagnostic figure
Figure n°5: Withdrawn diagnostic — not evidence of epistasis-map recovery
Description: Predicted and measured contrasts share purchased lower-order terms, so correlation can rise without better prediction of unmeasured components. See the evaluation methodology notes.[7] Point estimates only; y-axis scales differ by row. The absence of intervals is a data-availability constraint: the public TrpB artifact does not include pointwise confidence intervals.
What the study established
- Variant selection can be posed as measurement design: predicted fitness and expected experimental value are distinct objectives.
- Conjoint scoring avoids imposing additivity by separately scoring mutations on an unchanged reference context.
- Loop count defines a simple, reproducible allocation baseline, but it does not explicitly reward loop completion or guarantee identifiability.
- Historical downstream artifacts favored particular loop-count plates over fitness-greedy plates; the tracked evidence does not establish robustness across tie seeds.
- Masking dispersion has not yet shown added value beyond loop coverage in the tested v1 procedure.
- The original map-recovery interpretation is withdrawn because its correlation metric was confounded by shared measured terms, and the audit sharpened the question into a stricter Fourier-coefficient test.
These findings remain provisional and do not establish generality beyond the evaluated landscapes, learner, and protocol.
Why masking dispersion remains an open question
Sensitivity to an artificially masked sequence context is not the same as calibrated predictive uncertainty. A prediction can vary under masking and still be accurate; it can also remain stable and be wrong.
The v1 score also ignores error covariance between variants that share sequence context and pass through the same model. Current evidence therefore supports only a narrow conclusion: on these two four-site landscapes and with this fixed downstream learner, the registered analyses do not establish added value from masking dispersion beyond loop coverage. They do not show that contextual uncertainty signals can never be useful.
Next validation steps
The current version is a single-shot allocator: it ranks the complete candidate pool once and returns one plate of B variants.
A future extension could be sequential. It would measure a first plate, update its beliefs and uncertainty estimates for the remaining variants, and then select the next plate based on what the experiment resolved.
That would make the geodetic analogy literal. Once one part of the interaction network has been measured, related measurements may become less valuable because they provide redundant information. Measurements in unresolved parts of the network would become more valuable.
A future version should also keep three quantities separate:
- Predicted effect: how well the model expects a variant to perform.
- Calibrated predictive error: how uncertain that prediction is expected to be.
- Experimental value: how informative measuring the variant would be for the current scientific question, given what has already been measured.
The first helps predict, the second quantifies predictive uncertainty, and the third guides experimental design. One score should not stand in for all three.
Implications for experimental design
Protein language models make predictions cheap.
Experiments remain scarce.
As candidate generation scales, the bottleneck shifts from proposing sequences to deciding which evidence is worth producing.
epibudget is an early, static step toward that objective. It selects one plate before observing labels; a true closed-loop system would update its beliefs after every measured batch and use those results to design the next experiment.
The useful question is not “Which variants does the model like?” but “Which measurements make the interaction structure identifiable?”
The next generation of scientific AI should not merely predict biology from existing data. It should decide which experiment can change what we know.
References
- Phillips, P. C. Epistasis — the essential role of gene interactions in the structure and evolution of genetic systems. Nature Reviews Genetics (2008).
- Poelwijk, F. J. et al. The context-dependence of mutations: a linkage of formalisms. PLOS Computational Biology (2016).
- Faure, A. J. et al. An extension of the Walsh-Hadamard transform to calculate and model epistasis in genetic landscapes of arbitrary shape and complexity. PLOS Computational Biology (2024).
- Lin, Z. et al. Evolutionary-scale prediction of atomic-level protein structure with a language model. Science (2023).
- Wu, N. C. et al. Adaptation in protein fitness landscapes is facilitated by indirect paths. eLife (2016).
- Johnston, K. E. et al. A combinatorially complete epistatic fitness landscape in an enzyme active site. PNAS (2024).
- Frozen validation protocol, audit remediation notes, limitations, and tracked evidence artifact.