When a Lab Command Says SUCCEEDED, What Actually Happened?

Anthropic's Model Hardware Standard (MHS) gives AI agents a common interface to physical devices through standard primitives such as read and write.^[1]

That standardizes how an agent issues a command. It does not, by itself, establish what the command changed in the physical world.

Suppose an agent sends write("dispense", 40 µL) and communication fails halfway through. Before retrying, the system needs to distinguish between zero, partial, and complete transfer. A software status such as SUCCEEDED, FAILED, or UNKNOWN describes the execution attempt. It is not a measurement of the resulting physical state.

This article argues that reliable laboratory autonomy requires a separate effect state, supported by evidence explicitly linked to each physical action. The issue is not whether laboratories have sensors or readbacks. Experts already know they do. The issue is whether the system records which observation supports which action, what claim that observation justifies, and whether the resulting uncertainty makes a retry safe.

I tested this distinction against two public datasets:

• Flex-Cat, an autonomous catalysis platform built around Chemspeed automation
• a laboratory-scale Batch Distillation dataset containing induced anomalies and expert recovery annotations

The parsing code, methods, generated metrics, and deterministic artifacts are available in the public audit repository.

These datasets cannot establish a universal law of laboratory automation. They can show whether the proposed distinction explains concrete gaps in otherwise rich execution records.

1. MHS exposes the missing contract

At Genentech, researchers used MHS to coordinate a liquid handler, robotic arm, and plate reader for a BCA protein assay. Claude encountered tip-pickup and fluid-detection errors and recovered from some of them autonomously.^[1]

A harder failure appeared when bubbles occupied part of a pipette tip. Although the protocol requested 40 µL, the liquid transfer could be lower. Retrying created more agitation and foam, so the researchers had to treat this as a physical failure requiring physical correction.

This episode separates three questions: was the command accepted, did the software operation complete, and was the intended physical effect confirmed?

MHS is a research preview, with fault detection, safety evaluations, and richer device state still in development. It need not solve every sensing problem. Its command interface makes a complementary requirement visible: a contract from execution to physical evidence.

2. Flex-Cat: rich evidence, implicit relationships

Flex-Cat is a closed-loop autonomous catalysis platform published in Nature Communications.^[2] Its reproducibility package contains an example Chemspeed run with a main event log, task definitions, device configuration, volumetric transfer records, and low-level controller logs.

The high-level Eventlog.txt contained:

• 986 operation starts
• 986 operation ends
• 986/986 operations paired cleanly
• 0 unmatched operations
• 0 application cuts

The orchestration record is exceptionally clean. Lower layers add controller-position reads, pressure observations, and device states such as Controller ready and Move finished.

These signals can support physical-execution claims, but their relationship to individual actions remains implicit in timestamps, task definitions, and device context. The data does not provide a record such as:

EffectEvidence(
    action_id = "motion-123",
    source = "controller_position",
    claim = "target_position_reached"
)

Lower-level error and warning labels do not map cleanly to failed high-level operations; some precede later Controller ready and Move finished states. Severity alone cannot determine effect state without later readbacks and action context.

For liquid handling, end events largely repeat the requested parameters:

START
volume = 6.732 µL

END
volume = 6.732 µL

This confirms software completion, not liquid transfer. Separate records contain an actualVolume field, but all 60 observations match the requested value and the public data does not establish an independent measurement. The defensible claim is: command execution known, physical liquid effect independently unconfirmed.

AEGIS reaches the same boundary with visual monitoring on an Opentrons OT-2.^[3] It detects several visible failures, while transparent water remains difficult. There is no universal sensor for “the action happened.” Evidence must fit the effect being claimed.

3. Batch Distillation: recovery happened, but the log often cannot explain how

Arweiler and colleagues built a laboratory-scale batch distillation plant and ran 119 experiments, including fault-free runs and experiments with intentionally induced anomalies.^[4] The dataset combines sensor and actuator time series, expert annotations, video, audio, NMR data, and operation logs.

For the subset with operation logs, I found:

• 106 experiments
• 237 deduplicated anomaly records
• 137 ConfirmedAnomaly records
• 79 labelled recoveries

The metadata describes recovery actions such as Restore normal state, providing ground truth that a recovery occurred. I then inspected each recovery in an operation-log window from 60 seconds before to 120 seconds after the perturbation ended. When that timestamp was unavailable, I used the anomaly end as an explicit fallback.

In the original window, 34/79 = 43.04% of labelled recoveries had at least one parseable operation-log row nearby. This is a log-activity proxy, not evidence that the recovery itself was observed.

Five of the 79 recoveries are anchored outside the interval covered by their operational log. No window can match them. They are unanswerable rather than negative, so the comparison with random background uses the remaining 74.

At the original window, log activity is concentrated around the recovery labels at 2.60× the background obtained by random anchoring within the same experiment's log: 34/74 = 45.95% versus 17.67%, with empirical p = 0.0001 over 10,000 iterations using seed 20260830.

Widening the window raises raw headline coverage from 34/79 to 37/79, but random background rises faster. At ±600 s, the observed 50.0% is indistinguishable from chance (background 51.1%, ratio 0.98×, p = 0.63). The extra matches gained by widening are what chance alone would produce.

The anchor is the end of the perturbation. Removing event classes mechanically coupled to this boundary, including mode transitions and setpoint changes, leaves the enrichment intact at 2.80–2.85×. This rules out that precise explanation. But the residual signal is carried mainly by recipe-engine rows, which may be coupled to the same boundary when the controller resumes its step schedule. This remains a temporal association, not evidence that the labelled physical or operator recovery was itself observed.

A silent window does not mean no operator acted. The intervention may fall outside the selected window, appear in another modality, or remain outside the software logs. Recovery ground truth and recovery evidence are different records.

The two datasets expose complementary gaps. Flex-Cat preserves controller and process evidence without consistently binding it to the actions it could verify. Batch Distillation preserves expert recovery outcomes while its operation log often omits the intervention itself.

4. Execution state and effect state need separate semantics

A physical action has at least two states:

1. Execution state: what happened to the software command?
2. Effect state: what does the available evidence justify about the physical consequence?

A minimal action-linked evidence model could look like:

ActionIntent
    ↓
ExecutionAttempt
    ↓
ControllerAcknowledgement
    ↓
EffectEvidence[]
    ↓
EffectClaim
    ↓
RecoveryDecision

The resulting EffectClaim should preserve uncertainty rather than collapse it into command status:

full_effect_confirmed
partial_effect_confirmed
no_effect_confirmed
reported_only
effect_unknown

Evidence may come from an encoder, controller readback, process sensor, balance, flow sensor, analytical measurement, vision system, or operator confirmation. The model does not prescribe a universal sensor. It provides a common way to state which observation supports which claim about a specific action.

At minimum, such a record needs the action identifier, evidence source, observation time, observed value or event, and the effect claim it supports. The acceptance rule remains procedure-specific. An encoder may be sufficient to confirm a motor position, while a commanded pipette volume may require gravimetric, pressure, flow, visual, or analytical confirmation. The shared model standardizes provenance and uncertainty, not the scientific criterion for every instrument.

A result of SUCCEEDED should not automatically imply full_effect_confirmed. For a non-idempotent action, effect_unknown is fundamentally different from no_effect_confirmed. If 40 µL may already have been dispensed, retrying can compound the physical error.

5. This is an infrastructure contract, not a request for more logs

The evidence needed to verify an action often already exists somewhere in the automation stack. Flex-Cat shows controller positions, process values, device states, and detailed traces. The difficulty is that these observations are distributed across layers and are not consistently identified as evidence for a particular effect.

Simply retaining every raw message would increase storage and analysis costs without resolving that ambiguity. The useful contract is narrower:

MHS / device APIs
    intent → command

action-linked evidence
    command → justified claim about reality

MHS and an effect-evidence contract solve complementary problems. One standardizes how an agent asks hardware to act. The other states what the system is justified in believing after the attempt. Sometimes that evidence is a controller value; sometimes it comes from another sensor or an analytical result; and sometimes the correct state remains effect_unknown.

These two public datasets are not representative enough to show that action-to-evidence provenance is absent from laboratory automation generally. They do show that useful evidence can remain implicit even in rich, reproducible records. The proposal would be unnecessary where systems already expose explicit action identifiers, typed effect states, linked readbacks, and recovery outcomes.

Conclusion

The design requirement is not simply more logging. It is an explicit, machine-readable link between an action, the evidence bearing on its physical effect, and the claim the system is justified in making.

For non-idempotent operations, this distinction directly determines whether retrying is safe. Until an automation stack exposes it, SUCCEEDED should mean only that command execution completed, not that the intended physical effect is known to have occurred.

References

1. Anthropic. “Previewing the Model Hardware Standard.” August 27, 2026.
2. Bennett, J. A. et al. “An autonomous lab for data-driven homogeneous catalysis.” Nature Communications 17, 7783 (2026). Public execution package: Zenodo 18930287.
3. Setty, P. V., Ramanathan, A., Foster, I. & Stevens, R. “AEGIS: Assay-Aware Protocol Validation and Runtime Monitoring for Open-Source Liquid Handling Robots.” arXiv:2607.15620 (2026).
4. Arweiler, J. et al. “Batch Distillation Data for Developing Machine Learning Anomaly Detection Methods.” Scientific Data 13, 513 (2026). Dataset concept record: Zenodo 17395543; audit performed on versioned release 21535243.