Back
By Vivien Perrelle · 2026-05-11

Regulators Don't Accept Vibes: The Two Layers Pharma AI Is Missing

1. The Signal

Novo Nordisk built NovoScribe[1], an internal pipeline that drafts Clinical Study Reports with Claude. The numbers are public: drafting time down from 12 weeks to under 10 minutes, a team of ~50 medical writers reduced to 3, and a reported 90% cut in CSR writing time[2], with positive feedback from regulators. They experimented with other models for years before it worked.

The ROI math is brutal: by Novo Nordisk's own estimate, each day a medicine reaches market sooner is worth around $15M. AI is shrinking the market for regulatory medical writers while making each remaining one more critical.

The shift is happening, and regulators have noticed. In April 2026, the FDA issued its first Warning Letter citing improper reliance on AI[3]: a manufacturer had used AI to generate required documents without adequate human review. The agency's position is explicit: you remain fully responsible for every AI-generated output. The question is no longer whether AI writes regulatory documents. It is whether you can prove that what it writes is true.

2. The Ceiling

A Principal Medical Writer I interviewed runs AI daily on his own biohacking data: diet logs, biomarkers, time series. Roughly one response in three contains a timing error or conflates items across days. On his own data, with a 2-million-token context window, where he catches every mistake instantly.

Now project that failure rate onto a 500-page CSR headed for submission. Even the FDA's own internal AI tool, Elsa, was reported by agency officials to hallucinate nonexistent studies[4]. The stakes are not a wrong diet log entry. The stakes are patient safety and a $2B drug program.

3. Two Failure Points, Not One

The tempting fix is a single one: better fine-tuning, bigger context, smarter prompts.

The evidence says otherwise. A recent large-scale study (25,000 runs across three frontier models and eight scientific domains) found that scaffolding and prompt engineering explain 1.5% of the variance in AI performance, versus 41.4% for the base model itself. Better base models are what move the needle, and they keep arriving fast. But two constraints survive every model generation: unstructured inputs degrade even the best reasoner available, and a regulator will never accept "the model is good now" as evidence. Trust has to be demonstrated, claim by claim, whatever the model's raw accuracy.

Regulated document generation actually breaks in two distinct places:

  • •Upstream: unstructured inputs. Feed a model a messy PDF, a semi-structured CSR, or statistical output with inconsistent formatting, and it has no structural grounding. It infers relationships, and when inference fails, it fabricates. Cleaner structure reduces the error rate.
  • •Downstream: unverified outputs. Reduces, not eliminates. A model can hallucinate on perfectly structured data. Which means no amount of preprocessing makes an output submission-ready on its own. Every claim in the document must be checked against its source, by a layer independent of the one that generated it. A generator cannot be its own auditor.

Most "AI for regulatory" efforts fix neither. They demo a chatbot in the middle and hope.

4. The Architecture That Ships

What a submission-grade pipeline actually looks like:

  • •A standardization layer before the model. Structured extraction into validated, machine-readable schemas. FAIR data principles as a hard technical requirement, not an academic ideal. Inconsistencies caught at the data layer, where they are cheap.
  • •A verification layer after the model. Claim-level traceability from generated text back to source tables. Independent checks that fail loudly instead of shipping quietly. An audit trail a reviewer, human or regulatory, can actually inspect.

The generation step in the middle is the part everyone shares: the same handful of frontier models, available to every competitor. The two layers around it are where the differentiation lives. And note what the verification layer is made of today: expensive humans re-reading everything. That is the bottleneck left to systematize.

This is the unglamorous work that rarely makes the demo. It is easier to show a chatbot summarizing a CSR than to build the plumbing that makes the summary trustworthy. But without the plumbing, you are shipping a liability with good grammar.

Bottom Line

The model works. The system around it does not exist yet. Regulators do not accept confident text; they accept documents where every number has a lineage. Structure the inputs so the model reasons on solid ground. Verify the outputs so no claim leaves the pipeline unchecked. Companies that build both layers will own regulatory AI. Companies that skip them will keep demoing.

References

  1. MongoDB, "Novo Nordisk & MongoDB Atlas: A Clinical Study Report in Minutes."
  2. Anthropic, "Novo Nordisk customer story."
  3. Morgan Lewis, "FDA's Warning Letter Suggests Growing Scrutiny of AI Overreliance." April 2026.
  4. Futurism (citing CNN), "The FDA Is Using an AI to Speed Up Drug Approvals and Insiders Say It's Making Horrible Mistakes." July 2025.