ARVIND GURUPRASAD
Reading mode Plain
Degree Project · Biostatistics & Data Science Dept. of Medical Epidemiology & Biostatistics

Can AI agents standardise clinical data without losing its meaning?

A controlled study of whether large language model "agents" can turn messy hospital lab and medication records into a research ready format, while keeping every clinical fact intact.

Arvind Guruprasad · Author Supervisor · Vladimir Li, Arkus AI Models · GPT-5.4 & Kimi-K2.6
3
AI agent designs compared
2
Leading AI models benchmarked
5
Cumulative layers of "messiness"
100%
Lab accuracy, best agent
91%
Medication accuracy, best agent
§01 · The stakes

A converted table can look valid and still be clinically wrong

Every hospital records data its own way. To pool it for research, that data must be reshaped into a shared standard, and the danger is subtle: structural checks pass while clinical meaning quietly breaks.

OMOP

A widely used "common data model": shared tables and rules so different hospitals' data can be analysed together.

ETL

Extract, Transform, Load. The pipeline that pulls data from a source system, reshapes it, and loads it into the target.

Reshaping records into a shared model is not just renaming columns; it is a meaning preservation task. A lab result with the wrong unit, a local code mapped to the wrong standard, or a medication order mistaken for an administration each passes a structural check yet corrupts the science.

That is why this study targets lab results and medications: their meaning is spread across many fields at once (code, unit, value, dose, route, timing), so they are the highest stakes cases to get right. The blunt takeaway: "the pipeline ran" is not the same as "the data are correct."

What "losing meaning" looks like

Lab unit conversion safe

Glucose 65.5 mg/dL
↓ agent output
3.635213 mmol/L
Different text, same clinical value. A valid conversion, fine after review.

Medication instructions ambiguous

"2.5 mg QD"
↓ agent output
"2,5 mg ORAL daily TABLET"
Dose still readable, but route and form fold into the instructions, harder to compare.

Wrong drug specificity risky

Metformin ER (extended-release)
↓ mapped to
a less-specific drug code
The exact formulation may no longer be traceable, changing a drug exposure definition.
§02 · Method

A test built to be fair and repeatable

Judging the AI fairly needs a known-correct answer for every record, impossible with real patients. So the benchmark is fully synthetic and privacy safe: the right answer is fixed in advance, then the source data is made messier in controlled steps.

01

Generate

Synthetic patient records. No real people, fully shareable.

02

Enrich

Add the lab and medication detail a real benchmark needs.

03

Lock the truth

Fix the exact correct target answer to grade against.

04

Add messiness

Layer in realistic noise: codes, units, dates, formats.

05

Score

Grade each AI automatically, field by field.

Because the answer key is known and the data synthetic, the whole pipeline is open, auditable, and reproducible. A deliberate methodological choice, not just a privacy workaround.

⌘ Toggle "Technical" in the top bar for dataset internals, contracts, and statistics.
Technical detail · benchmark dataset

The source backbone is a Synthea-derived cohort (seed 42): 52 patients, 390 encounters, 1,222 lab rows, 213 medication orders, 6,450 medication administrations. Raw exports become five canonical tables, then deterministic OMOP CDM v5.4 truth for the measurement and drug_exposure targets (person and visit_occurrence act as controls).

Ground-truth artifacts (schema map, term map, transform map, attainable mask, provenance manifests) ship with the data so scoring never depends on after-the-fact human judgement. Lab rows retain LOINC truth; medications retain RxNorm truth, via deterministic local-code maps.

§03 · Difficulty

Five layers of real-world messiness

Real hospital data is rarely random noise; it is locally coherent but externally confusing. Five families of variation stack one on the next, so each agent faces a steadily harder version of the same records. Every layer stays recoverable, so the test rewards skill, not luck.

L1
Representation
Decimal commas, local null markers, odd date formats, local category codes.
L2
Schema and contract
Renamed fields, timestamps split apart, dose and reference text merged.
L3
Terminology
Local lab codes, formulary IDs and brand aliases instead of standard codes.
L4
Units and values
Unit swaps and scale changes: right concept, different magnitude on the page.
L5
Timing
Workflow timestamps shifted or redacted: which date is the true event time?

Out of scope by design: broken record linkage, pipeline glitches, and genuinely ambiguous data. These need human adjudication and are flagged as future work.

§04 · The agents

Three designs, each given more capability

The leader's question: does more tooling and autonomy actually make the AI better? Three designs add exactly one capability at a time.

A
Reasoning only

The baseline

Reads the schema and sample rows, then writes a mapping plan from its own knowledge. No lookups, no tools.

+ Schema reasoning
no external knowledge
B
+ Terminology lookup

The fact-checker

Everything A does, plus the ability to look up official medical codes for lab tests and drugs as it works.

+ Schema reasoning
+ Grounded code lookups
★ Best overall
C
+ Working environment

The analyst

A full workspace agent: it inspects files, writes and runs code, validates its own output, fixes mistakes, finalises.

+ Schema reasoning
+ Grounded code lookups
+ Run code and self-check
⌘ Toggle "Technical" for the exact architecture contracts.
Technical detail · architecture contracts

A emits a mapping spec applied by a deterministic executor (tests model-internal knowledge alone). B keeps A's prompt-to-spec and executor contract but adds UMLS/LOINC and RxNorm/RxNav tool paths, isolating terminology grounding as the only added variable. C shifts to a sandboxed single-agent workspace (shell, Python, local terminology tools, validation and finalisation) on a matched local-tool profile with web search disabled, so it gets no browsing advantage over A and B.

§05 · Results

What the numbers actually showed

More capability did help: the workspace agent (C) scored highest or tied-highest on both targets. But the gains were uneven, and came with operational costs a single accuracy score would hide.

Accepted match rate, by agent

Share of clinical fields preserved correctly, allowing documented clinically-equivalent conversions. Best model shown per agent.

Lab measurements near-solved
Agent A · baseline99–100%
Agent B · lookup99–100%
Agent C · workspace ★ best100%
Medications the harder target
Agent A · baseline89.9–90.6%
Agent B · lookup90.4–91.3%
Agent C · workspace ★ best91.4–91.5% ▲ +0.9

The 100% lab figure reflects documented "clinically equivalent" rules. It is not a claim of perfect, unconstrained conversion.

The trade-off leaders should see

Accuracy went up with capability. The workspace agent led on both data types.

Reliability went down. The simplest agent (A) never failed; tool-heavy agents hit rate limits and policy errors that aborted runs.

Cost climbed steeply for a roughly 1 to 2 point medication gain: most tools, tokens, and time.

Two models, near-tie on quality. GPT-5.4 and Kimi-K2.6 scored almost identically, but GPT-5.4 was more reliable and far more efficient.

The honest read: "more agentic" is not automatically "better." It depends on data type, reliability needs, and budget.

⌘ Toggle "Technical" for confidence intervals, robustness, and token cost.
Technical detail · accuracy, reliability and cost

Accepted (credits deterministic exceptions plus reviewed unit/value equivalence) vs exact (strict surface): measurement exact about 96 to 97% while accepted hit ceiling; drug exact about 90.7%. Pairwise bootstrap at full L1 to L5: C beat A on accepted drug_exposure by 1.25 pp (GPT, CI 0.64 to 1.86) and 1.96 pp (Kimi, CI 0.24 to 4.81); C vs B was only 0.24 pp for GPT (CI -0.57 to 1.25), a small layer-dependent edge, not a clean ordering.

Robustness: from L1 to the full stack, accepted-score retention stayed 0.992 to 1.012. Cumulative noise mostly changed which errors appeared rather than collapsing the score.

Reliability and cost: Agent A had zero failures; failures concentrated in tool-heavy B and C. A zero-imputation check (failures scored 0) flips Kimi's ordering toward A. Agent C tokens per source-row: Kimi 11,639 vs GPT 3,986 (drug), 1,644 vs 377 (measurement); average runtime 506.6s vs 137.1s.

§06 · Failure modes

Lab and medication data fail differently

A single accuracy number hides clinically different mistakes. Breaking errors down by type is the study's most decision-relevant finding.

L

Lab results: mostly cosmetic

  • safeUnit conversions that change the text but not the clinical value.
  • auditLabels swapped for codes: answer right, human-readable trail weaker.

Lookup tools cut likely-wrong code mappings sharply, from about 32% down to single digits.

M

Medications: consequential

  • doseDosage instructions (the "sig") reworded so they no longer compare cleanly.
  • drugProduct specificity lost, e.g. extended-release mapped to a generic code.
  • routeRoute of administration (e.g. "oral") dropped on some rows.
  • timeExposure dates shifted by a day, which can matter for duration-sensitive analyses.

The oversight lesson: medication data needs closer human review than lab data, even at high headline accuracy. A plausible-looking medication table is not proof it is correct.

§07 · So what

What this means for your organisation

"The pipeline ran" is not the same as "the data are correct." Quality must be measured at the level of clinical meaning.

For CMIOs and clinical leaders

  1. Promising, not deployable. Agents recover a lot of clinical meaning, but validated only on synthetic data with a known answer key.
  2. Keep humans in the loop for medications: dose, route, product specificity, and timing are where meaning slips.
  3. Insist on semantic checks, not just "did it load." Demand source-to-target traceability you can audit.

For CTOs and technology leaders

  1. More autonomy does not equal better ROI. The most capable agent added cost, latency, and failure modes for a small gain.
  2. Reliability is a first-class metric. Tool-heavy agents hit rate limits and policy errors; the simplest design never failed.
  3. Model choice is multi-dimensional. Two strong models tied on quality but differed up to about 3 times on token cost and runtime.

Before any patient-data use: source-quality controls, deterministic validation, governance review, and expert oversight remain non-negotiable.