Arvind Guruprasad, home

← All work

Master's thesis· 2026

Can AI reformat hospital data and keep its meaning?

A controlled benchmark of three AI agent designs that reshape messy, synthetic hospital records into a research standard (OMOP), graded field by field against a locked answer key.

The outcome

Agents kept lab results almost perfectly intact and got medications right about 91% of the time, on synthetic data. The agents with tools scored highest but sometimes failed to finish; the simplest agent never failed. Passing a schema check told me nothing about whether the meaning survived.

  • ~100%lab results kept their meaning, best agent
  • ~91%medications kept their meaning, best agent
  • 0failed runs for the simplest agent
  • 32% → <10%likely-wrong code mappings once agents could look codes up
Role
Sole author
When
2026 · MSc, KI, KTH and SU
With
Supervised by Vladimir Li, Arkus AI
Proves
Knowing what to check

Layers I owned

  1. Discovery
  2. Strategy
  3. Architecture
  4. Build
  5. Rollout

01Why I built it

Hospitals record data their own way, so research means reshaping it into a shared standard. That step is quietly risky: a lab value in the wrong unit, a drug mapped to a less specific code, or an order read as a dose given all pass structural checks. I wanted to know whether AI agents can do this job without changing what the data means.

02How I did it

I generated synthetic patients with a known correct answer for every record, then made the source data messier in five stacked layers: formats, schema, local codes, units and timing. Three agents, from reasoning only to a sandboxed workspace with lookup tools, ran each layer on two models, and a deterministic scorer graded every field.

  1. Messy records5 layers of noise
  2. Agent A, B or CGPT-5.4 · Kimi-K2.6
  3. Schema checkpasses
  4. Locked truthsame meaning?
Fig. 1 · The benchmark. Most pipelines stop at the schema check; this one keeps going.

ObservationLabs mostly survived every layer of messiness. Medications broke on dosing instructions, drug specificity, route and timing.

ObservationKimi-K2.6 used about three times the tokens of GPT-5.4 on the hardest agent, and took longer.

Medications kept their meaning, by agentGPT-5.4Kimi-K2.6

Fig. 2 · All five layers of messiness stacked. More autonomy helped a little on medications; labs were 99–100% for every agent.

See the numbers
AgentGPT-5.4Kimi-K2.6
A · reasoning only90.6%89.9%
B · with lookup tools91.3%90.4%
C · sandboxed workspace91.5%91.4%

03What got hard

  1. Agents with tools sometimes failed to finish a run.

    So I kept the reasoning-only agent as a reliable baseline and reported failures next to accuracy.

  2. Grading by exact match punished clinically equivalent answers, like the same glucose value in different units.

    So I scored exact and accepted separately, crediting only documented equivalences.

04What I didn't choose

  1. Real patient records

    There is no known-correct answer for every record, and they can't be shared. Synthetic data could be graded field by field and published.

  2. One agent design

    It wouldn't show what autonomy buys. A ladder of three shows where tools help and where they add failures.

  3. Web search for the agents

    Left off, so every run used the same local terminology tools and can be repeated.

05Shipping it, and running it yourself

Defended in June 2026. The benchmark code is public, and a plain-language summary sits alongside the full thesis.

Run it yourself

github.com/arvind0212/agentic-clinical-etl-benchmark

Setup steps are in the repository's README.

Read the long read

What this doesn't prove

  • Synthetic data in a lab setting, not a deployment.
  • Only lab results and medications.
  • Only messiness that can be recovered; truly ambiguous records are future work.
  • The ~100% counts documented, clinically equivalent conversions as correct.
  • Python
  • Synthea
  • OMOP CDM 5.4
  • LOINC
  • RxNorm
  • UMLS
  • GPT-5.4
  • Kimi-K2.6
Next caseAI reads the dosing notes. Code checks every row.