Delivery journal · The context behind the implementationDiscuss a similar challenge ↗

From Epic Clarity to an openEHR Clinical Trial Platform for IBD Research

November 18, 2026
16 min read
Written by
Jitendra Choudhary
Jitendra Choudhary

CTO & Co-Founder

CTO & Co-Founder at Nirmitee.io. Architects healthcare integrations across FHIR, SMART on FHIR, ABDM and NHCX, writing from production experience taking hospital software from sandbox to go-live.


A Dutch inflammatory bowel disease research program needed to reuse questionnaire responses and visit data recorded in Epic. The source data was extracted from Epic Clarity, Epic's SQL reporting database, and prepared as study-specific CSV and Excel files. The file-based handoff let the team remove direct patient identifiers before loading the data into the clinical trial platform.

Nirmitee built the ingestion service that validated the files, standardized Dutch questionnaire responses, and wrote coded records to the openEHR-based platform. FHIR Patient and Encounter references linked records across questionnaires and visits.

6clinical templates delivered
257fields mapped from Epic Clarity exports
114answer-to-code mapping rules
398total target fields across six openEHR templates

At a glance

Clinical studies need research data, not hospital exports

WhoAn IBD (inflammatory bowel disease: Crohn's disease and ulcerative colitis) research program in the Netherlands.
Where the data livesEpic, the hospital EHR. It is extracted from Epic Clarity and prepared as study-specific CSV and Excel files.
Privacy stepDirect patient identifiers are removed in the file handoff, before anything loads into the trial platform.
What they neededEvery intake, follow-up, vaccination, consent and patient-profile answer as a standard, coded record they can query across patients and reuse in the next study.
What we builtA mapping service that loads the exports, creates one FHIR patient and one visit per date, and writes every answer into the right openEHR template with the right code.
Where it standsIn production on Kubernetes. New study forms are added with a mapping file, without a new release.

About IBD research

A lifelong disease that research has to follow for years

Inflammatory bowel disease (IBD) covers Crohn's disease and ulcerative colitis: chronic inflammation of the gut that comes and goes over a lifetime. Patients are often treated with drugs that dampen the immune system, so their care, and the research on it, runs across many visits and many years.

577.6people with IBD per 100,000 in the Netherlands, Dec 31, 2020 [1]
20.1new diagnoses per 100,000 in 2020 [1]
8university hospitals in the Dutch IBD Biobank [2]

The Netherlands has a long tradition of IBD cohorts. The Dutch IBD Biobank, set up in 2007 under the Parelsnoer Institute across all eight university medical centers, records about 225 standardized data items per patient [2]. The IBD South Limburg cohort has followed newly diagnosed adults since 1991 [3].

Why vaccination and TB screening are in the data. European guidelines (ECCO) ask clinicians to screen for tuberculosis before anti-TNF therapy and to plan vaccinations around the start of immune-suppressing treatment [4][5]. That is why this hub has a full template for infections, TB tests and vaccinations, not just disease details.

What studies need from the data: the same answers, coded the same way, for every patient and every visit, so a researcher can compare disease type, location, behavior, surgery and treatment across thousands of people and many years.

The problem

An Epic export is a file, not a study dataset

Epic holds the clinical truth, but a research team gets a CSV. Each column is a Dutch questionnaire path, such as Peri-operatieve visit?/Operatie/1/Indicatie voor operatie (peri-operative visit / operation 1 / indication for surgery). Answers arrive as Dutch text ("Ja", "Nee", "Kaukasisch"), local labels, broken accents ("Sigmoïdoscopie"), and dates that are sometimes only a year. One patient appears in up to six files.

In that form the data cannot be queried across patients, compared between studies, or passed to another research system. Every study began with cleaning exports by hand.

Mapping a column is easy. Putting a Dutch questionnaire answer into the right openEHR path, with the right SNOMED code, across six templates, is the work.

Getting data out of Epic

How research data leaves Epic, and which route fits

Several Dutch university hospitals run Epic, including Amsterdam UMC, Radboudumc and UMCG [6][7]. Epic has more than one way to give data to researchers. Each answers a different question.

RouteWhat it isWhat a researcher getsGood for
Reporting WorkbenchReports run on Chronicles, Epic's live database [8]A report of current dataOperational lists, small extracts
ClarityEpic's SQL reporting database; access needs Epic data-model training and approval [8][9]Row-level tables, often pulled by an extraction teamStudy extracts such as questionnaire answers
CaboodleEpic's data warehouse, refreshed nightly [8]Integrated, identifiable warehouse dataAnalytics across sources
SlicerDicerSelf-service cohort tool on the Caboodle model [8]Counts and charts, not record-level extracts [8]Feasibility: how many patients fit a study
Bulk FHIRStandard FHIR export, asynchronous, NDJSON files [10][11]FHIR resources by typeStandard clinical data such as conditions and labs

Questionnaires are the key. Study forms like an IBD intake or follow-up are questionnaires in Epic. Their answers sit in their own table family: CL_QANSWER holds which questionnaire was answered, when, and a link to a research-study enrollment; CL_QANSWER_QA holds the answer to each question [12][13]. Flowsheet readings sit separately in IP_FLWSHT_MEAS [14]. A flat CSV export of these tables is what reaches the research team: one column per question path, in Dutch.

The route used here: Clarity. For the database itself, see our guide to Epic Clarity data integration; for the warehouse route, Epic Caboodle; and for the standards route, Bulk FHIR export from Epic.

The study data is extracted from Epic Clarity and prepared as study-specific CSV and Excel files. Working from files, not a live connection, gives the team a clear point to remove direct patient identifiers before the data reaches the trial platform. Our service starts from those files.

End to end

From a clinic visit to a research query

The full path one patient answer takes, from the moment it is recorded in Epic to the moment a researcher can count it.

Clinic visitquestionnaireanswered in Epic Study extractCSV / ExcelDutch question paths Mapping servicegroup by patientcode every answerfill 6 templates FHIRwho and which visit openEHRwhat was answered Researchquery acrosspatients and studies linked by encounter id HOSPITALHOSPITALNIRMITEERESEARCH PLATFORMRESEARCH TEAM

Fig. A · The full data journey, and who owns each step

Explore our healthcare interoperability solutions and our openEHR integration services for this kind of data exchange.

Why it was important

Without this step, the studies cannot start

Risk if data stays in exportsWhat it costs a studyHow the hub removes it
Each study re-cleans the same exportWeeks of data work before any analysisUpload once; the mapping is reused
Answers mean different things in different filesResults that cannot be comparedOne code per answer, the same in every record
A patient appears as several peopleWrong patient countsPatients grouped and looked up before creation
Consent is a separate spreadsheetData that cannot legally be usedConsent stored as a record for the same patient
TB and vaccination status in free textSafety questions hard to answer at scaleCoded screening and vaccine records

Why openEHR

Why not just keep working from Epic exports?

What a study needsEpic CSV exportopenEHR repository
Ask one question across all patientsJoin spreadsheets by handOne query over every record
Keep what an answer meansDutch column name and free textArchetype path plus a code (SNOMED CT, ICD-10, ATC, ISO)
Reuse data in the next studyExport and clean againAlready in the shared clinical model
Stay independent of one vendorTied to the hospital EHROpen standard, vendor-neutral
Add a new questionnaireNew export, new cleaningNew template plus one mapping file

FHIR sits beside it for identity: one Patient per person and one Encounter per visit, so every openEHR record ties back to the right person and visit.

What changed

What the research team works with now

Before

  • Epic CSV exports cleaned by hand for each study
  • Dutch free text and local labels
  • The same patient across six files
  • No way to query across studies

After

  • Upload the export; it processes in the background
  • Answers coded into six openEHR templates
  • One FHIR patient and one visit per date
  • A row-by-row error report for anything that did not load
  • New forms added by the client's own team
6clinical templates delivered
257fields mapped from Epic Clarity exports
114answer-to-code mapping rules
398total target fields across the six openEHR templates

Counted from the project code.

The templates

Six questionnaires, six openEHR templates

Each Epic questionnaire maps to one openEHR template. Two are mapped with rules files. Four needed their own code, because the openEHR validator rejects half-filled sections and those forms have many optional, nested parts.

FormTemplateAboutMapped byFieldsFrom the export
5ibd_specific_intake_visitFirst IBD visit: diagnosis, history, family, medicationBuilder7446
6vaccinatie_screeningInfections, TB screening, vaccinationsBuilder4938
7informed_consentConsent and study participationRules198
8generic_non_staticPatient data that changes: weight, lifestyle, social, mortalityBuilder11349
9generic_staticPatient data that does not change: birth, ethnicity, originRules4227
10ibd_specific_follow-up_visitEvery follow-up visit: disease activity, surgery, testsBuilder10189
Total398257

Fields = openEHR fields filled. The rest are fixed values each template requires, such as language, setting and workflow state.

One patientFHIR Patient 9 · Fixed patient dataonce: birth, ethnicity, origin 7 · Informed consentonce, before study data 5 · Intake visitonce: diagnosis and history 8 · Changing patient dataover time: weight, lifestyle, social 6 · Vaccination screeningbefore immune-suppressing drugs 10 · Follow-up visitevery visit: activity, surgery, tests

Fig. B · How the six templates sit around one patient

New formfrom the export Nested optionalsections? YAML rules fileforms 7 and 9 Python builderforms 5, 6, 8 and 10 YAML asfallback noyeson error

Fig. C · How we choose rules or a builder for each form

Our custom healthcare software development team builds clinical data tools like this one from the ground up.

Form 5

IBD intake visit

ibd_specific_intake_visit · Python builder · 74 fields

What it captures

  • IBD type and date of diagnosis
  • Location of active disease
  • Disease behavior: inflammatory, stricturing, penetrating
  • Perianal disease
  • Past IBD surgery: operation and date
  • IBD medication: product, start, stop, reason for changing or stopping
  • Family history: bowel cancer and IBD in the family, which relative, age

How the answers are handled

  • Dutch → SNOMED CT: Crohn's disease 34000006, ulcerative colitis 64766004; terminal ileum 85774003, rectum 34402009, sigmoid 60184004; stricturing 415582006, penetrating 118622000.
  • Yes/no normalized from Ja, Nee, 1, 0, y, n.
  • Section answers follow the detail: if a family history entry exists, the "is there family history" answer is set to yes, because the template requires it.
  • Coarse dates: a relative's age or a diagnosis given only as a year is kept, not rejected.
  • Empty sections are removed so the record passes openEHR validation.

Why a builder: nested family and medication sections with optional parts. A rules file filled half-sections that the validator rejected.

Form 6

Vaccination screening

vaccinatie_screening · Python builder · 49 fields

What it captures

  • Infections the patient has had
  • TB screening: Mantoux test, QuantiFERON, TB risk from history, chest X-ray, each with done flag, test and date
  • Yellow fever vaccination
  • Dutch national vaccination program (Rijksvaccinatieprogramma): vaccine, dose, date

How the answers are handled

  • Vaccines coded with ATC: for example J07AN01 BCG, J07CA09 Infanrix hexa.
  • TRUE()/FALSE() export values read as yes/no.
  • Month-and-year vaccination dates accepted.
  • Children dropped when the parent is no: no TB test details are written if the test was not done.
  • Codes outside the allowed set removed before submit.

Why it matters: IBD patients on immune-suppressing drugs must be screened for TB and vaccinated before treatment. Studies need this coded, not as free text.

Form 7

Informed consent

informed_consent · rules file · 19 fields

What it captures

  • Consent date and time
  • Study participation

How the answers are handled

  • Workflow state set by rule: "informed consent provided", state active, step active.
  • Fixed values for the parts openEHR requires but the export does not hold.

Why it matters: no study data is usable without a consent record linked to the same patient.

Form 8

Changing patient data

generic_non_static · Python builder · 113 fields

What it captures

  • Sex and gender identity
  • Weight, weight change and BMI
  • Tobacco and alcohol use
  • Other conditions (comorbidities) with dates
  • Education, work, household, religion or life philosophy
  • Mortality: death, date and cause

How the answers are handled

  • Units: kg and kg/m².
  • Smoking in SNOMED CT: daily smoker 449868002, former smoker 8517006, never smoked 266919005.
  • Cause of death in ICD-10 chapters: infections A00-B99, tumours C00-D48.
  • Comorbidity flags 1/0 turned into true/false.

Why a builder: the largest template, with many optional social and lifestyle sections that must be left out cleanly when empty.

Form 9

Fixed patient data

generic_static · rules file · 42 fields

What it captures

  • Date of birth, sex, gender identity
  • Ethnicity, for example Kaukasisch (Caucasian), Hindostaans (Hindustani), Turks (Turkish)
  • Height
  • Country of birth of the patient, both parents and all four grandparents

How the answers are handled

  • Countries to ISO 3166-1 codes from Dutch or English names: Albanië → AL, Algerije → DZ.
  • Height pulled from text by pattern and stored in cm.

Why it matters: origin across three generations is a research variable in IBD. Coded countries make it countable.

Form 10

IBD follow-up visit

ibd_specific_follow-up_visit · Python builder · 101 fields

What it captures

  • IBD type at the visit and type of outpatient visit
  • Peri-operative visit: operation, indication, approach, location, steroids around surgery
  • Pregnancy, birth weight, wish for children
  • Stoma, perianal disease and fistulas
  • Extra-intestinal manifestations
  • Imaging and endoscopy
  • Active disease on tests, new complications, side effects, supplementary nutrition

How the answers are handled

  • Manifestations in SNOMED CT: uveitis 128473001, erythema nodosum 32861005, PSC 197441003, arthritis 3723001.
  • Endoscopy coded: colonoscopy 73761001, sigmoidoscopy 24420007, video capsule 427595003.
  • Messy text accepted: both "code - label" and broken accents.
  • Units: birth weight in grams.
  • Sections skipped when the answer is empty or no.

Why a builder: the biggest and most varied form. Moved off rules on Dec 5, 2025, then about 25 field fixes followed between December 4 and 11.

FHIR

Patient and visit identity

FHIR Patient + Encounter

Patient

  • Identifier from the export's patient ID, normalized so 1, "1" and 1.0 match
  • Looked up before it is created, so no duplicates
  • Birth date and gender (M/F → male/female)

Visit

  • One Encounter per patient per visit date, ambulatory class
  • Rows from different sheets with the same date merge into one visit
  • Every openEHR composition stores its Encounter id, so records join back to the visit

Handling rules

Six kinds of rule cover most answers

RuleWhat it doesExampleUses
static_valueValue the template needs that the export does not havelanguage, setting, workflow state145
value_mappingDutch answer → code"Ziekte van Crohn" → SNOMED 34000006114
conditionalFill only when another answer allowsTB test details only if a test was done52
date_parseFull and partial dates"2019" → 2019-01-0151
numeric_extractNumber out of text"172 cm" → 1724
floatDecimal valuesweight, BMI4

Architecture

Who uses it, and what it talks to

Data managerExcel / CSV workbooks IBD mapping servicebuilt by Nirmitee FHIR server Composition APIplatform, over EHRbase Form-entry storePostgreSQL / GraphQL uploadPatient, Encountercompositionsread and update status

Fig. 1 · System context

IBD MAPPING SERVICE · one Docker container, port 5001 Web appReact 18, Vite, Tailwind Flask APIupload, jobs, error report Error reportExcel, row by row Job queue2 worker threads Mapping engineYAML rules + builder registry FHIR managerOAuth2 client credentials Sync jobAPScheduler, every 5 min openEHR submitflat compositions

Fig. 2 · Parts inside the service

Discover our healthcare product engineering approach to building clinical platforms that run in production.

Upload path

What happens to one export

Web appFlask APIWorkerFHIR serverComposition API 1 · upload workbook2 · job id3 · queue the job4 · sheet → form, group by patient5 · find or create Patient6 · create Encounter7 · flat composition8 · job status9 · results + error report

Fig. 3 · Upload sequence

Rows from every sheet are grouped by patient first, so one patient who answered four questionnaires becomes one FHIR Patient, not four.

Sync job

Every five minutes, saved forms catch up

Forms saved in the platform's form store do not always reach openEHR on the first try. Every 5 minutes a job picks up to 100 unsubmitted forms, oldest first, and sends each one. Success marks it done; failure marks it unsaved with the error kept, and the next run tries again.

60 runs = 5 hours. Orange = a retry.

Saved formstatus empty Sync runevery 5 min, up to 100 success unsavederror kept picked again next run

Fig. D · Life of a saved form in the sync job

Testing

Tests on the rules, checks on the output

  • 68 automated tests on value mapping, conditionals, dates and edge cases.
  • A script scans generated records for leftover placeholder values.
  • A script checks no two visits share one encounter.
  • 179 debug runs kept between December 5, 2025 and January 2, 2026 for field-by-field review.

Deployment

Built on every merge, running on Kubernetes

GitHub repo GitHub Actionson push to main Docker imageNode 18 build → Python 3.11 Kubernetesproduction cluster pushed to GitHub Container Registry

Fig. 4 · Build and run

Kubernetes deployment as reported by the Nirmitee team.

How it evolved

Seven phases, from field mapping to a platform the client extends

The project ran from June to December 2025 in 206 commits, then moved into production. It did not arrive in its final shape. Each phase fixed what the last one taught us.

13292812171790 JunJulAugSepOctNovDec Commits per month, 2025. December: field-by-field review and production hardening.

Fig. E · Where the work went

Phase 1 · Jun 2025

Map the fields

A Python mapper read each export column and printed where it went. First IBD fields mapped on June 23. Fast to start, but every change meant code.

Phase 2 · Jul 2025

Build real compositions

First openEHR compositions created. Intake and vaccination forms mapped. FHIR mapping added for patients and visits.

Phase 3 · Aug 2025

Move rules into YAML

Mapping moved from code to YAML configuration, one file per template. A generic FHIR mapper for Patient and Encounter, and a default location for visits.

Phase 4 · Sep 2025

Handle real workbooks

Real exports have many sheets. Uploads became background jobs with progress tracking. Practitioner and location data came from a provider instead of fixed values. OAuth2 token handling for the platform.

Phase 5 · Oct 2025

Let the server own identity

Patient and visit ids were left to the FHIR server, and every reference uses the server's patient id. Patient identifiers moved to UUIDs. Each running container now reports its exact code version.

Phase 6 · Nov 2025

Clean up for production

Old templates (forms 2 to 4) and old FHIR profiles removed. Invalid SNOMED and country codes fixed. Hardcoded test dates removed. Logging per run added so every upload can be traced.

Phase 7 · Dec 2025

Field review and hardening

90 commits. Output reviewed field by field against the templates. Complex forms moved to Python builders that skip empty sections. A builder registry replaced hardcoded template checks. Every composition now carries its visit id. The 5-minute sync job shipped on December 17.

2026

Production

Running on Kubernetes. The client's team adds new templates and mappers on the same engine.

Python mapperJun 2025 · change = code YAML rules per templateAug 2025 · change = config YAML + registered buildersDec 2025 · rules or code per form

Fig. F · How the mapping approach changed

Decisions

What we decided, in order

WhenTypeEntry
Jun 23, 2025StartedFirst field mapping.
Aug 25, 2025ShippedRows from different sheets on the same date merge into one visit.
Oct 25, 2025DecisionPatient identifiers moved to UUIDs.
Nov 18, 2025FixedInvalid SNOMED and country codes corrected; duplicate patients removed.
Dec 5, 2025DecisionComplex forms move from rules files to Python builders that skip empty sections, because the openEHR validator rejects half-filled structures.
Dec 2025DecisionHardcoded template checks across 15+ files replaced by a builder registry.
3–Dec 11, 2025ShippedField-level fixes on Dutch clinical fields: perianal disease, peri-operative visits, TB vaccination, cause of death, religion.
8–Dec 12, 2025WithdrawnThree changes reverted and redone: care-facility links on encounters, extra submission logging, a vaccination fix.
Dec 10, 2025ShippedEvery composition carries its FHIR Encounter id.
Dec 17, 2025ShippedScheduled sync job, every 5 minutes.
2026LiveIn production on Kubernetes; client adds templates.

Built to extend

The client adds new forms without waiting for us

To addWhat the team writesWhat does not change
A simple formOne YAML mapping file for the new templateAPI, router, FHIR code
A complex formOne Python builder, registered with one decoratorRouter and other builders
A new answer codeOne line in a value-mapping listEverything else

Sources

Where the background facts come from

  1. Prevalence, incidence and mortality of IBD in the Netherlands, Journal of Crohn's and Colitis, 2025. pure.eur.nl/en/publications/prevalence-incidence-and-mortality-of-inflammatory-bowel-disease-
  2. Cohort profile: the Dutch IBD Biobank (Parelsnoer Institute), 2017. research.rug.nl
  3. Cohort profile: the IBD South Limburg cohort. pubmed.ncbi.nlm.nih.gov/28430884
  4. ECCO guidelines on opportunistic infections in IBD, 2025 update. ecco-ibd.eu
  5. ECCO guidance on TB screening before anti-TNF therapy. pmc.ncbi.nlm.nih.gov/articles/PMC4986392
  6. Amsterdam UMC chooses Epic, 2013. computable.nl
  7. EPD market survey of Dutch hospitals, 2024. dutchitchannel.nl
  8. Vanderbilt VCLIC: Epic research data tools. vumc.org/vclic
  9. UC Davis Health data services: Clarity and Caboodle access. health.ucdavis.edu/data/services.html
  10. Epic on FHIR: Bulk Data specifications. fhir.epic.com/Specifications?api=10171
  11. FHIR Bulk Data export at scale. capminds.com
  12. open.epic.com EHI table: CL_QANSWER
  13. open.epic.com EHI table: CL_QANSWER_QA
  14. open.epic.com EHI table: IP_FLWSHT_MEAS

Project facts come from the project code and the Nirmitee team, not from these sources.

Planning an Epic, FHIR or openEHR research data project?

We turn EHR exports into coded, research-ready openEHR and FHIR records. Explore our EHR and EMR integration services, or talk to our team to scope your project.

Share this case study

Related Case Studies