From Epic Clarity to an openEHR Clinical Trial Platform for IBD Research
CTO & Co-Founder
CTO & Co-Founder at Nirmitee.io. Architects healthcare integrations across FHIR, SMART on FHIR, ABDM and NHCX, writing from production experience taking hospital software from sandbox to go-live.
A Dutch inflammatory bowel disease research program needed to reuse questionnaire responses and visit data recorded in Epic. The source data was extracted from Epic Clarity, Epic's SQL reporting database, and prepared as study-specific CSV and Excel files. The file-based handoff let the team remove direct patient identifiers before loading the data into the clinical trial platform.
Nirmitee built the ingestion service that validated the files, standardized Dutch questionnaire responses, and wrote coded records to the openEHR-based platform. FHIR Patient and Encounter references linked records across questionnaires and visits.
At a glance
Clinical studies need research data, not hospital exports
| Who | An IBD (inflammatory bowel disease: Crohn's disease and ulcerative colitis) research program in the Netherlands. |
|---|---|
| Where the data lives | Epic, the hospital EHR. It is extracted from Epic Clarity and prepared as study-specific CSV and Excel files. |
| Privacy step | Direct patient identifiers are removed in the file handoff, before anything loads into the trial platform. |
| What they needed | Every intake, follow-up, vaccination, consent and patient-profile answer as a standard, coded record they can query across patients and reuse in the next study. |
| What we built | A mapping service that loads the exports, creates one FHIR patient and one visit per date, and writes every answer into the right openEHR template with the right code. |
| Where it stands | In production on Kubernetes. New study forms are added with a mapping file, without a new release. |
About IBD research
A lifelong disease that research has to follow for years
Inflammatory bowel disease (IBD) covers Crohn's disease and ulcerative colitis: chronic inflammation of the gut that comes and goes over a lifetime. Patients are often treated with drugs that dampen the immune system, so their care, and the research on it, runs across many visits and many years.
The Netherlands has a long tradition of IBD cohorts. The Dutch IBD Biobank, set up in 2007 under the Parelsnoer Institute across all eight university medical centers, records about 225 standardized data items per patient [2]. The IBD South Limburg cohort has followed newly diagnosed adults since 1991 [3].
Why vaccination and TB screening are in the data. European guidelines (ECCO) ask clinicians to screen for tuberculosis before anti-TNF therapy and to plan vaccinations around the start of immune-suppressing treatment [4][5]. That is why this hub has a full template for infections, TB tests and vaccinations, not just disease details.
What studies need from the data: the same answers, coded the same way, for every patient and every visit, so a researcher can compare disease type, location, behavior, surgery and treatment across thousands of people and many years.
The problem
An Epic export is a file, not a study dataset
Epic holds the clinical truth, but a research team gets a CSV. Each column is a Dutch questionnaire path, such as Peri-operatieve visit?/Operatie/1/Indicatie voor operatie (peri-operative visit / operation 1 / indication for surgery). Answers arrive as Dutch text ("Ja", "Nee", "Kaukasisch"), local labels, broken accents ("Sigmoïdoscopie"), and dates that are sometimes only a year. One patient appears in up to six files.
In that form the data cannot be queried across patients, compared between studies, or passed to another research system. Every study began with cleaning exports by hand.
Mapping a column is easy. Putting a Dutch questionnaire answer into the right openEHR path, with the right SNOMED code, across six templates, is the work.
Getting data out of Epic
How research data leaves Epic, and which route fits
Several Dutch university hospitals run Epic, including Amsterdam UMC, Radboudumc and UMCG [6][7]. Epic has more than one way to give data to researchers. Each answers a different question.
| Route | What it is | What a researcher gets | Good for |
|---|---|---|---|
| Reporting Workbench | Reports run on Chronicles, Epic's live database [8] | A report of current data | Operational lists, small extracts |
| Clarity | Epic's SQL reporting database; access needs Epic data-model training and approval [8][9] | Row-level tables, often pulled by an extraction team | Study extracts such as questionnaire answers |
| Caboodle | Epic's data warehouse, refreshed nightly [8] | Integrated, identifiable warehouse data | Analytics across sources |
| SlicerDicer | Self-service cohort tool on the Caboodle model [8] | Counts and charts, not record-level extracts [8] | Feasibility: how many patients fit a study |
| Bulk FHIR | Standard FHIR export, asynchronous, NDJSON files [10][11] | FHIR resources by type | Standard clinical data such as conditions and labs |
Questionnaires are the key. Study forms like an IBD intake or follow-up are questionnaires in Epic. Their answers sit in their own table family: CL_QANSWER holds which questionnaire was answered, when, and a link to a research-study enrollment; CL_QANSWER_QA holds the answer to each question [12][13]. Flowsheet readings sit separately in IP_FLWSHT_MEAS [14]. A flat CSV export of these tables is what reaches the research team: one column per question path, in Dutch.
The route used here: Clarity. For the database itself, see our guide to Epic Clarity data integration; for the warehouse route, Epic Caboodle; and for the standards route, Bulk FHIR export from Epic.
The study data is extracted from Epic Clarity and prepared as study-specific CSV and Excel files. Working from files, not a live connection, gives the team a clear point to remove direct patient identifiers before the data reaches the trial platform. Our service starts from those files.
End to end
From a clinic visit to a research query
The full path one patient answer takes, from the moment it is recorded in Epic to the moment a researcher can count it.
Fig. A · The full data journey, and who owns each step
Explore our healthcare interoperability solutions and our openEHR integration services for this kind of data exchange.
Why it was important
Without this step, the studies cannot start
| Risk if data stays in exports | What it costs a study | How the hub removes it |
|---|---|---|
| Each study re-cleans the same export | Weeks of data work before any analysis | Upload once; the mapping is reused |
| Answers mean different things in different files | Results that cannot be compared | One code per answer, the same in every record |
| A patient appears as several people | Wrong patient counts | Patients grouped and looked up before creation |
| Consent is a separate spreadsheet | Data that cannot legally be used | Consent stored as a record for the same patient |
| TB and vaccination status in free text | Safety questions hard to answer at scale | Coded screening and vaccine records |
Why openEHR
Why not just keep working from Epic exports?
| What a study needs | Epic CSV export | openEHR repository |
|---|---|---|
| Ask one question across all patients | Join spreadsheets by hand | One query over every record |
| Keep what an answer means | Dutch column name and free text | Archetype path plus a code (SNOMED CT, ICD-10, ATC, ISO) |
| Reuse data in the next study | Export and clean again | Already in the shared clinical model |
| Stay independent of one vendor | Tied to the hospital EHR | Open standard, vendor-neutral |
| Add a new questionnaire | New export, new cleaning | New template plus one mapping file |
FHIR sits beside it for identity: one Patient per person and one Encounter per visit, so every openEHR record ties back to the right person and visit.
What changed
What the research team works with now
Before
- Epic CSV exports cleaned by hand for each study
- Dutch free text and local labels
- The same patient across six files
- No way to query across studies
After
- Upload the export; it processes in the background
- Answers coded into six openEHR templates
- One FHIR patient and one visit per date
- A row-by-row error report for anything that did not load
- New forms added by the client's own team
Counted from the project code.
The templates
Six questionnaires, six openEHR templates
Each Epic questionnaire maps to one openEHR template. Two are mapped with rules files. Four needed their own code, because the openEHR validator rejects half-filled sections and those forms have many optional, nested parts.
| Form | Template | About | Mapped by | Fields | From the export |
|---|---|---|---|---|---|
| 5 | ibd_specific_intake_visit | First IBD visit: diagnosis, history, family, medication | Builder | 74 | 46 |
| 6 | vaccinatie_screening | Infections, TB screening, vaccinations | Builder | 49 | 38 |
| 7 | informed_consent | Consent and study participation | Rules | 19 | 8 |
| 8 | generic_non_static | Patient data that changes: weight, lifestyle, social, mortality | Builder | 113 | 49 |
| 9 | generic_static | Patient data that does not change: birth, ethnicity, origin | Rules | 42 | 27 |
| 10 | ibd_specific_follow-up_visit | Every follow-up visit: disease activity, surgery, tests | Builder | 101 | 89 |
| Total | 398 | 257 |
Fields = openEHR fields filled. The rest are fixed values each template requires, such as language, setting and workflow state.
Fig. B · How the six templates sit around one patient
Fig. C · How we choose rules or a builder for each form
Our custom healthcare software development team builds clinical data tools like this one from the ground up.
IBD intake visit
ibd_specific_intake_visit · Python builder · 74 fieldsWhat it captures
- IBD type and date of diagnosis
- Location of active disease
- Disease behavior: inflammatory, stricturing, penetrating
- Perianal disease
- Past IBD surgery: operation and date
- IBD medication: product, start, stop, reason for changing or stopping
- Family history: bowel cancer and IBD in the family, which relative, age
How the answers are handled
- Dutch → SNOMED CT: Crohn's disease 34000006, ulcerative colitis 64766004; terminal ileum 85774003, rectum 34402009, sigmoid 60184004; stricturing 415582006, penetrating 118622000.
- Yes/no normalized from Ja, Nee, 1, 0, y, n.
- Section answers follow the detail: if a family history entry exists, the "is there family history" answer is set to yes, because the template requires it.
- Coarse dates: a relative's age or a diagnosis given only as a year is kept, not rejected.
- Empty sections are removed so the record passes openEHR validation.
Why a builder: nested family and medication sections with optional parts. A rules file filled half-sections that the validator rejected.
Vaccination screening
vaccinatie_screening · Python builder · 49 fieldsWhat it captures
- Infections the patient has had
- TB screening: Mantoux test, QuantiFERON, TB risk from history, chest X-ray, each with done flag, test and date
- Yellow fever vaccination
- Dutch national vaccination program (Rijksvaccinatieprogramma): vaccine, dose, date
How the answers are handled
- Vaccines coded with ATC: for example J07AN01 BCG, J07CA09 Infanrix hexa.
- TRUE()/FALSE() export values read as yes/no.
- Month-and-year vaccination dates accepted.
- Children dropped when the parent is no: no TB test details are written if the test was not done.
- Codes outside the allowed set removed before submit.
Why it matters: IBD patients on immune-suppressing drugs must be screened for TB and vaccinated before treatment. Studies need this coded, not as free text.
Informed consent
informed_consent · rules file · 19 fieldsWhat it captures
- Consent date and time
- Study participation
How the answers are handled
- Workflow state set by rule: "informed consent provided", state active, step active.
- Fixed values for the parts openEHR requires but the export does not hold.
Why it matters: no study data is usable without a consent record linked to the same patient.
Changing patient data
generic_non_static · Python builder · 113 fieldsWhat it captures
- Sex and gender identity
- Weight, weight change and BMI
- Tobacco and alcohol use
- Other conditions (comorbidities) with dates
- Education, work, household, religion or life philosophy
- Mortality: death, date and cause
How the answers are handled
- Units: kg and kg/m².
- Smoking in SNOMED CT: daily smoker 449868002, former smoker 8517006, never smoked 266919005.
- Cause of death in ICD-10 chapters: infections A00-B99, tumours C00-D48.
- Comorbidity flags 1/0 turned into true/false.
Why a builder: the largest template, with many optional social and lifestyle sections that must be left out cleanly when empty.
Fixed patient data
generic_static · rules file · 42 fieldsWhat it captures
- Date of birth, sex, gender identity
- Ethnicity, for example Kaukasisch (Caucasian), Hindostaans (Hindustani), Turks (Turkish)
- Height
- Country of birth of the patient, both parents and all four grandparents
How the answers are handled
- Countries to ISO 3166-1 codes from Dutch or English names: Albanië → AL, Algerije → DZ.
- Height pulled from text by pattern and stored in cm.
Why it matters: origin across three generations is a research variable in IBD. Coded countries make it countable.
IBD follow-up visit
ibd_specific_follow-up_visit · Python builder · 101 fieldsWhat it captures
- IBD type at the visit and type of outpatient visit
- Peri-operative visit: operation, indication, approach, location, steroids around surgery
- Pregnancy, birth weight, wish for children
- Stoma, perianal disease and fistulas
- Extra-intestinal manifestations
- Imaging and endoscopy
- Active disease on tests, new complications, side effects, supplementary nutrition
How the answers are handled
- Manifestations in SNOMED CT: uveitis 128473001, erythema nodosum 32861005, PSC 197441003, arthritis 3723001.
- Endoscopy coded: colonoscopy 73761001, sigmoidoscopy 24420007, video capsule 427595003.
- Messy text accepted: both "code - label" and broken accents.
- Units: birth weight in grams.
- Sections skipped when the answer is empty or no.
Why a builder: the biggest and most varied form. Moved off rules on Dec 5, 2025, then about 25 field fixes followed between December 4 and 11.
Patient and visit identity
FHIR Patient + EncounterPatient
- Identifier from the export's patient ID, normalized so 1, "1" and 1.0 match
- Looked up before it is created, so no duplicates
- Birth date and gender (M/F → male/female)
Visit
- One Encounter per patient per visit date, ambulatory class
- Rows from different sheets with the same date merge into one visit
- Every openEHR composition stores its Encounter id, so records join back to the visit
Handling rules
Six kinds of rule cover most answers
| Rule | What it does | Example | Uses |
|---|---|---|---|
| static_value | Value the template needs that the export does not have | language, setting, workflow state | 145 |
| value_mapping | Dutch answer → code | "Ziekte van Crohn" → SNOMED 34000006 | 114 |
| conditional | Fill only when another answer allows | TB test details only if a test was done | 52 |
| date_parse | Full and partial dates | "2019" → 2019-01-01 | 51 |
| numeric_extract | Number out of text | "172 cm" → 172 | 4 |
| float | Decimal values | weight, BMI | 4 |
Architecture
Who uses it, and what it talks to
Fig. 1 · System context
Fig. 2 · Parts inside the service
Discover our healthcare product engineering approach to building clinical platforms that run in production.
Upload path
What happens to one export
Fig. 3 · Upload sequence
Rows from every sheet are grouped by patient first, so one patient who answered four questionnaires becomes one FHIR Patient, not four.
Sync job
Every five minutes, saved forms catch up
Forms saved in the platform's form store do not always reach openEHR on the first try. Every 5 minutes a job picks up to 100 unsubmitted forms, oldest first, and sends each one. Success marks it done; failure marks it unsaved with the error kept, and the next run tries again.
60 runs = 5 hours. Orange = a retry.
Fig. D · Life of a saved form in the sync job
Testing
Tests on the rules, checks on the output
- 68 automated tests on value mapping, conditionals, dates and edge cases.
- A script scans generated records for leftover placeholder values.
- A script checks no two visits share one encounter.
- 179 debug runs kept between December 5, 2025 and January 2, 2026 for field-by-field review.
Deployment
Built on every merge, running on Kubernetes
Fig. 4 · Build and run
Kubernetes deployment as reported by the Nirmitee team.
How it evolved
Seven phases, from field mapping to a platform the client extends
The project ran from June to December 2025 in 206 commits, then moved into production. It did not arrive in its final shape. Each phase fixed what the last one taught us.
Fig. E · Where the work went
Map the fields
A Python mapper read each export column and printed where it went. First IBD fields mapped on June 23. Fast to start, but every change meant code.
Build real compositions
First openEHR compositions created. Intake and vaccination forms mapped. FHIR mapping added for patients and visits.
Move rules into YAML
Mapping moved from code to YAML configuration, one file per template. A generic FHIR mapper for Patient and Encounter, and a default location for visits.
Handle real workbooks
Real exports have many sheets. Uploads became background jobs with progress tracking. Practitioner and location data came from a provider instead of fixed values. OAuth2 token handling for the platform.
Let the server own identity
Patient and visit ids were left to the FHIR server, and every reference uses the server's patient id. Patient identifiers moved to UUIDs. Each running container now reports its exact code version.
Clean up for production
Old templates (forms 2 to 4) and old FHIR profiles removed. Invalid SNOMED and country codes fixed. Hardcoded test dates removed. Logging per run added so every upload can be traced.
Field review and hardening
90 commits. Output reviewed field by field against the templates. Complex forms moved to Python builders that skip empty sections. A builder registry replaced hardcoded template checks. Every composition now carries its visit id. The 5-minute sync job shipped on December 17.
Production
Running on Kubernetes. The client's team adds new templates and mappers on the same engine.
Fig. F · How the mapping approach changed
Decisions
What we decided, in order
| When | Type | Entry |
|---|---|---|
| Jun 23, 2025 | Started | First field mapping. |
| Aug 25, 2025 | Shipped | Rows from different sheets on the same date merge into one visit. |
| Oct 25, 2025 | Decision | Patient identifiers moved to UUIDs. |
| Nov 18, 2025 | Fixed | Invalid SNOMED and country codes corrected; duplicate patients removed. |
| Dec 5, 2025 | Decision | Complex forms move from rules files to Python builders that skip empty sections, because the openEHR validator rejects half-filled structures. |
| Dec 2025 | Decision | Hardcoded template checks across 15+ files replaced by a builder registry. |
| 3–Dec 11, 2025 | Shipped | Field-level fixes on Dutch clinical fields: perianal disease, peri-operative visits, TB vaccination, cause of death, religion. |
| 8–Dec 12, 2025 | Withdrawn | Three changes reverted and redone: care-facility links on encounters, extra submission logging, a vaccination fix. |
| Dec 10, 2025 | Shipped | Every composition carries its FHIR Encounter id. |
| Dec 17, 2025 | Shipped | Scheduled sync job, every 5 minutes. |
| 2026 | Live | In production on Kubernetes; client adds templates. |
Built to extend
The client adds new forms without waiting for us
| To add | What the team writes | What does not change |
|---|---|---|
| A simple form | One YAML mapping file for the new template | API, router, FHIR code |
| A complex form | One Python builder, registered with one decorator | Router and other builders |
| A new answer code | One line in a value-mapping list | Everything else |
Sources
Where the background facts come from
- Prevalence, incidence and mortality of IBD in the Netherlands, Journal of Crohn's and Colitis, 2025. pure.eur.nl/en/publications/prevalence-incidence-and-mortality-of-inflammatory-bowel-disease-
- Cohort profile: the Dutch IBD Biobank (Parelsnoer Institute), 2017. research.rug.nl
- Cohort profile: the IBD South Limburg cohort. pubmed.ncbi.nlm.nih.gov/28430884
- ECCO guidelines on opportunistic infections in IBD, 2025 update. ecco-ibd.eu
- ECCO guidance on TB screening before anti-TNF therapy. pmc.ncbi.nlm.nih.gov/articles/PMC4986392
- Amsterdam UMC chooses Epic, 2013. computable.nl
- EPD market survey of Dutch hospitals, 2024. dutchitchannel.nl
- Vanderbilt VCLIC: Epic research data tools. vumc.org/vclic
- UC Davis Health data services: Clarity and Caboodle access. health.ucdavis.edu/data/services.html
- Epic on FHIR: Bulk Data specifications. fhir.epic.com/Specifications?api=10171
- FHIR Bulk Data export at scale. capminds.com
- open.epic.com EHI table: CL_QANSWER
- open.epic.com EHI table: CL_QANSWER_QA
- open.epic.com EHI table: IP_FLWSHT_MEAS
Project facts come from the project code and the Nirmitee team, not from these sources.
Planning an Epic, FHIR or openEHR research data project?
We turn EHR exports into coded, research-ready openEHR and FHIR records. Explore our EHR and EMR integration services, or talk to our team to scope your project.


