Building a FHIR-Native RAG Pipeline: Embedding Clinical Data for Agent Retrieval

Part of our complete guide to AI agents across the care pathway.

Hypothetical example: a document retriever returns only one part of a medication list split across PDF chunks. Relevant information may exist in structured FHIR resources but remain absent from the retrieved context. This illustrates a failure mode to test; it is not a measured client incident.

If you are planning this for production, our healthcare data engineering team can build it with you.

This is the fundamental problem with healthcare RAG today: teams embed PDFs and clinical notes while ignoring the structured FHIR data that already contains the clinical semantics they need. A Condition resource with a SNOMED CT code of 73211009 tells you the patient has diabetes mellitus. A chunk of text that mentions "blood sugar has been running high" might or might not mean the same thing. FHIR resources carry clinical meaning in their structure, not just their text.

This guide shows you how to build a RAG pipeline that embeds FHIR resources directly, preserves clinical semantics through the embedding process, and evaluates retrieval quality with metrics that matter for clinical applications.

For a production example beyond clinical records, see how Nirmitee built a multi-source data integration platform with governed AI, where 54 live integrations form the context layer behind two AI assistants that cite their sources.

Why Standard RAG Pipelines Fail on Clinical Data

Standard RAG pipelines were designed for documents: PDFs, web pages, knowledge base articles. They assume text is the primary carrier of meaning. In healthcare, structured data carries as much clinical meaning as narrative text, often more.

Problem 1: Naive Chunking Destroys Clinical Context

A FHIR Bundle containing a patient's problem list might include 15 Condition resources, each with coded diagnoses, onset dates, clinical status, and verification status. A standard text chunker splitting at 512 tokens will cut this list at arbitrary points, separating a condition from its onset date or splitting a medication from its dosage instructions. Our FHIR Condition resource guide explains how clinicalStatus and verificationStatus work together, which is exactly the context a chunker must not drop.

Compare document-based and resource-aware retrieval on representative questions. Record missed evidence and context loss; no universal recall improvement is assumed here.

Problem 2: General Embeddings Miss Clinical Semantics

When you embed the text "Patient has type 2 diabetes mellitus" using a general-purpose embedding model like text-embedding-3-small, the vector captures the semantic meaning of the English sentence. But it does not capture the clinical relationship between diabetes and its associated conditions (diabetic nephropathy, retinopathy), its medication implications (metformin as first-line), or its SNOMED CT hierarchy (73211009 is a descendant of Endocrine disorder).

Clinical embedding models trained on medical literature and EHR data encode these relationships in vector space, making retrieval more clinically relevant rather than just textually similar.

Problem 3: Retrieval Evaluation Ignores Clinical Relevance

Standard RAG evaluation measures like Recall@K and NDCG tell you whether the retriever found textually relevant chunks. They do not tell you whether the retrieved chunks contain the clinically correct information for the query. A chunk mentioning "elevated glucose" is textually relevant to a diabetes query but clinically incomplete without the HbA1c value, diagnosis date, and current treatment plan.

Step 1: FHIR Resource Extraction and Normalization

The first step is extracting FHIR resources and converting them into embeddable text that preserves clinical semantics. This is not a simple JSON-to-text serialization. Each resource type needs a specific extraction strategy. Clinical notes need a different path: our guide to fetching clinical notes via FHIR DocumentReference covers retrieving the attachment and turning it into LLM-ready text.

The FHIR-to-Text Pipeline

import json
from typing import List, Dict, Optional
from dataclasses import dataclass

@dataclass
class ClinicalChunk:
    text: str
    resource_type: str
    fhir_resource_id: str
    patient_id: str
    snomed_codes: List[str]
    loinc_codes: List[str]
    effective_date: Optional[str]
    metadata: Dict

def extract_condition(resource: dict) -> ClinicalChunk:
    """Extract embeddable text from a FHIR Condition resource."""
    code = resource.get("code", {})
    display = code.get("coding", [{}])[0].get("display", "Unknown condition")
    snomed = code.get("coding", [{}])[0].get("code", "")

    clinical_status = resource.get("clinicalStatus", {}).get(
        "coding", [{}])[0].get("code", "unknown")
    verification = resource.get("verificationStatus", {}).get(
        "coding", [{}])[0].get("code", "unknown")
    onset = resource.get("onsetDateTime", "unknown date")
    severity = resource.get("severity", {}).get(
        "coding", [{}])[0].get("display", "")

    # Structured clinical text — not just the display name
    text_parts = [
        f"Diagnosis: {display}",
        f"Clinical status: {clinical_status}",
        f"Verification: {verification}",
        f"Onset: {onset}",
    ]
    if severity:
        text_parts.append(f"Severity: {severity}")

    return ClinicalChunk(
        text=" | ".join(text_parts),
        resource_type="Condition",
        fhir_resource_id=resource["id"],
        patient_id=resource.get("subject", {}).get("reference", "").split("/")[-1],
        snomed_codes=[snomed] if snomed else [],
        loinc_codes=[],
        effective_date=onset if onset != "unknown date" else None,
        metadata={"category": resource.get("category", [{}])[0].get(
            "coding", [{}])[0].get("code", "")}
    )

Step 2: Clinical Chunking Strategies

Once you have extracted text from individual FHIR resources, the next decision is how to group them into chunks for embedding. There are three strategies, each with different tradeoffs.

Strategy 1: Resource-Level Chunking (Recommended Starting Point)

One FHIR resource equals one chunk. A single Condition resource becomes one embedding, a single Observation becomes one embedding. This is the simplest approach and preserves resource integrity perfectly.

Pros: Clean boundaries, easy to trace back to source, simple to update when resources change.
Cons: Misses cross-resource relationships. A query about "diabetes management" would need to retrieve the Condition, related MedicationRequest resources, and recent Observation values (HbA1c, glucose) separately.

Strategy 2: Encounter-Level Chunking

Group all resources from a single encounter into one chunk. A clinic visit produces one chunk containing the conditions addressed, observations recorded, medications prescribed, and procedures performed during that visit.

Pros: Captures the clinical context of a single visit, which is often the relevant unit for clinical questions.
Cons: Chunks can be very large for complex encounters (ICU stays, surgical episodes). May exceed embedding model context limits.

Strategy 3: Clinical Concept Chunking (Highest Retrieval Quality)

Group resources by clinical concept using SNOMED CT hierarchies. All resources related to a patient's diabetes (the Condition, HbA1c Observation values, metformin MedicationRequest, diabetic retinopathy screening Procedure) become one chunk.

Pros: Highest retrieval relevance for clinical questions. A single query about diabetes retrieves the complete clinical picture.
Cons: Requires a clinical ontology mapper to group resources by concept. Harder to implement but significantly better retrieval quality.

Step 3: Choosing Your Embedding Model

Evaluate candidate embedding models on representative terminology, questions and data from the intended workflow. Compare retrieval quality, latency and operating cost; no universal performance advantage is assumed.

Our Recommendation

For most healthcare RAG pipelines, start with PubMedBERT or BioLORD-2023. They offer the best balance of clinical accuracy, dimension size (768, manageable for pgvector), and inference cost. Move to larger models like GTE-Qwen2 only if you need maximum retrieval accuracy and can afford the compute and storage costs of 3584-dimensional vectors.

Step 4: Vector Store Setup with pgvector

For healthcare RAG, pgvector is the pragmatic choice. You already have PostgreSQL for your FHIR data store and application data. Adding vector search to the same database simplifies HIPAA compliance (one database to audit, one set of access controls) and enables hybrid queries that combine vector similarity with structured FHIR metadata filters.

Schema Design

-- Enable pgvector extension
CREATE EXTENSION IF NOT EXISTS vector;

-- Clinical embeddings table
CREATE TABLE clinical_embeddings (
    id SERIAL PRIMARY KEY,
    patient_id UUID NOT NULL REFERENCES patients(id),
    resource_type VARCHAR(50) NOT NULL,
    fhir_resource_id VARCHAR(255) NOT NULL,
    embedding vector(768) NOT NULL,
    clinical_text TEXT NOT NULL,
    snomed_codes TEXT[] DEFAULT '{}',
    loinc_codes TEXT[] DEFAULT '{}',
    effective_date TIMESTAMP,
    embedding_model VARCHAR(100) NOT NULL,
    chunk_strategy VARCHAR(50) NOT NULL,
    created_at TIMESTAMP DEFAULT NOW(),
    UNIQUE(fhir_resource_id, embedding_model)
);

-- HNSW index for approximate nearest neighbor search
CREATE INDEX idx_embeddings_hnsw
    ON clinical_embeddings
    USING hnsw (embedding vector_cosine_ops)
    WITH (m = 16, ef_construction = 128);

-- Filtered search indexes
CREATE INDEX idx_embeddings_patient ON clinical_embeddings(patient_id);
CREATE INDEX idx_embeddings_resource ON clinical_embeddings(resource_type);
CREATE INDEX idx_embeddings_snomed ON clinical_embeddings USING gin(snomed_codes);

Hybrid Retrieval Query

The power of pgvector in a FHIR context is combining vector similarity with structured metadata filters in a single query:

-- Find relevant clinical context for a medication review agent
SELECT
    ce.clinical_text,
    ce.resource_type,
    ce.fhir_resource_id,
    ce.snomed_codes,
    1 - (ce.embedding <=> query_embedding) AS similarity
FROM clinical_embeddings ce
WHERE
    ce.patient_id = $1
    AND ce.resource_type IN ('Condition', 'MedicationRequest', 'AllergyIntolerance')
    AND ce.effective_date > NOW() - INTERVAL '2 years'
ORDER BY ce.embedding <=> query_embedding
LIMIT 20;

This query retrieves the 20 most semantically similar clinical chunks, filtered to only active conditions, medications, and allergies from the last 2 years. A pure vector store cannot do this without post-retrieval filtering, which wastes retrieval budget on irrelevant results.

Step 5: Clinical Retrieval Evaluation

Standard RAG evaluation is not sufficient for clinical applications. You need metrics that capture clinical relevance, not just textual similarity. Based on systematic reviews of RAG in biomedical applications, we use a three-tier evaluation framework.

Tier 1: Standard Retrieval Metrics

Tier 2: Clinical Relevance Metrics

Tier 3: Safety Metrics

Pinecone Alternative: When to Choose Managed Vector Search

Choose a vector store after testing representative data, metadata filtering, latency and operational requirements. Confirm vendor terms, deployment options and any required agreements directly; do not assume a particular retrieval time or compliance status from the product name.

Production Considerations

Embedding Updates When FHIR Resources Change

FHIR resources are not static. A Condition can change from active to resolved. A MedicationRequest can be cancelled. Your embedding pipeline needs to detect these changes and update vectors accordingly. Use Change Data Capture (CDC) on your FHIR server to trigger re-embedding when resources are created or updated.

PHI in Vector Embeddings

Embeddings derived from PHI are themselves PHI under HIPAA. Your vector store needs the same access controls, encryption at rest, and audit logging as your FHIR server. This is another reason pgvector is attractive: it inherits PostgreSQL's existing security infrastructure.

Model Versioning

When you upgrade your embedding model, you need to re-embed your entire corpus. The embedding_model column in our schema supports running both old and new embeddings simultaneously, with a gradual cutover after validating retrieval quality. Track this in your model registry alongside your inference models.

Getting Started

Begin with resource-level chunking using PubMedBERT and pgvector. This combination ships fast, integrates with your existing PostgreSQL infrastructure, and gives you a solid baseline to iterate from. Add clinical concept chunking and more sophisticated embedding models as your retrieval evaluation metrics tell you where the pipeline is falling short.

The key insight is that FHIR resources already carry clinical semantics in their structure. Your RAG pipeline should embed that structure, not flatten it into unstructured text and hope the embedding model reconstructs the meaning.

At Nirmitee, we build FHIR-native data pipelines for healthcare AI systems. From FHIR agent integration to production RAG pipelines with clinical evaluation, our team has the healthcare data engineering expertise to get your clinical AI from prototype to production.

Related reading

For more insights, explore our guides on AI agents in healthcare and FHIR in Modern Healthcare.

You may also find value in EHR Software Development Guide and Building a Healthtech App.