When Regex Meets Fuzzy Matching: Building a Medical Document Classifier That Knows Who's Who

Medical offices still run on faxes. Every day, insurance companies, labs, and specialists send multi-patient PDF batches—100 pages, 15 patients, all jumbled together. Staff spend hours manually sorting pages by squinting at headers, cross-referencing MRNs, and hoping they don't mix up John Smith #1 with John Smith #2.

Inspired by Tennr's approach to medical document automation, I built a document processing pipeline that ingests multi-patient PDFs, extracts identifiers from each page, clusters them into patient profiles, and outputs per-patient PDFs with confidence scores. The core insight: patient identity isn't a binary classification—it's a weighted similarity problem across multiple identifier types.

Here's what I learned building it.

The Problem: One PDF, Many Patients

A typical incoming fax looks like this:

• Pages 1-3: Lab results for Mary Johnson, MRN 123456
• Page 4: Prescription for Robert Garcia, DOB 05/15/1980
• Pages 5-7: Referral for Mary Johnson (same patient, no MRN this time)
• Pages 8-12: Mixed records, some with phone numbers only

The challenges stack up quickly. Some pages have MRNs, others only have name and DOB. Scanned documents introduce OCR noise—typos, artifacts, and formatting variations. And then there's ambiguity: when you see "M. Johnson" on page 6, is that Mary Johnson or someone else entirely?

Manual sorting fails at scale. Rule-based automation using exact string matching fails on real-world data. I needed something smarter—a system that could reason about degrees of similarity across multiple identifier types and make probabilistic judgments about page ownership.

The Architecture: A Six-Stage Pipeline

┌─────────────────────────────────────────────────────────────────────────────┐
│                          DOCUMENT INGESTION                                  │
│  ┌──────────────┐                                                           │
│  │ Multi-Patient│  100+ pages, 10+ patients                                 │
│  │ PDF Input    │  mixed labs, prescriptions, referrals                     │
│  └──────┬───────┘                                                           │
└─────────┼───────────────────────────────────────────────────────────────────┘
          │
          ▼
┌─────────────────────────────────────────────────────────────────────────────┐
│                          PREPROCESSING LAYER                                 │
│                                                                              │
│  ┌────────────────────┐      ┌────────────────────┐                         │
│  │  Page Extractor    │      │  OCR Processor     │                         │
│  │  ─────────────     │ ───▶ │  ─────────────     │                         │
│  │  Poppler/pdf2image │      │  Tesseract (default)│                        │
│  │  DPI: 200          │      │  OlmOCR (optional)  │                        │
│  └────────────────────┘      └─────────┬──────────┘                         │
└─────────────────────────────────────────┼───────────────────────────────────┘
                                          │ raw text + word positions
                                          ▼
┌─────────────────────────────────────────────────────────────────────────────┐
│                          ENTITY EXTRACTION                                   │
│                                                                              │
│  ┌────────────────────────────────────────────────────────────────────┐     │
│  │  Entity Extractor (Regex-based)                                     │     │
│  │  ┌──────────┐  ┌──────────┐  ┌──────────┐  ┌──────────┐            │     │
│  │  │  Name    │  │   MRN    │  │   DOB    │  │  Phone   │            │     │
│  │  │ "Patient:│  │ "MRN:    │  │ "DOB:    │  │ "(555)   │            │     │
│  │  │  John.." │  │  123456" │  │  1/1/80" │  │  123-.." │            │     │
│  │  └──────────┘  └──────────┘  └──────────┘  └──────────┘            │     │
│  └────────────────────────────────────────────────────────────────────┘     │
└─────────────────────────────────────────┬───────────────────────────────────┘
                                          │ identifiers + confidence
                                          ▼
┌─────────────────────────────────────────────────────────────────────────────┐
│                          PATIENT CLUSTERING                                  │
│                                                                              │
│  ┌──────────────────────┐         ┌──────────────────────┐                  │
│  │  Fuzzy Matcher       │         │  Entity Linker       │                  │
│  │  ─────────────       │  ────▶  │  ─────────────       │                  │
│  │  RapidFuzz scoring   │         │  Anchor-based        │                  │
│  │  Per-entity strategy │         │  patient clustering  │                  │
│  └──────────────────────┘         └──────────┬───────────┘                  │
└─────────────────────────────────────────────────┼───────────────────────────┘
                                          │ patient profiles
                                          ▼
┌─────────────────────────────────────────────────────────────────────────────┐
│                          PAGE ASSIGNMENT                                     │
│                                                                              │
│  ┌────────────────────────────────────────────────────────────────────┐     │
│  │  Page Assigner (Weighted Scoring)                                   │     │
│  │                                                                     │     │
│  │  Score = 0.4×name + 0.3×mrn + 0.2×dob + 0.1×phone                  │     │
│  │                                                                     │     │
│  │  ┌─────────────┐  ┌─────────────┐  ┌─────────────┐                 │     │
│  │  │ Assigned    │  │ Ambiguous   │  │ Unassigned  │                 │     │
│  │  │ (conf ≥60%) │  │ (close race)│  │ (conf <60%) │                 │     │
│  │  └─────────────┘  └─────────────┘  └─────────────┘                 │     │
│  └────────────────────────────────────────────────────────────────────┘     │
└─────────────────────────────────────────┬───────────────────────────────────┘
                                          │
                                          ▼
┌─────────────────────────────────────────────────────────────────────────────┐
│                          OUTPUT GENERATION                                   │
│                                                                              │
│  ┌────────────────────┐      ┌────────────────────┐                         │
│  │  Document Splitter │      │  Outputs:          │                         │
│  │  ─────────────     │ ───▶ │  • patient_001.pdf │                         │
│  │  PyPDF2            │      │  • patient_002.pdf │                         │
│  │                    │      │  • metadata.json   │                         │
│  │                    │      │  • unassigned.pdf  │                         │
│  └────────────────────┘      └────────────────────┘                         │
└─────────────────────────────────────────────────────────────────────────────┘

The pipeline is modular by design. The Page Extractor renders PDF pages to images at configurable DPI using Poppler. The OCR Processor runs text extraction with pluggable backends—Tesseract by default, but swappable for OlmOCR or custom handlers.

The Entity Extractor runs regex patterns over OCR output to detect four identifier types: patient names, Medical Record Numbers (MRNs), dates of birth, and phone numbers. Each extracted identifier carries a confidence score based on pattern match quality.

The Fuzzy Matcher and Entity Linker work together to cluster page-level identifiers into patient profiles. Fuzzy matching determines whether "Mary Johnson" on page 3 is the same person as "MARY JOHNSON" on page 7, while the linker builds coherent patient clusters using anchor-based logic.

The Page Assigner scores each page against all known patient clusters using weighted entity matching, flagging ambiguous cases for human review. Finally, the Document Splitter generates per-patient PDFs with accompanying metadata.

Each stage is independently testable. The orchestrator wires them together but delegates all logic to the components.

Lesson 1: Not All Identifiers Are Equal

My first approach weighted all identifiers equally. A name match counted the same as an MRN match. That was wrong.

MRNs are unique identifiers—if two pages share an MRN, they're almost certainly the same patient. A 95% MRN match is nearly definitive evidence. Names, in contrast, are not unique at all—there might be three Robert Garcias in the practice. A perfect name match by itself proves little.

DOBs fall in between. They're unique per patient but commonly appear across multiple pages for the same person. Phone numbers are similar—moderately unique, but patients might share family phone numbers or update their contact information.

The fix was weighted scoring where each identifier type contributes proportionally to its reliability:

weights = {
    "name": 0.4,   # High coverage, low uniqueness
    "mrn": 0.3,    # Low coverage, high uniqueness
    "dob": 0.2,    # Moderate on both
    "phone": 0.1,  # Least reliable
}

for identifier in page.identifiers:
    match_score = fuzzy_match(identifier, patient)
    total_score += match_score * weights[identifier.kind]

Why is name weight highest despite being least unique? Because names appear on almost every page, while MRNs don't. A name match alone means little, but when you combine name + DOB + phone, you get strong signal even when MRN is absent. The weights encode how much incremental confidence each identifier type provides.

The lesson: Domain knowledge should inform feature importance. Don't assume all signals are equal—weight them based on reliability and frequency of occurrence.

Lesson 2: Fuzzy Matching Requires Per-Entity Strategy

Exact string matching fails immediately on real OCR output. "MARY JOHNSON" might come through as "MARY J0HNSON" (zero instead of O) or "Mary Johnson" (extra space) or "Johnson, Mary" (reordered). You need fuzzy matching, but not all fuzzy matching is the same.

I used RapidFuzz, but with different comparison strategies per entity type:

from rapidfuzz import fuzz

def score_name(a: str, b: str) -> float:
    # Token sort handles "John Smith" vs "Smith, John"
    return fuzz.token_sort_ratio(normalize(a), normalize(b)) / 100.0

def score_dob(a: str, b: str) -> float:
    # Dates must match exactly after normalization
    return 1.0 if normalize_date(a) == normalize_date(b) else 0.0

def score_phone(a: str, b: str) -> float:
    # Partial matching handles missing area codes
    return fuzz.partial_ratio(digits_only(a), digits_only(b)) / 100.0

For names, token sort ratio handles word reordering, so "John Smith" matches "Smith, John" correctly. MRNs get stricter treatment—after stripping non-alphanumeric characters, I expect near-exact matches. DOBs require exact match after normalization, handling format variations like "5/15/1980", "05-15-80", and "5.15.1980". Phone numbers use partial matching to catch missing area codes.

The lesson: Fuzzy matching isn't one-size-fits-all. Each entity type has different semantics—names tolerate reordering, dates require exact match after normalization, phone numbers can be partial.

Lesson 3: Clustering Is Position-Aware

Here's a subtle problem: what happens when a page has multiple patients' information?

Consider a referral letter. The referring doctor mentions their patient (the subject of the referral), but also references a patient at the receiving clinic for context. Naive clustering treats all identifiers on a page as belonging to one patient—which would incorrectly merge two distinct people.

The solution is anchor-based clustering. MRNs are designated as "anchors"—high-confidence identifiers that seed patient clusters. Other identifiers attach to the nearest preceding anchor on the same page:

for idx, identifier in enumerate(page.identifiers):
    if identifier.kind == "mrn":
        # MRNs are anchors—create or match to a cluster
        cluster = find_or_create_cluster(identifier)
        anchor_positions.append((idx, cluster))
    else:
        # Supportive identifiers attach to nearest anchor
        nearest_anchor = find_nearest_anchor(idx, anchor_positions)
        attach_to_cluster(identifier, nearest_anchor)

This models how medical documents are typically structured: patient identifiers appear in blocks, with the MRN or header coming first. So when a page has MRN 123456 followed by "Mary Johnson", then later MRN 789012 followed by "Robert Garcia"—the two names attach to their respective anchors rather than merging into one confused patient.

The lesson: Document structure matters. When clustering entities, consider their position on the page, not just their content.

Lesson 4: Ambiguity Should Be Flagged, Not Hidden

Early versions of the assigner tried to be "helpful" by assigning every page to some patient, even when confidence was low. That's dangerous in healthcare—a misassigned prescription is worse than an unassigned one. The wrong routing could delay critical treatment or violate HIPAA by exposing records to the wrong patient's file.

I added explicit ambiguity detection with two layers:

scores = [(patient, score_page(page, patient)) for patient in patients]
scores.sort(key=lambda x: x[1], reverse=True)

top_patient, top_score = scores[0]
manual_review = False

# Layer 1: Low confidence → unassigned
if top_score < MIN_CONFIDENCE:  # default 0.6
    return PageAssignment(patient_id=None, manual_review=True)

# Layer 2: Close race → flag for review
if len(scores) > 1 and (top_score - scores[1][1]) < AMBIGUITY_MARGIN:
    manual_review = True

return PageAssignment(patient_id=top_patient, manual_review=manual_review)

The output now has three explicit categories:

• Assigned with high confidence: Automatically routed, no human intervention needed
• Assigned with manual review: Routed but flagged for human verification
• Unassigned: Low confidence, requires human classification

The CLI and API both surface these metrics prominently. You can track your unassigned rate over time and tune thresholds accordingly.

The lesson: In high-stakes domains, "I don't know" is a valid output. Systems that hide uncertainty are systems that create liability.

Lesson 5: Make OCR Pluggable from Day One

Tesseract is the default OCR engine, and it's good enough for most scanned documents. But some PDFs have handwriting, complex table layouts, or degraded quality (fax-of-a-fax quality) that Tesseract struggles with.

Rather than hardcoding Tesseract, I built backend pluggability into the OCR processor. The configuration accepts three modes: tesseract (the default), olmocr (for integration with newer vision-language models), or a custom module path like my_module:ocr_function.

The custom mode uses Python's import machinery to load any callable that conforms to the expected interface—accepting an image and returning extracted text with optional bounding boxes. This means you can experiment with cloud OCR APIs, newer open-source models, or domain-specific extractors without touching the core pipeline code.

Switching backends is a single environment variable change. The pipeline doesn't care where the text comes from—it just needs text.

The lesson: When building pipelines, make the most likely-to-change components pluggable. You don't need to implement alternatives on day one—just ensure swapping is painless when you do need them.

Lesson 6: The Config Decides Everything

Medical workflows vary enormously. Some clinics need 95% confidence before auto-routing; others accept 60%. Some receive clean PDFs from modern systems; others get fourth-generation photocopy faxes. A one-size-fits-all threshold would fail for at least half of use cases.

I exposed every tunable as an environment variable. Entity matching thresholds control how similar two values must be to match: 85% for names, 95% for MRNs, exact match for DOBs, 90% for phones. Page assignment has its own knobs: minimum confidence for assignment, ambiguity margin for flagging, and whether to allow unassigned pages at all.

The entity weights—how much each identifier type contributes to the overall score—are also configurable. A clinic that always has MRNs on every page might weight them higher; one that never uses MRNs might set that weight to zero.

There's a --show-settings CLI flag that dumps the current configuration before processing. This makes debugging configuration issues trivial and ensures operators always know what thresholds are active.

The lesson: Hardcoded thresholds are tech debt. Every magic number should have a config knob, even if you only expose it to power users.

What I'd Do Differently

1. Add layout-aware extraction: Medical forms have predictable structure—headers, patient info blocks, tables, signature lines. A layout model (like LayoutLM or DONUT) could extract identifiers more reliably than regex over raw OCR text, especially for structured forms where position matters.

2. Use contextual embeddings for entity disambiguation: When two "Robert Garcia" entries exist in the system, semantic embeddings of the surrounding context could help determine if they're the same person. "Robert Garcia, cardiology patient, MRN 123456" is different from "Robert Garcia, referred for dermatology" even if the name matches.

3. Implement active learning: Every page flagged for manual review is a labeling opportunity. Human corrections could feed back into improved extraction patterns, clustering thresholds, and even regex refinements over time.

4. Add document type classification: Knowing that a page is a "lab result" vs "prescription" vs "referral" would improve entity expectations and weighting. Lab results almost always have MRNs; referral letters often mention multiple patients; prescriptions might only have name and DOB.

Key Takeaways

This project is open source and designed for high-volume medical office workflows. The test suite runs with mocked OCR by default, with real Tesseract available via environment flag.