Medical Reasoning-Service

In the world of automated decision-making, "Retrieval-Augmented Generation" (RAG) has become the standard answer for connecting LLMs to private data. The recipe is usually simple: chunk your text, embed it into vectors, and retrieve the top-k chunks based on cosine similarity.

But when I set out to build a Prior Authorization Reasoning Service: a system that determines if a patient meets the strict clinical criteria for a medical procedure: I found that standard vector RAG wasn't enough. Medical policies aren't just collections of semantic similarity; they are hierarchical rule sets where context, parent-child relationships, and specific exclusionary clauses matter more than general topic overlap.

This is the technical story of how Ibuilt a reasoning engine that moves beyond flat vectors to hierarchical tree search, auditable ReAct agents, and a custom Go database built from scratch.

The Problem: "Is this MRI covered?"

The core question our service answers is: "Is this patient referral ready to file according to the current policy?"

To answer this, a human (or machine) must:

```
Patient Packet ──► Policy PDF ──► Structured Notes
      │                │                 │
      ├─ demographics  ├─ tables         ├─ coverage clauses
      │                │                 │
      ▼                ▼                 ▼
   Facts DB     Section map        Rule references
      │                │                 │
      └───────────► Decision Rationale ◄─┘
```

Standard vector search fails here because it flattens the document. It might retrieve a paragraph about "exclusion criteria" without retrieving the parent header that says "For pediatric patients only," leading to hallucinations.

The Solution Architecture

Our system is composed of three specialized components, each handling a distinct part of the reasoning pipeline:

┌─────────────┐     ┌──────────────┐     ┌──────────────┐     ┌─────────────────────┐
│ Policy PDFs │ ──▶ │  PageIndex   │ ──▶ │  TreeStore   │ ──▶ │ ReAct Reasoning Loop │
└─────────────┘     │  (structure) │     │ (hierarchy)  │     │  (decide & cite)    │
                     └──────┬───────┘     └──────┬───────┘     └──────┬──────────────┘
                            │                    │                       │
                            ▼                    ▼                       ▼
                      JSON policy tree     Node-oriented reads     Approval decision

1. PageIndex: Structure First, Text Second

Instead of "chunking" text, Iuse PageIndex to convert visual PDFs into a hierarchical JSON tree.

• Input: A raw PDF.
• Process: OCR + Layout Analysis + LLM-based Tree Construction.
• Output: A structured tree where every node knows its parent, children, page range, and summary.

PDF Page
   │
   ├─► OCR Blocks
   │      │
   │      ├─► Layout Detector (coordinates, fonts)
   │      │        │
   │      │        └─► Section Classifier
   │      │                 │
   │      │                 ▼
   │      └──────────────► Node Builder ─────────┐
   │                                            │
   ▼                                            ▼
Tree Root ──► Policy 1.0 ──► 1.1 Indications ──► 1.1.4 Exceptions
                │                │                     │
                │                └─► Child summaries   └─► Tokens + cite span
                └─► Page anchors & parent pointers

This allows us to perform LLM Tree Search. Instead of searching for keywords, the model traverses the document tree:

"I need coverage criteria. I'll open node 1.0 (Policy). Now I see 1.1 (Indications) and 1.2 (Limitations). I'll expand 1.1..."

This mimics how a human reads a table of contents, preserving the vital context that vector databases destroy.

2. TreeStore: A Database Built for Trees

Irealized Ineeded a data store that treats "parent-child" relationships as first-class citizens. Relational DBs (SQL) are clunky for deep trees; Graph DBs are overkill.

So, Ibuilt TreeStore in Go. (Also because I wanted to understand the foundations of building a database)

• Core Tech: A custom B+Tree implementation with ACID transactions.
• Design: Optimized for GetNode, GetChildren, and GetSubtree operations.
• Performance: Capable of 100k+ reads/sec, ensuring that our reasoning agent can traverse complex policies with near-zero latency.
• Features: Native support for document versioning (vital for medical policies that change yearly) and metadata indexing.

It’s a single-binary, embeddable database that bridges the gap between a file system and a document store.

          [Root]
          /   \\
     [Index] [Leaf]
             / | \\
        Node refs  ► On-disk pages

3. The ReAct Controller: Auditable Reasoning

The heart of the Python service is the ReAct (Reason + Act) Controller. Imoved away from "one-shot" prompts to an agentic loop.

The Controller doesn't just guess; it investigates:

This generates a reasoning trace:a step-by-step log of how the decision was reached. In healthcare, this auditability is non-negotiable.

┌─────────┐    THINK    ┌────────────┐    ACT     ┌──────────────┐
│ Context │ ───────────▶│ Policy Tree│────────────▶│ Tool Calls   │
└─────────┘             └────┬───────┘            └─────┬────────┘
     ▲                       │                         │
     │                 OBSERVE│                         │RESULTS
     │                       ▼                         ▼
┌─────────┐◀────────────┌────────────┐       ┌────────────────────┐
│ Patient │   UPDATE     │ Evidence   │◀─────│ Trace + Confidence │
│  Facts  │              │  Buffer    │       └────────────────────┘
└─────────┘              └────────────┘                  │
                                                       DECIDE
                                                         │
                                                 Approve / Uncertain / Deny

Key Technical Decisions

Safety & "Uncertainty"

Iimplemented a strict safety gate. If the LLM's confidence score (a composite of retrieval relevance and evidence alignment) drops below 0.65, the system refuses to guess. It returns a status of UNCERTAIN, flagging the case for human review. Iprefer a "I don't know" over a hallucinated "Yes."

Confidence Meter
┌───────────────┬───────────────┐
│ 0.0 ────────► │ 0.65 threshold│ ────────► 1.0
└───────────────┴───────────────┘
        │                 │
        ▼                 ▼
  Human review        Automatic decision

Hybrid Retrieval

While Ifavor the Tree Search, Iacknowledge its limits. If a specific node is massive (e.g., a 20-page list of CPT codes), the agent can trigger a local SQLite FTS5 fallback. This performs a classic BM25 keyword search inside the selected node to pinpoint the exact line, combining the best of structural navigation and keyword precision.

Tree Traversal
     │
     ▼
Large Node? ──No────► Continue expanding children
     │
    Yes
     │
     ▼
 [SQLite FTS5]
     │
     ▼
Precise line hit ──► return span to controller

The "Shadow Mode" Rollout

Deploying LLM agents is scary. To mitigate risk, Ibuilt a Shadow Mode into our controller. The production traffic is handled by a heuristic (rule-based) controller, while the LLM agent runs in the background. Ilog and compare their decisions. Only when the LLM matches or beats the heuristic on our evaluation dataset do Iflip the switch.

        ┌─────────────────────────────┐
        │ Incoming Prior Auth Request │
        └──────────────┬──────────────┘
                       │
             ┌─────────▼─────────┐
             │ Heuristic Control │──► Production decision
             └─────────┬─────────┘
                       │ mirroring inputs
             ┌─────────▼─────────┐
             │  LLM Shadow Loop  │──► Logged reasoning
             └─────────┬─────────┘
                       │
                Offline comparison
                       │
                 Trigger cutover

Conclusion

The reasoning-service project demonstrates that for complex, high-stakes domains, structure is all you need. By respecting the document's hierarchy with PageIndex and TreeStore, and forcing the LLM to show its work via ReAct, we've built a system that doesn't just retrieve information: it actually understands it.

Check out the full code on GitHub