ReadmitRisk

Overview

A full-stack care management platform combining ML-driven readmission risk scoring with retrieval-augmented generation over 10,000+ chunks from a real clinical corpus (MTSamples). A custom MCP server (9 tools) exposes both XGBoost predictions and semantic search to Claude through an agentic conversational interface. Features inline citation rendering with expandable source chunks, a multi-tool agentic loop, and a retrieval evaluation harness with documented baseline metrics (P@5, Recall@5, MRR).

Key Features

• Multi-source ML pipeline processing 280K+ patient records from MIMIC-IV (ICU), UCI Diabetes, and CMS HRRP datasets
• Gradient Boosting classifier with SMOTE oversampling to handle severe class imbalance (8.8% → 50% positive class)
• 61-feature clinical model with demographic normalization and comprehensive feature importance analysis
• RAG over 10,000+ chunks from the MTSamples clinical corpus, indexed in Postgres with pgvector HNSW (cosine) and embedded with Voyage AI voyage-3-lite (512-dim)
• Inline citation rendering with expandable source chunks — a global citation counter spans multiple RAG tool calls per turn, rendered as collapsible <details> components in the chat widget
• Multi-tool agentic loop (up to 10 turns) that lets Claude interleave ML predictions with semantic search across 2 RAG tools (search_clinical_notes, find_similar_cases) and 7 structured-data tools
• Retrieval evaluation harness with seeded question sets (factual, comparative, case_similarity, edge_case) computing P@5, Recall@5, MRR, irrelevant-violation counts, and latency per run
• Remote MCP server deployed on Railway exposing 9 tools (patient risk lookup, live ML predictions, hospital metrics, dataset comparison, feature importance, risk distribution, model info, plus the two RAG tools) that any AI assistant can call via SSE transport
• Interactive care management dashboard with risk stratification tiers (60%, 70%, 80% thresholds), cost estimation, ROC-AUC and precision-recall visualizations, and a chat widget connecting React → FastAPI → Claude → MCP → ML models and vector store

System Architecture

1. Chat widget embedded in the ReadmitRisk frontend (Next.js on Vercel)
2. FastAPI proxy backend deployed on Railway that runs the agentic loop, handles Anthropic API calls, and routes tool execution through MCP
3. MCP server deployed on Railway exposing 9 tools — 7 structured-data tools (patient risk, live predictions, dataset comparison, feature importance, risk distribution, hospital metrics, model info) and 2 RAG tools (search_clinical_notes, find_similar_cases)
4. Postgres with pgvector HNSW index storing 10K+ MTSamples chunks; queries are embedded with Voyage AI voyage-3-lite and ranked by cosine similarity with Python-side deduplication
5. Claude processes natural language queries, selects MCP tools across multiple agentic turns, and emits inline [N] citations tied back to expandable source chunks
6. Responses stream back via SSE for a real-time conversational experience

Challenges & Solutions

The project tackled significant ML and AI engineering challenges including handling severe class imbalance in readmission data using SMOTE oversampling, extracting 211K ICU admissions from Google BigQuery with PhysioNet credentials, engineering 61 clinical features from raw MIMIC-IV data, building a measurement-driven RAG stack on Postgres + pgvector with Voyage AI embeddings, and orchestrating a multi-turn agentic loop where Claude can interleave structured ML tools with semantic search and emit inline citations. The eval harness drove a concrete improvement story: P@5 went from 0.080 → 0.290 (3.6×), Recall@5 from 0.200 → 0.492 (2.5×), and MRR from 0.141 → 0.613 (4.3×) after the eval surfaced a chunk-deduplication bug (MTSamples ingestion produced one row per medical_specialty, and search_chunks wasn't deduplicating) — fixed by over-fetching and deduping in retrieval.py.