AquaOracle
Terminal-Based Local RAG Platform for Water Safety & Regulatory Compliance
About the Project
AquaOracle is a fully local, terminal-based retrieval-augmented generation platform built for water safety and regulatory compliance. It lets facilities managers and EHS professionals query CDC guidelines, OSHA standards, and WHO water quality requirements through a CLI — getting cited, factual answers from their own document corpus with zero data leaving the machine. Built on a four-stage retrieval pipeline (vector search → cross-encoder re-ranking → graph expansion → context assembly) feeding a local Llama 3.1 instance via Ollama.
The Challenge
Facilities responsible for Legionella prevention and water system compliance sit on a pile of regulatory PDFs — CDC protocols, OSHA monitoring requirements, WHO standards — that are scattered, version-fragmented, and full of tables and structured data that generic tools mishandle. The real constraint: these environments often cannot use cloud AI due to data sovereignty, HIPAA, or internal security policy. Teams were spending hours manually cross-referencing documents to answer questions like "What are the CDC control measures for Legionella in cooling towers?" — in a domain where wrong answers are a life-safety failure.
The Solution
Built a 100% local RAG system: no external API calls, no cloud storage, no telemetry. The ingestion pipeline uses PyMuPDF for layout-aware PDF parsing, pdfplumber for table structure preservation, and a hybrid semantic + sliding-window chunker (1024-token max, 20% overlap) that keeps document structure intact. Embeddings (768-dim) are generated locally via Ollama and stored in Qdrant with HNSW indexing. Queries run through a four-stage retrieval pipeline: vector search retrieves 50–100 candidates, a cross-encoder re-ranker (ms-marco-MiniLM-L-6-v2) scores each pair for true relevance, graph expansion (NetworkX, k=1–2 hops) surfaces related sections the vector search missed, and context assembly deduplicates and trims to an 8192-token window before hitting Llama 3.1:8b at temperature 0.1 for grounded, cited responses. The whole system runs from the terminal — designed for practitioners, not demos.
Impact & Results
- Ingested 18 regulatory PDFs (~56 MB) covering CDC, OSHA, and WHO water safety standards — end-to-end pipeline runs one-time in ~60 minutes, then queries resolve in ~3 seconds p95.
- Four-stage retrieval pipeline (vector → re-rank → graph → assembly) achieves materially higher answer precision than naive RAG: cross-encoder re-ranking alone eliminates false positives that embedding similarity misses in dense regulatory text.
- Zero data egress: 100% local inference with Ollama + Qdrant + SQLite — built for environments where cloud AI is blocked by compliance policy, exactly the constraint most RAG demos ignore.
- Designed to the actual domain constraints: table extraction preserves regulatory tables that naive PDF-to-text destroys; citations in every response meet the auditability bar required in compliance contexts.
- Demonstrated the Forward Deployed pattern: identified the real constraint (not "AI for water" but "local AI for a data-sovereignty-constrained compliance use case"), made the hard architectural call (fully local over easier cloud), and built to production-grade retrieval quality rather than demo quality.