vocal-stack
Open-Source npm Library for Production Voice AI Agents — Published on npm
About the Project
vocal-stack is a published npm utility library that solves the three most common production failures in voice AI pipelines: unnatural LLM output reaching TTS, silent pauses breaking conversation flow, and zero latency observability. It ships three composable modules — a Speech Sanitizer, Flow Controller, and Voice Auditor — that drop into any LLM-to-TTS pipeline with a single import.
The Challenge
Every team building a voice AI agent hits the same three walls: LLM output is full of markdown, URLs, and code blocks that sound broken when spoken aloud; processing pauses create awkward silence that kills the conversational feel; and there is no built-in way to measure time-to-first-token, p95/p99 latency, or throughput. These are not edge cases — they are the default failure modes of every production voice agent. Each team was solving them independently with fragile, one-off boilerplate.
The Solution
Instead of solving this once inside a single product, I packaged the solution as a provider-agnostic open-source library and published it to npm. The Speech Sanitizer strips markdown, URLs, and code from LLM streams before they reach TTS. The Flow Controller detects stream stalls at a configurable threshold (700ms default) and injects natural filler phrases to hold the conversation. The Voice Auditor wraps any stream and tracks TTFT, total duration, token counts, and p95/p99 percentiles. Each module is independent but composes via a clean async pipeline: LLM Stream → Sanitizer → Flow Control → Monitor → TTS. TypeScript-first, ESM + CJS, 90%+ test coverage, and ~2–3ms per-chunk overhead at 1000+ chunks/second.
Impact & Results
- Shipped a production-ready open-source library to npm (v1.0.2) — provider-agnostic, works with OpenAI, Claude, Gemini, ElevenLabs, Azure, and any streaming LLM/TTS pair.
- Eliminated hours of boilerplate for voice AI developers: text sanitization, stall detection, and latency monitoring in one composable package.
- Designed the architecture to be modular and tree-shakeable — developers import only what they need via subpath imports (vocal-stack/sanitizer, /flow, /monitor).
- Demonstrated the Forward Deployed instinct: identified a recurring infrastructure gap across voice AI teams, built the reusable solution, and shipped it publicly rather than keeping it internal.