Deep Dives

VoiceStudio: The Free, Local ElevenLabs Alternative — 646 Languages, 27 Engines, Zero Cloud

VoiceStudio is the open-source, fully-local ElevenLabs alternative: voice cloning from a 3-second sample, voice design from a text description, video dubbing with per-speaker preservation, dictation, and EPUB/PDF-to-audiobook — across a 646-language catalog, with no account, subscription, or usage meter. The trick is architectural: it's not one model but a workbench over 16 TTS and 11 ASR engines, where a new open voice model becomes a menu item rather than a migration. It runs on Apple Silicon, CUDA, ROCm, or CPU, doubles as a local audio server with an OpenAI-compatible API and an MCP server for agents, and ships AGPL-3.0. The honest context: hardware matters, it's an active beta, and voice cloning is dual-use — local doesn't change the law or the ethics.

OpenAI's New Voice Model Doesn't Stop Talking to Think

OpenAI's GPT-Live-1 running on LiveKit introduces full-duplex voice: the model listens and speaks at the same time over one always-open audio stream, deciding on its own when to reply. Reasoning and tool calls are handed to a separate backend model — GPT-6 Astra or Luna, or your own harness — so the voice layer never has to stop talking to think. This short walkthrough covers the airline-rebooking demo and the drop-in code (GPTLiveModel + AgentSession).

I Fully Designed an AI Voice Studio in Rust — Why Won't I Build It?

Aarambh Dev Hub follows its pure-Rust LLM (Aarambh Studio) with Aarambh Voice Studio: a complete AI audio studio — speech, music, singing, and full-song composition — architected end-to-end in Rust on Candle, with zero Python, zero PyTorch, and zero external APIs. This 46-minute blueprint walks the whole system: a 12.5 Hz neural audio codec, three generative engines on a shared transformer core, GRPO/DPO alignment, online self-learning, and safety enforced in the Rust type system — open-sourced as a 24-crate, 28-phase roadmap for the community to build.

AI Voices Sound Entirely Human. What's Next? — Inworld TTS-2

In 2021 the Blizzard Challenge saturated — human listeners could no longer tell AI speech from a real actor. This deep dive covers how voice AI labs define, measure, and train for the new frontier: expressiveness. Covers the Mean Opinion Score (MOS) and its ITU-standardization history, LLM-style arenas with Elo rankings (Artificial Analysis), AI judges like UTMOS/DNSMOS (and why speech language models can't yet replace human raters), Inworld's deterministic DSP proxy (pitch range + variation + loudness variation), and how DPO/GRPO shape a voice — including a Goodhart's-law case where optimizing for intelligibility flattened prosody, and a reward-hacking case where a model rambled forever. Closes with the Stephen Hawking lesson: sounding human isn't always the goal.

Three of the Four Stages Are Nearly Free: Inside Hugging Face's Open Voice Loop

A 7-minute architectural breakdown of huggingface/speech-to-speech that does the rare thing: it names the bottleneck. VAD, transcription and speech synthesis are all effectively free, which leaves the language model holding the entire perceived delay by itself - so most voice-agent optimisation targets the wrong stage. Verified against the repo (10,161 stars, Apache-2.0), NVIDIA's Parakeet card (6.34% WER, 25 languages) and OpenAI's pricing page ($32/$64 per million audio tokens, exact). Two things the video gets right that most coverage does not: time-to-first-audio beats total synthesis time, and you should hunt the 95th percentile rather than the average. It also corrects itself twice - the repo is from August 2024, and you own four ways to fail with no support contract.

How Far Are We from "Her"?

Julia Turc explores the evolution from cascade voice assistants to full-duplex models — featuring Moshi (Kyutai), GPT Live, and Thinking Machines' interaction models, with insights from Neil Zeghidour on inner monologues, audio tokenization, and why end-to-end voice AI will dominate in 1–2 years.

Trelis Tiron - State-of-the-Art Multi-speaker Meeting Transcription and Attribution (Open Weights!)

I Tried the Open Source ElevenLabs Alternative (Voicebox)

Better Stack explores Voicebox, a free open-source local AI voice studio that bundles voice cloning, TTS, system-wide dictation, and MCP agent integration into one desktop app — positioned as the Ollama moment for voice AI.

Beyond Transcription — Building Voice AI That Understands Conversations

Hervé Bredin (pyannote.ai) on speaker diarization: why 'who said what' matters as much as 'what was said', the gap between benchmark numbers and real-world performance, and how pyannote reconciles STT with diarization at the word level.

State of the Art Meeting Transcription with Overlapping Speakers and Speaker-Identification

Trelis Research benchmarks Chorus Pro against Gemini Pro 3.1 on meetings with overlapping speakers — revealing that a dedicated, smaller transcription model can outperform general-purpose LLMs when 4+ participants talk over each other.

Voice Loop — A Local Voice Agent in ~500 Lines of Python — Trelis Research

Building a local voice agent with Whisper, Kokoro TTS, and an LLM in ~500 lines of Python.

NVIDIA's New Voice AI Is Absolutely WILD — PersonaPlex

NVIDIA released PersonaPlex — an open-source full-duplex conversational AI that listens and speaks simultaneously with near-zero latency. Unlike traditional voice assistants, it backchannels with "uh-huh" and "right" while you're still talking.