Hands-Free Two-Way Voice Approval with Home Assistant Voice (NC-VK-9727) & RobOS Home Speaker

Why agents need a physical, zero-cloud voice intercom on your desk. When an autonomous coding agent finishes a task or reaches a deployment gate (like pushing to main), you aren't always staring at a terminal window. With the Nabu Casa Home Assistant Voice Preview Edition (NC-VK-9727) connected to RobOS Home Speaker (packages/home-speaker, port :19200), any agent or script can speak a question out loud in natural Kokoro-82M neural voice, automatically open the speaker's dual-microphone array as soon as the question finishes playing, transcribe your answer locally on your workstation using Whisper (Xenova/whisper-tiny.en), and confirm your decision out loud—all over your local LAN with zero cloud dependencies.

How hands-free auto-listen works under the hood (ESPHome msgType 119 field 4). Normally, smart speakers require you to say a wake word ("Okay Nabu") or press the physical center button before they stream microphone audio. By subscribing as a native ESPHome voice assistant pipeline over TCP port 6053 (VoiceAssistantSubscribeRequest, msgType 89 with flags = 1) and sending VoiceAssistantAnnounceRequest (msgType 119) with field 1 set to the HTTP FLAC stream URL and field 4 (start_conversation) set to true, the NC-VK-9727 plays the spoken question through its TI DAC and—at the exact millisecond playback finishes—automatically triggers VoiceAssistantRequest (msgType 90, start=true) and streams 16 kHz 16-bit mono PCM chunks (VoiceAssistantAudio, msgType 106) back to RobOS.

Acoustic echo guard & RMS silence detection. To prevent the room's acoustic reverb tail from the speaker's own TTS prompt from false-triggering the voice detector, esphome-voice-service.js applies a 550 ms post-playback echo guard before arming RMS speech detection, then automatically closes the mic stream with VoiceAssistantEventResponse (msgType 92) after 950 ms of post-speech silence and runs local Whisper transcription.