July 1, 2025
Voice AI hardware + WebRTC + Pipecat
Here's a @pipecat_ai SmallWebRTCTransport client for the ESP32-S3 family of embedded devices.
The SmallWebRTCTransport is a serverless WebRTC connection designed for voice AI. Link to code and resources below ...
@aconchillo wrote the ESP32 client code, building on work by @_pion, @pbbakkum, and sepfy's libpeer library.
The device in the video is the ESP32-S3-BOX-3B developer kit, with a microphone, speaker, and touch screen. It's connecting to a voice agent running on my laptop. The voice agent uses @openai GPT-4o and the @cartesia_ai Sonic voice model.
Some ideas for things to do next:
- write all the voice agent output to the screen as text
- write LLM-generated images to the screen
- design a little micro-UI kit library for the touch screen and dynamically generate the user interface
ESP32 SmallWebRTCTransport client code is here:
https://t.co/ImPAPH6KrF
You'll also need this Pipecat branch to run the examples/foundational demos with the ESP32 client (until the next Pipecat release):
https://t.co/jmgeXmpNns
Here's the hardware device in the video. (It should be fairly easy to support any device in the S3 family.)
https://t.co/cPERpT65iw
The voice bot in the video is just one of the standard Pipecat test examples:
https://t.co/I74ikUEhEI
