Voice-to-voice AI using Open Source tools

August 22, 2024

Voice-to-voice AI using Open Source tools.

I did a small demo at @AITinkerers last night.

✅ zero-UI HTML circa 1997
✅ talks like a pirate
✅ function calling demo "get_weather()"
✅ joke about Blaise Pascal
✅ fast, natural voice conversation with an LLM in ~100 lines of code.

The tech stack:
- @DeepgramAI transcription
- @AnthropicAI Claude 3.5 Sonnet
- @cartesia_ai voice
- @trydaily transport
- @pipecat_ai voice-to-voice toolkit

Thank you to @jheitzeb for putting on such great events and recording the demos. And to @pk_iv for being such a fantastic emcee.

Lots of good stuff in Joe's thread of the demos. Go check it out! It's always fun to see @Altimor demo @getlindy. The @humanlayer_dev demo made me think, too — a kind of "inversion of control" view of agents. And @yawnxyz's AI formulas library for Google Sheets felt like a view into the future of spreadsheets.

The repo for my demo code is here: https://t.co/vSkySdSciv

Clone it and run your own voice-to-voice bot that can talk like a pirate or tell you what it sees in your camera feed.

Joe Heitzeberg@jheitzeb

@kwindla showing their very very new voice to voice APIs that make voice bots easy to create… reminds me of Twilio back in the day! @trydaily