Govind-S-B@violetto96April 8, 2025
I definitely agree that a bot manager is a useful pattern. There are examples floating around that call this a "bot runner."
Pipecat itself is un-opinionated about deployment/scaling architectures.
The basic model is "one Python process per agent." Beyond that you can build whatever scaffolding makes sense for your use case. ... 🧵
Govind-S-B@violetto96@kwindla @mem0ai love pipecat, good to see yall pluggin in with other things. only issue is while i like ur higher level features. the subprocess bots kinda thing sound a nightmare for scaling. shouldnt there be a bot manager instance and then bots could be individual workers that can spin up
tldr: right now most people are building new voice AI clusters in-house using Kubernetes. You can also check out Pipecat Cloud, which tries to make production deployments of open source voice AI as easy as `docker push`
https://t.co/UhucUrq7VF
These days, if you're building out infra for scalable, long-running processes, you're either:
1. Building on top of Kubernetes
2. Building on top of a higher-level or differently opinionated infra provider like Fly, Modal, or Cloudflare workers.
One important thing to note is that many infrastructure products don't support long-running processes, UDP networking, or both. So you can't use AWS Lambda or Google Cloud Run for voice AI today.
Both (1) and (2) above are a significant amount of work. If you have a lot of k8s experience on your team, you'll be able to set up voice agent clusters in a few days. But some of the building blocks are going to be different from what you've got in production for your other workloads. (How you do capacity planning, think about cold starts, do rolling deployments so you don't drop current sessions, etc.) So factor in extra work specific to the maintenance and evolution of your voice AI clusters.
None of the new school cloud providers yet have setups optimized for realtime AI. But I think that will change.
We think Pipecat Cloud sits at the sweet spot between giving you the full flexibility of building your own agents while also taking all the devops headaches away. Let us know what you think.