This point about the serverless tooling from the big cloud providers not being…

March 17, 2025

This point about the serverless tooling from the big cloud providers not being a good fit for stateful workloads is important.

We've been thinking about this from the perspective of long-running, interactive conversation use cases for a long time. (WebRTC). We've never been able to use the higher-level infrastructure tooling from any of the big cloud providers. (Kubernetes or bust!)

WebRTC was always a niche use case. Big in terms of traffic, but small in terms of number of developers who would need to or could stand up their own infrastructure.

But now, with conversational LLM use cases growing, this is not so niche anymore. The serverless infra products will have to evolve to support:
- very long-lived connections
- UDP network transport
- better global routing

Jesse Ezell@jezell

So MCP is upgrading it's SSE transport. Building higher level tool calling for LLMs is something I've spent a lot of time on, so here are some thoughts:

1) Most HTTP APIs are stateless, so requiring a stateful wrapper around them wastes a lot of resources.

2) Actually stateful