cerebriumai@cerebriumaiSeptember 17, 2024
This a really nice example of how to build an AI voice and video conversational agent that uses function calling.
Function calling is an increasingly important component of production real-time AI applications.
The latest LLMs are now quite good at calling functions reliably, which has greatly expanded the use cases that conversational AI is well suited to.
For example, you can use function calling:
➡️ as part of a dynamic RAG system that allows flexible access to a knowledge base,
➡️ to help an AI follow a script,
➡️ for saving information gathered from a user during a conversation,
➡️ to implement lightweight lookup of dynamic information
That last one is the basis for everyone's favorite docs example of function calling, the `get_weather()` tool!
Function calling is still a fairly new capability and still a little bit challenging to implement from scratch for a new app.
The first challenge is that you have to implement a control flow that glues together the LLM request/responses with code that makes the function calls and formats the function return values.
The second challenge is that function calls add to latency, and low latency — fast response times — is critical for voice AI applications.
The example in the tweet below uses state-of-the-art, low-latency AI tools from @heytavus and @cartesia_ai, and then goes the extra mile by hosting a @MistralAI model specialized for function calling on @cerebriumai's excellent serverless GPU infrastructure.
Go try out the demo. It's very, very responsive. It's also super-impressive that this is a pretty small amount of code — ~400 lines of Python, including all the function calling and scripting of the AI agent.
cerebriumai@cerebriumaiWe wanted to push the boundary and see if we could create realistic, human-like situations in order to train teams to perform better.
For example: Handling an angry customer on a sales call or preparing for the notoriously stressful YC interview👀
#ai #startups #aiavatar #llms
