LLMs are making it possible to automate a huge surface area of software…

August 13, 2024

LLMs are making it possible to automate a huge surface area of software interactions that were originally designed for use only by humans.

This voice agent is a really nice example.

If you call the DMV in California, a computer answers the phone. @omooretweets built a low-code voice agent (using @Vapi_AI and @OpenAI) that can talk to the DMV for you to schedule an appointment.

There's no DMV "API" for appointment scheduling. You have to talk to the DMV computer's voice interface. But now you don't need an API to automate your interaction with the DMV's back-end system!

Today's LLMs can turn things like voice conversations, graphical user interfaces, and web pages into "APIs" that you can write code against.

For other examples, check out what @browserbasehq and @AdeptAILabs are doing, and the papers "Language Models Can Solve Computer Tasks" and "UFO: A UI-Focused Agent for Windows OS Interaction."

Olivia Moore@omooretweets

I built an AI voice agent that called the DMV to schedule an appointment for me!

....and you can too. It took <30 minutes and two tools.

(And, it was incredibly satisfying to hear my AI deal with the DMV's AI instead of sitting on the phone myself😂)

How I did it 👇