Kathryn Wu@kathrynwu1April 15, 2026
I love this framing, and am also not sure it's right. (But it might be! It's great food for thought.)
In some ways, these "natural language machines" (LLMs) that we're starting to redesign all our software around, take us back to the days of the CLI. And CLIs are both famously productive for power users and have very steep learning curves.
A huge difference, though, is that these new CLIs can introspect and explain things interactively. If we prompt them well.
So there are at least a couple things here that need to be for an application to pass @kathrynwu1's AI-native test:
1. You have to prompt the LLMs embedded in your AI-native app to actually know about the app so they both get their own capabilities right for non-trivial tasks, and so they can help the user understand what they can do. This is not easy!
2. Users have to learn that they can use computers in this mode. "Hey, application, how do we do foo together?"
We launched a big, AI-native (I hope), experimental app yesterday. A game. And we have spent a lot of time trying to embed game knowledge into the game's primary voice agent context. Lots of iteration on the prompt. Things like micro-testing small changes in the tool definitions, adding and removing few shot examples, building little mini-evals (but not enough of them). We built a progressive skills loading system. But still ... I think we're at "good but not great" in terms of the main AI interaction surface being able to introspect/explain things really well.
But one thing that surprised me is that a lot of new players don't even think to ask these kinds of questions. I'm sure that will change, but I answered a lot of questions on Twitter yesterday, where the real (but not super polite answer) would have been: just ask your ship AI in the game.
I was kind of against building a tutorial for the game. But I was wrong. For this reason, and also a few others, I'm sure!
Kathryn Wu@kathrynwu1AI-native product insight:
If your AI needs a tutorial, you’re not AI-native yet.
Relevant quote here from the Pipecat discord.
If you want to play the game, it's here[1]
Source code is here (including all the prompts)[2]
