The Smarts Gap
I love seeing bake offs between Google Assistant and Alexa (and other AI assistants) because it makes me nostalgic for the development of the Ubi when we first launched that product. The issue with bake offs is that they very quickly push the questioner into asking the obscure and then seeing what percentage of questions were answered correctly.
This misses the point as there is much more than a Pareto distribution when it comes to questions and answers to AI assistants. Probably 95% of questions asked will make up <1% of the total questions asked (this is a hypothesis but I’d love to see actual data).
For the Ubi, it was these questions (under the domain of “information requests”):
- How’s the weather? (and many variations of this)
- What’s the time? (and time in other timezones)
- Who’s the President (of the United States)? (very distant 3rd)
All other requests were a long tail.
One time, we were burned alive by a client because we weren’t able to get Kevin Costner’s age right.
Amazon doesn’t have to be as good at Google at knowing everything that Google knows. It has to win at delivering a better answer to the most common questions. With context and follow up capabilities, Alexa might be there very soon.
