Thought to Intent
Forget about Though To Text. We need machines to understand.
About a year ago, researchers at Chang Lab at UCSF released papers about thought to text. It’s a big deal if we can send thoughts to each other. However, to interface to a machine, sending thought-based text is slow and likely requires the same effort as vocalizing or writing.
It might even require more effort to think. With speaking or writing, we can curate the thoughts and only output what we need to be translated. With thoughts, we might be interrupted by many competing thoughts. Maybe after some training, writing could become more meditative but for instances where we’re not writing a manuscript and where we’re relying on speed and accuracy, this skipping the mouthing of words part might not get us where we want to go.
What we really want when communicating with machines is for them to know our intents. We want the robot to move over there, or pick up that thing, or to turn on the light switch or open the blinds. We want to express actions and assign entities to these.
When interacting machines, we’ve relied more on natural language understanding (NLU), a subset of natural language processing (NLP). Typically, this involves taking text and extracting entities and intents. Intents are usually related to actions and entities might be “slots” to be filled or the objects of these actions.
Humans existed without spoken languages before they touched the Monolith. We could grunt our intents or show it with body language. Anything more nuanced might have been lost. However, we retained the ability to communicate intents this way, even without needing to point. Eye movements, expressions, curling of the mouth, turning of the head are all effective communication tools.
The next move for machines to understand us would be to skip the language middle man, or at least combine it with single words, to understand our intents. For example, looking at some dirt on the ground and thinking “clean” could get the Roomba to clean. Looking at the blinds first at the side and then at the top could signal to open them. Looking at the oven could signal for the light to come on inside of it to check on what’s cooking.
Many of these things can be done without requiring an cranial implant. Eye tracking, facial expression detection, gesture recognition, and more context awareness could make many of these scenarios possible today.
