Researchers at the Mechatronics Research Laboratory of the Massachusetts Institute of Technology (MIT) have come up with a new approach to helping robots help humans, allowing them to guess what tools or other objects a human might need and automatically make them available: Relevance.
“Relevance can guide the robot to generate seamless, intelligent, safe, and efficient assistance in a highly dynamic environment,” claims first author Xiaotong Zhang of the team’s work. “I would want to test this system in my home to see, for instance, if I’m reading the paper, maybe it can bring me coffee. If I’m doing laundry, it can bring me a laundry pod. If I’m doing repair, it can bring me a screwdriver. Our vision is to enable human-robot interactions that can be much more natural and fluent.”
Relevance, with a capital R, could help to make future robots more helpful — without even telling them what you need. (📷: Zhang et al.)
“This approach of enabling relevance could make it much easier for a robot to interact with humans,” adds senior author Kamal Youcef-Toumi, a professor of mechanical engineering at MIT. “A robot wouldn’t have to ask a human so many questions about what they need. It would just actively take information from the scene to figure out how to help.”
Relevance, with a capital R, works by taking in audiovisual cues and processing them for contextual clues as to what a human might do next. If they’re reaching for a cup of coffee, for example, the robot might pass them a creamer and a stirrer; if two people are discussing hunger, it might pass them a piece of fruit.
The system relies on a machine learning model, which combines a large language model (LLM) with classification algorithms, designed to mimic the human brain’s reticular activating system, or RAS — neurons responsible for subconsciously pruning unnecessary stimuli so you can better focus on what’s relevant to the task at hand. “The amazing thing is, these groups of neurons filter everything that is not important, and then it has the brain focus on what is relevant at the time,” Youcef-Toumi says. “That’s basically what our proposition is.”
Relevance also improved safety, reducing collisions by better predicting the likely path of a human’s hand. (đź“·: Zhang et al.)
In testing, Relevance proved able to predict humans’ objectives with 90 percent accuracy and to identify items relevant to those objectives with 96 percent accuracy. Interestingly, the increased focus also improved safety — delivering, the researchers found, a reducing in collisions by over 60 percent compared to carrying out the same tasks without Relevance running in the background by creating a virtual ellipsoid in the predicted path of a human hand that “repulsed” the robotic hand.
A preprint of the team’s work is available under open-access terms on Cornell’s arXiv server.