Mastering physical contact with new algorithm for robots

Penn Engineers have developed a new algorithm that allows robots to react to complex physical contact in real time, making it possible for autonomous robots to succeed at previously impossible tasks, like controlling the motion of a sliding object.

The algorithm, known as consensus complementarity control (C3), may prove to be an essential building block of future robots, translating directions from the output of artificial intelligence tools like large language models, or large language models, into appropriate action.

"Your large language model might say, 'Go chop an onion,'" says Michael Posa, assistant professor in mechanical engineering and applied mechanics (MEAM) and a core faculty member of the General Robotics, Automation, Sensing and Perception(GRASP) Lab. "How do you move your arm to hold the onion in place, to hold the knife, to slice through it in the right way, to reorient it when necessary?"

One of the greatest challenges in robotics is control, a catch-all term for the intelligent use of the robot's actuators, the parts of a robot that move or control its limbs, like motors or hydraulic systems. Control of the physical contact that a robot makes with its surroundings is both difficult and essential. "That kind of lower- and mid-level reasoning is really fundamental in getting anything to work in the physical world," says Posa.

Humans, of course, rarely have to think twice about how they interact with objects. In part, the challenge for robots is that something as simple as picking up a cup actually involves many different choices-from the correct angle of approach to the appropriate amount of force. "Not every one of these choices is so terribly different from the ones around it," Posa points out. But, until now, no algorithm has allowed robots to assess all those choices and make an appropriate decision in real time.

To solve the problem, the researchers essentially devised a way to help robots "hallucinate" the different possibilities that might arise when making contact with an object. "By imagining the benefits of touching things, you get gradients in your algorithm that correspond to that interaction," says Posa. "And then you can apply some style of gradient-based algorithm and in the process of solving that problem, the physics gradually becomes more and more accurate over time to where you're not just imagining, 'What if I touch it?' but you're actually planning to go out and touch it."

Read more at Penn Engineering.