Continuing to work on a humanoid helper robot called ARMAR, the Collaborative Research Center 588: Humanoid Robots at the University of Karlsruhe began planning ARMAR-IIIa (blue) in 2006. It has 43 degrees of freedom (torso x3, 2 arms x7, 2 hands x8, head x7) and is equipped with position, velocity, and force sensors.  The upper-body has a modular design based on the average dimensions of a person, with 14 tactile sensors per hand.  Like the previous versions, it moves on a mobile platform.  In 2008 they built a slightly upgraded version of the robot called ARMAR-IIIb (red).  Both robots use the Karlsruhe Humanoid Head, which has 2 cameras per eye (for near and far vision).  The head has a total of 7 degrees of freedom (neck x4, eyes x3), 6 microphones, and a 6D inertial sensor.

Besides developing mechatronic components for the robots, the researchers also want the robot to be capable of interacting naturally with people.  They are focusing on interactive learning (through observation, verbal instruction, and gesture) which requires multimodal interaction (speech recognition, dialog processing, and visual perception of the human instructor).  The robot is therefore able to learn  to recognize new words, objects, and people.  Using a motion-capture set-up, the robot is also learning to recognize how a person moves and manipulates objects by tracking and analyzing key points on the human body (such as joint positions) as the person performs a task.

Manipulating objects is one of the most complex tasks for robots. Motion trajectory, grasping, and manipulation planning, both in simulation and in a realistic kitchen setting, is being carried out with ARMAR-III.  First, a simulation of the robot’s hand tests the various ways it can firmly hold 3D models of standard household objects.  At the time there was no simulation software suitable for their needs, so the team had to build their own!  They called it OpenGrasp, which is based on the modular simulation toolkit OpenRave.  The software is compatible with a variety of physics engines and programming languages.

OpenGrasp allowed them to simulate the kinematics of the robot hand, the object’s shape, material properties, forces and moments, as well as obstacles near the object.  The simulation can then try to determine the most stable grip (out of a very large number of possibilities) taking into account exactly where the robot’s fingers will contact the object.  Once satisfied with the simulation results, they’re able to reliably perform collision-free one-handed and two-handed manipulation tasks with the real robot using visual servoing.

As part of an international 4-year project called PACO-PLUS (Perception, Action, and Cognition through learning of Object-Action Complexes), the researchers have tried a new approach to embodied cognition that couples objects with the actions that can be performed with them.  Object-Action Complexes or OACs, a system of object recognition and classification, not only allow the robot to build a kind of understanding of objects, but can also lead to the development of language, as OACs (mutually grounded symbols) are shared between other robots or people.

Children learn to classify objects through all sorts of interactions, such as putting things into their mouth.  Much can be learned about an object that way; its size and weight, taste, texture, temperature, and whether it is hard or soft.  Much in the same way, ARMAR-III can use all of its sensors to detect and classify objects in a process called Proprioceptive Learning.  Its cameras can detect features, shapes, and colors, while its tactile sensors can measure an object’s rigidity or softness.  Microphones can listen to the sounds an object makes as it is manipulated (such as pebbles rolling inside a bottle), and in the future, smell and taste may play a role.  Like children interacting with a parent or teacher, this learning process can be sped up by a human instructor that tells the robot what color an object is, or by guiding their arm when opening a drawer.  In time, the robot will have built a large library of OACs that make use of an object’s unique properties.

ARMAR-III was able to fetch a cereal box from an unknown location, and stack cups by color, in the laboratory kitchen.  It also learned  more complex tasks such as how to load and unload a dishwasher (see videos below).  These sorts of behaviors will be required of household helper robots in the future, and the PACO-PLUS approach seems to be a successful method of developing them.

[source: Karlsruhe Institute of Technology] via [IEEE Spectrum]

• KIT Collaborative Research Center
• KIT @ YouTube
• PACO-PLUS: Perception, Action, & Cognition
• ARMAR III @ ICT Results, (2)

Media

Video (Mirror):

Video (Mimicking human poses) (Mirror):

Video (Unloading a dishwasher) (Mirror):

Video (Haptic object verification and deformation investigation) (Mirror):

Video (Dual-arm manipulation) (Mirror):

Video (Visually searching for Kellogg’s Frosties cereal) (Mirror):

Images: