Synopsis

The research and realization of  “seeing” machines and “intelligent” robots has been a main focus of the Intelligent Robots Lab (formerly known as “Institute of Measurement Science”) since 1977. Our aim is both a basic understanding of vision, autonomy and intelligence of technical systems, and the realization of intelligent, seeing robots. These should be able to function dependably in the unpredictable and ever-changing real world and to fulfill various tasks autonomously or in cooperation with humans.

From the very beginning our research work has been essentially guided by the rule that all research results have to be verified and demonstrated by practical experiments under real-world conditions. While this approach is very demanding, it has the great advantage over simulations that it leads to more reliable and practice-oriented results. The fact that our research projects were often carried out in cooperation with industrial partners also led to a better applicability and practicability.

Similar to most animals, all our robots use machine vision as an effective sensor modality. The hardware of our vision systems often includes a PC, in some cases augmented by simple microprocessors, digital signal processors or transputers. A vision system architecture ("object-oriented vision") we have developed enables the systems to be so efficient that real-time vision became possible as early as 1980 with the then available 8-bit microprocessors. Communication bottlenecks are avoided by wideband video busses and high-speed links between the processors. When using modern hardware in combination with an efficient and robust feature extraction on the basis of controlled correlation as well as a situation-dependent control of gaze direction and camera sensitivity, these systems give our robots a highly developed visual sense.

 

Main Research Topics

  • architecture and realization of robot vision systems 
  • motion stereo for accurate range measurement and spatial interpretation of image sequences 
  • calibration-free robots (i.e., robots not needing any quantitatively correct model of themselves or the environment)
  • object- and behavior-oriented stereo vision as a basis for the control of such robots by direct transition from sensor data to motor control commands 
  • system architecture for behavior-based mobile robots 
  • recognition of dynamically changing situations in real time as the basis for behavior selection by robots and for man-machine-communication
  • machine learning, e.g., for object recognition, motion control, and knowledge acquisition for navigation and interaction with humans
  • integration of very complex robotic systems, such as our humanoid robot HERMES, a prototype of a personal assistant robot, able to see, hear, speak, and feel, as well as move about, localize itself, build maps and manipulate various objects
  • dependability and long-term tests of complex robotic systems

 

Important Milestones

  • 1980   real-time vision system BVV 1; its novel, later known as “object-oriented”, architecture enabled it to interpret dynamic scenes in real time (cycle time 40 ms) by using ordinary 8-bit microprocessors with 3 MHz clock frequency. Data thus obtained can be used for controlling machines almost at human speed.
  • 1982   application of the BVV 1 for stabilizing inverted pendula down to 40 cm of length with computer vision (real-time image processing) as the only sensor modality
  • 1983   real-time image processing system BVV 2; same architecture as BVV 1, but ten times more powerful by using 16-bit microprocessors with 8 MHz clock frequency
  • 1985   controlled (“intelligent”) correlation, an extraordinarily efficient, flexible and robust method for feature extraction from noisy images which resembles the human visual system in some respects
  • 1987   vision system for the world's fastest fully autonomous road vehicle (96 km/h) on the basis of the real-time vision system BVV 2 and controlled correlation (cycle time of image processing: 17 ms)
  • 1989   real-time vision system BVV 3; same architecture as BVV 1, but a thousand times more powerful, due to 32-bit microprocessors with 20 MHz clock frequency and coprocessors specially designed for feature extraction
  • 1989   vision-guided mobile robot ATHENE with a novel situation-oriented behavior-based system architecture and the abilities to chart networks of corridors and to localize itself accurately relative to landmarks and obstacles
  • 1990   novel aproach to motion stereo, allowing highly accurate range measurements in real time using an uncalibrated video camera
  • 1992   vision system for the real-time recognition of traffic scenes, allowing a driverless motor car to run at normal speed in ordinary freeway traffic; basis: real-time vision system BVV 3 and numerous recognition modules for traffic-relevant objects
  • 1994   vision-guided mobile robot ATHENE II, able to explore and chart networks of corridors and to navigate in them autonomously
  • 1995   calibration-free vision-guided learning robot (stationary articulated arm), being extremely robust against unmodeled disturbances of its characteristics, e.g., the camera arrangement
  • 1998   humanoid mobile robot HERMES with a highly modular hard- and software structure, based on perception, situation recognition and skills, demonstrated at the Hannover Industrial Fair
  • 1999   demonstration of vision-guided handling of various objects as well as learning, incremental learning and forgetting by a calibration-free robot
  • 2000   HERMES is capable of performing a variety of transportation and other service tasks and to communicate with humans in a situation-dependent way by gestures and by spoken and written natural language.
  • 2002   HERMES has developed into a versatile personal assistant robot and completed a long-term dependability test by serving, together with intelligent robots from all over the world, for 6 months in the special exhibition "Computer.Brain" in the Heinz Nixdorf MuseumsForum in Paderborn, the world's largest computer museum. There it chatted with visitors in natural language in English, French and German, answered questions and performed services as requested by the visitors. HERMES survived the daily hard work (usually 2 hours, sometimes 12 hours per day) far away from its "fathers" where no easy access to repair and maintenance was available, and even with presenters who did not know much about robot technology. According to the museum staff, HERMES was one of the few experimental robots that could regularly be demonstrated in action, and among them they considered it the most intelligent and most dependable one.