The aim of designing and optimizing human-machine interfaces is not to faithfully represent the real world in the new world of interaction but to create a world for the operator that ensures immersion and performance. Immersion means that the user can be immersed in the system, potentially to the point of no longer being able differentiate between the real world and virtual reality. Performance is a measure of the quality of task accomplishment. Basic research of ways to provide such an environment has included feedback via different information channels:
- haptic/kinesthetic feedback,
- vision, visual feedback,
- acoustic feedback.
Haptics/Kinesthetics
Haptic feedback is a central challenge in designing interfaces in the field of telerobotics. Put simply, haptics represent the sense that is relevant for grasping or touching objects. Haptic feedback, for example, allows us to grip a glass firmly enough to not drop it but gently enough to not break it. Kinesthesia or proprioception is the sense of self-movement and position of the joints, muscles and tendons. Fine-motor coordination requires both haptic and kinesthetic feedback.
Thermal display
Cybertouch
Cybergrasp
Literature
Hannig, Gunter: Modellierung multi-modaler haptischer Wahrnehmung; Dissertation Universität der Bundeswehr München, 2012.
Vision/Visual Feedback
Visual conditions and thus visual information in telepresence applications is affected by different parameters. Cameras cannot capture all views and angles. Transmission bandwidth affects image quality. If the operator sees the scene through a head-mounted display (HMD) – which has many advantages – the field of vision is limited.
Head Mounted Display (HMD)
This has negative effects on the assessment of distances, performance quality in search tasks, identifying important objects and the operator’s wellbeing. One study examines whether supplementing video images with virtual reality can improve performance, situational awareness and the feeling of telepresence.
Literature
Acoustic Feedback
Humans are “multisensory information processors”. Despite the dominance of the visual sense, haptic and acoustic information also play an important role. Studies on the influence of acoustic information focus on localization and navigation tasks. For one thing, they demonstrate that when it comes to these specific tasks, both auditive and haptic feedback are inferior to visual feedback. What is more, the combination of haptic or acoustic information with visual information does not lead to better performance on this type of task compared with purely visual information, either.
Literature
Data Compression and Transmission
If the operator and the teleoperator are located far apart and bandwidth is limited, there will be lags or gaps in data transmission. Delayed or missing feedback leads to impaired performance. This problem can be solved with what we know about perception.
Weber’s law states that the just-noticeable increase in the intensity of a stimulus is a constant proportion of the intensity of the initial stimulus.
If you light a candle in a dark room, the difference in brightness is obvious. If, however, 50 candles are already burning and you add one more, the difference will not be noticeable.
Perception-Based Data Compression
This principle can be applied to the compression of haptic data in order to reduce the data transmission rate. Only the black values are transmitted (see above). Only when there is a significant change will a new value be transmitted. This makes it possible to reduce the data rate by more than 90% without impairing perceived quality or task performance.
Literature