What if a computer knew what you see?
What if a computer knew what you are interested in at the moment?
DFKI GmbH
Knowledge Management
Trippstadter Strasse 122
D-67663 Kaiserslautern
Gaze-based Human-Computer Interaction (HCI) is now greatly drawing people's interest. Today, users can use their eyes to move a pointer, to type texts and to control other interactive functionalities by simply gazing at a computer display. In addition to these traditional gaze-based interactions, an advent of a glasses type of mobile eye tracker enables us to design a gaze-based interactive system in a real environment, not only in a computer display. Museum Guide 2.0 is one of the applications at the cutting edge of such computer technology.
In CeBIT 2013, we present a new type of gaze interaction application, called Talking Places.
Please visit us on the CeBIT, H9 F50 from March 5th to 9th 2013

Museum Guide 2.0 is a sophisticated automatic guide system for a museum visitor. It analyzes eye movements of the visitor and recognize to which object he/she pays an attention. When the visitor looks at a particular object for a while, Museum Guide 2.0 supplies him/her with the information about the object with several different forms, such as audio playback, displaying on a smartphone and so on.
This system is realized by combining a mobile eye tracker (We use the SMI Eye tracking Glasses) with an object recognition engine. We also implemented an algorithm for distinguishing attentive gaze from non-attentive gaze in order to detect the viewer's interests on the objects.


We present another type of gaze interaction application in CeBIT 2013.
You get information of the town in augmented reality by simply looking at a miniature model of Kaiserslautern

Eye tracking is the process of finding out a point of gaze of an animal. Owing to recent advances of eye tracking technologies, an eye tracker nowadays becomes quite compact and easy-handling. We use a mobile eye tracker for detection of the gaze position of a museum visitor.
![]()
Furthermore, an object recognition engine is also integrated in Museum Guide 2.0. While the visitor looks at an object in a museum, a computer connected to an eye tracker analyzes the scene image from a camera and recognize to which object the visitor gaze is located.

In order to investigate the efficiency and the usability of Museum Guide 2.0, we plan to conduct a user study in a real museum.
Currently, we have offered to a local museum in Kaiserslautern to cooperate with us to do that.
Takumi Toyama; Thomas Kieninger; Faisal Shafait; Andreas Dengel
Gaze Guided Object Recognition Using a Head-Mounted Eye Tracker. In: The Seventh ACM Symposium on Eye Tracking Research & Applications. Symposium on Eye Tracking Research & Applications (ETRA-2012), March 28-30, Santa Barbara, California, United States, ACM, 2012.
Toyama-Gaze-Guided-Object-Recognition-ETRA12.pdf
Takumi Toyama; Thomas Kieninger; Faisal Shafait; Andreas Dengel
Museum Guide 2.0 - An Eye-Tracking based Personal Assistant for Museums and Exhibits. In: L. Ciolfi; K. Scott; S. Barbieri (Hrsg.). Re-Thinking Technology in Museums 2011: Emerging Experiences. International Conference on Re-Thinking Technology in Museums, May 26-27, Limerick, Ireland, ISBN 1-905952-31-7 978-1-905952-31-1, University of Limerick, 5/2011.
Takumi-MuseumGuide-2011.pdf
Takumi Toyama
Object Recognition System Guided by Gaze of the User with a Wearable Eye Tracker. In: Rudolf Mester; Michael Felsberg (Hrsg.). DAGM'11 Proceedings of the 33rd International Conference on Pattern Recognition. Annual Symposium of the German Association for Pattern Recognition (DAGM-11), 33rd, August 30 - September 2, Frankfurt, Germany, ISBN 978-3-642-23122-3, Springer, Berlin, 2011.
We use a mobile eye tracker (SMI Eye Tracking Glasses) produced by SensoMotoric Instruments GmbH (SMI).
SMI Eye Tracking Glasses
http://www.eyetracking-glasses.com/
SensoMotoric Instruments GmbH (SMI)
http://www.smivision.com/

We conduct a user study in a real museum in cooperation with a local museum in Kaiserslautern.
Museum Pfalzgalerie Kaiserslautern
http://www.mpk.de/ ![]()
Museumsplatz 1
67657 Kaiserslautern
Telefon: 0631 3647-201
Telefax: 0631 3647-202
E-Mail: info@mpk.bv-pfalz.de