Social robots may be able to understand where people are looking without expensive eye-tracking equipment. AI scientist Linlin Cheng of Vrije Universiteit Amsterdam (VU) investigated whether this is possible using only ordinary cameras. Eye trackers are accurate but expensive, can feel intrusive and are hard to deploy outside the lab. Cheng therefore focused on appearance-based gaze estimation, where gaze direction is predicted from standard camera images. “With an ordinary camera, robots can get a useful picture of where someone is looking, without needing special eye-tracking equipment.”

The system is particularly good at recognizing someone’s general attention pattern, for example whether they are looking at the robot or at a task, at a distance of roughly one to two meters. Quick glances and sudden shifts in attention are harder to capture. The technology therefore does not yet replace highly precise eye trackers everywhere, but offers a useful and affordable indication.

Cheng compared models against reference data from eye trackers and manual annotations, and tested them in experiments where participants performed structured tasks with real robots. She also developed a real-time system on low-cost robotics hardware that detects when someone has completed a task. A care robot, for example, can see whether an older person is looking at the medication tray or is distracted or confused, and a supportive robot in a workshop can tell that a task is finished. According to Cheng, the lower cost and complexity make real-world use realistic within a few years, for patients in hospitals, students in classrooms and workers in factories and warehouses.