Accelerating eye movement research via accurate and affordable smartphone eye tracking

Eye tracking, the measurement of eye movements, has been a crucial tool in various fields, including vision research, language studies, and usability testing. However, traditional eye trackers are expensive and require specialized hardware, limiting their accessibility and scalability. This research addresses these limitations by developing a novel method for accurate eye tracking using the ubiquitous smartphone. Smartphones are pervasive, with billions of users worldwide spending significantly more time on mobile devices than desktops or TVs. Accurate and affordable smartphone-based eye tracking would revolutionize eye movement research by allowing studies to scale to thousands of participants, generating insights across diverse populations and unlocking applications previously infeasible due to cost and hardware constraints. While recent machine learning approaches have shown promise for eye tracking using smartphone cameras, their accuracy has been insufficient for rigorous research. This study aims to overcome these accuracy limitations by leveraging a sophisticated machine learning model and a novel personalization technique.

The researchers reviewed existing literature on eye movements and their applications. They highlighted the importance of eye movements as a measure of overt spatial attention and their relevance to various research areas, including visual search, scene perception, and reading. The literature also underscored the limitations of current eye-tracking technology, namely the high cost and lack of scalability due to specialized hardware requirements. The authors discuss prior work on machine learning-based eye tracking using smartphone and laptop cameras but noted the accuracy limitations of these methods for rigorous research. They establish the context of the study, emphasizing the need for an accurate and affordable smartphone-based eye-tracking solution to enable large-scale eye movement research and diverse applications.

The researchers developed a multi-layer feed-forward convolutional neural network (ConvNet) to estimate gaze from images captured by the smartphone's front-facing camera. The model takes RGB images cropped to the eye region as input and uses convolutional layers to extract gaze features. These features are combined with automatically extracted eye corner landmarks to produce a final on-screen gaze estimate. A key aspect of their method is the personalization of the model for each participant using calibration data. During calibration, participants fixate on a green circular stimulus at various screen locations, and the model is fine-tuned using these data points. A support vector regression (SVR) model is fitted to the output of the ConvNet, further personalizing the gaze estimates. Model accuracy is evaluated by computing the error in centimeters between the stimulus locations and estimated gaze locations. The study involved several experiments to validate the accuracy and utility of the method. Study 1 compared the accuracy of their smartphone-based eye tracker to that of a high-end, expensive mobile eye tracker (Tobii Pro 2 glasses). Study 2 replicated key findings from previous eye-movement research on oculomotor tasks (prosaccade, smooth pursuit, and visual search). Study 3 validated the method using natural images, comparing gaze patterns with those obtained using specialized desktop eye trackers. Study 4 investigated the potential of smartphone gaze for detecting reading comprehension difficulties.

The researchers' smartphone-based eye tracker achieved a mean accuracy of 0.46 cm on the phone screen (0.6-1° viewing angle) using less than 30 seconds of calibration data per user. This accuracy was comparable to the state-of-the-art mobile eye tracker (Tobii Pro 2 glasses), which is at least 100 times more expensive. The model's accuracy was consistent across different screen locations, with slightly larger errors towards the bottom of the screen. The researchers successfully replicated key findings from previous eye-movement research using their smartphone-based system. In the oculomotor tasks, they observed saccade latencies consistent with previous studies (mean 210ms), and in the visual search tasks, they replicated the effects of target saliency and set size on search performance. Their analyses of gaze patterns on natural images were qualitatively similar to those obtained using expensive desktop eye trackers, indicating the potential for scaling saliency analyses on mobile content. Finally, the study showed that smartphone gaze could be used to detect reading comprehension difficulty by analyzing gaze patterns during the reading of passages, observing that gaze was more dispersed for interpretive questions compared to factual ones. Participants who answered factual questions correctly focused more on relevant parts of the passage than those who answered incorrectly.

This study demonstrates the feasibility of accurate and affordable smartphone-based eye tracking using machine learning. The achieved accuracy, comparable to that of significantly more expensive systems, along with the successful replication of prior research findings, validates the methodology. The potential of using the system to detect reading comprehension difficulty highlights its utility beyond basic research. The significance of this work lies in its potential to dramatically increase the scalability of eye tracking research. This opens new opportunities to study eye movements in diverse populations and contexts and enables the development of novel applications in fields like accessibility and healthcare. The findings address the research question by demonstrating a highly accurate and scalable eye-tracking solution that overcomes previous limitations.

The researchers successfully developed and validated a highly accurate and affordable smartphone-based eye-tracking system using machine learning. This system achieved accuracy comparable to state-of-the-art mobile eye trackers at a fraction of the cost. The successful replication of key findings from existing literature and the demonstration of its utility in detecting reading comprehension difficulties highlight its potential for wide-ranging applications. Future research could focus on improving the robustness of the model across different head poses, devices, and demographics, as well as exploring real-time on-device processing for applications like gaze-based interaction for accessibility.

The study had some limitations. Participants were recruited from a specific pool of user study volunteers and studies were conducted in lab settings using a fixed device stand. This controlled environment might not fully represent real-world usage scenarios, where participants might hold the phone differently or experience varying lighting conditions. The temporal resolution of the system, limited by the phone's camera specifications, restricted the precision of certain eye movement measurements. Future research should address these limitations by conducting studies in more naturalistic settings and exploring the performance of the system under different conditions.