Our current standardisation work is on the topic of eye data quality (EDQ) - how eyetrackers compare in their performance across different types of participants/users and usage scenarios, their suitability for valid mesurement of eye movement classes and types, designing methods and measures for valid comparison. This project is led and administered by EMRA in collaboration with COGAIN, whose role in the project is to report results as they are relevant to users and designers of gaze-interaction interfaces. You can read about the aims of the project here, and see who the committee members are here.
In April 2014 the first large scale test of commercial eyetrackers eye data quality, using the methods developed by the standardisation committee, was hosted by The Humanities Laboratory,
Lund Univeristy, Sweden. This is one of several sites used by the project for eye movement data collection. With help of visiting students and researchers from the EDQ committee member's institutions, and the support of manufacturers in supplying systems, we recorded 192 participant's fixational and smooth pursuit eye movements with participants from around the world. We recorded each participant on on 17 commercially available eyetrackers including high-end lab-based equipment, wearable eye tracking glasses, and low cost plug and play eye trackers. We recorded fixation, saccade and smooth pursuit data across the full screen area with a high degree of environmental control, alongside a large number of individual characteristics and experimenter measurements. The purpose of the large dataset is to assess individual versus system effects on eye data quality and its effect on valid measurement of eye movements in research results. We also use this dataset to test and refine sample selection and data quality measures to ensure unbiased system comparison through consensus across researchers and various participating manufacturers. In this sense, the large dataset, when released, is intended to provide a control and comparison resource for eye data quality to the research community, along with the tools and standard measures to use exactly the same tests in a regular research recording - increasing the size of the dataset and the systems represented with new recordings over time. The first recording provides the case where a large number of the same individuals were recorded on 17 eye trackers, based on power estimations for predicting effects of individual variance as well as system variance.
Rochester Institute of Technology, NY, has also been a test centre, where recording has focussed on artificial eyes and the development of robotics. First results from the project were presented at ETRA 2012, ECEM 2013, VSS 2014, and at ECEM 2015 via an EDQ project symposium. These methods will be published and both analysis scripts and data made available in the future, but any interested researchers who would like to get involved in project in the meantime is welcome to join the sub committee and contribute with full attribution. Membership of the main technical committee is through election and open positions are advertised via call for nominations from the subcommittee and among existing external collaborators, and/or through relevant eye movement mailing lists.