Eye Gaze Tracking & Pupillometry

Eye-based sensors provide two complementary cognitive monitoring signals: gaze position (where the operator is looking) and pupil diameter (how aroused or cognitively loaded they are). Both are captured non-invasively with camera-based systems requiring no body-worn sensor.

Eye Gaze Tracking

What It Measures

A camera-based system estimates the 2D gaze coordinates \((x, y)\) on the display surface as a function of time \(t\), typically by tracking the corneal reflection of infrared light sources.

Cognitive Relevance

In the COLET dataset, eye-tracking data is the primary modality for cognitive load estimation. Gaze features computed from 21 image-viewing trials across 47 participants provide ground truth for cross-modal alignment with EEG-based representations.

Application in Operational Settings

In complex operational environments, gaze analysis reveals critical information:

• Which elements an operator is monitoring at any given moment.
• Whether the operator has visually checked critical areas of the display.
• Whether the gaze distribution suggests awareness of the full situation or fixation on a narrow subset.

Scan path irregularity and sudden changes in visual scanning strategy can precede critical errors and are valuable early warning signals.

Pupillometry

What It Measures

The measurement of pupil diameter (left and right separately) over time. Pupil size is controlled by the autonomic nervous system via the locus coeruleus–norepinephrine (LC-NE) system - a key modulator of arousal and cognitive effort.

Cognitive Relevance

Pupil dilation is a highly reliable, non-contact, real-time workload proxy. It is one of the few physiological markers that directly reflects the LC-NE system, which regulates signal-to-noise ratio in neural computation - providing a link between peripheral physiology and neural processing efficiency.

Data Format

pupil_data = {
    "t":   [...],   # timestamp in seconds
    "LPD": [...],   # left pupil diameter (mm)
    "RPD": [...],   # right pupil diameter (mm)
}

LPD and RPD are tracked separately and may be averaged or used independently. Blink-induced dropouts require interpolation before analysis.

Role in Brain Foundation Models

Both eye gaze and pupillometry are underrepresented modalities - large pre-training corpora for these signals do not exist, and collecting them requires specialised eye-tracking hardware and controlled viewing conditions.

The Underrepresented Modalities research thread addresses this by aligning eye-tracking encoders with the pre-trained EEG encoder. The alignment can use semantic-level pairing: windows of eye-tracking data labelled with high workload are aligned with EEG representations of high-workload states, even if the two were not recorded simultaneously.

Hardware

Consumer-grade wearable eye trackers provide gaze and pupillometry in mobile, ecologically valid settings.

Datasets Including Eye Data