Physiological Sensors

The Brain FM system ingests signals from multiple physiological monitoring modalities, collectively referred to as PPMs (Psychophysiological Monitors). Each modality captures a different window into the operator's cognitive and physiological state.

Modality Summary

Invasiveness and Practicality

The modalities vary significantly in how intrusive they are to wear and operate. EEG provides the richest cognitive information - millisecond-resolution neural dynamics - but requires a head-worn device that is uncomfortable for long shifts and conspicuous in operational settings.

Speech is the most practical modality for voice-heavy environments: it requires only a microphone already present in the environment, produces no body-worn sensor burden, and is continuously available whenever the operator is communicating.

PPG offers a middle ground: a wristband is comfortable enough for all-day wear, and while the cognitive information content is lower than EEG, it provides reliable heart rate and HRV features that correlate with stress and workload.

Multimodal Combination

Combining multiple modalities improves prediction robustness:

• EEG + PPG: Neural and autonomic signals are complementary; high workload increases both frontal theta power and sympathetic HRV markers.
• EEG + Eye gaze: Neural and attentional signals together characterise situational awareness more completely than either alone.
• Speech + EEG: Speech provides near-zero-overhead monitoring; EEG can be activated as a higher-fidelity backup during critical events.

See Multimodal Learning for the technical approach to combining modalities.

Data Availability by Modality

The Brain FM's cross-modal alignment strategy directly addresses the availability imbalance: