Cognitive States
The Brain FM system targets four cognitive states that are most operationally relevant for high-stakes environments: cognitive workload, stress, fatigue, and situational awareness.
Why These Four
These states were selected because they:
- Have measurable physiological correlates - they are detectable from EEG, ECG, PPG, and other biosignals with validated methods.
- Directly affect performance - degradation in any of these states increases the probability of operational error.
- Have validated subjective assessment instruments - NASA-TLX (workload), SAM/valence-arousal scales (stress), Karolinska Sleepiness Scale (fatigue), and SART (situational awareness) provide ground-truth labels for training.
- Are practically monitorable - the sensor requirements (wearable EEG, wristband PPG, microphone) are feasible in training and operational environments.
State Definitions
| State | Core Definition | Optimal Level | Degraded Performance When |
|---|---|---|---|
| Cognitive Workload | Mental demands imposed by a task relative to available cognitive capacity | Moderate; manageable | Too high (overload) or too low (underload / complacency) |
| Stress | Physiological and psychological response to perceived threat or demand | Low | Elevated; prolonged stress impairs decision quality |
| Fatigue | Reduction in alertness and processing speed due to sustained effort or sleep deprivation | Alert | Accumulated; microsleeps and attention lapses become probable |
| Situational Awareness | Accurate mental model of the current operational situation | High | Degraded; operator acts on an outdated or incomplete picture |
Interrelationships
These states are not independent: high sustained workload accelerates fatigue; high workload also triggers stress; stress impairs situational awareness; and acute stress can temporarily increase perceived workload. Accumulated fatigue further degrades situational awareness. A highly loaded controller who is also fatigued and stressed is in a compounding risk state where none of the four states are monitored adequately by focusing on only one.
Measurement and Labelling
Subjective Assessment Instruments
Subjective assessments provide the ground-truth labels for supervised fine-tuning. They are administered at intervals during experiments (continuous administration during actual task performance is not feasible).
| State | Primary Instrument | Format |
|---|---|---|
| Workload | NASA-TLX | 6-dimension 1–21 scale; administered post-section |
| Workload | ISA (Instantaneous Self-Assessment) | Single 5-point real-time scale |
| Stress | SAM (Self-Assessment Manikin) | 9-point arousal + 9-point valence scales |
| Fatigue | KSS (Karolinska Sleepiness Scale) | 9-point scale |
| Situational Awareness | SART | Multi-dimension SA ratings |
Physiological Ground Truth
Subjective labels have limitations: they require interrupting the operator, they reflect a retrospective estimate, and they are influenced by mood and personality traits. Objective performance metrics (error rates, reaction times) provide complementary labels:
- Task error rate → proxy for SA + workload degradation.
- Reaction time → proxy for fatigue and workload.
- Secondary task performance → standard dual-task workload metric.
Decoder Architecture
Each cognitive state has its own lightweight decoder head attached to the shared BFM encoder - a small MLP that maps the encoder's latent vector to the target output space: three classes (Low / Medium / High) for workload, a continuous valence-arousal pair for stress, and a continuous scalar for fatigue and situational awareness.
Multiple decoders run in parallel on the same latent representation, enabling simultaneous multi-state monitoring with a single BFM forward pass.