Fatigue Assessment using Eyetracking for People with Special Needs

In this thesis, we have shown the feasibility of using eye-tracking to detect mental fatigue during typing using gaze control. Mental fatigue is an emerging problem in the working population, with work transforming from physically to cognitively challenging, as well as in people with neurological disorders, as fatigue affects their communication via augmented systems.

The communication system often utilizes gaze-tracking, on account of the people with neurological disorders having disabilities, resulting in loss of control of muscles, including the muscles for speech. Eye-tracking, which can now be implemented using just a camera, is becoming ubiquitous. Detection of mental fatigue is the first step towards fatigue management and health monitoring.

Eye-tracking allows the recording of various characteristics of the eye movements and pupils, which are affected by the cognitive state of a person, including mental fatigue. First, we collected first- and second-hand qualitative data on the experience of fatigue with people with neurological disorders and special education teachers, respectively.

Based on this data, we reached the goal of the thesis – to investigate mental fatigue. Next, we researched the literature for the eye-based metrics most useful in the detection of mental fatigue. Using these features, we predicted mental fatigue on a continuous scale, with an accuracy 20% higher than baseline.

We also discovered posture to be an important feature for the prediction of mental fatigue. This was corroborated by the special education teachers, who observed posture changes as an effect of fatigue. Finally, we determined a novel method of mental fatigue detection, based on natural eye movement – smooth-pursuit, generated by following an object with the eyes, implemented using an eye-typing task on a tablet.

We concluded that mental fatigue was influenced by expending time and effort on a task, but was not influenced by the time of day. Mental fatigue prediction can be a potential application included in the augmented communication systems, to indicate the fatigue level of the users, as well as in the computers or phones, to reflect the fatigue level of the population without disability.

In summary, this PhD thesis showed that mental fatigue detection and prediction is feasible using eye-tracking data during eye-typing using an eye-tracker in communication systems or a camera from a tablet.