Fatigue detection using EEG in industrial environment

Hi All,

I know that my idea is somewhat… ambitious, but as a newcomer to the EEG topic and independent component analisys I would like to ask about your opinion.

So… the general idea is that I would like to build a personal fatigue profile for industrial workers.

human workforce is working on the production line or using the production cells. He or she has specific periodic tasks. According to the endproduct tasks and sub-tasks have a high repetition rate with very specific arm and torso movements.

I record following data:

  • Eye tracking, using the frequency and duration of the blink and gaze estimation for 3D heatmaps.
  • EMG to know what task is the most tiring in terms of muscle activity.
  • ECG combined with body impedance for stress level measuring.
  • EEG for measuring the cognitive load the task represents.

It’s clear that I have to remove movement artifacts, but my main question is:
Is it feaseble if I run the ICA in several rounds to separate the specific artefacts so I can extract specific data from the same EEG recording?

Sorry if it bacame too long, regards, Zoltan

Hi Zoltan. I’m not sure what you mean by running ICA in “several rounds.” Do you mean running ICA once, removing the data, then running ICA on the cleaned data in hopes of further cleaning? This is possible and can sometimes be beneficial for particularly noisy data.

On the other hand, the best fatigue indicator will probably be in the alpha band (according to literature). So it might be beneficial to filter the data, e.g., 5-15 Hz. That will already remove quite a few artifacts, and might give cleaner results.

Otherwise, sounds like an interesting project!

Dear Mike,

What I mean in with “several rounds”: you put your reference electrodes above one of the eyes and on on the temporal on both sides. This gives you the possibility to have a reference signal form to extract the blinking artefact from the cognitive data when you run the ICA. This is the standard procedure. I completely understand.

But…

1: as the subject does not sit still, but does a specific series of moves with his/her hand and torso, the EEG signal will have artefacts due to this hand and torso movements. This hand and torso movements will be measured with EMG as I would like to have a time course data how the task (the series of movements) make the subject physically tired. This also can act as reference data (I assume) to the ICA to clean the EEG data.

2: I intend to use eye tracking, combined with a stereo camera so I can make a 3D gaze heat map that I can use in the digital twin of the real production environment -with a 1:1 relation- to analyse the concentration/mental workload of the subject. As I use eye tracking, the frequency and the duration of the blinks can give data how tired the subject mentally is. this is also visible in the raw EEG data.

My assumption: if the blinks are clearly identified because of the reference electrodes, I can extract the frequency and the duration of the blinks as useful data, so I don’t have to use computer vision to get this data.

round 1: I first run ICA I can clean the raw EEG data to have efficient data for ERP, focusing on brain data. Classic cleaning by the book, I save the data.

round 2: I run a second, differently weighed ICA so I have optimal weights to extract the blinking frequency and duration as a sufficient data, suppressing all the other data to have a “metronome” to sync the gaze data from the separately recorded data from the eye tracker and 3D stereo cam. I save the data.

round 3: I run a third, differently weighed ICA so I have optimal weights to extract the hand and torso movement so I can have a “metronome” to sync the separately recorded EMG time course data, to the cleaned brain data and the blinking data

This way I’ll have optimal, cleaned and synced data that I can use for build a fatigue profile of the subject, prior to the task.

This data then can be used to optimise the work where collaborative industrial robots and humans work together.

But maybe I’m just overthinking the entire thing as I’m new to cognitive neuroscience. :slight_smile:

assumption2: Eye tracking, blinking data and alpha waves combined together give different aspects of the same fatigueness, but maybe can provide a deeper insight what to optimise in the assembly process, so the subject will have a better performance.

OK, thanks for the clarifications.
1: If you do common average referencing, then there shouldn’t be much (if any) contamination by body movements in the EEG. This should work as long as you have enough (e.g., >31) electrodes.

2: Again, if you have enough electrodes, you don’t need a reference electrode for the eye movements – they should be represented well enough in the EEG data for ICA to isolate them. If you have EOG, then that’s probably better than ICA. So you can use ICA to isolate and remove the eye blink artifacts from the EEG, but you wouldn’t necessarily need ICA to determine eye blinks if you have electrodes around the eyes.

So in short, your “round-1” sounds good. I don’t think you need to do a “round-2” because you might already have good data from the EOG, not to mention that an eye-movement IC might already be present from “round-1”.

For “round-3” do you mean you’d do an ICA only on the body EMG signals? That might be interesting. I don’t think there will be too much contamination in the scalp EEG though.

Blockquote For “round-3” do you mean you’d do an ICA only on the body EMG signals? That might be interesting. I don’t think there will be too much contamination in the scalp EEG though.

Round 3 would be performed on the EEG data as well, just to suppress the EEG specific data, in order to get a “sync signal” which is actually the body movement artefact. This sync signal would be used to align the separated EMG recording to get EEG and EMG in the same “rhythm”.

Blockquote1: If you do common average referencing, then there shouldn’t be much (if any) contamination by body movements in the EEG. This should work as long as you have enough (e.g., >31) electrodes.

I intend to use OpenBCI which is although daisy-chainable, but I do not really know if it is good to go up to 32 channels. Still, OpenBCI is the only affordable option for me up to date as far as I know the market. Maybe there is a different R+D EEG around 1500-2000 USD that handles 32 channels out of the box and can be used as a wearable device on the shop-floor.

Back to your question:

BlockquoteFor “round-3” do you mean you’d do an ICA only on the body EMG signals? That might be interesting. I don’t think there will be too much contamination in the scalp EEG though.

That didn’t go through my mind, but it could clean the EMG signals very efficiently, so I could get a much cleaner EMG data to synchronise with the separately recorded EEG.

Thanks, Zoltan

I think my main advice from this discussion so far is that you should try several different things and see what works best. Data cleaning is often equipment- and experiment-specific.

A good strategy in general is to collect one full dataset, spend a lot of time working with that one dataset, and get a feel for what is likely to be promising and what isn’t. And then start collecting more data once you feel more confident about developing a custom pipeline. I’m happy to give more specific advice as you go along, but because I’ve never done your experiment with your data and your setup, all I can do is offer some vague suggestions and positive encouragement :smiley:

Regarding the electrodes, if you are an engineer or if you are feeling ambitious, you can build your own recording setup for cheap, e.g., this.

I absolutely understand, and these advices and suggestions are a kind of lighting tower for me. It’s my first project with EEG and for every information I get I’m really thankful. Now I have a direction to head at.