Dear Prof. Cohen,
I have a question concerning source separation for steady state responses in MEG using RESS.
MEG (at least Neuromag systems) have two different sensors types (with two different sensitivity profiles). Does this affect the way we want to calculate our generalized eigendecomposition?
More in detail, would you advise to run GED on both sensors type or would you keep them separate, thus calculating GED first on e.g. magnetometers and then on planar gradiometers?
Kind regards,
Lorenzo
Hi Lorenzo. Please call me Mike 
One way to proceed is to normalize all the sensors (e.g., z-transform over time) and then combine them into the same GED. This is useful because it incorporates all of the information you have across all the sensors. When visualizing the spatial filter forward models, youâll need to separate the gradiometers and magnetometers, thus making two topoplots for each filter.
Hi Mike,
I think I managed to perform RESS on MEG data.
Here some preliminary results obtained on one experimental condition and four participants.
Visual stimuli were presented at 6 Hz, with one every 5 stimuli being an âoddâ stimulus. Thus what we would expect (assuming odd stimuli are processed) is a peak at 1.2 Hz and a peak at 6 Hz. 4 Hz is a frequency of no interest that I used to check for overfitting.
In red the results of RESS on the real data time series.
In black instead are the results of a permutation test where I randomly permute the data time series for 500 times.
RESS nicely separates the 6Hz peak.
4Hz, if we look at the SNR, it looks like an overfitting problem (there was no 4Hz stimulation). This is confirmed by the eigenspectrum, which does not seem to differ between real and permuted data.
1.2 Hz regrettably doesnât reach significance, although we can see maybe a trend in the eigenspectrum⌠maybe with more participantsâŚ
This procedure however, took a lot of time. Few days of computation for only four participants. Permuting the data, filtering and new GED takes a lot of time (306 channels, 5 trials per participant [if we consider only one condition, I have more], each trial lasted 60 seconds).
Do you have any suggestion in this respect?
Do leave-one-out procedures take care of the overfitting problem as, in this case, permutation testing?
Also, would you change filters parameters between 1.2 and 6Hz?
Best,
Lorenzo
https://ibb.co/MkQRwL6
Hi Lorenzo. Looks nice overall. âOverfittingâ is a difficult term here, because there probably is real (non-SSVEP-related) brain activity at 4 Hz. Itâs therefore also useful to look at the topographies of the filters (the eigenvector times the S matrix) â you might find that the 6 Hz component is posterior/visual whereas the 4 Hz component is anterior.
But days to run the analysis!?!?! Yikes, sounds like something went wrong or could be optimized. If you would like to share your code (you can also do this off-forum), then I can take a look and see if something can be improved. In theory, RESS should take tens of seconds to minutes. Unless youâre running this on one of these classics.
1 Like
OK, based on the code you sent me, there are a few issues:
- The loop over channels doesnât permute anything. Also, one of the advantages of eigendecomposition is that it isnât based on any physical locations. So shuffling the channel order doesnât do anything, unless the channel order is different on each trial (but this isnât a good idea, see below).
- Randomly shuffling time points also isnât a great idea. Thatâs changing the temporal autocorrelation structure of the signal, including the spectral structure.
In general, the idea of permutation shuffling is to keep the data as intact as possible, and change only the one key factor that is directly related to the hypothesis. In this case, the null hypothesis is that S=R. Thus, the appropriate way to shuffle is to randomly sort data into S vs. R. Hereâs how I would do that:
- Filter the data in the three bands (SSVEP frequency and the two neighbors). Keep all of the filtered data. This will give you 3x the number of trials.
- Concatenate all filtered signals into one matrix that is chans X time X [3 x trials].
- To create the S matrix, randomly draw from 1/3 of those âtrialsâ. Same for R_lower and R_upper.
- Then repeat the GED.
Notice that youâre not actually re-filtering the data on each iteration. You only need to filter all the data once, but the assignment of a given signal to S and R is randomized 1000 (or whatever) times.
Also keep in mind that for permutation testing, you donât actually need to store the time series or maps on each iteration. You just need the sorted eigenvalues. That should also save computation time.
Hi Mike, thank you for your prompt response. I did as you suggested and is indeed much faster in terms of computational time.
I obtain however some puzzling results.
https://ibb.co/SJ0QmGQ
In black the average of the eigenvalues from the permutation test (500 permutations).
In red the eigenvalues, SNR and power without permutation.
Why would the average of the permuted eigenvalues be higher than that of the data where I donât permute the filtered data from S and R?
hmm, yes that is strange. The null hypothesis values should be around 1, which you can see for the red line after the first few components. My first guess is that the shuffling isnât quite correct. Please double-check that the data are roughly evenly randomly distributed into the S and R matrices. Feel free to email me the code again if youâre unsure.
Hi Lorenzo. Thanks for sending me your code and sample data. Really nice data!
I donât think youâve done anything wrong in the code. With only 5 trials, sorting the data randomly into S and R matrices gives N=10 for k=2 groups. That means that thereâs only 2^9=512 unique ways of grouping the data. But that includes a lot of suboptimal groupings, e.g., when 4/5 single-trial covariance matrices are in the correct S or R matrix.
So in conclusion, I donât think you can do permutation testing with such a small N. Indeed, the estimation error of the empirical p-value is roughly 1/N (ref), which means your p-value cannot be trusted to greater than a precision of .1 
We discussed in the RESS paper several ways of doing statistics, only one of which was based on permutation testing for eigenvalues. Do you have multiple conditions with the same stimulus frequency? Then you can include all trials and look at power differences between conditions.
Hope that all helps,
Mike
hi Mike thank you for your help!
I indeed have multiple conditions with the same stimulus frequency. However, I expect a marginally different topographical distribution of the effects in the different conditions. Wouldnât this be a problem?
Nah, I donât think so. The thing is that you have light stimulation at 1.2 Hz, so no one will be surprised that you get a strong brain response at 1.2 Hz. You donât need a p-value for everything.