On Figure 34.2_A (Analyzing Time Series Data)

#1

Dear Prof. Cohen,
Hi,
I am about to finish reading your insightful book on analyzing time series data. And I am really grateful that you have shared your knowledge by writing such an easy-to-understand book.

In chapter 34, In the section you are talking about comparing task-related power changes relative to baseline, I found a discrepancy between the method figure 34.2(A) is representing and the method the actual matlab code is using in the code section demarked as figure 34.3 (lines 93-98).
Judging from Fig 34.2, it appears you shift data points around a cut-off point and then you calculate the baseline, values of which vary on each iteration. However, the code on lines 93-98 is comparing the time-shifted signal against predefined, fixed baseline values denoted by “realbaselines”.

Since these two methods produce different results, I was wondering which one is more correct in the context of what book is explaining and how these two methods of permutation are different conceptually? (I feel there is a difference between null hypotheses in the two methods, but I can’t envisage it correctly)

Thanks in advance,
Best regards,
Amir

B.S.

Instead of codes on line 93-98, you can insert these lines and comment or uncomment to see the difference on the final results

for permi=1:n_permutes

% Original code
cutpoint = randsample(2:nTimepoints-diff(baseidx)-2,1);
permuted_vals(permi,:,:) = 10*log10(bsxfun(@rdivide,mean(eegpower(:,[cutpoint:end 1:cutpoint-1],:),3),mean(realbaselines,2)) );


% This is based on fig 34.2 A
%newdat = mean(eegpower(:,[cutpoint:end 1:cutpoint-1],:),3);
%permuted_vals(permi,:,:) = 10*log10(bsxfun(@rdivide, newdat, mean(newdat(:,baseidx(1):baseidx(2)),2) ));

end

#2

Hi Amir. Yes, I see what you mean. I couldn’t say whether I did that intentionally or whether that was a mistake (I wrote the book, and that code, in 2012). As for which one is “correct,” I also don’t know. Sometimes with permutation testing, there is a clear optimal way to shuffle the data; other times, there are several ways to shuffle data. Testing power for significance against a baseline period falls into the latter category, and I’ve seen several different approaches.

You are also correct that the results are slightly different, but results of permutation testing can also differ from run to run. And you can get differences when doing cluster correction vs. extreme-pixel correction. And you can get somewhat different results by using different wavelet parameters. Unfortunately, the complexity of the data coupled with noise and (often) not enough data means that it’s normal to get somewhat different results from different methods.

1 Like
#3

Dear Prof. Cohen,
Thank you very much for your explanations. I totally understand your point.
Best regards,
Amir