Non-parametric permutation tests

[JBlanchette]

Hi,

I received this comment from a reviewer on my paper: "In your EEG analyses. you used sample-by sample t-tests, corrected for multiple comparisons only at the electrode, but not at the sample level. This analysis procedure considerably inflates false discovery rate. If you want to stick to a sample-by sample analysis strategy, you should use non-parametric cluster permutation tests, which naturally control for false discovery rate. Alternatively, you should analyze EEG data in selected time windows at selected electrodes."

Here's what we've done: "Brainstorm software© [58] was thus used to statistically compare EEG signals from both intuitive and counterintuitive conditions, and make comparisons on all electrodes, at a p < .01 threshold with a Bonferroni correction for the number of electrodes. Only differences for specific time frames lasting at least 15 ms were considered at this step, in order to validate that the main differences in the signal were indeed localized only in the components of interest, and not on the whole signal. As a second step, top-down comparisons (paired sample t-tests) were run between conditions for the two components of interest. For exhaustivity purposes and since previous research found differences between conditions in various scalp regions [45], we ran comparisons for each main scalp region. We used the average of the electrodes located on a scalp region for both N2 and P3 components [45]."

I'm very new in this field and with Brainstorm, so I'm not sure if the reviewer is talking about our first or second step. Could somebody help me with that please?

Thank you!

[Sylvain]

The Reviewer recommends you perform a spatio-temporal cluster statistics analysis to avoid reporting an uncontrolled number of false positives across time. This approach is described here:
https://neuroimage.usc.edu/brainstorm/Tutorials/Statistics#Example_3:_Cluster-based_correction

[JBlanchette]

Thank you very much for the clarification and the link.

Could you also tell me if the reviewer is refering to the first or second step of the analysis we described?

[JBlanchette]

Hi again,
I tried the analysis you suggested, but I get an error report (see below).

I don't understand because I've got the same number of files in A and B.

Here's what I did:

Thanks again for your help,

[Sylvain]

Make sure the sampling rates are the same for all files (I suppose yes) and that they all display properly as EEG traces, meaning that the data time series are properly recognized by Brainstorm as of EEG type. (simply double click on one of the "raw" files shown in your data tree.)

[Raymundo.Cassani]

The issue is related to the way FieldTrip identify the sensor types.

Be sure, the channels have the right type, and their names are uniform across files.

Can should you share a screenshot of your channel file?
(right-clink on the channel file > Edit channel file)

[JBlanchette]

Thank you so much for your help. Here's the channel file:

[JBlanchette]

Hi,
Instead of the analysis you proposed, I tried the permutation test (see screenshot), and it worked (no error report). First, can you tell me if this makes sense (that it worked with this test), and second, is this test appropriate in my situation?

Thank you very much for your insights,