Hi,
I am currently performing sources analyses on my data and I am a bit lost in statistics. I basically want to look for differences between two groups during a the retention period of a working memory task (my time window is 500ms long). I used minimum norm, constrained sources. I normalized my subject averages in z-scores. I have 22 participants in my 1st group and 27 participants in my second group.
What settings should I use when performing NON-PARAMETRIC PERMUTATION T-TESTS?
- Equal or unequal variance? (I can’t find how to verify if variance is equal or unequal between groups)
- Number of randomizations? (what is the difference if I use 1000, 2000 …)
Also, if I decide to perform t-test on scouts, how do I calculate the p-value associated with the t-value provided in the output? Is the formula identic to the one used for PARAMETRIC t-tests?
this is the formula for parametric t test : p = 2*(1 - tcdf(abs(t),df)) (with the statistics toolbox)
Thank you!
Hi Aubree,
Non-parametric permutation tests do not make distributional assumptions. Thus, selecting equal or unequal variance will not affect the validity of the procedure, just the power to detect an effect since you are using a different statistic. The effect will probably be minimal. You may choose either equal or unequal variance and I expect mostly similar results.
Regarding the number of randomizations, it depends on the p-value you will eventually select for statistical significance. 1000 will suffice assuming you aim for a p<0.05 effect. If you need more conservative thresholds, e.g. p<0.001, then you need a finer estimation of the distribution, so use 10,000 randomizations. The more randomizations you perform, the finer you estimate the tails of the distribution, which are critical for the p-value.
Ideally, you want a non-parametric test to directly give you the p-values. But if you obtain t-values on scouts, the formula p = 2*(1 - tcdf(abs(t),df)) should work fine for a two-sided test. Following the p-value conversion, you may consider using FDR correction to control for multiple comparisons across time.
A good resource to understand non-parametric statistical procedures is here:
Eric Maris, Robert Oostenveld, Nonparametric statistical testing of EEG- and MEG-data, Journal of Neuroscience Methods, Volume 164, Issue 1, 15 August 2007, Pages 177-190, ISSN 0165-0270, https://doi.org/10.1016/j.jneumeth.2007.03.024.
Best,
Dimitrios
1 Like