Tutorial 25: Statistics

[TUTORIAL UNDER DEVELOPMENT: NOT READY FOR PUBLIC USE]

Authors: Francois Tadel, Dimitrios Pantazis, Elizabeth Bock, Sylvain Baillet

Until now we have been computing measures of the brain activity in time or time-frequency domain. We were able to observe clear effects or slight tendencies, but what we were observing was always dependent on an arbitrary amplitude threshold or on the configuration of the colormap. With appropriate statistical tests, we can go beyond these empirical observations and assess what are the significant effects in a more formal way.

We are typically interested in comparing different groups of samples. We want to know what is significantly different in the brain responses of two experimental conditions or two groups of subjects. So we will be essentially estimating differences and testing if these differences are significantly different from zero.

Difference deviant-standard

In this auditory oddball experiment, we can test for the significant differences between the brain response to the deviant beeps and the standard beeps, time sample by time sample.

Before running complicated statistical tests that will take weeks of computation, you can start by checking what the difference of the average responses looks like. If in this difference you observe obvious effects that are clearly not what you are expecting, it's not worth moving forward with finer analysis: either the data is not clean enough or your initial hypothesis is wrong.

We are going to use the Process2 tab, at the bottom of the Brainstorm figure. It works exactly like the Process1 tab but with two lists of input files, referred to as FilesA (left) and FilesB (right).

• In Process2, drag and drop the non-normalized deviant average on the left (FilesA) and the non-normalized standard average on the right (FilesB).

• Run the process "Difference > Difference A-B".
  Select the option "Use absolute values", which will convert the unconstrained source maps (three dipoles at each cortex location) into a flat cortical map by taking the norm of the three dipole orientations before computing the difference.
  
  

• Rename the new file in "Deviant-Standard", double-click on it and explore it in time.
  
  

• We are looking at the difference (Deviant-Standard) so positive/red regions indicate higher activity levels for the deviant beeps, and negative/blue regions higher activity for the standard beeps.

  • Before 50ms: The motor activity in the deviant is probably due to the previous stims.

  • P50: Maybe a stronger response in the primary auditory cortex for the standard condition.

  • MMN (125ms): Stronger response for the deviant (left temporal, inferior frontal).

  • 150ms: Stronger response in the auditory system for the standard condition.

  • 175ms: Stronger response in the motor regions for the standard condition (motor inihibition).

  • After 200ms: Stronger response in the deviant condition.

Difference of means

Another process can compute the average and the difference at the same time. We are going to compute the difference of all the trials from both runs at the sensor level. This is usually not recommended because the subject might have moved between the runs. Averaging the recordings across runs is not accurate but can give a good first approximation, in order to make sure we are on the right tracks.

• In Process2, select all the deviant trials (Files A) and all the standard trials (Files B).

• Run the process "Test > Difference of means", select the option "Arithmetic average".
  
  

• Rename the new file in "Deviant-Standard", double-click on it to display it. The difference deviant-standard does not show anymore the early responses (P50, P100) but emphasizes the difference in the later process (MMN/P200 and P300).
  
  

Parametric Student's t-test

Using a t-test instead of the difference of the two averages, we can reproduce similar results but with a significance level attached to each value.

• In the Process2 tab, select the following files:

  • Files A: All the deviant trials, with the [Process sources] button selected.

  • Files B: All the standard trials, with the [Process sources] button selected.

• Run the process "Test > Student's t-test", Equal variance, Average of absolute value.
  We have to select this last option if we want to do the test on the norm of the three dipole orientations for unconstrained sources. The second option (absolute value of average) is better indicated for constrained source maps, but here it would return one value per orientation, which is then difficult to interpet.
  
  

• Double-click on it for displaying it. With the new tab "Stat" you can control the p-value threshold and the correction you want to apply for multiple comparisons.
  
  

• Set the options in the Stat tab: p-value threshold: 0.01, Multiple comparisons: Uncorrected.
  We can make similar observations than with the difference of means, but without the arbitrary amplitude threshold (this slider is now disabled in the Surface tab). If at a given time point a vertex is red in this view, the mean of the deviant condition is significantly higher than the standard conditions (p<0.01).
  
  

• This approach considers each time sample and each surface vertex separately. This means that we have done Nvertices*Ntime = 15000*361 = 5415000 t-tests. The threshold at p<0.01 controls correctly for false positives at one point but not for the entire cortex. We need to correct the p-values for multiple comparisons. The logic of two types of corrections available in the Stat tab (FDR and Bonferroni) are explained in Bennett et al (2009).

FieldTrip: Cluster-based correction

We have the possibility to call some of the FieldTrip functions from the Brainstorm environment. For this, you need first to install the FieldTrip toolbox on your computer and add it to your Matlab path.

For an introduction to non-parametric cluster-based statistics in FieldTrip, watch this video:

Export to SPM

An alternative to running the statical tests in Brainstorm is to export all the data and compute the tests with an external program (R, Matlab, SPM, etc). Multiple menus exist to export files to external file formats (right-click on a file > File > Export to file).

Two tutorials explain to export data specifically to SPM:

• Export source maps to SPM8 (volume)

• Export source maps to SPM12 (surface)

On the hard drive

Right click one of the first TF file we computed > File > View file contents.

References

• Maris E, Oostendveld R, Nonparametric statistical testing of EEG- and MEG-data
  J Neurosci Methods (2007), 164(1):177-90.

• Pantazis D, Nichols TE, Baillet S, Leahy RM. A comparison of random field theory and permutation methods for the statistical analysis of MEG data, Neuroimage (2005), 25(2):383-94.

• Bennett CM, Wolford GL, Miller MB, The principled control of false positives in neuroimaging
  Soc Cogn Affect Neurosci (2009), 4(4):417-422.

• FieldTrip video: Non-parametric cluster-based statistical testing of MEG/EEG data:
  https://www.youtube.com/watch?v=vOSfabsDUNg

Additional discussions on the forum

• Forum: Multiple comparisons: http://neuroimage.usc.edu/forums/showthread.php?1297

• Forum: Cluster neighborhoods: http://neuroimage.usc.edu/forums/showthread.php?2132

Delete all your experiments

Before moving to the next tutorial, delete all the statistic results you computed in this tutorial. It will make it the database structure less confusing for the following tutorials.

---