6406
Comment:
|
11289
|
Deletions are marked like this. | Additions are marked like this. |
Line 1: | Line 1: |
= Tutorial 20: Advanced scripting = | = Tutorial 28: Scripting = '''[TUTORIAL UNDER DEVELOPMENT: NOT READY FOR PUBLIC USE] ''' |
Line 3: | Line 5: |
The previous tutorials explained how to use Brainstorm in an interactive way to process one subject with two acquisition runs. In the context of a typical neuroimaging study, you may have tens or hundreds of subjects to process in the same way, it is unrealistic to do everything manually. Some parts of the analysis can be processed in batches with no direct supervision, others require more attention. This tutorial introduces tools and tricks that will help you assemble an efficient analysis pipeline. Requirements: You need a license for the Matlab environment in order to use these tools and execute custom scripts. If you are running the compiled version of Brainstorm with the MCR library, the only custom code you can run is through the menu File > Matlab console and the process "Run Matlab command". |
|
Line 6: | Line 12: |
= From CTF = The main window includes a graphical batching interface that directly benefits from the database explorer: files are organized as a tree of subjects and conditions, and simple drag-and-drop operations readily select files for subsequent batch processing. Most of the Brainstorm features are available through this interface, including pre-processing of the recordings, averaging, time-frequency decompositions, and computing statistics. A full analysis pipeline can be created in a few minutes, saved in the user’s preferences and reloaded in one click, executed directly or exported as a Matlab script. |
== Starting a new script == The easiest way to get started with a new Brainstorm script is to use the script generator, already introduced in the tutorial [[http://neuroimage.usc.edu/brainstorm/Tutorials/PipelineEditor#Saving_a_pipeline|Select files and run processes]]. Select some files in the Process1 or Process2 tabs, select a list of processes, and use the menu '''Generate .m script'''. The example below should work on the the protocol "TutorialIntroduction" created during the introduction tutorials. |
Line 9: | Line 15: |
The available processes are organized in a plug-in structure. Any Matlab script that is added to the plug-in folder (brainstorm3/toolbox/process/functions/) and has the right format will be automatically detected and made available in the GUI. This mechanism makes the contribution from other developers to Brainstorm very easy. | * In the Process1 tab, leave the selection box empty and click on [Run]. Instead of selecting the files from the Brainstorm interface, we will select them directly from the database using a script. * Select process '''File > Select files: Recordings''': <<BR>>Subject='''Subject01''', Condition='''[Empty]''', File comment='''Avg: deviant''' (the space is important). <<BR>>This will select the averages of the deviant condition for both runs (total of 2 files).<<BR>><<BR>> [SCREEN CAPTURE] * Add process '''Pre-process > Band-pass filter''': Lower cutoff='''0Hz''', Upper cutoff='''30Hz''', Mirror. <<BR>> '''Add process File > Save snapshot''': Recordings time series, Sensor type=MEG. <<BR>>This will apply a low-pass filter at 30Hz and save a screen capture of the signals in the report. <<BR>><<BR>> [SCREEN CAPTURE] * Do not run the pipeline, instead select the menu '''Generate .m script'''. It saves a new .m file and opens it in the Matlab editor. Close the pipeline editor window and look at the script.<<BR>><<BR>> [SCREEN CAPTURE x2] |
Line 11: | Line 20: |
== Creating a pipeline == | |
Line 13: | Line 21: |
=== List of processes === 1. Click on Run. The Process selection window appears, with which you can create an analysis pipeline (ie. a list of process that are applied on the selected files one after the other). The first button in the toolbar shows the list of processed that are currently available. If you click on a menu, it's added to the list. |
|
Line 16: | Line 22: |
{{http://neuroimage.usc.edu/brainstorm/Tutorials/TutProcesses?action=AttachFile&do=get&target=pipeline1.gif|pipeline1.gif|class="attachment"}} 1. Some menus appear in grey (example: Sources > Spatial smoothing). This means that they are not meant to be applied to the type of data that you have in input, or at the end of the current pipeline. The "spatial smoothing" process may only be run on source files. 1. When you select a process, a list of options specific to this process is shown in the window. * To delete a process: Select it and press the ''Delete'' key, or the big cross in the toolbar. * With the "up arrow" and "down arrow" buttons in the toolbar, you can move up/down a process in the pipeline. 1. Now add the following processes, and set their options: * '''Pre-process > Band-pass filter''': 2Hz - 30Hz * In some processes, you can specify the type(s) of sensors on which you want to apply the process. This way you can for instance apply different filters on the EEG and the MEG, if you have both in the same files. * '''Extract > Extract time''': 40.00ms - 49.60ms, overwrite initial file * This will extract from each file a small time window around the main response peak. * Selecting the overwrite option will replace the previous file (bandpass), with the output of this process (bandpass+extract). This option is usually unselected for the first process in the list, then selected automatically. * '''Average > Average over time''': Overwrite initial file * Compute the average over this small time window. |
[SCREEN CAPTURE] |
Line 30: | Line 24: |
{{http://neuroimage.usc.edu/brainstorm/Tutorials/TutProcesses?action=AttachFile&do=get&target=pipeline2.gif|pipeline2.gif|class="attachment"}} 1. Save your pipeline: Click on the last button in the toolbar > Save > New... > Type "process_avg45". |
Starting Brainstorm |
Line 33: | Line 26: |
=== Saving/exporting a pipeline === The last button in the the toolbar offers a list of menus to save, load and export the pipelines. |
. - gui / nogui - selecting protocol - delete existing protocol |
Line 36: | Line 28: |
. {{http://neuroimage.usc.edu/brainstorm/Tutorials/TutProcesses?action=AttachFile&do=get&target=pipeline3.gif|pipeline3.gif|class="attachment"}} | Selecting files |
Line 38: | Line 30: |
* '''Load''': List of processes that are saved in the user preferences * '''Load from file''': Import a pipeline from a pipeline_...mat file (previously saved with the menu "Save as Matlab matrix") * '''Save''': Save the pipeline in the user preferences, to be able to access it really fast after * '''Save as Matlab matrix''': Exports the pipeline for a Matlab structure in a .mat file. Allows different users to exchange their analysis pipelines (or a single user between different computers). * '''Generate .m script''': This option generates a human-readable Matlab script that can be re-used for other purposes or modified. * '''Delete''': Remove a pipeline that is saved in the user preferences. * '''Reset options''': Brainstorm saves automatically for each user the options of all the processes. This menu removes all the saved options, and set them back to the default values. |
- Inputs / outputs |
Line 46: | Line 32: |
Here is the Matlab script that is generated automatically for this pipeline. | - Select processes |
Line 48: | Line 34: |
. {{http://neuroimage.usc.edu/brainstorm/Tutorials/TutProcesses?action=AttachFile&do=get&target=script.gif|script.gif|class="attachment"}} . |
- Adding tags to help with the file selection later |
Line 51: | Line 36: |
Click on Ok, in the pipeline window. After a few seconds, you will see two new files in the database, and the "Report viewer" window. | == File manipulation == * Modify a structure manually: Export to Matlab/Import from Matlab * File manipulation: file_short, file_fullpath, in_bst_*... * Documentation of all file structures: point at the appropriate tutorials * Select files from the database (with bst_get and processes) |
Line 53: | Line 42: |
=== Report viewer === Each time the pipeline editor is used to executed to run a list of processes, a report is generated and saved in the user home folder (/home/username/reports/). The report viewer shows as an HTML page some of the information saved in this report structure: the date and duration of execution, the list of processes, the input and output files. It reports all the warning and errors that happen during the execution. |
== Loop over subject and runs == Creating loops is not supported yet by the script generator, but relatively easy to do from a script without having to know too much about Matlab programming. |
Line 56: | Line 45: |
The report viewer does not necessarily appear automatically at the end of the last process: it is shown only when more than one processes were executed, or when any of the processes returned an error or a warning. | 1) Fill the cell array SubjectNames with all your subjects names, with the same dimensions as the list of input raw files (sFiles) |
Line 58: | Line 47: |
When running processes manually from a script, the calls bst_report(Start, Save, Open) explicitly indicate when the logging of the events should start and stop. | . 2) Add a "for" loop that includes all the bst_process() calls (leave the bst_report() calls and input definition outside) 3) Inside the loop, replace SubjectNames with SubjectNames{i} and sFiles with sFiles(i) |
Line 60: | Line 50: |
You can add images to the reports for quality control using the process "File > Save snapshot". | == How to process many subjects == This section proposes a standard workflow for processing a full group study with Brainstorm. It contains the same steps of analysis as the introduction tutorials, but separating what can be done automatically from what should be done manually. This workflow can be adapted to most ERP studies (stimulus-based). |
Line 62: | Line 53: |
{{http://neuroimage.usc.edu/brainstorm/Tutorials/TutProcesses?action=AttachFile&do=get&target=report.gif|output.gif|class="attachment"}} | * '''Prototype''': Start by processing one or two subjects completely '''interactively''' (exactly like in the introduction tutorials). Use the few pilot subjects that you have for your study to prototype the analysis pipeline and check manually all the intermediate stages. Take notes of what you're doing along the way, so that you can later write a script that reproduces the same operations. * '''Anatomical fiducials''': Set NAS/LPA/RPA and compute the MNI transformation for each subject. * '''Segmentation''': Run FreeSurfer/BrainSuite to get surfaces and atlases for all the subjects. * '''File > Batch MRI fiducials''': This menu prompts for the selection of the fiducials for all the subjects and saves a file __fiducials.m__ in each segmentation folder. You will not have to redo this even if you have to start over your analysis from the beginning. * '''Script''': Write a loop that calls the process "Import anatomy folder" for all the subjects. * '''Alternatives''': Create and import the subjects one by one and set the fiducials at the import time. Or use the default anatomy for all the subjects (or use [[Tutorials/TutWarping|warped templates]]). |
Line 64: | Line 60: |
After you close the report window, you can re-open the last report with the main menu of the Brainstorm window: '''File > Report viewer'''. | * '''Script #1''': Pre-processing: Loop on the subjects and the acquisition runs. * '''Create link to raw files''': Link the subject and noise recordings to the database. * '''Event markers''': Read and group triggers from digital and analog channel, fix stimulation delays * '''Evaluation''': Power spectrum density of the recordings to evaluate their quality. * '''Pre-processing''': Notch filter, sinusoid removal, band-pass filter. * '''Evaluation''': Power spectrum density of the recordings to make sure the filters worked well. * '''Cleanup''': Delete the links to the original files (the filtered ones are copied in the database). * '''Detect artifacts''': Detect heartbeats, Detect eye blinks, Remove simultaneous. * '''Compute SSP''': Heartbeats, Blinks (this selects the first component of each decomposition) * '''Compute ICA''': If you have some artifacts you'd like to remove with ICA (no default selection). * '''Screenshots''': Check the MRI/sensors registration, PSD before and after corrections, SSP. * '''Export the report''': One report per subject, or one report for all the subjects, saved in HTML. * '''Manual inspection #1''': * '''Check the reports''': Information messages (number of events, errors and warnings) and screen captures (registration problems, obvious noisy channels, incorrect SSP topographies). * '''Mark bad channels''': Open the recordings, select the channels and mark them as bad. Or use the process "Set bad channels" to mark the same bad channels in multiple files. * '''Fix the SSP/ICA''': For the suspicious runs: Open the file, adjust the list of blink and cardiac events, remove and recompute the SSP decompositions, manually select the components. * '''Detect other artifacts''': Run the process on all the runs of all the subjects at once (select all the files in Process1 and run the process, or generate the equivalent script). * '''Mark bad segments''': Review the artifacts detected in 1-7Hz and 40-240Hz, keep only the ones you really want to remove, then mark the event categories as bad. Review quickly the rest of the file and check that there are no other important artifacts. * '''Additional SSP''': If you find one type of artifact that repeats (typically saccades and SQUID jumps), you can create additional SSP projectors, either with the process "SSP: Generic" or directly from a topography figure (right-click on the figure > Snapshot> Use as SSP projector). * '''Script #2''': Subject-level analysis: Epoching, averaging, sources, time-frequency. * '''Importing''': Process "Import MEG/EEG: Events" and "Pre-process > Remove DC offset". * '''Averaging''': Average trials by run, average runs by subject (registration problem in MEG). * '''Noise covariance''': Compute from empty room or resting recordings, copy to other folders. * '''Head model''': Compute for each run, or compute once and copy if the runs are co-registered. * '''Sources''': Compute for each run, average across runs and subjects in source space for MEG. * '''Time-frequency''': Computation with Hilbert transform or Morlet wavelets, then normalize. * '''Screenshots''': Check the quality of all the averages (time series, topographies, sources). * '''Export the report''': One report per subject, or one report for all the subjects, saved in HTML. * '''Manual inspection #2''': * '''Check the reports''': Check the number of epochs imported and averaged in each condition, check the screen capture of the averages (all the primary responses should be clearly visible). * '''Regions of interest''': If not using predefined regions from an atlas, define the scouts on the anatomy of each subject (or on the template and then project them to the subjects). * '''Script #3''': Group analysis, ROI-based analysis, etc. * '''Averaging''': Group averages for the sensor data, the sources and the time-frequency maps. * '''Statistics''': Contrast between conditions or groups of subjects. * '''Regions of interest''': Any operation that involve scouts. |
Line 66: | Line 96: |
With the buttons in the toolbar, you can go back to the previous reports saved from the same protocol. | |
Line 68: | Line 97: |
<<EmbedContent("http://neuroimage.usc.edu/bst/get_prevnext.php?prev=Tutorials/Connectivity")>> | == Final script == The following script from the Brainstorm distribution reproduces the introduction tutorials ("Get started"): '''brainstorm3/toolbox/script/tutorial_introduction.m''' <<HTML(<div style="border:1px solid black; background-color:#EEEEFF; width:720px; height:500px; overflow:scroll; padding:10px; font-family: Consolas,Menlo,Monaco,Lucida Console,Liberation Mono,DejaVu Sans Mono,Bitstream Vera Sans Mono,Courier New,monospace,sans-serif; font-size: 13px; white-space: pre;">)>><<EmbedContent("http://neuroimage.usc.edu/bst/viewcode.php?f=tutorial_introduction.m")>><<HTML(</div >)>> <<BR>>For an example of a script illustrating how to create loops, look at the tutorial [[Tutorials/VisualSingle|MEG visual: single subject]]. '''brainstorm3/toolbox/script/tutorial_visual_single.m''' <<HTML(<div style="border:1px solid black; background-color:#EEEEFF; width:720px; height:500px; overflow:scroll; padding:10px; font-family: Consolas,Menlo,Monaco,Lucida Console,Liberation Mono,DejaVu Sans Mono,Bitstream Vera Sans Mono,Courier New,monospace,sans-serif; font-size: 13px; white-space: pre;">)>><<EmbedContent("http://neuroimage.usc.edu/bst/viewcode.php?f=tutorial_visual_single.m")>><<HTML(</div >)>> == Report viewer == Click on Run to start the script. As this process is taking screen captures, do not use your computer for something else at the same time: if another window covers the Brainstorm figures, it will not capture the right images. At the end, the report viewer is opened to show the status of all the processes, the information messages, the list of input and output files, and the screen captures. The report is saved in your home folder ($home/.brainstorm/reports). If you close this window, you can get it back with the menu File > Report viewer. <<EmbedContent("http://neuroimage.usc.edu/bst/get_prevnext.php?prev=Tutorials/Workflows")>> |
Tutorial 28: Scripting
[TUTORIAL UNDER DEVELOPMENT: NOT READY FOR PUBLIC USE]
Authors: Francois Tadel, Elizabeth Bock, Sylvain Baillet
The previous tutorials explained how to use Brainstorm in an interactive way to process one subject with two acquisition runs. In the context of a typical neuroimaging study, you may have tens or hundreds of subjects to process in the same way, it is unrealistic to do everything manually. Some parts of the analysis can be processed in batches with no direct supervision, others require more attention. This tutorial introduces tools and tricks that will help you assemble an efficient analysis pipeline.
Requirements: You need a license for the Matlab environment in order to use these tools and execute custom scripts. If you are running the compiled version of Brainstorm with the MCR library, the only custom code you can run is through the menu File > Matlab console and the process "Run Matlab command".
Contents
Starting a new script
The easiest way to get started with a new Brainstorm script is to use the script generator, already introduced in the tutorial Select files and run processes. Select some files in the Process1 or Process2 tabs, select a list of processes, and use the menu Generate .m script. The example below should work on the the protocol "TutorialIntroduction" created during the introduction tutorials.
- In the Process1 tab, leave the selection box empty and click on [Run]. Instead of selecting the files from the Brainstorm interface, we will select them directly from the database using a script.
Select process File > Select files: Recordings:
Subject=Subject01, Condition=[Empty], File comment=Avg: deviant (the space is important).
This will select the averages of the deviant condition for both runs (total of 2 files).
[SCREEN CAPTURE]Add process Pre-process > Band-pass filter: Lower cutoff=0Hz, Upper cutoff=30Hz, Mirror.
Add process File > Save snapshot: Recordings time series, Sensor type=MEG.
This will apply a low-pass filter at 30Hz and save a screen capture of the signals in the report.
[SCREEN CAPTURE]Do not run the pipeline, instead select the menu Generate .m script. It saves a new .m file and opens it in the Matlab editor. Close the pipeline editor window and look at the script.
[SCREEN CAPTURE x2]
[SCREEN CAPTURE]
Starting Brainstorm
- - gui / nogui - selecting protocol - delete existing protocol
Selecting files
- Inputs / outputs
- Select processes
- Adding tags to help with the file selection later
File manipulation
- Modify a structure manually: Export to Matlab/Import from Matlab
- File manipulation: file_short, file_fullpath, in_bst_*...
- Documentation of all file structures: point at the appropriate tutorials
- Select files from the database (with bst_get and processes)
Loop over subject and runs
Creating loops is not supported yet by the script generator, but relatively easy to do from a script without having to know too much about Matlab programming.
1) Fill the cell array SubjectNames with all your subjects names, with the same dimensions as the list of input raw files (sFiles)
- 2) Add a "for" loop that includes all the bst_process() calls (leave the bst_report() calls and input definition outside)
3) Inside the loop, replace SubjectNames with SubjectNames{i} and sFiles with sFiles(i)
How to process many subjects
This section proposes a standard workflow for processing a full group study with Brainstorm. It contains the same steps of analysis as the introduction tutorials, but separating what can be done automatically from what should be done manually. This workflow can be adapted to most ERP studies (stimulus-based).
Prototype: Start by processing one or two subjects completely interactively (exactly like in the introduction tutorials). Use the few pilot subjects that you have for your study to prototype the analysis pipeline and check manually all the intermediate stages. Take notes of what you're doing along the way, so that you can later write a script that reproduces the same operations.
Anatomical fiducials: Set NAS/LPA/RPA and compute the MNI transformation for each subject.
Segmentation: Run FreeSurfer/BrainSuite to get surfaces and atlases for all the subjects.
File > Batch MRI fiducials: This menu prompts for the selection of the fiducials for all the subjects and saves a file fiducials.m in each segmentation folder. You will not have to redo this even if you have to start over your analysis from the beginning.
Script: Write a loop that calls the process "Import anatomy folder" for all the subjects.
Alternatives: Create and import the subjects one by one and set the fiducials at the import time. Or use the default anatomy for all the subjects (or use warped templates).
Script #1: Pre-processing: Loop on the subjects and the acquisition runs.
Create link to raw files: Link the subject and noise recordings to the database.
Event markers: Read and group triggers from digital and analog channel, fix stimulation delays
Evaluation: Power spectrum density of the recordings to evaluate their quality.
Pre-processing: Notch filter, sinusoid removal, band-pass filter.
Evaluation: Power spectrum density of the recordings to make sure the filters worked well.
Cleanup: Delete the links to the original files (the filtered ones are copied in the database).
Detect artifacts: Detect heartbeats, Detect eye blinks, Remove simultaneous.
Compute SSP: Heartbeats, Blinks (this selects the first component of each decomposition)
Compute ICA: If you have some artifacts you'd like to remove with ICA (no default selection).
Screenshots: Check the MRI/sensors registration, PSD before and after corrections, SSP.
Export the report: One report per subject, or one report for all the subjects, saved in HTML.
Manual inspection #1:
Check the reports: Information messages (number of events, errors and warnings) and screen captures (registration problems, obvious noisy channels, incorrect SSP topographies).
Mark bad channels: Open the recordings, select the channels and mark them as bad. Or use the process "Set bad channels" to mark the same bad channels in multiple files.
Fix the SSP/ICA: For the suspicious runs: Open the file, adjust the list of blink and cardiac events, remove and recompute the SSP decompositions, manually select the components.
Detect other artifacts: Run the process on all the runs of all the subjects at once (select all the files in Process1 and run the process, or generate the equivalent script).
Mark bad segments: Review the artifacts detected in 1-7Hz and 40-240Hz, keep only the ones you really want to remove, then mark the event categories as bad. Review quickly the rest of the file and check that there are no other important artifacts.
Additional SSP: If you find one type of artifact that repeats (typically saccades and SQUID jumps), you can create additional SSP projectors, either with the process "SSP: Generic" or directly from a topography figure (right-click on the figure > Snapshot> Use as SSP projector).
Script #2: Subject-level analysis: Epoching, averaging, sources, time-frequency.
Importing: Process "Import MEG/EEG: Events" and "Pre-process > Remove DC offset".
Averaging: Average trials by run, average runs by subject (registration problem in MEG).
Noise covariance: Compute from empty room or resting recordings, copy to other folders.
Head model: Compute for each run, or compute once and copy if the runs are co-registered.
Sources: Compute for each run, average across runs and subjects in source space for MEG.
Time-frequency: Computation with Hilbert transform or Morlet wavelets, then normalize.
Screenshots: Check the quality of all the averages (time series, topographies, sources).
Export the report: One report per subject, or one report for all the subjects, saved in HTML.
Manual inspection #2:
Check the reports: Check the number of epochs imported and averaged in each condition, check the screen capture of the averages (all the primary responses should be clearly visible).
Regions of interest: If not using predefined regions from an atlas, define the scouts on the anatomy of each subject (or on the template and then project them to the subjects).
Script #3: Group analysis, ROI-based analysis, etc.
Averaging: Group averages for the sensor data, the sources and the time-frequency maps.
Statistics: Contrast between conditions or groups of subjects.
Regions of interest: Any operation that involve scouts.
Final script
The following script from the Brainstorm distribution reproduces the introduction tutorials ("Get started"): brainstorm3/toolbox/script/tutorial_introduction.m
For an example of a script illustrating how to create loops, look at the tutorial MEG visual: single subject. brainstorm3/toolbox/script/tutorial_visual_single.m
Report viewer
Click on Run to start the script.
As this process is taking screen captures, do not use your computer for something else at the same time: if another window covers the Brainstorm figures, it will not capture the right images.
At the end, the report viewer is opened to show the status of all the processes, the information messages, the list of input and output files, and the screen captures. The report is saved in your home folder ($home/.brainstorm/reports). If you close this window, you can get it back with the menu File > Report viewer.