|
Size: 12867
Comment:
|
Size: 12036
Comment:
|
| Deletions are marked like this. | Additions are marked like this. |
| Line 4: | Line 4: |
| The Brainstorm window includes a graphical batching interface. With the two tabs Process1 and Process2 in the lower part of the window, you can easily select files from the database explorer and assemble a processing pipeline. Most of the operations available in the interface can also be executed this way, including everything we've been doing with Brainstorm so far. | The Brainstorm window includes a graphical batching interface. With the two tabs Process1 and Process2 in the lower part of the window, you can select files from the database explorer and assemble a processing pipeline. Most of the operations available in the interface can also be executed this way, including everything we've been doing with Brainstorm so far. |
| Line 6: | Line 6: |
| On the other hand, some features are only available this way. It is the case for the frequency filters we will need for the pre-processing of our auditory recordings. This tutorial is a parenthesis to explain how to select files and run processes on them, the cleaning of the recordings will resume in the next tutorial. | On the other hand, some features are only available this way. It is the case for the frequency filters we will need for the pre-processing of our auditory recordings. This tutorial is a parenthesis to explain how to select files and run processes, we will resume with the cleaning of the recordings in the next tutorial. |
| Line 13: | Line 13: |
| First thing to do is to define the files you are going to process. This is done easily by picking files or folders in the database explorer and dropping them in the empty list of the Process1 tab. | * Try to drag and drop in Process1 all the nodes you currently have in your database explorer. * You will see that it accepts all the folders and all the recordings, but not the channel files. * When you add a new node, the interface evaluates the number of files of the selected type that each of them contain. The number in the brackets next to each node represents the number of data files that where found in them. * On top of the list, a comment shows the total number of files that are currently selected. <<BR>><<BR>> {{attachment:process1_example.gif}} * The buttons on the left side allow you to select what type of file you want to process: Recordings, sources, time-frequency, other. When you select another button, all the counts are updated to reflect the number of files of the selected type that are found for each node. * Right now, if you select another file type, it would show only "0" everywhere because there are no sources or time-frequency decompositions available in the database yet. <<BR>><<BR>> {{attachment:process1_sources.gif||height="159",width="464"}} |
| Line 15: | Line 20: |
| 1. Drag and drop the following nodes in the Process1 list: Right/ERF (recordings), Right (condition), and Subject01 (subject) . {{http://neuroimage.usc.edu/brainstorm/Tutorials/TutProcesses?action=AttachFile&do=get&target=files1.gif|files1.gif|class="attachment"}} * The number in the brackets next to each node represents the number of data files that where found in each of them. The node ERF "contains" only itself (1), Subject01/Right contains ERF and Std files (2), and Subject01 contains 2 conditions x 2 recordings (4). * The total number of files, ie. the sum of all those values, appears in the title of the panel "Files to process [7]". 1. The buttons on the left side allow you to select what type of file you want to process: Recordings, sources, time-frequency, other. Now select the second button "Sources". All the counts are updated and now reflect the number of sources files that are found for each node. . {{http://neuroimage.usc.edu/brainstorm/Tutorials/TutProcesses?action=AttachFile&do=get&target=files2.gif|files2.gif|class="attachment"}} 1. If you select the third button "Time-frequency", you would see "0" everywhere because there are no time-frequency decompositions in the database yet. . {{http://neuroimage.usc.edu/brainstorm/Tutorials/TutProcesses?action=AttachFile&do=get&target=files3.gif|files3.gif|class="attachment"}} 1. Now clear the list from all the files. You may either right-click on the list (popup menu ''Clear list''), or select all the nodes (holding ''Shift ''or ''Crtl ''key) and then press the ''Delete ''key. 1. Select both files Left/ERF and Right/ERF in the tree (holding ''Ctrl ''key), and put the in Process list. We are going to apply some functions on those two files. You cannot distinguish them after they are dropped in the list, because they are both referred as "ERP". If at some point you need to know what is in the list, just leave you mouse over a node for a few seconds, and a tooltip would give you information about it. Just like in the database explorer. . {{http://neuroimage.usc.edu/brainstorm/Tutorials/TutProcesses?action=AttachFile&do=get&target=files4.gif|files4.gif|class="attachment"}} |
* To remove files from the Process1 list: * Select the nodes to remove (holding Shift or Ctrl key) and press the '''Delete''' key. * Right-click on the list > '''Clear list''' |
| Line 27: | Line 24: |
| = Selecting files in the database = ''Authors: Francois Tadel'' |
== Filter by name or comment == When you have lots of files in a folder, like multiple source reconstructions or time-frequency files for each trial, it is difficult to grab just the ones you are interested in. After selecting your folders in the Process1 box, you can refine the selection with the '''Filter search box''' at the bottom-right corner of the window. |
| Line 30: | Line 27: |
| When you have lots of files in a folder, like multiple source reconstructions of time-frequency files for each trial, it is difficult to grab just the ones you are interested in. Two mechanisms can help you: the Filter box and the process "File > Select files with tag". | * The example below shows how select the data file corresponding to the noise recordings: by typing "Noise" in the search box and selecting the option "Search file names". We cannot perform the search "by comment" because all the data files have the same comment "Link to raw file".<<BR>><<BR>> {{attachment:process1_search.gif||height="148",width="436"}} |
| Line 32: | Line 29: |
| ==== Filter text box ==== You want to select only the averages in the data files in the protocol TutorialRaw. Drag and drop all your files there, then use the Filter search box at the bottom-right of the Brainstorm window. Type the string "Avg" to keep only the files that contain this string. {{http://neuroimage.usc.edu/brainstorm/SelectFiles?action=AttachFile&do=get&target=allFiles.gif|allFiles.gif|class="attachment"}} |
* Reminder: To see the file name corresponding to a node in the database, leave your mouse over it for a few seconds. You can do this both in the database explorer and the Process1 list.<<BR>><<BR>> {{attachment:process_filename.gif||height="148",width="490"}} |
| Line 39: | Line 33: |
| * '''Search comments''': Look for the string in the comments of the files, ie. what is displayed in the database explorer to represent them (the .Comment field) * '''Search file names''': Look for the string in the full file names (including their relative path) * '''Select files''': Only the files that contain the string are selected * '''Exclude files''': Only the files that DO NOT contain the string are selected * '''Reset filters''': Removes the current file filters applied on Process1 and Process2 * '''Case insensitive''': Note that the search is not sensitive to case. If you try to select the constrained sources using the string "Constr" in the comments, it would return all the files, because this matches at the same time "Constr" and "Unconstr". But you can do the opposite instead: exclude the files that contain "Unconstr". |
* '''Search comments''': Look for the string in the comments of the files, ie. what is displayed in the database explorer to represent them (the .Comment field). * '''Search file names''': Look for the string in the full file names (including their relative path). * '''Select files''': Only the files that contain the string are selected. * '''Exclude files''': Only the files that DO NOT contain the string are selected. * '''Reset filters''': Removes the current file filters applied on Process1 and Process2. * '''Case insensitive''': Note that the search is not sensitive to case. |
| Line 46: | Line 40: |
| ==== Process: Select files with tag ==== | == Running a process == * Clear the file list and the search filters. * Select the Link to raw file for run '''AEF#01'''. * Click on the '''[Run]''' button at the bottom-left corner of the Process1 tab. * The '''Pipeline editor''' window appears. You can use it to create an analysis pipeline, ie. a list of process that are applied on the selected files one after the other. The first button in the toolbar shows the list of processes that are currently available. If you click on a menu, it's added to the list. * Some menus appear in grey. This means that they are not meant to be applied to the type of data that you have in input, or at the end of the current pipeline. * In the current example, we have a file with the type "continuous raw recordings", so we have access mostly to menus to manipulate event markers, run cleaning procedures and import data blocks. You can recognize a few operations that we execturd in the previous tutorials: "Event > Read from channel" and "Event > Detect analog triggers". <<BR>><<BR>> {{attachment:process1_addprocess.gif||height="480",width="473"}} * When you select a process, a list of options specific to this process is shown in the window. * To delete a process: Select it and press the '''Delete''' key, or use the '''[X]''' button in the toolbar. 1. With the "up arrow" and "down arrow" buttons in the toolbar, you can move up/down a process in the pipeline. 1. Now add the following processes, and set their options: * '''Pre-process > Band-pass filter''': 2Hz - 30Hz * In some processes, you can specify the type(s) of sensors on which you want to apply the process. This way you can for instance apply different filters on the EEG and the MEG, if you have both in the same files. * '''Extract > Extract time''': 40.00ms - 49.60ms, overwrite initial file * This will extract from each file a small time window around the main response peak. * Selecting the overwrite option will replace the previous file (bandpass), with the output of this process (bandpass+extract). This option is usually unselected for the first process in the list, then selected automatically. * '''Average > Average over time''': Overwrite initial file * Compute the average over this small time window. {{http://neuroimage.usc.edu/brainstorm/Tutorials/TutProcesses?action=AttachFile&do=get&target=pipeline2.gif|pipeline2.gif|class="attachment"}} 1. Save your pipeline: Click on the last button in the toolbar > Save > New... > Type "process_avg45". == Creating a pipeline == === Saving/exporting a pipeline === The last button in the the toolbar offers a list of menus to save, load and export the pipelines. . {{http://neuroimage.usc.edu/brainstorm/Tutorials/TutProcesses?action=AttachFile&do=get&target=pipeline3.gif|pipeline3.gif|class="attachment"}} * '''Load''': List of processes that are saved in the user preferences * '''Load from file''': Import a pipeline from a pipeline_...mat file (previously saved with the menu "Save as Matlab matrix") * '''Save''': Save the pipeline in the user preferences, to be able to access it really fast after * '''Save as Matlab matrix''': Exports the pipeline for a Matlab structure in a .mat file. Allows different users to exchange their analysis pipelines (or a single user between different computers). * '''Generate .m script''': This option generates a human-readable Matlab script that can be re-used for other purposes or modified. * '''Delete''': Remove a pipeline that is saved in the user preferences. * '''Reset options''': Brainstorm saves automatically for each user the options of all the processes. This menu removes all the saved options, and set them back to the default values. Here is the Matlab script that is generated automatically for this pipeline. . {{http://neuroimage.usc.edu/brainstorm/Tutorials/TutProcesses?action=AttachFile&do=get&target=script.gif|script.gif|class="attachment"}} . Click on Ok, in the pipeline window. After a few seconds, you will see two new files in the database, and the "Report viewer" window. == Process: Select files with tag == |
| Line 58: | Line 96: |
| ==== How to control the output file names ==== | == How to control the output file names == |
| Line 64: | Line 102: |
== Creating a pipeline == === List of processes === 1. Click on Run. The Process selection window appears, with which you can create an analysis pipeline (ie. a list of process that are applied on the selected files one after the other). The first button in the toolbar shows the list of processed that are currently available. If you click on a menu, it's added to the list. . {{http://neuroimage.usc.edu/brainstorm/Tutorials/TutProcesses?action=AttachFile&do=get&target=pipeline1.gif|pipeline1.gif|class="attachment"}} 1. Some menus appear in grey (example: Sources > Spatial smoothing). This means that they are not meant to be applied to the type of data that you have in input, or at the end of the current pipeline. The "spatial smoothing" process may only be run on source files. 1. When you select a process, a list of options specific to this process is shown in the window. * To delete a process: Select it and press the ''Delete'' key, or the big cross in the toolbar. * With the "up arrow" and "down arrow" buttons in the toolbar, you can move up/down a process in the pipeline. 1. Now add the following processes, and set their options: * '''Pre-process > Band-pass filter''': 2Hz - 30Hz * In some processes, you can specify the type(s) of sensors on which you want to apply the process. This way you can for instance apply different filters on the EEG and the MEG, if you have both in the same files. * '''Extract > Extract time''': 40.00ms - 49.60ms, overwrite initial file * This will extract from each file a small time window around the main response peak. * Selecting the overwrite option will replace the previous file (bandpass), with the output of this process (bandpass+extract). This option is usually unselected for the first process in the list, then selected automatically. * '''Average > Average over time''': Overwrite initial file * Compute the average over this small time window. {{http://neuroimage.usc.edu/brainstorm/Tutorials/TutProcesses?action=AttachFile&do=get&target=pipeline2.gif|pipeline2.gif|class="attachment"}} 1. Save your pipeline: Click on the last button in the toolbar > Save > New... > Type "process_avg45". === Saving/exporting a pipeline === The last button in the the toolbar offers a list of menus to save, load and export the pipelines. . {{http://neuroimage.usc.edu/brainstorm/Tutorials/TutProcesses?action=AttachFile&do=get&target=pipeline3.gif|pipeline3.gif|class="attachment"}} * '''Load''': List of processes that are saved in the user preferences * '''Load from file''': Import a pipeline from a pipeline_...mat file (previously saved with the menu "Save as Matlab matrix") * '''Save''': Save the pipeline in the user preferences, to be able to access it really fast after * '''Save as Matlab matrix''': Exports the pipeline for a Matlab structure in a .mat file. Allows different users to exchange their analysis pipelines (or a single user between different computers). * '''Generate .m script''': This option generates a human-readable Matlab script that can be re-used for other purposes or modified. * '''Delete''': Remove a pipeline that is saved in the user preferences. * '''Reset options''': Brainstorm saves automatically for each user the options of all the processes. This menu removes all the saved options, and set them back to the default values. Here is the Matlab script that is generated automatically for this pipeline. . {{http://neuroimage.usc.edu/brainstorm/Tutorials/TutProcesses?action=AttachFile&do=get&target=script.gif|script.gif|class="attachment"}} . Click on Ok, in the pipeline window. After a few seconds, you will see two new files in the database, and the "Report viewer" window. |
Tutorial 9: Select files and run processes
Authors: Francois Tadel, Elizabeth Bock, Sylvain Baillet
The Brainstorm window includes a graphical batching interface. With the two tabs Process1 and Process2 in the lower part of the window, you can select files from the database explorer and assemble a processing pipeline. Most of the operations available in the interface can also be executed this way, including everything we've been doing with Brainstorm so far.
On the other hand, some features are only available this way. It is the case for the frequency filters we will need for the pre-processing of our auditory recordings. This tutorial is a parenthesis to explain how to select files and run processes, we will resume with the cleaning of the recordings in the next tutorial.
Contents
Selecting files to process
The tab Process1 contains a empty box in which you can drag and drop any number of files or folders from the database explorer. The easier way to understand how it works is to try it.
- Try to drag and drop in Process1 all the nodes you currently have in your database explorer.
- You will see that it accepts all the folders and all the recordings, but not the channel files.
- When you add a new node, the interface evaluates the number of files of the selected type that each of them contain. The number in the brackets next to each node represents the number of data files that where found in them.
On top of the list, a comment shows the total number of files that are currently selected.
- The buttons on the left side allow you to select what type of file you want to process: Recordings, sources, time-frequency, other. When you select another button, all the counts are updated to reflect the number of files of the selected type that are found for each node.
Right now, if you select another file type, it would show only "0" everywhere because there are no sources or time-frequency decompositions available in the database yet.
- To remove files from the Process1 list:
Select the nodes to remove (holding Shift or Ctrl key) and press the Delete key.
Right-click on the list > Clear list
Filter by name or comment
When you have lots of files in a folder, like multiple source reconstructions or time-frequency files for each trial, it is difficult to grab just the ones you are interested in. After selecting your folders in the Process1 box, you can refine the selection with the Filter search box at the bottom-right corner of the window.
The example below shows how select the data file corresponding to the noise recordings: by typing "Noise" in the search box and selecting the option "Search file names". We cannot perform the search "by comment" because all the data files have the same comment "Link to raw file".
Reminder: To see the file name corresponding to a node in the database, leave your mouse over it for a few seconds. You can do this both in the database explorer and the Process1 list.
The options offered in the Filter menu are:
Search comments: Look for the string in the comments of the files, ie. what is displayed in the database explorer to represent them (the .Comment field).
Search file names: Look for the string in the full file names (including their relative path).
Select files: Only the files that contain the string are selected.
Exclude files: Only the files that DO NOT contain the string are selected.
Reset filters: Removes the current file filters applied on Process1 and Process2.
Case insensitive: Note that the search is not sensitive to case.
Running a process
- Clear the file list and the search filters.
Select the Link to raw file for run AEF#01.
Click on the [Run] button at the bottom-left corner of the Process1 tab.
The Pipeline editor window appears. You can use it to create an analysis pipeline, ie. a list of process that are applied on the selected files one after the other. The first button in the toolbar shows the list of processes that are currently available. If you click on a menu, it's added to the list.
- Some menus appear in grey. This means that they are not meant to be applied to the type of data that you have in input, or at the end of the current pipeline.
In the current example, we have a file with the type "continuous raw recordings", so we have access mostly to menus to manipulate event markers, run cleaning procedures and import data blocks. You can recognize a few operations that we execturd in the previous tutorials: "Event > Read from channel" and "Event > Detect analog triggers".
- When you select a process, a list of options specific to this process is shown in the window.
To delete a process: Select it and press the Delete key, or use the [X] button in the toolbar.
- With the "up arrow" and "down arrow" buttons in the toolbar, you can move up/down a process in the pipeline.
- Now add the following processes, and set their options:
Pre-process > Band-pass filter: 2Hz - 30Hz
- In some processes, you can specify the type(s) of sensors on which you want to apply the process. This way you can for instance apply different filters on the EEG and the MEG, if you have both in the same files.
Extract > Extract time: 40.00ms - 49.60ms, overwrite initial file
- This will extract from each file a small time window around the main response peak.
- Selecting the overwrite option will replace the previous file (bandpass), with the output of this process (bandpass+extract). This option is usually unselected for the first process in the list, then selected automatically.
Average > Average over time: Overwrite initial file
- Compute the average over this small time window.
Save your pipeline: Click on the last button in the toolbar > Save > New... > Type "process_avg45".
Creating a pipeline
Saving/exporting a pipeline
The last button in the the toolbar offers a list of menus to save, load and export the pipelines.
Load: List of processes that are saved in the user preferences
Load from file: Import a pipeline from a pipeline_...mat file (previously saved with the menu "Save as Matlab matrix")
Save: Save the pipeline in the user preferences, to be able to access it really fast after
Save as Matlab matrix: Exports the pipeline for a Matlab structure in a .mat file. Allows different users to exchange their analysis pipelines (or a single user between different computers).
Generate .m script: This option generates a human-readable Matlab script that can be re-used for other purposes or modified.
Delete: Remove a pipeline that is saved in the user preferences.
Reset options: Brainstorm saves automatically for each user the options of all the processes. This menu removes all the saved options, and set them back to the default values.
Here is the Matlab script that is generated automatically for this pipeline.
Click on Ok, in the pipeline window. After a few seconds, you will see two new files in the database, and the "Report viewer" window.
Process: Select files with tag
You can get to the exact same result by adding the process "File > Select files with tag" before the process you want to execute, to keep only a subset of the files that were placed in the Process1 list. It is less convenient in interactive mode because you don't see immediately the effect of your file filter, but it can be very useful when writing scripts.
To see the list of files that was selected by the process, you can open the report viewer (File > Report viewer). This lists the input files (all the files) and the final files (the selected files), with a summary of how many files where selected on the top.
After
This process is useless if it is not followed immediately by another process that would do something with the selected files. An application can be the deletion of some files, by adding the process "File > Delete files".
How to control the output file names
If you are running two processes with different parameters but that produce exactly the same file names and file comments, you wouldn't be able to select them with this process. But immediately after calling any process, you can add the process "File > Add tag" to tag one specific set of files, so that you can re-select them easily later.
Example: you run twice the time-frequency decomposition with different options on the same files, tag the files after calculating them with different tags.
Report viewer
Each time the pipeline editor is used to executed to run a list of processes, a report is generated and saved in the user home folder (/home/username/reports/). The report viewer shows as an HTML page some of the information saved in this report structure: the date and duration of execution, the list of processes, the input and output files. It reports all the warning and errors that happen during the execution.
The report viewer does not necessarily appear automatically at the end of the last process: it is shown only when more than one processes were executed, or when any of the processes returned an error or a warning.
When running processes manually from a script, the calls bst_report(Start, Save, Open) explicitly indicate when the logging of the events should start and stop.
You can add images to the reports for quality control using the process "File > Save snapshot".
After you close the report window, you can re-open the last report with the main menu of the Brainstorm window: File > Report viewer.
With the buttons in the toolbar, you can go back to the previous reports saved from the same protocol.
PROCESSES AND PLUG-INS
The available processes are organized in a plug-in structure. Any Matlab script that is added to the plug-in folder (brainstorm3/toolbox/process/functions/) and has the right format will be automatically detected and made available in the GUI. This mechanism makes the contribution from other developers to Brainstorm very easy.
REPORT: OTHER TUTORIALS