Import different parts of total continuous file (raw file) as raw file and apply ICA projectors attained by that total file

Discussions
1

Hi,
I had a long EEG data as raw file for which the ICA projectors were computed. Now, I want to import different parts of this data as raw file and apply the aforementioned projectors on each part, separately. Each part should be located in a separate folder (condition) with a given name for example "concentrate1". I want to do this by scripting. I tried somewhat but it did not work. My codes are:

clear;
clc;

Num_sub=1;
Num_data=2;
TP=1:10000;
ChannelReplace=2;
ChannelAlign=0;
Tolerance = 0;

SubjectNames = {'Subject001'};
sFiles=[];
sFiles = bst_process('CallProcess', 'process_select_files_data', sFiles, [], 'subjectname', SubjectNames{Num_sub});

[sInput, nSignals, iRows] = bst_process('LoadInputFile', sFiles(Num_data).FileName);

DataMat = in_bst(sFiles(Num_data).FileName, [], 0);
ChannelFlag = DataMat.ChannelFlag;

sMat=db_template('datamat');
sMat.F=sInput.Data(:,TP);
sMat.Comment = 'Test';
sMat.ChannelFlag=ChannelFlag;
sMat.Time=sInput.Time(TP);
% % % sMat.DataType='raw';

iStudy = db_add_condition(SubjectNames{Num_sub}, '@rawConcentrate1');
sStudy = bst_get('Study', iStudy);
OutputFile = db_add(iStudy, sMat);
db_set_channel( iStudy, sFiles(Num_data).ChannelFile, ChannelReplace, ChannelAlign, Tolerance);

I would be thankful if anyone could help me to get this job done.

Regards,
Talesh

2

:warning: Never call clear, clear functions, clear classes or clear all while Brainstorm is running.
It will clear variables, functions and classes that are needed by Brainstorm.

Raw data consist of two files, one is the file () in the Brainstorm database, that indicates where the file with the raw recordings is actually. This is the reason a raw files says Link to

To achieve what you describe, it could be simpler to split the long EEG raw data.

  1. Create a .csv file with events (name, beginningTime, duration) for the parts you want to obtain:
segment_01, 1.609, 1.122
segment_02, 2.609, 4.122
segment_03, 5.609, 8.122

  1. Add those events to the long EEG raw data. Plot the file, and in the Record tab, Events subsection: File > Add events from file... choose the .csv file with file format CSV text file, close to save modifications.

  2. Drag-and-drop the long EEG raw to the Process tab and run the process Import > Import recordings > Split raw file and indicate the name of the event for your segments, e.g., segment_01, uncheck Keep segments outside ...
    Run the process. it will create a new raw folder with the data for the indicated segment, it has the same channel file as the long recording.

  3. Repeat step 3 for all the desired segments.

For scripting: Step 2 can be done with the process Events > Import from file

3

Thanks Raymundo,

I will try your proposed simpler method. However, I want to explain my reason for persisting on scripting. I have 78 EEG data (43 controls and 31 athletes) each about 1.5 hours data and with about 20 events. Manually, segmenting (20×78) segments can not be very efficient. Also, I want that each segment has a name corresponding to its event name. Is it possible to extract 20 event data as raw file from total raw file with 20 different names and with projectors of total raw file using scripting?

Thanks in advance,

4

Yes, you can do that with scripting. For each subject:

  1. Create the .csv file for each subject with an script.
  2. Import the .csv file with the process (as script)
  3. Use a FOR loop for all the events in the file, this will create the 20 mini-raw files

In a different reason, why do you want to have 20 mini-raw files per subject?
Why don't you import the EEG long file for the 20 segments of interest, in that way the projectors are applied when imported.

5

One ICA component consists for example emg artifact for one event of eeg where as this component offer eeg data for another event of eeg data. Thus, if I remove this component, it remove emg from that event but it also removes eeg data from many other event data. Hence, it is better to analyse active projectors for each event data separately. Continuous long data offer enough samples for projector estimation but for applying I think it is better to apply on each event data separately to more efficiently remove artifacts with less concern about removing eeg data. For evoked data, it is not appropriate to remove one component from only some epochs because these epochs are averaged and probably getting into source imaging process. However, for continuous events (for example focusing on sable shooting) , before and after fucus training, this can not be a problem when the ICAs are applied on before and after data separately and these two events are analysed separately.
My question about writing script is that what commands of brainstorm with what order should I use to be able to extract raw file of event data from continuous long eeg data? In real, I know how to extract datatype 'recordings' (as my shared code do, thanks the bst community for offering related tutorials by which I could write this code). However, I were confused about datatype 'raw' an currently I didn't know how to script extracting raw event file from long original file ans save them in different condition folders with the name of event. For event file I am able to prepare the event file even in brainstorm.mat event format. However, for dealing with raw file, getting the original raw file and related link, extracting a desired segment as raw file from that original file, saving the new segmented raw file in a new raw condition folder I didn't what should I do and what commands with what orders should I write in matlab script? These are my challenges, else I know how to update the event and time of new segmented raw condition or adding channel file.
At the end, I appreciate it if you could help me to overcome this challenge.

Thanks in advance,

6

Your approach makes sense with those considerations on the data.

Use the process Import > Import recordings > Split raw file, you can convert it into code with one click, check this section of the scripting tutorial. I assumed you were familiar with it as your script above uses bst_process(...

https://neuroimage.usc.edu/brainstorm/Tutorials/Scripting#Starting_a_new_script

7

Thank you a lot.
I know it is unbelievable and ridiculous that I say I read that page for many hours and study the bold face commands and their matlab codes, codes with gray color background in that page, and the example of adding new condition "Script to Test" to deal with my data and write my shared matlab code. However, I didn't pay any attention to the initial parts of this page and the "Generate .m" part and the first paragraph starting with "The easiest way ...".
Your advise would be a great help for me. I will try it during the next days.

The best regards,

8

I performed splitting the raw file and generating script as you explained. However, I could not save the results by a specific name for example @raw_precct. I found that the line 119 of "process_split_raw_file" function is responsible for this name changing (I changed the content of newCondition and it worked). However, I want to change the name in the generated script file to apply "for" loop for all events and subjects. Also, such blind manipulating one line of one code that i did may get me into problems that I currently do not know).
My question is how to assign my desired name to the raw folder of a event through scripting?

9

Renaming raw conditions is not recommended, thus it is not available. But you can change the name once the data is imported. More info of this behaviour here:
Bugfix: Allow renaming rawcondition nodes by rcassani · Pull Request #506 · brainstorm-tools/brainstorm3 · GitHub

About writing the FOR loop, there is not real need to have an specific name, you can always use functions such as contains() or regexp (regular expressions) to identify studies. Take makes the code more flexible.

It is not recommended, nor implemented in Brainstorm. If you really want to do it, create a copy of process_split_raw_file, add a new input to select the name and use it to create the file and condition. Store the modified process in the User Processes dir, it will override the process in Brainstorm.

:warning: As you may have already seen, your script will not be compatible for other users without your modified version of the process.

https://neuroimage.usc.edu/brainstorm/Tutorials/TutUserProcess#Process_folders

10

A post was split to a new topic: PSD for components from ICA