Scripting switching back and forth between raw and epoched data

Discussions
1

Hello,

Here's what I need to do on my very large data with scripting:
[link to raw]
Discard a list of channels in each dataset with process_channel_setbad
import --> [data]
Discard overly bad/flat chunks of data with process_detectbad
back to raw --> [link to raw]
Detect blinks with process_evt_detect_eog
Compute and apply SSP based on blinks process_ssp_eog
import --> [data]
...

The order matters to me, since I would like to remove very bad chunks of data before computing SSPs, so they don't contaminate the projectors with noise. The problem as far as I understand is that I need to import and epoch data before detectbad, and can only compute SSPs on raw data (a "link to raw file"). I saw there is a way to click through the back and forth switch, but I do need to script this since I have hundreds of datasets to process in this way.

I tried to use process_import_data_raw, but can't figure out how to pass my input data.

Is there a way to do this?

2

Hi Maximilien,

Thanks for bringing this topic. Indeed, an easier way to do the described processing should be available in Brainstorm. Jumping between imported and raw data seems over complicated, and a second import (of clean data) would be needed.

Process process_detectbad would label as bad the segments of X seconds (user-defined) around every moment time the channel exceeds the threshold. Once the segments are labelled bad, these will not be used to compute the SSP.

In addition, a checkbox in the process (only for raw) would allow to label as bad the entire channel if it has at least one bad segment.

How do you like this implementation? Would it fit on your current workflow?

3

Hi Raymundo,

Thanks, this makes sense to me.
We should either be able to compute SSPs on imported data, or detectbad on raw data. You suggest to adapt detectbad to be able to run on raw data.

What you suggest makes sense but I don't really understand the last point, with the checkbox.

The way I understand it, as soon as process_detectbad marks a bad segment (which applies to all channels), all channels are going to be bad, but I'm sure that's not what you mean.

Thanks!
Max

4

Hi Max,

Now it is possible to run the process_detectbad on raw (continuous) data. From Brainstorm 19-04-2024

image
image385×683 18.3 KB

The analysis is done in segments of length indicated by the user. The process generates events for the bad segments in two ways:

  • Per channel: Bad events are channel wise and last for the segment where the signal was bad (not in the criteria). So when the data is imported, channels that were bad in a segment will be labelled as bad channels in the trials/epochs that correspond to the data from that segment.

  • All channels: Bad events are not channel wise and last for the segment where any channel was bad. So when the data is imported, trials/epochs that correspond to the data from that segment will be marked as bad epoch/trials.

Best,
Raymundo

1 Like
5

Fantastic. I'll be testing it shortly. Thanks!