Trouble running `process_import_data_event` from Python

Discussions
1

Hi Raymundo,

Lately, I have been having problems with the function process_import_data_event.

When I run it in MATLAB it works just fine; the folders are generated and I obtain the sFilesEpochs1 struct. In Python, however, even though it generates the folders the output file is empty ([ ])
Is there a reason behind this issue?

Thank you!

Problems with importing EVENTS from EDF
2

Hi @claudiarodgadeam, I move this question to its own thread as the problem is not with Brainstorm source code, but with the way Python interacts with it.

It is not possible for me to give you an answer with this little information. Please provide relevant information. For example which Python code are you using?

4

Hi Raymundo,

Thanks for your message.

Below is the Python code I am using to import events via Brainstorm using the MATLAB engine:

imported_spikes = self.eng.bst_process(
    'CallProcess', 'process_import_data_event',
    output_eeg, [],
    'subjectname', subject_name,
    'condition',   '',
    'eventname',   eventname_str,
    'timewindow',  matlab.double([]),
    'epochtime',   matlab.double([-0.300, 0.300]),
    'createcond',  1,   # Create a new folder for each matching event
    'ignoreshort', 1,
    'usectfcomp',  1,
    'usessp',      1,
    'freq',        [],
    'baseline',    matlab.double([-0.1, -0.0017]),
    nargout=1)

Here, output_eeg is the output file from process_import_data_raw, and eventname_str is a string containing the event names separated by commas (e.g., "f4, c4").

The function correctly creates the folders corresponding to the different event time instants in Brainstorm, but the returned variable imported_spikes is empty ([]).

After that, my intention is to compute the average of the imported events using:

spikesAvg = self.eng.bst_process(
    'CallProcess', 'process_average',
    imported_spikes, [],
    'avgtype',    3,
    'avg_func',   1,
    'weighted',   0,
    'keepevents', 0,
    nargout=1)

However, since imported_spikes is empty, no average is generated.

So my main issue is that although process_import_data_event appears to run correctly and creates the epochs in Brainstorm, it does not return any output file to Python.

Thank you!

5

Some debugging is needed from your side to understand what is going on.

Like this other issue with the Python engine, my guess is that the trouble lies in casting data from Python to Matlab.

As sanity check, create a Matlab function pipeline_test.m that does what you need:

% Process: Create link to raw files
sFiles = bst_process('CallProcess', 'process_import_data_raw', [], [], ...
...
...
...);

% Process: Import MEG/EEG: Events
sFilesEpochs = bst_process('CallProcess', 'process_import_data_event', sFiles, [], ...
...
...
...);

% Process: Average
sFilesAvg = bst_process('CallProcess', 'process_average', sFilesEpochs, [], ...
...
...
...);

% Print outcome
sFilesAvg

and call with a simple Python script

import matlab.engine
eng = matlab.engine.start_matlab()
res = eng.pipeline_test()
print(res)

If this works as expected the issue is indeed how the arguments for the processes are passed from Python to Matlab. E.g., try to pass all numeric data as float

Also, you can redirect the standard output and standard error, to verify is an error was raised in Matlab:

https://www.mathworks.com/help/matlab/matlab_external/redirect-standard-output-and-error-to-python.html

6

Hello Raymundo,

Thank you for your recommendations.

I did debug the script, and apparently, the function 'process_import_data_event' does create the folders with the different events in it, but the problem is that imported_spikes is returned empty (matlab.double([])). Therefore, as that is empty, no average is generated and the function 'process_average' returns an empty outcome as well.

I tried creating the Matlab function, as you suggested, however, I got the same results. I also tried obtaining all the events files from the folder in brainstorm_db and importing them in 'process_average', but that did not worked either.

Is there something else I could try?

Thank you,

Claudia

7

Yes, you check the different variables (elements) that are in running the script

We know that:
A. All the processes involved in the script are working fine in Brainstorm. Becuase they are used in other pipelines without trouble

B. The approach of a Matlab script failed. So, before testing the calls from Python this must be solved. Did you try to run this script directly on Matlab?

To figure out where the problem in the script is, I would perform each of these actions in order:

  1. In Brainstorm GUI, do the steps that lead desired result: Link raw, Import epochs, Average epochs

  2. In Brainstorm GUI, use the Process tab create the 3-step pipeline that gives the desired results

    1. Process Import > Import recordings > Create link to raw file
    2. Process Import > Import recordings > Import MEG/EEG: Events
    3. Process Average > Average files

  3. Generate a .m script from the 3-step pipeline (see how). Run this generated script from Matlab.

  4. If step 3 was successful, call the same .m script from Python

  5. If step 4 was successful, make a Python scripts that makes the same process calls as the script generated in step 3.

Please give it a try, and let us know if you encounter any trouble in the 3 first steps.

P.S. I have tested calling the tutorial_introduction.m from Python, and it runs without any trouble.

My guess is that the problem in your case is not the interaction of Python calling Matlab, but the commands that are called. This will be discarded if you can successfully perform the step 3 from the list above.