Making brainstorm figures from python

Hi,

The following question steamed from interacting with Brainstorm from an IPython notebook, but it probably (not sure, need to check) applies to any use of Brainstorm through a Matlab server (rather than the usual Matlab interface). Basically, in my notebook, I’m opening scout mat files, changing the colors of the scouts according to statistical calculation of scouts average activity (power, connectivity, etc.), saving the modified scout mat file, loading it in Brainstorm, displaying it and saving the resulting figure. However, I’m meeting some difficulties for the plotting part of this pipeline… From Python’s side, I use python-matlab-bridge (https://arokem.github.io/python-matlab-bridge/) to open a socket to a Matlab server:

    from pymatbridge import Matlab
    mlab = Matlab() #Matlab(matlab='C:/Program Files/MATLAB/R2015a/bin/matlab.exe')
    mlab.start()

    matlabCode = \
    """
        rootPath = 'C:/Users/Orechr/Dropbox/code/analyses/John/notebooks/'; 
        addpath(rootPath) 
    """

    result = mlab.run_code(matlabCode)

Then, I would run the following Matlab code each time I want to make a figure (with different arguments for the Matlab function of course) :

    matlabCode = \
    """
        show_scouts('AaCi/tess_cortex_pial_low.mat', 'left', 'conditionEffect_inter', 'test.png')
    """

    result = mlab.run_code(matlabCode)

First annoyance is that I have to add “brainstorm nogui” each time I want to run this last piece of code, making it in fact

    matlabCode = \
    """
        brainstorm nogui
        show_scouts('AaCi/tess_cortex_pial_low.mat', 'left', 'conditionEffect_inter', 'test.png')
    """

    result = mlab.run_code(matlabCode)

Of course, directly in Matlab, I would only have to run “brainstorm nogui” once but for some reason I don’t understand, here it has to be ran at every call. It should not be the case since Matlab’s session is always the same (e.g., I don’t need to re-set the path at every call… its valid for the whole session, until I hit “mlab.stop()”). If I don’t run this “brainstorm nogui” command at every call, it throws an exception on first Brainstorm call and the reason seems to be associated with the a blank Brainstorm’s global variable “GlobalData” (i.e. GlobalData == []).

The second issue is that I get the following error:

which I don’t get when I execute this code in Matlab environment.

The third issue is that when I run this code in the Matlab environment, I have opaque scouts (alpha=0; as should be according to the code of the “show_scouts” function) but when I run it through the Matlab server, I get translucent scouts as if there was something overriding my specification that alpha=0. Here is the code of the “show_scouts” function (in which I’m probably messing with Brainstorm in ways a gentleman should not):

function show_scouts(SurfaceFile, orient, atlasName, figName)
% orient : 'left', 'right_intern', 'right', 'left_intern', 'back', 'front', 'bottom', 'top'

sSurf              = bst_memory('LoadSurface', SurfaceFile);
[sSubject, iSubject, iSurf] = bst_get('SurfaceFile', SurfaceFile);

iAtlas = 0;
for i=1:size(sSurf.Atlas, 2)
    if strcmp(sSurf.Atlas(i).Name, atlasName)
        iAtlas = i;
    end
end

global GlobalData;
GlobalData.Surface(iSurf).iAtlas = iAtlas;
GlobalData.Surface(iSurf).isAtlasModified = 1;

SurfAlpha=0.0;

% Get GlobalData DataSet associated with subjectfile (create if does not exist)
SubjectFile = sSubject.FileName;
iDS = bst_memory('GetDataSetSubject', SubjectFile, 1);
iDS = iDS(1);


% Prepare FigureId structure
FigureId = db_template('FigureId');
FigureId.Type     = '3DViz';
FigureId.SubType  = '';
FigureId.Modality = '';
% Create figure
[hFig, iFig, isNewFig] = bst_figures('CreateFigure', iDS, FigureId, 'AlwaysCreate');

if isempty(hFig)
    bst_error('Could not create figure.', 'View surface', 0);
    return;
end

% Set application data
setappdata(hFig, 'SubjectFile',  SubjectFile);
   
%% ===== DISPLAY SURFACE =====
% Add surface to figure
iSurf = panel_surface('AddSurface', hFig, SurfaceFile);
if isempty(iSurf)
    return
end
% Set transparency
if ~isempty(SurfAlpha)
    panel_surface('SetSurfaceTransparency', hFig, iSurf, SurfAlpha);
end

% Set figure as current figure
%bst_figures('SetCurrentFigure', hFig, '3D');
% Show all scouts for this surface (for cortex only)
panel_scout('ReloadScouts', hFig);

% Make sure to update the Headlight
camlight(findobj(hFig, 'Tag', 'FrontLight'), 'headlight');
% Camera basic orientation
figure_3d('SetStandardView', hFig, orient);
set(hFig, 'Name', 'Atlas');
set(hFig, 'Visible', 'off');
out_figure_image(hFig, figName);
bst_figures('DeleteFigure', hFig, 'NoUnload');
%bst_memory('UnloadAll', 'Forced');

Any idea on what could cause these three issues? I have the feeling that they are probably all linked with Brainstorm’s global variable. My guess is that global variables are not necessarily handled in the same way in these two contexts (global variables are the root of all computational evil after all). Although I know this question is very specific, does anyone had any experience with similar issues?

Hi Christian,

I haven’t looked at this pymatbridge yet, so I cannot give any good advice… I’ll ask around to some Python lovers if they have any additional advice for you.

From problem #1, it seems clear that successive calls to mlab.run_code do not keep the global variables of the session. Maybe there is an implicit “clear” or “clear global” that is executed at the end of the matlab call. Maybe this is something you can configure, you could ask the developers of this package.
Running Brainstorm without global variables is impossible… As Matlab was not really designed to be a real programming language, there was no other easy way to store efficiently large instance variables attached to the interface. Dirty language, dirty coding…

Issue #2: I don’t see the attachment, can you post it again?

Issue #3: There is probably no opengl support in your Matlab instance, so no transparency in the 3D figures… Maybe you could test this with simpler examples (without brainstorm) to make sure that it is not brainstorm which fails at detecting the most appropriate figure renderer. You can also ask about this to the developers.

Cheers,
Francois

Hi François,

#1 Strangely, it is not that simple. Each call of "run_code" does propagate global variable values. For example, if I run

matlabCode = """
global test3
test3 = 'some string'
"""
mlab.run_code(matlabCode)

matlabCode = """
global test3
disp(test3)
"""
result = mlab.run_code(matlabCode)
print(result)

in results, I see that the second call printed "some string" showing that the assignment from the first call was still effective in the second call. There is probably something specific in the way that the global variable is treated within Brainstorm...

#2 Here:


Hope is shows now.

#3 Actually, it's the other way around. Call through the Matlab server DOES show transparency (when it should not because I set alpha = 0 in "show_scouts") whereas call within the Matlab normal interface shows no transparency (as it should). So, the problem is actually that transparency is used although it should not so it cannot be a failure from OpenGL (unless such a failure could cause transparency!).

I'll continue to work on these issues today and will let you know if I get a way to fix them.

#1: I don’t think there is anything special in the management of the global variable GlobalData… It is defined in bst_startup, lines 72-73:
global GlobalData;
GlobalData = db_template(‘GlobalData’);
And then reset in bst_exit, line 90:
GlobalData = [];

#2: I have no idea… this would require some more intense debugging.
Do you have access to the output to the command window? If so you could add disp() calls to track what is happening…

#3: Ok, it is definitely not an OpenGL problem.
> panel_surface(‘SetSurfaceTransparency’) sets the surface transparency
To change the scouts transparency:
> panel_scout(‘SetScoutTransparency’, alpha);

Thank you for investigating this, I’m sure it’s going to be useful for other people!

There is something very funny with the way the gobal variable GlobalData behave in Brainstorm. For example, if I make a small matlab function


function func()
global globalFromFunc;
globalFromFunc = 'test from func';

and then I run in Python

matlabCode = """
global globalFromFunc global2;
func_test2();
global2 = 'test';
disp('1 ########### globalFromFunc:')
disp(globalFromFunc)
disp('2 ########### global2:')
disp(global2)
"""

result = mlab.run_code(matlabCode)
print(result["content"]["stdout"])


matlabCode = """
global globalFromFunc global2;
disp('3 ########### globalFromFunc:')
disp(globalFromFunc)
disp('4 ########### global2:')
disp(global2)
"""

result = mlab.run_code(matlabCode)
print(result["content"]["stdout"])



matlabCode = """
global globalFromFunc global2;
func_test2();
global2 = globalFromFunc;
"""

result = mlab.run_code(matlabCode)


matlabCode = """
global globalFromFunc global2;
disp('5 ########### globalFromFunc:')
disp(globalFromFunc)
disp('6 ########### global2:')
disp(global2)
"""

result = mlab.run_code(matlabCode)
print(result["content"]["stdout"])


matlabCode = """
global globalFromFunc global2;
func_test2();
global2 = globalFromFunc;
"""

result = mlab.run_code(matlabCode)
print(result["content"]["stdout"])

I get the expected behavior, with an output

1 ########### globalFromFunc:
test from func
2 ########### global2:
test

3 ########### globalFromFunc:
test from func
4 ########### global2:
test

5 ########### globalFromFunc:
test from func
6 ########### global2:
test from func

Now, if instead of using “func”, I use “brainstorm nogui” such that

matlabCode = \
"""
rootPath = 'C:/Users/Orechr/Dropbox/code/analyses/John/notebooks/';
addpath(rootPath) 
"""
result = mlab.run_code(matlabCode)


matlabCode = """
global GlobalData global2;
brainstorm nogui
global2 = 'test';
disp('1 ########### GlobalData:')
disp(GlobalData)
disp('2 ########### global2:')
disp(global2)
"""

result = mlab.run_code(matlabCode)
print(result["content"]["stdout"])


matlabCode = """
global GlobalData global2;
disp('3 ########### GlobalData:')
disp(GlobalData)
disp('4 ########### global2:')
disp(global2)
"""

result = mlab.run_code(matlabCode)
print(result["content"]["stdout"])



matlabCode = """
global GlobalData global2;
brainstorm nogui
global2 = GlobalData;
"""

result = mlab.run_code(matlabCode)

matlabCode = """
global GlobalData global2;
disp('5 ########### GlobalData:')
disp(GlobalData)
disp('6 ########### global2:')
disp(global2)
"""

result = mlab.run_code(matlabCode)
print(result["content"]["stdout"])


I get

[Warning: Function isrow has the same name as a MATLAB builtin. We suggest you rename the function to avoid a potential name conflict.] 
[> In path (line 109)
  In addpath (line 86)
  In brainstorm>bst_set_path (line 273)
  In brainstorm (line 119)
  In pymat_eval (line 31)
  In matlabserver (line 24)] 
 
BST> Starting Brainstorm:
BST> =================================
BST> Version: 04-Aug-2015
BST> Compiling main interface files...
BST> Emptying temporary directory...
BST> Deleting old process reports...
BST> Loading configuration file...
BST> Initializing user interface...
BST> Starting OpenGL engine... hardware
BST> Reading plugins folder...
BST> Loading current protocol...
BST> =================================
 
BST> Warning: Brainstorm is already running from a different Matlab session.
1 ########### GlobalData:
                 Program: [1x1 struct]
                DataBase: [1x1 struct]
                 DataSet: [0x0 struct]
                     Mri: [0x0 struct]
                 Surface: [0x0 struct]
          UserTimeWindow: [1x1 struct]
          FullTimeWindow: [1x1 struct]
         UserFrequencies: [1x1 struct]
           ChannelEditor: [1x1 struct]
             HeadModeler: [1x1 struct]
                Clusters: [0x0 struct]
           CurrentFigure: [1x1 struct]
              DataViewer: [1x1 struct]
    CurrentScoutsSurface: ''
    VisualizationFilters: [1x1 struct]
               Colormaps: [1x1 struct]
             Preferences: [1x1 struct]
         ChannelMontages: [1x1 struct]
               Processes: [1x1 struct]
          ProcessReports: [1x1 struct]
          Interpolations: []

2 ########### global2:
test

3 ########### GlobalData:
4 ########### global2:
test

5 ########### GlobalData:
6 ########### global2:
                 Program: [1x1 struct]
                DataBase: [1x1 struct]
                 DataSet: [0x0 struct]
                     Mri: [0x0 struct]
                 Surface: [0x0 struct]
          UserTimeWindow: [1x1 struct]
          FullTimeWindow: [1x1 struct]
         UserFrequencies: [1x1 struct]
           ChannelEditor: [1x1 struct]
             HeadModeler: [1x1 struct]
                Clusters: [0x0 struct]
           CurrentFigure: [1x1 struct]
              DataViewer: [1x1 struct]
    CurrentScoutsSurface: ''
    VisualizationFilters: [1x1 struct]
               Colormaps: [1x1 struct]
             Preferences: [1x1 struct]
         ChannelMontages: [1x1 struct]
               Processes: [1x1 struct]
          ProcessReports: [1x1 struct]
          Interpolations: []

as can be seen, in 3 and 5, GlobalData takes no values as a normal global variable does… Maybe for some reason “bst_exit” gets triggered at the end of the “run_code” call… I have no clue. Meanwhile, for the problem #1, we can get around this issue by first calling

matlabCode = """
global GlobalData GlobalData_tmp;
brainstorm nogui
GlobalData_tmp = GlobalData
"""

result = mlab.run_code(matlabCode)

and then calling

matlabCode = """
global GlobalData GlobalData_tmp;
GlobalData = GlobalData_tmp
show_scouts('AaCi/tess_cortex_pial_low.mat', 'left', 'conditionEffect_inter', 'test.png')
"""
result = mlab.run_code(matlabCode)

whenever I want to plot a brain (without having to call again the “brainstorm nogui” script. Works well… but it nonetheless looks as if there is something not kosher with GlobalData for it to behave in this way.

#2 : The problem seem to have vanished by itself. Go figure.

#3 : As you noticed, the code for “show_scouts” was setting transparency to the surface, not to the scouts. Still, scouts transparency was different when ran from matlab server and matlab interface, which is not expected. Also, the change of atlas was not taken into account. The lines

global GlobalData;
GlobalData.Surface(iSurf).iAtlas = iAtlas;
GlobalData.Surface(iSurf).isAtlasModified = 1;

seemed to have no effect when using the matlab server, but working perfectly through the standard matlab interface. All sorts of bizarre behaviors that did led me to believe that interaction with the global variable GlobalData was still much problematic. The get around here was to copy the code of ReloadScouts, GetScoutPosition, PlotScouts, SetScouts, and UpdateScoutsDisplay functions directly in my show_scouts.m file and get rid of all the use of the GlobalData variable in the local copies of these functions. All works perfectly now. Of course, it is not a perfect solution. Sadly , it is not a truly genuine Brainstorm-python interaction anymore because it requires special versions of Brainstorm functions (which have been stripped of many features that were not necessary for my use case).

Hi Christian,

Thanks for this very detailed review.
My guess is indeed that something is causing Brainstorm to close after the execution, and bst_exit is explicitly called by the Matlab server after the end of your call.

The callback function that closes Brainstorm (function closeWindow_Callback in gui_brainstorm.m) is called in two cases only:

  1. when the Brainstorm JFrame is closed (the user closes the Brainstorm window)
  2. when the Brainstorm hidden Matlab figure is closed (Matlab is closed, or the hidden figures are closed, like when calling “close all”)

Maybe the Matlab server is a bit more aggressive at closing leftover stuff than the regular Matlab environment. It would make sense, because if all the executed scripts keep figures and Java objects alive, the server may quickly lack of memory.
So maybe after each execution of “mlab.run_code” the server runs the Java VM garbage collection with more aggressive parameters (causing #1), or maybe it simply does a “close all” to close all the figures (causing #2).

The first issue happens for instance when executing Brainstorm in a compiled environment. Just having the Java JFrame is not enough to keep the environment alive when running the compiled version of Brainstorm.
This is why I have to keep looping explicitly in the first main call (bst_startup.m, lines 580-587) in the compiled case. If not the script returns and Matlab exits.

You could try several things:

  • edit bst_startup.m at line 580 and set isMatlabRunning=0, so that it keeps looping (your call to run_code might never return…)
  • edit gui_brainstorm at line 429, add a return statement so that Brainstorm is never killed, then call explicitely bst_exit when you’re done.
  • make the Brainstorm hidden figure visible so that it stays alive: hBst = bst_mutex(‘get’, ‘Brainstorm’); set(hBst, ‘Visible’, ‘on’);

There are probably lots of other things that could be tested… Hopefully you’ll find a combination that works.
Be aware that the behavior you are observing might be specific of one unique combination of versions of software (Python/pymatbridge/Matlab server/Matlab/Java/Operating system)…

Good luck!
Francois

Hi François,

  • edit bst_startup.m at line 580 and set isMatlabRunning=0, so that it keeps looping (your call to run_code might never return…)
    ==> Indeed, the Brainstorm splash screen stays on and the run_code function never returns.

  • edit gui_brainstorm at line 429, add a return statement so that Brainstorm is never killed, then call explicitly bst_exit when you’re done.
    ==> Works very well. I’m pretty happy about this solution. I can now interact with Brainstorm without the need of modifying any specific function except for the one exception (“edit gui_brainstorm at line 429, add a return statement”). Regarding that exception, would it be possible to add a switch to the brainstorm command so that we can call for example “brainstorm nogui keep_alive” which would make brainstorm to enter in an “if” block that would return at line 429 of gui_brainstorm? This way, we could use Brainstorm from Python (and from a Matlab server in general) without the need to tinker it at all!

Thanks for you help.

Christian

Hi Christian,

I added an option “brainstorm server” that should display even less stuff than “brainstorm nogui”, and that doesn’t have the closing callbacks enabled.
Call “brainstorm stop” to quit.

Cheers,
Francois