Running Brainstorm on a cluster

Hi François!

I post some of the issues we have been discussing here, in case they may help other people:

1- Running Brainstorm on a cluster:
From Macos/Linux:
brainstorm3.command <MATLABROOT> <script.m> <parameters>

It's important to include this matlab root, which can also be the Matlab RunTime (in case you want to run it without Matlab license). This info is now included in the tutorial. Thanks!
.

2- To avoid being asked for “brainstorm_db” every time add: local
Example:
brainstorm3.command /usr/local/MATLAB/MATLAB_Runtime/v98 main.m local

This will use a local_db automatically created at: $HOME/.brainstorm/local_db
.

3- With all these, the following lines in the tutorial example script are not longer needed:

% Set up the Brainstorm files
pathStr = '/home/user/brainstorm3/';
addpath(genpath(pathStr));
BrainstormDbDir = '/home/user/brainstorm_db';
% Start Brainstorm
if ~brainstorm('status')
brainstorm server
end

Since brainstorm is already started with the command call (maybe worth commenting it in the tutorial example)

With this way of using brainstorm we have to be very careful of the projects created in the database.
If brainstorm gives an error, and the ProtocolX has been created but not properly registered in Brainstorm. Then, it won’t allow you to create a new one with same name or delete it using brainstorm functions:
gui_brainstorm('DeleteProtocol', ProtocolName); or
gui_brainstorm('CreateProtocol', ProtocolName, 0, 1);

A few more questions:

** Can we change the local_db location [automatically created at: $HOME/.brainstorm/local_db]??
If we create a container with brainstom3.command+matlab-runtime, and multiple users are running it, their databases may collide, therefore, I think it would be useful if their brainstorm_db folder could be specifically changed to be in their home directory. With the local option, it seems this is not possible. And without using the local option, it keeps asking on the command line about the brainstorm_db folder location, which can’t be entered while running automated scripts.

** Where is the information related to Brainstorm registered protocols stored??

Thanks a lot!

(Still need to try the new container)

Hi Guio,

Thanks for the useful summary.
I restructured the section "Running scripts on a cluster" in the Scripting tutorial, since I realized it contained some wrong instructions, which probably didn't you in the first place:
https://neuroimage.usc.edu/brainstorm/Tutorials/Scripting#Running_scripts_on_a_cluster

I integrated some of your comments in this new subsection "Without Matlab".
Please feel free to edit directly the tutorial to add the information you think is still missing or unclear.

Can we change the local_db location [automatically created at: $HOME/.brainstorm/local_db]??

Of course.
Start Brainstorm so that it doesn't bother you at startup because BrainstormDbDir is missing.
Then change it to wherever you want from your script:
bst_set('BrainstormDbDir', BrainstormDbDir); => only sets the folder for future protocol creations
or
db_import(BrainstormDbDir); => Sets BrainstormDbDir + loads all the protocol it contains.

Where is the information related to Brainstorm registered protocols stored??

$HOME/.brainstorm/brainstorm.mat
=> ProtocolsListInfo + ProtocolsListSubjects + ProtocolsListStudies
=> ProtocolsListInfo : Include the absolute paths to the anat and data folders
=> Most of the entries of ProtocolsListSubjects and ProtocolsListStudies are empty, which means that the info is saved in the local Protocol/data/protocol.mat instead.

Dear Guio, dear BST team, dear all,

I am getting back to you on this thread because I am struggling sending BST computations on a cluster.

Following your documentation and François's (thank you both!!) I could write and execute a script for processing one participant or several ones sequentially successfully. Now we would like to run several participants (at once) on several computing cores of our cluster using slurm job scheduler. For this we wrote a batch script (attached), so that an index corresponding to the participant is passed as an input of the BST script and that every single participant is processed on a distinct computing core.
Something went wrong…. The resulting BST database is incomplete. Looking at the output files, it seems that some participants processing never ended and one of the output was :

In this output log everything looks OK until the line “Warning : File not found…”

(the markers skipped are excepted.. we are epoching on another existing one)

Do you have experience with such failure? Do you have any advice to continue in this direction (or not!)?

Do not hesitate to ask me any clarification.

Thank you.
Best,
AnneSo

SubmitBSTcluster_asd.sh (578 Bytes)