Brainstorm's Database Structure

Authors: Martin Cousineau, Raymundo Cassani

Understanding how data is organized and managed in Brainstorm is paramount for scripting your own pipelines, and adding new processes in the software. This page describes the structure of the Brainstorm database, and the metadata that is necessary to manage it.

On the hard drive

Brainstorm database resides on the hard drive at the database directory (brainstorm_db/). The database consists in an organized collection of protocols, where each protocol each with its own directory, which has a strict hierarchy, explained in the bullet list below:

• anat: This is where Anatomy files (e.g. MRI, surfaces) and subject's metadata are stored.

  • One folder per subject

    • brainstormsubject.mat: Metadata of the subject, see db_template('subjectmat')

    • subjectimage_*.mat: MRI files, see db_template('mrimat')

    • tess_*.mat: Surface files, see db_template('surfacemat')

• data: This is where Functional files (e.g. channels, recordings) and study's metadata are stored.

  • protocol.mat: All metadata of this protocol, as Matlab structs. See next section for details. Refer to variable ProtocolMat of function db_save().

  • protocol.db: All metadata of this protocol, as an SQLite database. See next section for details. Refer to sql_generate_db().

  • One folder per subject

    • One folder per study

      • brainstormstudy.mat: Metadata of the study, see db_template('studymat')

      • channel_*.mat: Channel files, see db_template('channelmat')

      • data_*.mat: Data (recording) files, see db_template('datamat')

      • Etc.

The protocol metadata files (protocol.mat and protocol.db) contain all the metadata that is required to manage the protocol data. In case that any of these files is accidentally deleted, or gets corrupted, the protocol metadata can regenerated from all the brainstormsubject.mat and brainstormstudy files in the protocol directory.

All required metadata should always be saved on the hard drive outside of the protocol.mat / protocol.db files, such that if they get corrupted, deleted or one does a fresh reload of the database, there is no loss of information. This is why things like: which cortex file is selected, or which trials are marked as bad are also saved in separate files (brainstormsubject.mat and brainstormstudy.mat).

The filename of each file should always clearly indicate the basic type of the file, hence the required prefixes (e.g. data_*.mat).

For the other anatomy files and other functional data files, the prefix in the filename indicates its content. For example, a file surface_*.mat contains cortex surface information. The data is stored as a Matlab structure, according to the different file types.

The content of the structure of each .mat files is defined by the function db_template(). The mat suffix is used to differentiate from the in-memory metadata structure and the in-momery data structure of each type. For example:

• db_template('surfacemat'): Structure saved on the hard drive as a surface_*.mat file.

• db_template('loadedsurface'): Structure saved in memory containing all the data of a loaded surface_*.mat file.

• db_template('surface'): Structure saved in memory containing all the metadata of a surface_*.mat file, loaded from the protocol.db / protocol.mat file rather than the surface_*.mat file itself.

Edit conflict - other version:

Terms that need definition

• Brainstorm database: Collection of Protocols

• Protocol:

• Anatomy data:

• Functional data:

• Subject:

• Study:

• Dataset:

• Folder:

• Sub-folder:

Edit conflict - your version:

End of edit conflict

Protocol metadata

A brief paragraph describing why there are two versions, the main differences, and when is the user expected to see one or the other.

Matlab structure: protocol.mat

Once a protocol is loaded, Brainstorm has Matlab structures in memory that contain the metadata of all the files in this protocol. These are located in the GlobalData.DataBase global variable, such that it is easily accessible in any function of the software. It is defined db_template('GlobalData'). Let's walk through some of the important variables in this structure:

• GlobalData.DataBase.ProtocolInfo: Basic information about the protocols in the Brainstorm database folder, refer to db_template('ProtocolInfo').

  • Comment: Name of the protocol

  • STUDIES: Absolute path to the folder containing the functional data (/data/).

  • SUBJECTS: Absolute path to the folder containing the anatomical data (/anat/).

  • iStudy: Currently selected study, if applicable.

  • UseDefaultAnat: Protocol setting, whether new subjects should use the default anatomy by default.

  • UseDefaultChannel: Protocol setting, whether new subjects should share channel files by default.

• GlobalData.DataBase.ProtocolSubjects: Metadata of all subjects and their anatomical files of each loaded protocol, refer to db_template('ProtocolSubjects').

  • Subject: Metadata of all regular subjects, refer to db_template('subject').

  • DefaultSubject: Special subject used to store default anatomy. Same structure as regular subjects.

• GlobalData.DataBase.ProtocolStudies: Metadata of all studies and their functional files of each loaded protocol, refer to db_template('ProtocolStudies').

  • Study: Metadata of all regular studies, refer to db_template('study').

  • DefaultStudy: Special study used to store Common files, e.g. a global channel file. Same structure as regular studies.

  • AnalysisStudy: Special study to store Inter-subject files, e.g. stats results. Same structure as regular studies.

• GlobalData.DataBase.isProtocolLoaded: List of bool, whether the metadata of the protocol is already in memory or not. This is useful when you're running Brainstorm from the command line.

• GlobalData.DataBase.isProtocolModified: List of bool, whether the metadata of the protocol was modified and therefore the protocol.mat needs to be overwritten when Brainstorm closes.

• GlobalData.DataBase.iProtocol: The index of the currently selected protocol.

• GlobalData.DataBase.BrainstormDbDir: Absolute path to the brainstorm database directory.

• GlobalData.DataBase.DbVersion: Version number of the currently loaded protocol. Used to update the protocol structure when new fields are added to a specific file for example. Refer to db_update().

• GlobalData.DataBase.isReadOnly: Flag whether the user has write access to the protocol database directory. Used in tree_callbacks() to remove GUI option menus that would modify files.

• GlobalData.DataBase.LastSavedTime: Timestamp of the last time the protocol.mat file was overwritten. There used to be an automatic save when it had been a few hours.

• GlobalData.DataBase.Searches: Information about the search tabs of the database:

  • iActive: Index of the currently selected active search tab (0 if none).

  • Active: List of all active search tabs, with the first one being the whole database.

    • Name: Name of the tab, or empty if whole database

    • SearchNode: Tree structure representing the search query

    • AnatRootNode/FuncRootNode: The root node of the database tree, to avoid rebuilding the Java tree when you switch between tabs.

    • AnatSelNode/FuncSelNode: The currently selected node, to remember the last selected object before you switched between tabs.

  • All: If searches were saved between Brainstorm sessions, they are stored here.

    • Name: Name of the search

    • Search: Tree structure representing the search query

Edit conflict - other version:

Edit conflict - your version:

The two main variables of interest are ProtocolSubjects and ProtocolStudies. Indeed, together they contain the metadata of every single file in the protocol! Let's dive a bit deeper into both of them.

ProtocolSubjects

This structure contains the metadata of every subjects and anatomy files.

• Name: Name of the subject. Two special subjects have specific names:

  • @default_subject: Default anatomy subject.
  • Group_analysis: Subject for statistical results or projections to a common space.

• Comments: Additional information about the subject set manually by the user.

• FileName: Path to the brainstormsubject.mat file where the subject specific metadata is stored in case the protocol's metadata file is corrupted or needs to be reloaded.

• DateOfAcquisition: Unused, legacy field.

• Anatomy: List of all MRI files, refer to db_template('Anatomy').

• Surface: List of all surface and fibers files, refer to db_template('Surface').

• iAnatomy: Index of the MRI file in the Anatomy list that is currently selected for this subject. This will show in green in the database viewer. Any operation related to this subject that requires an MRI file will load this selected one.

• iScalp: Index of the selected scalp surface file from the Surface list.

• iCortex: Index of the selected cortex surface file from the Surface list.

• iInnerSkull: Index of the selected inner skull surface file from the Surface list.

• iOuterSkull: Index of the selected outer skull surface file from the Surface list.

• iFibers: Index of the selected fibers file from the Surface list.

• iFEM: Index of the selected surface file used for FEM from the Surface list.

• iOther: Index of the selected sub-structure surface file from the Surface list.

• UseDefaultAnat: Whether this subject uses the default anatomy files rather than its own.

• UseDefaultChannel: Whether this subject shares its channel file across all runs or uses the global channel file.

ProtocolStudies

This structure contains the metadata of every studies (also known as first level folders, or conditions) and every functional files inside each study.

• Name: Name of the study. If it starts with "@raw", this is a special study that can only contain unprocessed files and is created when you create a new link to a raw file. Three special studies exist:

  • @default_study: Common files shared either by the whole protocol (see @default_subject) or the whole subject. Typically channel files.
  • @inter: Inter-subject statistical results.
  • @intra: Intra-subject statistical results.

• FileName: Path to the brainstormstudy.mat file where the study specific metadata is saved should the protocol metadata file get corrupted or need to be reloaded.

• DateOfStudy: If this is a raw study, this is the acquisition date of the raw file. Otherwise, this is the date this study was created by the user.

• BrainstormSubject: Path to the file of the metadata of the subject this study belongs to.

End of edit conflict

TODO: db_template('Subject'), db_template('Study'), db_template('Data'). etc

SQLite structure: protocol.db

TODO.