What's next

A roadmap to the future developments of Brainstorm.

Current topics

Source modeling

• Implementation of a new unified minimum norm/beamformer framework (work in progress)

Documentation

• Standard workflows for different types of data and experiments (work in progress)

• Cross-validation with MNE and FieldTrip:
  http://martinos.org/mne/dev/auto_tutorials/plot_brainstorm_auditory.html

Functional connectivity

• Significance thresholding of the connectivity matrices (not started)

Computation

• Removing the dependence to the Java interface to run in headless mode (not started)

Recordings

• 2D topographies:
  • Nicer topographies with circle around 2DDisc (similar to EEGLAB plots)

  • Standardized plots (using FieldTrip .lay files?)

  • Aligned on the midline at least
  • Contour lines sometimes messed up with Elekta recordings
  • Make the surface on which the values are interpolated simpler
• MEG/EEG registration: Apply the same transformation to multiple runs
• Montages:
  • Separators (" : MLC11" already works, could be just " : "), should add an empty space.
• 2DLayout:
  • Use the same standard positions, too much space between sensors (Recordings + TF)
  • Overlay multiple conditions

  • RAW files: Doesn't work when changing page => need refresh of GlobalData.Preferences.TopoLayoutOptions.TimeWindow

  • Same shortcuts as the raw file viewer (right-click + move for gain)
  • Add support for montages
• RAW file viewer:
  • Pre-load next page of recordings
  • Faster display: Downsample time series before plotting
  • Add field "comment" to markers: For clinicians to add notes (Marcel)
  • Events: Change the category of a selected event easily, instead of deleting/marking new

  • Events: Advanced process for recombining.
    Example: http://www.erpinfo.org/erplab/erplab-documentation/manual/Binlister.html

Interface

• Start Brainstorm without Java (-nodesktop)
• Move code to github

• Allow FieldTrip functions in compiled version

• Generalize the use of the units (field .DisplayUnits): Rewrite processes to save the units correctly

• Colormaps:
  • Allow brightness/contrast manipulations on the custom colormaps
  • Create a colormap similar to MNE, where extrema are bright
  • Global colormap max: Should get the maximum across all the open files
• Open new figures as tab (docked in the Figures window)
• Copy figures to clipboard (with the screencapture function)
• Removing all the CTRL and SHIFT in the keyboard shortcuts
• Display warning before opening files that are too big
• Smooth display from figure_image (ERPimage, raster plot...)

• Contact sheets & movies: use average of time windows instead of single instants, for each picture.

Connectivity

• Thresholding and stat tests the connectivity matrices
• Connectivity on unconstrained sources: how to group the three orientations?

• Review by Jan-Mathijs: http://journal.frontiersin.org/article/10.3389/fnsys.2015.00175/full

• Connectivity based on band limited power (Sylvain):
  • Compute Hilbert/Bandpass + correlation of the envelopes
  • Bandpass envelopes before computing correlations?
  • Compute Hilbert(sensors) and then project to source space?
• Graph view:
  • Does not display negative values correctly (correlation or difference of coherence)
  • Re-write using pure Matlab code and smoothed graphics
  • Fixed scales for intensity sliders
  • Text bigger
  • Too much data in appdata
  • Fixed scales for intensity sliders
  • Add "=" shortcut for having graphs with similar configurations
  • Disable zoom in one region (serious bugs)
  • NxN on sensors: does not place the sensors correctly in space
• Coherence:

  • Average cross-spectra instead of concatenating epochs (to avoid discontinuities)
    Explore inter-trial approaches (Esther refers to chronux toolbox)

• Granger:
  • Crashes sometimes: improve stability
  • Check for minimum time window (Esther: min around 500-1000 data points)
  • Re-write and optimize code
  • Add progress bar
• PLV:
  • Add p-values
  • Remove evoked
  • Optimize code
  • Add time integration
  • Unconstrained sources
  • Add warning when running of short windows (because of filters)
• PAC:
  • Add input TF , to disconnect TF decomposition and PAC computation (Peter)
  • Refine frequency vector of low frequencies
  • How many central frequencies to use in bst_pac?
  • Change filters: no chirplet functions
  • bst_freqfilter: Use nfcomponents like in bst_pac
  • Esther recommended a larger frequency binning of the PAC estimation
  • PAC maps: Display all sensors at once (like TF and DynamicPAC)
• Hui-Ling's PAC:

  • https://bsp.hackpad.com/Cross-Frequency-Coupling-cChe95lhDHz

  • https://github.com/NCTU-BSP/MEEG

• Time-resolved correlation/coherence: Display as time bands
• Other metrics:
  • Coherence by bands: bst_coherence_band_welch.m
  • Granger by bands: bst_granger_band.m
  • Inter-trial coherence

• Tutorial coherence [1xN] : Reproduce FieldTrip results?

• Connect NxN: Display as time series > Display warning before trying to open too many signals

Processes

• Explore Matlab-Python bridge:
  https://mail.python.org/pipermail/neuroimaging/2016-June/001001.html

• Allow processes in Python and Java

• Add FieldTrip functions:

  • ft_freqanalysis
  • ft_combineplanar
  • ft_volumesegment
  • ft_computeleadfield
  • ft_prepare_heamodel
• Pipeline editor:
  • Optimize speed: Opening the window and the showing menu "Add process" are slow
  • Add loops over subjects/conditions/trial groups
• PLS:

  • https://www.rotman-baycrest.on.ca/index.php?section=84

  • meg-pls dot weebly dot com

  • Krishnan 2011: http://www.ncbi.nlm.nih.gov/pubmed/20656037

  • Cheung 2015: http://www.sciencedirect.com/science/article/pii/S1053811915007648

  • McIntosh 2012: http://www.ncbi.nlm.nih.gov/pubmed/22804773

  • McIntosh 2004: http://www.ncbi.nlm.nih.gov/pubmed/15501095

• ICA:
  • Exploration: Add window with spectral decomposition (useful for muscle artifacts)
  • Export IC time series (and then compute their spectrum): solves the problem above

  • Comparison JADE/Infomax:
    http://journals.plos.org/plosone/article?id=10.1371/journal.pone.0030135

  • Add methods: SOBI, Fastica, AMICA/CUDICA (recommended by S Makeig)

  • Dimension reduction with PCA adds artifacts: Not done by default in EEGLAB
    Contact: Stephen Shall Jones ( shall-jones@infoscience.otago.ac.nz )
    Student Carl Leichter detailed this in his thesis

  • S Makeig: Use ICA to select the IC of interest instead of only removing artifacts
  • Display of spectrum for components (PSD/FFT)
  • Use FastICA (algo crashing)
  • Understand why EEG/Epilepsy tutorial data crashes if we don't limit the number of components
  • Add components preselection: Correlation with EOG/ECG
  • Import ICA matrices available in EEGLAB .set files

  • EEGLAB recommends ICA + trial rejection + ICA again: Impossible right now with Brainstorm
    (http://sccn.ucsd.edu/wiki/Chapter_09:_Decomposing_Data_Using_ICA)

• SSP:

  • Display warning if changing the ChannelFlag while there is a Projector applied

  • Show where the attenuation is projected:
    (sum(IK,2)-sum(SSP(k,:)*IK,2)./sum(IK,2)

• SSS/tSSS: Get implementation from MNE
• Bandpass:
  • Mark with an event the possible edge effects
  • Offer option: bst_bandpass_fft / bst_bandpass_filter
  • Rewrite without the forced low-pass filter at Fs/3
  • Show warning when using inappropriate high-pass filter (precision too high)
  • Use FIR filter
• Spectral flattening (John):
  • ARIMA(5,0,1): Apply on the signal before any frequency/connectivity/PAC analysis
• PSD:
  • Rewrite to have the same input as coherence (frequency resolution instead of window length)
  • Use the progress bar

• Remove line noise: http://www.nitrc.org/projects/cleanline

• Use Matlab Coder to optimize some processes: Wavelets, bandpass filter, sinusoid removal
• Reports: Click on link reopens exactly the figure
• Time-frequency:
  • Multi-tapers

  • Short-time Fourier transform: http://www.mikexcohen.com/lectures.html

  • Matching pursuit: http://m.jneurosci.org/content/36/12/3399.abstract?etoc

  • Bug: Display logs as negative
  • Bug: 3D figures: Colormaps with "log" option doesn't work
  • Bug: Difference of power displayed in log: problems (Soheila)
  • 2D Layout in spectrum
  • Make much faster and more memory efficient (C functions coded by Matti ?)
  • TF scouts: should display average of TF maps
  • Impossible to keep complex values for unconstrained sources
  • Pad short epochs with zero values for getting lower frequencies
  • Hilbert with time bands very slow on very long files (eg. 3600s at 1000Hz) because the time vector is still full (10^7 values): save compressed time vector instead.
  • Extend clusters tab to display of TF to overlay TF signals (Svet)
  • When normalizing with baseline: Propagate with the edge effects marked in TFmask
  • Allow running TF on montages
  • Review continuous files in time-frequency space (for epilepsy)
• Artifact detection:
  • Artifact rejection like SPM: if bad in 20%, bad everywhere
  • Test difference between adjacent samples
• Events detection: Add option "std" vs "amplitude"
• Co-registration of MEG runs:
  • SSP: Group projectors coming from different files
  • Finish validation of the method
  • Apply to continuous recordings for correcting head movements
• Other processes:
  • Moving average
  • Max
  • Median
• Simulation:

  • Fix units in simulation processes => no *1e-9 in "simulate recordings"

  • Use "add noise" process from Hui-Ling (in Work/Dev/Divers)
• Use field process field "Group" to separate Input/Processing/Output options
• Use new Matlab functions: movmean, movsum, movmedian, movmax, movmin, movvar, movstd

Database

• Add buttons to sort files: by name, by comment, by date
• MEG protocols: More flexible organization of the database; sub-conditions to allow different runs X different conditions.
• Navigator: Use F2/F3 to explore the entire database (right now F2 works in a weird way)
• Matrix files: Allow to be dependent from other files
• Add notes in the folders (text files, visible as nodes in the tree)
• Screen captures: save straight to the database
• Rename multiple files
• Default headmodel lost when reloaded: Keep selection on the hard drive (in brainstormstudy.mat)
• protocol.mat can be too big: do not store the results links in it (and recreate when loading)

Distributed computing

• Options from FieldTrip:

  • Loose collection of computers: https://github.com/fieldtrip/fieldtrip/tree/master/peer

  • Alternative, with less limitations: http://research.cs.wisc.edu/htcondor/

  • Single multicore machine: https://github.com/fieldtrip/fieldtrip/tree/master/engine

  • Batch system: https://github.com/fieldtrip/fieldtrip/tree/master/qsub

  • Documentation: http://fieldtrip.fcdonders.nl/faq#distributed_computing_with_fieldtrip_and_matlab

• PSOM: http://psom.simexp-lab.org/

• Various initiatives: http://samirdas.github.io/Data_sharing.html#/

Source modeling

• Mixed head models:
  • Project to templates (scouts or source maps)

  • Create scout form maximum doesn't work (menu Sources > Max value)

  • Display in MRI doesn't work
  • Smooth display: do not smooth subcortical structures
  • Export as .nii volume: doesn't work
  • Set loose parameter from the interface
• Project individual dipoles files on a template
• Dipoles:
  • panel_dipoles: Doesn't work with multiple figures
• Panel Get coordinates: Add button "find maximum"
• BEM single sphere: Get implementation from MNE
• Stenroos 2014 paper: Include the following methods

  • Inner and outer skull surfaces generator from FieldTrip (needs SPM)

  • Nolte corrected-sphere model (good model re:Alex)
  • Fast BEM models
• Visualize Beamformer results:
  • Read CTF SAM .svl
  • Display as layers in the MRI viewer
• Unconstrained sources:
  • Stat and connectivity: what to do? (re-send email John+Sylvain)
• Overlapping spheres: improve the estimation of the spheres for the frontal lobes
• Volume grid:
  • Test volume sources with all the subsequent processes (timefreq, stat...)
  • Optimize: 3D display (better than 9x9 cubes)
  • Optimize: vol_dilate (with 26 neighbors)
• Magnetic extrapolation: Do the same thing with EEG
• Noise covariance matrix:
  • Display with figure_image()
  • Storage of multiple noise covariance matrices (just like the head models)
  • Always save as full, then at inversion time, we can decide between full, heteroskedastic (diagonal) or homoskedastic (i.i.d, scalar)

  • Problem of having inividual trials + averages in the condition => Display warning or not?

  • Save nAvg in noisecov file, to make it easier to scale to other recordings

• Sources on surface: Display peak regions over time (time = color) => A.Gramfort

• Calculate ImagingKernel * Gain for a scout

• Beamformers from FieldTrip (LCMV, SAM)

• Time-frequency beamformers:
  • Band-pass everything in different frequency bands + Source estimation + TF
  • Ask data to Sarang where he sees effects that cannot be extracted with MN followed by TF
• Process "Extract scouts time series": Add PCA option (replace isnorm with choice PCA/Norm)
• BEM: Fix unstable results when one vertex is too close from the layers (5mm ?)
• Hui-Ling beamformers:
  • More explanations about what is in NAI and Spatial filters

  • Explain that is this is better to study effects extended in time (Ntime > Nsensors)

  • Group LCMV+MCB
  • Condition LEFT median nerve: very bad results

• Menu Sources > Simulate recordings:

  • Do not close the 3D figures after generating a new file
  • Add a process equivalent to this menu
• Keep options for inverse computation
• OpenMEEG: Post example datasets for the remaining issues:

  • https://github.com/openmeeg/openmeeg/issues/64

  • https://github.com/openmeeg/openmeeg/issues/68

  • Example protocol ECOG: doesn't work

Anatomy

• MNI coordinates:

  • Extend to non FreeSurfer volumes (BrainSuite volumes that are not 256x256x256)

  • Compute with FieldTrip: ft_prepare_sourcemodel (linear and non-linear) => Peter

• Project all sub-cortical structures to default anatomy (check code from Denis S)

  • Output is not created in the code => Crashes

  • Reloading at the last line (db_reload_protocol) doesnt work

  • Make sure to update all the fields in GridAtlas (Vert2Grid, Grid2Source) when projecting

• FreeSurfer:

  • Do not save "mid" surface unless specified explicitely with the process version
  • Add cerebellum to default model generated with "Import FS anatomy"
• Import MRIs with different resolutions: re-interpolate automatically
• Edit fiducials: Replace 6 text boxes with 1 for easy copy-paste (see fiducials.m)

• BrainVISA: Add support for MarsAtlas (Guillaume A)

• BrainSuite:

  • Use BrainSuite inner skull for surface generation

  • Use same colors for left and right for anatomical atlases
• Warping: Scale option has to be fixed, it is currently very unstable
• Scouts:
  • Display edges in the middle of the faces instead of the vertices
  • Display scouts in a tree: hemisphere, region, subregion
  • Downsample to atlas: allow on timefreq/connect files
  • Sort scouts by region in process options

• Menu head model > Copy to other conditions/subjects (check if applicable first)

• Generate mixed density surfaces
• Optimize computation interpolation MRI-surface (tess_tri_interp)
• Render surface envelope in the MRI as a thin line instead of the full interpolation matrix
• Major bug when importing surfaces for an MRI that was re-oriented manually
• Smooth surface: Fix little spikes due to irregularities in the mesh
• Add eyes models to attract eye activity
• tess_mrimask: Not robust enough to major holes in the brain
• Bug: Hide scouts in the preview of the grid for volume head models

ECOG/SEEG

• Contact positions: Import / set / detect
  • Import/export electrodes positions in MNI/SCS/MRI coordinates

  • Co-register MRI and CT for electrodes marking in the MRI Viewer
    => Mutual information: BrainVISA > Outils > Recalage rigide

  • Automatic segmentation of CT scans: See project of Rodrigo Paz (Marseille/Timone)

  • Import from FreeSurfer/FreeView: Problem of difference between RAS and TkRegRAS: http://neuroimage.usc.edu/forums/showthread.php?1958-SEEG-electrodes-and-subject-s-anatomy-are-not-alligned

  • Add history: Save modifications and transformations applied to the channel files (Marcel)
  • ECOG: How to handle cases where not all the grid contacts are in the channel file? (Marcel)
  • Project contact positions across subjects (Marcel)
• Display:
  • Display SEEG+ECOG contacts at the same time
  • Better way to represent ECOG strips that are not in contact with the skull (new type?)
  • SEEG length: count from the most superficial contact
  • SEEG default view: transparent cortex instead of MRI viewer (Marcel)
  • ECOG: Project on cortex instead of inner skull, just for display (Marcel)
  • Color each strip/grid differently
  • Bad channels: Contacts greyed out instead of ignored (Marcel)
  • ECOG: Topography display without the 3D coordinates (Marcel)
  • 2DLayout: Should not try to use the actual position of the sensors, just the grids/strips.
  • Display time in H:M:S

• Alternatives to OpenMEEG: SimBio/FieldTrip? Matti Stenroos? NFT/NIST?

• Prepare tutorial with public dataset (Marcel)

Statistics

• ANOVA:
  • Use LENA functions(?)
  • Output = 1 file per effect, all grouped in a node "ANOVA"
  • Display several ANOVA maps (from several files) on one single figure, using a "graphic accumulator", towards which one can send any type of graphic object
• New process to test for Gaussianity using swtest
• PLS: Partial Least Squares
• Simulate recordings with specific properties, for stat validation

• Quality control before statistics, on condition averages across subjects:
  mean(baseline)/std(baseline): shows bad subject quickly.

• Use SurfStat: Impements interesting things, like an analytical cluster-based p-value correction (Random-field theory which is used in SPM) - Peter

Input / output

• 4D file format:
  • Use reader from MNE-Python: mne.io.read_raw_kit (doesn't require Yokogawa slow library)
  • Reference gradiometers: Keep the orientation of the first or second coil?
  • Reference gradiometers: Add the sensor definition from coil_def.dat
  • Validate with phantom recordings that noise compensation is properly taken into account
  • References at too far from the head sensors in Marseille 4D system
  • The noise compensation is considered to be always applied on the recordings, not sure this assumption is always correct
  • 4D phantom tutorial (JM Badier?)
• Export to EDF
• EEG File formats:
  • Nihon Koden
  • Nicolet EEG: Get from from MNE

  • EEG CeeGraph

  • EGI: Finish support for epoched files (formats 3,5,7)
• BCI2000 Input (via EEGLAB plugin)
• EEGLAB import:
  • Support for binary AND epoched files (now it's one or the other)
  • Allow epoched files with recordings saved in external files
• BST-BIN: Add compressionto .bst
• BIDS-MEG: Automatic loading of BIDS-structured databases
• Review raw on all the file formats (ASCII EEG and Cartool missing)
• gTec EEG recordings: Read directly from the HDF5 files instead of the Matlab exports.
• Use new Matlab functions readtable/writetable (2006b): for Excel and text files

Distribution & documentation

• Workflows FieldTrip: http://www.fieldtriptoolbox.org/faq/what_types_of_datasets_and_their_respective_analyses_are_used_on_fieldtrip

• Reference tutorials on Google scholar + ResearchGate

• Google Analytics: Create template and update the section of the Community page
• Finish existing tutorials:
  • Dipoles
• New tutorials:
  • MEG steady-state / high-gamma visual
  • Intra-cranial recordings
  • MEG connectome (impossible without the head shapes)
  • Coherence (cortico-muscular ?)
  • Co-register MEG runs (Beth)
• Missing in the introduction tutorials:
  • Description of the .pos files (different folder when in .ds folder or when imported after)
  • Volume scouts
  • Time-frequency: Description of "log freq scale" option
  • Detect bad channels: Peak-to-peak
• Missing in page "Cite Brainstorm": Add all the methods used in the software

• Brainstorm File Exchange: Distribute scripts that users would like to share, but that are too specific to be included in the Brainstorm distribution. => On GitHub

• Deface the MRIs of all the tutorials

Current bugs

• Screen capture:
  • Bug on Win8/Win10: doesn't capture the correct part of the screen
  • Window managers with fading effect: captures the top window
• Image viewer:
  • Difficult to get to 100%
  • Buggy on some systems
• 2DLayout:
  • (TF) Images are too far apart with EEG 20 channels
  • (TF) Units are weird with % values
  • (TF) Difficult to navigate in frequencies: Scaling+changing frequency resets the scaling
  • (time series) Sometimes the lines are not visible
  • (time series) Does not work when DC offset is not removed
• Progress bar:
  • Doesn't close properly on some Linux systems
  • Focus requests change workspace when processing constantly (Linux systems)
• MacOS bugs:
  • Buttons {Yes,No,Cancel} listed backwards
  • Record tab: Text of epoch number is too big
  • Colormap menus: Do not work well on compiled MacOSX 10.9.5 and 10.10
• Matlab bugs:
  • Interface looks small on screens with very high resolutions: Reduce the resolution
• Event markers are not visible anymore with the sequence: Open MEG, open EOG, close MEG
• in_bst_data_multi: If trials have different sizes, output is random (the one of the first file)
• Edit scout in MRI: small modifications cause huge increase of the scout size
• Canolty maps computation: Fix progress bar
• Change of default anatomy does not reset the interpolation matrices in all the subjects.
  • After projecting sources once from subject to default anat, the interpolation is saved and not updated. Interpolations need to be removed manually before projecting again on a new template.

Geeky programming details

• Hide Java panels instead of deleting them
• Processes with "radio" and "radio_line" options: Replace with "radio_label" and "radio_linelabel"
• Interpolations: Use scatteredInterpolant, griddedInterpolant, triangulation.nearestNeighbor (2014b)
• bst_warp and channel_project: Use tess_parametrize_new instead of tess_parametrize
• Shared kernels: "get bad channels" operation in a different way (reading all the files is too slow)
• Optimize bst_get:
  • Now study and subject have necessarily the same folder name
  • Replace big switch with separate functions
• Progress bar:
  • Add different levels (to handle sub-processes)
  • Make work correctly with RAW on resting tutorial
  • Uniformize calls in bst_process/Run
  • Add a "Cancel" button
• Fix all the 'todo' blocks in the code
• Error message: Add a link to report directly the bug on the forum