Importing a Brainstorm protocol from an older Brainstorm version and opening raw files

Bugs
1

First, apologies; I am unsure whether this is actually a bug as there are multiple compounding factors.

In 2016 we analyzed some MEEG data in brainstorm (on a unix server with brainstorm also running in unix). This week I wanted to reopen some of the files for further analyses, and as I had a copy of the protocol folder on a hard drive, opened the protocol on a windows 10 machine with the latest Brainstorm version (02-May-2019) and Matlab R2018b (file->load protocol->load from folder). Mostly, everything works perfectly except I cannot view the raw .fif files. First, the links to raw .fif files needed updating. When trying to open the file (i.e. MEG (all) -> display time series and then finding the correct file using the brainstorm prompt that pops up), it clearly loads the file to some extent as, for example, the Events window updates and shows the correct event structure of the file. However, the 'View time series' window is stuck with the message "Loading data..." and an error message is printed in the Matlab's Command Window (please see below; also please see https://imgur.com/BRvha79 for a screencap). A similar message (see link at the very bottom of the page) pops up when I try to, for example, filter the raw data file. Any advice would be appreciated - is this an error between keyboard and chair or is there something else that has gone awry?

###ERROR MESSAGE WHEN ATTEMPTING TO VIEW TIME SERIES

Error using fiff_read_tag (line 232)
Cannot handle other than dense or sparse matrices yet

Error in fif_read_raw_segment (line 100)
tag = fiff_read_tag(sfid,this.ent.pos);

Error in in_fread_fif (line 54)
[F, TimeVector] = fif_read_raw_segment(sFile, sfid,
SamplesBounds, iChannels);

Error in in_fread (line 84)
[F,TimeVector] = in_fread_fif(sFile, sfid,
iEpoch, SamplesBounds, iChannels);

Error in panel_record>ReadRawBlock (line 1173)
[F, TimeVector] = in_fread(sFile, ChannelMat, iEpoch,
smpBlock, [], ImportOptions);

Error in panel_record (line 30)
eval(macro_method);

Error in bst_memory>LoadRecordingsRaw (line 888)
F = panel_record('ReadRawBlock',
GlobalData.DataSet(iDS).Measures.sFile, ChannelMat,
iEpoch, TimeRange, 1, RawViewerOptions.UseCtfComp,
RawViewerOptions.RemoveBaseline, UseSsp);

Error in bst_memory>LoadRecordingsMatrix (line 822)
DataMat.F = LoadRecordingsRaw(iDS);

Error in bst_memory>GetRecordingsValues (line 1940)
LoadRecordingsMatrix(iDS);

Error in bst_memory (line 72)
eval(macro_method);

Error in figure_timeseries>GetFigureData (line 2269)
[Fall, StdAll] = bst_memory('GetRecordingsValues',
iDS, [], [], isGradMagScale);

Error in figure_timeseries>PlotFigure (line 2352)
[F, TsInfo, Std] = GetFigureData(iDS, iFig);

Error in figure_timeseries (line 39)
eval(macro_method);

Error in view_timeseries (line 241)
isOk = figure_timeseries('PlotFigure', iDS, iFig);

Error in
tree_callbacks>@(h,ev)view_timeseries(filenameRelative,AllMod{iMod},[],'NewFigure')
(line 1275)
gui_component('MenuItem',
jMenuModality, [], 'Display
time series',
IconLoader.ICON_TS_DISPLAY,
[],
@(h,ev)view_timeseries(filenameRelative,
AllMod{iMod}, [],
'NewFigure'));

###ERROR MESSAGE WHEN FILTERING THE DATA, IN A BRAINSTORM POPUP WINDOW

https://imgur.com/ZtGviJN

2

Hello,

For link to raw files, the metadata of the file such as events is saved in the Brainstorm protocol but not the actual data of the file. This is why you had broken links and had to find the file again on your hard drive, but the events were still present. Are you sure you chose the exact same raw file you used in 2016? From your error message, it seems to me like the new FIF file you chose is not supported by our FIF reader.

What I would try first to make sure the issue is not with the raw file you chose would be to try to review as raw this same file on a new protocol/subject and see if it works properly. If it does, then there is a more complex issue that will need further debugging. If you have issues reading the file, it's probably because our FIF reader is quite outdated (we're using MNE's Matlab implementation dating from 2008 so it is very possible...)

Let us know,
Martin

3

Thank you for the tips and explanation, Martin!

I do not have issues opening the raw file in another protocol, or even within the same protocol. It seems that the issue arises only when I attempt to fix an existing link, not when I review a raw file and add it to an another subject, for example (see figures below where I do exactly that).

1 - attempt to fix the link: https://www.mv.helsinki.fi/home/ejpartan/brainstorm/1-relink.JPG
2 - choosing the file to relink: https://www.mv.helsinki.fi/home/ejpartan/brainstorm/2-filechoose.JPG
3 - the error message: https://www.mv.helsinki.fi/home/ejpartan/brainstorm/3-fail.JPG
4 - choosing another subject to review the raw file: https://www.mv.helsinki.fi/home/ejpartan/brainstorm/4-reviewraw.jpg
5 - selecting the same file as in above: https://www.mv.helsinki.fi/home/ejpartan/brainstorm/5-openraw.jpg
6 - after the event channel popup displaying raw MEG data: https://www.mv.helsinki.fi/home/ejpartan/brainstorm/6-openimported.jpg
7 - success: https://www.mv.helsinki.fi/home/ejpartan/brainstorm/7-fileopen.jpg

4

Then I suggest you delete the old file links and create new ones.

5

I agree that this would be a solution but we would lose all the SSP projectors and other preprocessing we've done on the files by hand, right? There are epoched files available as well, but we are actually going back to the old data specifically so that we would not have to do all the preprocessing steps again. =S

6

Indeed, it would be better if you could read the files with the older links.
Maybe there were some changed in the way these links are created since 2016, it's difficult to say.

If you would like us to have a look at this, please prepare an example dataset so we can reproduce the issue:

  1. Right-click on a subject > File > Duplicate subject
  2. In the duplicated subject, delete everything but the faulty link
  3. Right-click on the duplicated subject > File > Export subject
  4. Upload the .zip file somewhere and post the download link here
  5. Rename the duplicated subject
  6. Delete the faulty link
  7. Link again the same FIF file (the link should now work)
  8. Export as .zip, upload, post the download link
  9. Upload the .fif file somewhere and post the download link here.

With these three files, I should be able to reproduce the error on my end.

7

Thank you for looking into this; the files should be found here:

https://filesender.funet.fi/?s=download&token=7da829d8-f1b9-43dc-9474-876e614650fb

In short, the duplicated subject is in the file labeled WordMemoryTraces_S04block1_copy
The duplicated subject where the link was deleted and the same file linked again is the file WordMemoryTraces_S04block1_copy_linkedAgain
the .fif file is attached.

8

Hello,

I looked at the links you sent, and noticed that the Unix timestamps in the files headers (LinkMat.F.header.tree.id.secs) indicate that the old link and the new link refer to different files.

Even if they have the same name and very similar sizes, the file was initially link was created on Tuesday 24 February 2015, while the FIF file you sent me was created on Saturday 25 June 2016.
The two files have slightly different header sizes, making it impossible to reuse the header saved previously with the new file: the various data blocks are not at the same positions in the two files.

In conclusion, the cause of the error is that you are not selecting the correct FIF file when trying to fix the link. Or that your FIF file was modified between the moment you linked it initially in Brainstorm and now.

If you are sure the two files are the same and want to reuse the SSP projectors, you can probably copy-paste the old channel file to the new folder. To copy the events: load one file, export them to a text file, load the other file, load the events from the text file.

9

Thank you again for looking into this! The files have to contain the same information; it is just that originally the link refers to the file that was on our Unix server, the file I am trying to relink is a backup of the same file. I say that they must have the sam information because I have the whole backuped folder of the original data on a separate hard drive. Perhaps this copying from the server and to a different OS has had an effect, resulting in differences in file header? =S

Nevertheless, I will have a try to copy-paste the old channel file to the new folder to spare the SSP projectors etc., thank you for the suggestion!

10

Copying files between file systems may change the dates of creation/modification, but not the actual contents of the file. These files were saved in different ways, or one is a re-processed version of the other (any Elekta program could have made this change).