Process_average outputs 2 files instead of 1

Hi Brainstorm,

I have a list of 30 files, all from the same condition and from different subjects, that I want to average together, but the process outputs 2 averages instead of 1, one average that is computed from

Sorry, the message was posted before I finished writing.

I was explaining one average is computed from 12 out of the 30 file, and the other average from the remaining 18.

Any idea why this happens?

I attach the corresponding report report_BBvsCLIC_scripted_200121_2140.html (2.5 KB)

Let me know what kind of additional useful information I can provide.

Dorothée

Hi Dorothée,

I can't see what process options you chose from your report, but if you want to average all of the input files together you need to make sure you select "Group files: Everything" (the very first option).

The different options available in the Average process are detailed in the following tutorial: https://neuroimage.usc.edu/brainstorm/Tutorials/Averaging#Process_options:_Average

I hope this helps,
Martin

Thank you Martin for your reply,

I did not use the option you mentioned, I will try this as soon as I retrieve access to my database. However the behavior I observed still seems unexpected to me, so maybe it is worth understanding what happens? I had used the following matlab code:

sFilesGrandAvg = bst_process('CallProcess', 'process_average', sFilesAvg, [], ...
'avgtype', 7, ... % By trial group (grand average)
'avg_func', 1, ... % Arithmetic average: mean(x)
'weighted', 0, ...
'keepevents', 0);

I realize choosing 'avgtype' as 'By folder (grand average)' was maybe more appropriate, and I will have to check what happens with this setting too. But why does the option 'By trial group (grand average)' result in 2 files, and not 1 or 30, or any other number? I don't understand how the files get grouped here.

The files are grouped by names.
Try with the interface first to prototype your analysis, and use the scripted version only when you have defined all the options and you already know what you are doing.

Hi Francois,

The files are grouped by names.

That's indeed what I read in the online tutorial, but here all file names have the same prefix and they all differ by the time suffix (see the report), so the grouping process remains obscur to me.

Try with the interface first to prototype your analysis, and use the scripted version only when you have defined all the options and you already know what you are doing.

That's in fact what I did : I prototyped an analysis using a set of files, and wanted to replicate the same analysis on a different set of files, so I scripted it and ran the scripted version on the two different input sets (which have the same number of files each, coming from the same 30 subjects). But with the first input set I get a single file as output (as I expected) whereas with the second I get two. Hence my post.

Hi Dorothée,

The reason you are getting more than one output is that in your script you chose to group by trial group (AvgType = 7), so Brainstorm is returning one average file per trial group. You probably had a single trial group in your first input set and two in the second, hence the discrepancy (I can't see the name of the files in the report, just their path).

If you simply want to average everything together regardless of input, you can adjust the average type to 1 in your script, as follows:

sFilesGrandAvg = bst_process('CallProcess', 'process_average', sFilesAvg, [], ...
'avgtype', 1, ... % Everything
'avg_func', 1, ... % Arithmetic average: mean(x)
'weighted', 0, ...
'keepevents', 0);

Like François said, the best way to adjust your script is with prototyping your analysis from the pipeline editor.

I hope this helps!
Martin

Thanks again Martin for replying,

I have already sorted this using the option 'avgtype'=1, as you suggested earlier. My point insisting was only to notify to you guys what seems to me as an unwanted behavior. If 'file name" refers to the 'Comment' field of the matlab structure, then all 30 input file names are exactly the same in both of the previously described cases, so there is no obvious reason to me why they would get sorted into two groups.

Sorry if this all bothers you, I'm just trying to contribute.

You mention a "time suffix" in the comments of the file.
This is maybe formatted in a way that is confusing for the parser that decides how to group epochs in groups.

The files should be grouped by the averaging process in the same they are in the database explorer.
Could you post a screen capture to illustrate how the files are grouped in the database, so we can evaluate whether we should change anything in this algorithm?

Thank you for trying to help us improve the software!

You mention a "time suffix" in the comments of the file.

I was referring to the suffix in the name of the actual .mat file (the file path).

Ok, I found the cause of the splitting. When the input files were generated, some had a suffix _02 added by Brainstorm to the comment, and some did not. This happened non-homogeneously across subjects because some subjects already had an average computed with the exact same number of trials and therefore the same name (with the number of files between brackets), whereas the others also already had an average computed but with a different number of trials.

I apologize I had not noticed this earlier. I must have been very tired when I checked earlier because this now looks blindingly obvious to me. As you can see in the screenshot, I added the tag ‘EqualN’ to these files after they were generated, this is how I select them as input to the grand average process. Obviously, the problem did not occur when I prototyped the analysis because I prototyped it when computing the grand average of the initial subject-level averages (the ones that do not have the tag ‘EqualN’).

So I’m not sure this is a case that the parser should generally identify. The parser is not sensitive to the number of files indicated between brackets in the name, which is a good thing, but I guess it makes sense that it can discriminate between files that hold a _02, or _0n suffix, or none.