I am currently making a custom BST process based on the process_example_customavg.m from the "how to write your own process" tutorial. While I was working on this project I realised that the "process_example_customavg.m" assumes that all individual epochs have the SAME number of time samples. The below is an excerpt from "process_example_customavg.m" which shows that BST uses the first file in the list to initialise the loop which will serve as the template for generating the AllMat matrix. Meaning that the load matrix: [Nchannels x Ntime x Nepochs] will be constrained by the first file of the list.
% ===== LOAD THE DATA =====
% Read the first file in the list, to initialize the loop
DataMat = in_bst(sInputs(1).FileName, [], 0);
epochSize = size(DataMat.F);
Time = DataMat.Time;
% Initialize the load matrix: [Nchannels x Ntime x Nepochs]
AllMat = zeros(epochSize(1), epochSize(2), length(sInputs));
% Reading all the input files in a big matrix
for i = 1:length(sInputs)
% Read the file #i
DataMat = in_bst(sInputs(i).FileName, [], 0);
% Check the dimensions of the recordings matrix in this file
if ~isequal(size(DataMat.F), epochSize)
% Add an error message to the report
bst_report('Error', sProcess, sInputs, 'One file has a different number of channels or a different number of time samples.');
% Stop the process
return;
end
% Add the current file in the big load matrix
AllMat(:,:,i) = DataMat.F;
end
Unfortunately, the epochs that I am working with do NOT have the same number of time samples.
Each "epoch" (screenshot shown below) is actually a visually identified sleep spindle (imported from a raw continuous file as events) with each spindle having a different time duration and, therefore, a different number of time samples.
Therefore my question is how can I amend the above code so that BST can generate a matrix similar to "AllMat" but it can accommodate the addition of individual epochs with different time samples?
Many thanks in advance!
Which is a reasonable assumption as the process computes the average across epochs, which may synch by an event as in the case of ERPs.
In the example code, the variable AllMat exist to aggregate the data from all the epochs, and be able to run the average across the trial dimension. If you goal is to average the data around a given event in the epochs, it would make sense to do something like this in the process:
Make a for loop to get all the time durations
Find the longest common duration
Extract the data from these longest common duration for each epoch
Now data can be stored in matrix as AllMat, and averaged
Thank you for these ideas Raymundo. However, I had something different in mind.
My intention is to input every single epoch (which in this case is a sleep spindle) and run process_X (which will calculate A, B and C) which will do the following.
Input X number of epochs (number of time points variable in each epoch).
Run process_X to calculate A, B and C using data from epoch 1 and put the resulting measurements into the first columns of matrices A, B and C respectively containing data only from measurements A, B and C respectively .
Run process_X to calculate A, B and C using data from epoch 2 and put the resulting measurements into the second columns of matrices A, B and C respectively containing data only from measurements A, B and C respectively and so on and so on for X number of epochs.
Therefore eventually you will end up with 3 matrices (and output 3 files) A, B and C each with 6 channels and X number of epochs on the x-axis with the 3 matrices each containing a different measurement (e.g. amplitude, frequency, duration).
After looking at your answer and the "how to write your own process tutorial" it seems that the type of process that I need is actually not "custom" but either "filter" or "file" (as my intention is to treat each file independently in the input list).
While this method is useful (and serves the purpose of calculation) the downside is that it quickly generates a LOT of files. If you have 40 epochs (or 40 sleep spindles in my case) that means I would have 40 X 3 (3 calculations) = 120 files.
Therefore I need some way to not end up with individual files but with something like the below where each individual result from each calculation is progressively saved in a big matrix containing all the calculated measurements from all epochs. Meaning, at the end of process_X, I would have three files A, B and C each containing all the individually calculated measurements A, B and C from all epochs (hope the diagram below makes sense).
Just wondering if this can be done through sProcess.Category = 'File'; ?
Or something else has to be done?
Thank you immensely for your time!
Thanks for the detailed overview on the projected process.
No, this cannot be done with category File, as this category treats each file independently.
There are at least two ways you can handle this:
With category File, then concatenate the output files, and delete the individual files.
Thus this will imply to call process_concat() and process_delete() three times: for A files, for Bfiles and for C files. This option is equivalent to your first diagram.
With a category * Custom, as there is no need of the individual A, B and C matrices, the concatenation of these can be inside the same process. This is more appropriated as it will not save intermediate files. This option is equivalent to your second diagram.
If individual A, B and C files are needed, this process can be called once for each file.
In option 2, the idea is the have a function extract_a_b_c() in the process, then the process will concatenate those results and save them in a file. Something similar to that happens in the Extract function in process_extract_values.m
With option 2 of using the custom process category, I am not sure if this will work at all because each of my individual epochs have unique time samples therefore it will be impossible for the process to concatenate all epochs into one big matrix?
The idea is to concatenate the results of processing each epoch, not concatenating the epochs.
Thus the Run function of the process, may look something like this:
% Initialize variables to concatenate results
finalMatrixA = zeros(nChannels, nInputs);
finalMatrixB = zeros(nChannels, nInputs);
finalMatrixC = zeros(nChannels, nInputs);
% Process each Input
for iInput = 1 : nInputs
% Compute a,b, and c per file
[matrixA, matrixB, matrixC] = compute_a_b_c(sInputs(iInput));
% Concatenate computed matrices
finalMatrixA(:, iInput) = matrixA;
finalMatrixB(:, iInput) = matrixB;
finalMatrixC(:, iInput) = matrixC;
end
% Save the three final matrices
% Register the three final matrices
!