Custom process retunrs outputfiles of different types

Discussions
,
1

Hello,

I am currently writing a process to simulate NIRS data. The process takes as input some data and outputs the data with some added activity + the ground truth map on the cortex

however, I faill to have the file properly registered in the database as I get the following warning after running the process :

BST> Warning: 3 file(s) not found in database.

If I reload the condition, then the file appears. It is however long to reload the condition (esp when simulating a lot of signals)

The weird thing is :

  • If I save only the data, then everything works fine
  • If I save only the map, then it works fine too (if we omit the warning about the map missing its data file)

but when I save both, then the file don't get registered.

Here are the relevant piece of code:

'''matlab

% Definition of the input accepted by this process
sProcess.InputTypes  = {'data', 'data'};
sProcess.OutputTypes = {'data', 'results'};
sProcess.nInputs     = 1;
sProcess.nMinFiles   = 1;




    sDataOut = db_template('data');
    sDataOut.F            = data_simul; 
    sDataOut.Comment      = sprintf('simul | %s | %s  SNR =  %.2fdb',Atlas_name,ROI_name, activation.options.SNR) ;
    sDataOut.ChannelFlag  = ones(size(data_simul, 1), 1);
    sDataOut.Time         = sData.Time;
    sDataOut.DataType     = 'recordings'; 
    sDataOut.nAvg         = 1;
    sDataOut.Events       = event;
    sDataOut = bst_history('add', sDataOut, 'process', sProcess.Comment);
    sDataOut.DisplayUnits = 'delta OD';

    % Generate a new file name in the same folder
    OutputFile_data = bst_process('GetNewFilename', bst_fileparts(sStudy.FileName), 'data_sim');
    sDataOut.FileName = file_short(OutputFile_data);
    bst_save(OutputFile_data, sDataOut, 'v7');
    % Register in database
    db_add_data(iStudy, OutputFile_data, sDataOut);
    OutputFiles{end+1} = OutputFile_data;

    ResultsMat = db_template('resultsmat');
    ResultsMat.Comment       = 'Ground Truth';
    ResultsMat.DataFile      = file_short(OutputFile_data);
    ResultsMat.Function      = '';
    ResultsMat.Time          = sData.Time;
    ResultsMat.ImageGridAmp  = groundTruth;
    ResultsMat.ChannelFlag   = [];
    ResultsMat.GoodChannel   = [];
    ResultsMat.DisplayUnits  = 'delta OD';
    ResultsMat.SurfaceFile   = nirs_head_model.SurfaceFile;
    ResultsMat.simulation_options    = activation;
    % % Save new file structure
    OutputFile = bst_process('GetNewFilename', bst_fileparts(sStudy.FileName), 'results_ground_truth_simul');
    bst_save(OutputFile, ResultsMat, 'v6');
    % Update database
    db_add_data(iStudy, OutputFile, ResultsMat);
    OutputFiles{end+1} = OutputFile;

'''

The complete process is here: process_nst_simul_nirs.m ยท GitHub

thanks a lot,
edouard

2

The issue is mixing types of files in OutputFiles. After running the process, its OutputFiles are prepared to be the input of a following process (if any). In your case the outputs are from different type, then some cannot be found in the DB as it looks per type (the first output type).

Files are already registered in the DB, but not shown in the tree.
Registration happens in db_add_data()

A solution is to return an empty OutputFiles, and prompt the redraw of the Study node (with the registered data) as last thing in your process Run() function.

% Update tree 
panel_protocols('UpdateNode', 'Study', iStudy);

Yes, that is why we are 'redrawing' the Study node, it's content is already in the database.
Thus, reloading is not needed. Reload is slower as it goes to the HDD and regenerates the database entry for the Study with the files in there.

3

thanks a lot,

Its working