Search and select files/nodes faster

Discussions
1

Hello,

I am currently running some simulations and some metrics.

Insite my database, I have a bunch of simulations:

image
image361×637 4.81 KB

Under each simulation, I have a bunch of source localization results and a ground truth:

image
image490×697 17 KB

What I want is for each simulation to get the list of the files under the simulation so I can run the code computing the metrics :

What I have done is first get the list of simulations, then grab all the files that have parents of those simulations:

The script looks like this


tic
sFiles = bst_process('CallProcess', 'process_select_search', [], [], ...
    'search', '(([name CONTAINS "Avg: Task"] AND [parent EQUALS "simulation_task_0db_small"]))');

toc


MetricFile = '/Users/edelaire1/Documents/Project/wMEM-fnirs/data/metrics/metrics_simul-task_00db_small.csv';

for iFile = 1:length(sFiles)



    % Process: Select files using search query
    sMaps = bst_process('CallProcess', 'process_select_search', [], [], ...
        'search', [ sprintf('([parent CONTAINS "%s"] ' ,   sFiles(iFile).Comment), ...
                   'AND' , ...
                   '(  [name CONTAINS "Ground Truth"]   OR  [name CONTAINS "cMEM"] ))']);


    % Process: Compute statistics
    bst_process('CallProcess', 'process_nst_wMEM_metrics', sMaps, [], ...
                                         'textFile', {MetricFile, 'csv'});

end

However, each call to process_select_search is taking from 1 to 2 minutes. so it will take between 1 and 2 hours to run the script just because of the file selection.

Is there a faster way to achieve what I want to do?

Thanks a lot,
Edouard

2

You can query directly the database:

This should do the trick:

[sStudy, iStudy, iResults] = bst_get('ResultsForDataFile', DataFile);
ResultFiles = sStudy.Result(iResults);
Comments = {ResultFiles};
1 Like
3

Thanks. it works.

Here is the updated script:




sFiles = bst_process('CallProcess', 'process_select_search', [], [], ...
    'search', '(([name CONTAINS "Avg: Task"] AND [parent EQUALS "simulation_task_0db_small"]))');



bst_report('Start', sFiles);

MetricFile = '/Users/edelaire1/Documents/Project/wMEM-fnirs/data/metrics/metrics_task/metrics_cMEM_simul-task_00db_small.csv';

for iFile = 1:length(sFiles)

    % Get results files 
    [sStudy, iStudy, iResults] = bst_get('ResultsForDataFile', sFiles(iFile).FileName);
    ResultFiles = sStudy.Result(iResults);
    

    % Select Ground truth and cMEM
    sMaps = ResultFiles( contains({ResultFiles.Comment}, {'Ground Truth', 'cMEM'}));

    % Process: Compute statistics
    bst_process('CallProcess', 'process_nst_wMEM_metrics', {sMaps.FileName}, [], ...
                                         'textFile', {MetricFile, 'csv'});

end

% Save and display report
ReportFile = bst_report('Save', sFiles);
bst_report('Open', ReportFile);