New error in bst_process('CallProcess', 'process_channel_addloc', sFiles, [],

Bkavanaugh · June 26, 2024, 8:02pm

Okay, so I ran it on my personal laptop (not the virtual environment) and it worked without issues...Using the same script I sent you, "script_new".

Yes, you are right, the sInputs is different between the personal laptop that worked and the virtual environment that didn't work.

When it worked (personal comp): sInputs is a 1x30 struct, reflective of the 30 epochs.
When it didn't work (virtual): sInputs is a empty [].

Importantly, I checked the sInputs for the step prior to eeg channel positions (i.e., importing epochs). In both situations, the sInputs is correctly 1x30 struct.

So it looks like that during eeg channel positions (on the virtual environment), the sInputs is switched from 1x30 struct to empty? Does that make any sense?

(I deleted and re-downloaded brainstorm for both environments before re-running everything today.)

Thanks Raymundo!

Raymundo.Cassani · July 3, 2024, 4:51pm

Do they have the same OS?

What do you call a "virtual environment"?

Bkavanaugh · July 9, 2024, 3:00pm

No, they are different OS.

Virtual environment is brown's oscar open on demand portal:

The Virtual Desktop runs XFCE on Linux (Red Hat EL 7.7).

Here are my dell laptop system specifics:

If that helps? I'm also checking with the virtual environment help group to see if they have insights

Raymundo.Cassani · July 10, 2024, 7:22pm

It seems there is an issue with resolving paths in the function GetInputStruct() in bst_process, when it is called in L278

sInputs = GetInputStruct(OutputFiles);

as there are OutputFiles but the sInputs is empty.

Unfortunately, I have not been able to replicate it, and it seems you cannot replicated either outside of the virtual environment. To find the origin of this issue, in the virtual environment, you would need to debug GetInputStruct(). Place a checkpoint in it to verify how it is executed.

github.com

brainstorm-tools/brainstorm3/blob/b20aa3cb3766bd91b49170f9e2cc0876c5ffa3c4/toolbox/process/bst_process.m#L1488


      
              OutputFiles{1} = GetNewFilename(bst_fileparts(sStudy.FileName), fileTag);

              % Save on disk

              bst_save(OutputFiles{1}, sOutput, 'v6');

              % Register in database

              db_add_data(iStudy, OutputFiles{1}, sOutput);

          end

          

          

          

          %% ===== INPUT STRUCTURE =====

          function sInputs = GetInputStruct(FileNames)

              % If single filename: convert to a list

              if ischar(FileNames)

                  FileNames = {FileNames};

              end

              % Output structure

              sInputs = repmat(db_template('processfile'), 1, length(FileNames));

              % Get file type for the first file

              FileType = file_gettype(FileNames{1});

              % Remove the full path

              ProtocolInfo = bst_get('ProtocolInfo');

Bkavanaugh · September 3, 2024, 2:17pm

Hi Raymundo,

Sorry for the delay.

I've been working with the brown IT group to help me figure this out. It seems like the issue emerges when I set my "brainstorm_db" to our larger group's data directory, but things work correctly when I set brainstorm_db to my home directory. Which makes sense looking back on when my issue emerged in June, in that I had to move to the data directory because I had run out of space in my home directory.

We specifically see that that individual epoched files do not get created when saved in the data directory (but are created when running in the home directory).

brown IT is wondering if some of the files in brainstorm are hard coded to the database path in home directory? Even though I know how to move the database path manually in brainstorm.

Curious about your thoughts.

Thanks again. This has been a long road but hoping we are close to solving

Raymundo.Cassani · September 3, 2024, 3:31pm

@Bkavanaugh, thank you for checking on your end.
I'll double-check all the steps at the creation of data files to find out the trouble

Bkavanaugh · September 4, 2024, 12:54pm

Thanks Raymundo!

Bkavanaugh · September 6, 2024, 12:32pm

Quick follow-up.

Here is a screen shot of when it works correctly in the home directory

Below, is when I put a symbolic link (in the home directory) to the brainstorm_db in the data directory. THe goal was to see if brainstorm woudl allow that while I could still store it in the data directory for space issues.

Weirdly, it worked, but it all the epoched files were deleted at some point in the process.

The brown IT team is confused as to why the epoched files would have been deleted in one directory but not deleted in the other directory.

Raymundo.Cassani · September 10, 2024, 3:17pm

The [deleted] legend is printed either if they were really deleted, or if they were not created in first place.

I created these pipeline (script_new_print_info.m) that uses (print_bst_info.m) to test the directory info and paths, file creating and file deletion while running your pipeline. Run the script_new_print_info while having print_bst_info in the same folder. Test your two setups and let us know the information that is printed in the Matlab command window.

script_new_print_info.m (2.9 KB)

print_bst_info.m (1.5 KB)

Bkavanaugh · September 12, 2024, 1:33pm

Hi Raymundo, okay great. Here is what they look like in the home directory (where it works) and data directory (where it does not work). What else can I provide?

Raymundo.Cassani · September 12, 2024, 2:29pm

From the paths, it seems you are setting the Brainstorm database to be:

For the home folder: * /oscar/home/bkavanau/brainstorm_240829_scr/brainstorm_db
For the data folder: * /oscar/bkavanau/data/bkavanau/brainstorm_240829_scr

If the folder brainstorm_240829_scr is the one containing the Brainstorm code, this is not advised. Please set a different directory for the program and for the database
https://neuroimage.usc.edu/brainstorm/Tutorials/CreateProtocol#Brainstorm_folders

When running in the data directory, once the error is produced.
What are the contents of these directories?

/oscar/bkavanau/data/bkavanau/brainstorm_240829_scr/Protocol01_tryingtoproblemsolve_ddata/data
/oscar/bkavanau/data/bkavanau/brainstorm_240829_scr/Protocol01_tryingtoproblemsolve_ddata/data/Subject002/
/oscar/bkavanau/data/bkavanau/brainstorm_240829_scr/Protocol01_tryingtoproblemsolve_ddata/data/Subject002/TB2_002_Rain_P1V11_T999_epoch_357dot_1

P.S. Provide the entire text displayed in the Matlab command window. Preferentially, share a text rather than image so we can inspect it easier.

Bkavanaugh · September 12, 2024, 4:10pm

Okay yes, I've re-run with the brainstorm_db folder.

i'm attaching a word doc copy of the matlab command window output.

I'm also including here pics of the three directory inquiries

Thanks Raymundo!

Raymundo.Cassani · October 4, 2024, 2:56pm

The logs look good, there are no problems creating files in those folders.

Did you have any luck with IT figuring out this behaviour?

I wonder if it could be related to the very long path that is generated.
Could you do a quick test renaming the protocol to something like Pro?

Bkavanaugh · October 15, 2024, 8:36pm

Hi Raymundo, Thanks again for all your help. I did not figure it out. But I found a temporary work around. It worked if I ran small batches of files in my home directory (this is the directory with a cap on space), and after each batch I deleted the brainstorm_db and started the next batch. Seemed to work for now!