Goodness of fit

Dear all

I want to know how I could optimize the parameters for the head model. Could you let me know how to compute the goodness of fit? I use EEG data (123ch, Neuroscan).

Regards
Masahiro

Hi Masahiro,

It was great to meet you in Quebec!

Once you have a source file available, right click over it and select “Model Evaluation”. this will compute the forward model of the source time series and save it as a new data file named : Simulation: <name of the source file>. To evaluate the goodness of fit to the original data, I suggest you drag and drop the original data file and the simulated data file in the Process2 panel and select to compute “A-B”. this will compute the difference between the data and the model. You can then export the contents of this file to Matlab (right click: File > Export to Matlab) and compute any metric of GOF you like (eg RMS).

Hope this helps,

Dear Sylvain,

Thank you very much for your reply.
I have tried to find the icon named “Model Evaluation”, but I could not find such an icon (attachment.jpg).
Could you check if my approach is correct?

Regards
Masahiro

Thanks for your screenshot.

Indeed the 'model evaluation' does not show up in your environment. Please see attached screenshot of what should be available.


  1. Please make sure you are running the latest version of Brainstorm (updated last June or later).
  2. It is possible that model evaluation is not available for 'full image models' of the sources, which would be a bug. Please make sure that when computing sources, you check the option of computing the 'Imaging Kernel' only. Right-click over the source results file and check whether the option for model evaluation is available.

Let me know how this goes and we'll move from there. Thank you!

Dear Sylvain,

Thank you very much for your advice and support.
Bellow is my results.

First, I tried computing source with using the version 3.0, and the “Model evaluation” was avairable in one subject data. Then, I tried computing the average (average everything) of the subjects, and also the “Model evaluation” showed up (version3.0_1).

In the version 3.1, when I followed your advice (I used the latest version and checked the option of computing the ‘Imaging Kernel’ only), the “Model evaluation” showed up in one subject data. However, the “Model evaluation” did not show up when I averaged the two subjects’ data, and imported the zip file data, which showed up the “Model evaluation” in version 3.0 (version3.1_1).

Now, I have other problems in the version 3.1.
(1) When I import the channel file, which was available in the version 3.0 (version3.0_2), the channel file does not work in the version 3.1 (version3.1_2).

(2) I import the averaged data in each subject. When I compute the subject grand average (average everything), the version 3.1 displays the average reference though I did not check “AVG REF”. In version 3.0, only if I check “AVG REF”, the average reference of EEG is displayed.

I hope I inform you enough about the current situation and the additional problems.

Regards
Masanhiro

Dear Masahiro,
Sorry for the late reply, I was away for a few months.

In the version 3.1, when I followed your advice (I used the latest version and checked the option of computing the 'Imaging Kernel' only), the “Model evaluation” showed up in one subject data. However, the “Model evaluation” did not show up when I averaged the two subjects’ data, and imported the zip file data, which showed up the “Model evaluation” in version 3.0 (version3.1_1).

I had disabled this menu in the "Intra-subject" and "Inter-subject" nodes because I was not exactly sure of the meaning of it.
I enabled those menus again, you can use them after downloading the latest brainstorm version (click on Help > Update Brainstorm).

(1) When I import the channel file, which was available in the version 3.0 (version3.0_2), the channel file does not work in the version 3.1 (version3.1_2).

When you say "import a channel file", what operation do you exactly refer to? And how is it "not working" ? Are you referring to the fact that the electrodes are located directly "on" the brain (which is, indeed, a little awkward)?
Can you describe exactly what you expected, what you did, and what you got at the end?

(2) I import the averaged data in each subject. When I compute the subject grand average (average everything), the version 3.1 displays the average reference though I did not check “AVG REF”. In version 3.0, only if I check “AVG REF”, the average reference of EEG is displayed.

[Post edited on 09.08.11, see following post]

Let me know if you have any further questions.

Cheers,
Francois

I apologize for publishing erroneous information yesterday.

Brainstorm is indeed supposed to keep the original reference of the EEG recordings during all the pre-processing and averaging process. Recordings files (in the database) are not necessarily converted to average reference, unless specified explicitly by applying the process “EEG Average reference”.

However, the source estimation process requires EEG data in average reference. When needed, the recordings are converted on the fly to average ref with the function bst_avgref.

As we are not intensively working with EEG ourselves, there might be some residual bugs in those processes, so please pay attention to what is happening. To get rid of all possible ambiguities, it can be good idea to force the recordings to be saved in the database in average reference, as part of the pre-processing pipeline.

So your initial problem remains unsolved: why is your average file saved in average reference?
Can you do it again, and try to identify what could be the step that converts the recordings to average ref ? Please let me know how it goes. If you observe the same thing, can you describe better what you are doing, and send a screen shot of your database explorer?

Thanks
Francois