ENH: Deep source modeling

[payamsash]

Describe the new feature or enhancement

looking at roadmap and #6784:

now (since 7.x FS) segment_subregions gives proper subsegmentations (hippo, amygdala, thalamic nuclei, and brainstem), also FS 8.x now runs SynthMorph as part of recon-all so every subject already have subject-to-template deformation field saved. So for morphing problem, instead of trying to inflate surfaces to a sphere, we could just take the saved deformation field and apply it directly to the vertex coordinates and then we can find the corresponding template mesh with nearest neighbour or ICP. no?

I mean these files:

mri/transforms/synthmorph.1.0mm.1.0mm/warp.to.mni152.1.0mm.1.0mm.nii.gz        # subject → MNI152
mri/transforms/synthmorph.1.0mm.1.0mm/warp.to.mni152.1.0mm.1.0mm.inv.nii.gz    # MNI152 → subject

Describe your proposed implementation

if yes, then:

1. we can modify setup_source_space or new setup_subcortical_source_space to create SourceSpaces
2. morph with saved volumetric warp, not spherical inflation
3. and a plotting support in Brain

Describe possible alternatives

.

Additional context

Happy to open PRs for that, but better to discuss first. @larsoner

[ttaiv]

Hi @payamsash! I am currently further developing this Cere-MEG-Bellum (CMB) package that extends the functionality of MNE-Python by adding cerebellum to the surface source space. It fits a custom high-resolution template of the cerebellum to the subject's MRI and then uses this fitted template to construct a source space that includes the cerebellum.

To better integrate with MNE-Python, my plan is to follow the API proposal laid out in #6784, meaning that instead of having custom source space creation logic, CMB would output FreeSurfer-compatible .surf files to subjects_dir/subject... and then setup_source_space (or the new setup_subcortical_source_space which might make more sense for tighter subcortical spacing) would consume them.

I think leveraging the subsegmentations from FreeSurfer and the fitted cerebellum template from CMB for source space creation have many things in common, so we might be able to build support for both at the same time. I haven't started to implement MNE-Python's side of this yet, so I'm happy to let you take the lead if you have time now.

[payamsash]

Thanks @ttaiv, I'd be happy to take the mne side. Here is my suggestion:

1. setup_subcortical_source_space(subject, label, aseg=...) -> SourceSpaces similar to Brain.add_volume_labels (only for the aseg labels, for the 1st PR)
2. once settled, we can put surface= for FS .surf file or even (rr, tris), so at this point we can also plug in meshes from CMB.
3. Morphing, which I'm more interested in SynthMorph warp rather than spherical inflation ... but could be discussed later ...

wdyt?

[ttaiv]

Sounds very good to me! But would it actually be easier to implement surface=... first (or at the same time)? Because in that scenario we already have the mesh ready, which could simplify things. I would be happy to act as a tester for that PR using CMB cerebellum meshes (and I can also share some with you).

Regarding morphing: Because all the meshes created by CMB are essentially the same template mesh just warped to the subject space, my understanding is that there is already direct 1:1 correspondence between the vertices of different subjects (assuming that the same subsampling of the template mesh was used). So I think there is no need to do warping (but still probably need to smooth/interpolate). But for aseg-based meshes leveraging SynthMorph sounds very smart!

I wonder if @larsoner has any insights?

[larsoner]

If CMB already warps a template mesh then the 1:1 is convenient to use directly there. For everything else (?) and maybe also sometimes Cerebullum (if people want) then we can figure out how to make mne.SourceMorph "do the right thing". I think it makes sense to first proceed with a proof-of-concept on sample, then once that works, add the same surfaces for fsaverage, once that works, draw up the morphing mechanism. It's often easier to see the best API once you actually start trying to use it. And it's not too much work to rename and move code if it's factored properly into helper functions or classes to start.