Error in harmony: 'dimension cannot be changed'

[jeremymsimon]

Previously ran some scATAC integration with harmony using snapATAC v2.8, recently updated to 2.9 and installed everything in a fresh mamba environment. Unfortunately now I'm getting a puzzling error:

snap.pp.harmony(obj, batch="sample")
[...]

Traceback (most recent call last):
  File "integrationClustering.py", line 158, in <module>
    main()
    ~~~~^^
  File "integrationClustering.py", line 28, in main
    snap.pp.harmony(obj, batch="sample")
    ~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/jsimonlab/tools/miniforge3/envs/snapatac29/lib/python3.13/site-packages/snapatac2/preprocessing/_harmony.py", line 107, in harmony
    adata.obsm[use_rep + "_harmony"] = mat
    ~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^
RuntimeError: dimension cannot be changed from 226129 to 30

The harmony part converged after 5 iterations and seemingly works up to the point where the matrix gets saved in the AnnDataSet .obsm, but fails at this step

My dimensionality reduction was with spectral embedding, so the default use_rep='X_spectral' should work fine here, and nothing appears to be wrong with my .obs['sample'] variable either

Any ideas what may be going on here?

[jeremymsimon]

Hi Kai-
With Claude's help, I pinpointed the error here due to a change in harmonypy >= 2.0.0

In short, we need to add a transpose step before writing the harmony matrix into the .obsm.

So in place of:

snap.pp.harmony(data, batch="batch")

Users facing the same issue can work around it as follows:

mat = snap.pp.harmony(data, batch="batch", inplace=False)
if mat.shape[0] != data.n_obs:
    mat = mat.T
data.obsm["X_spectral_harmony"] = np.ascontiguousarray(mat)

Claude's suggested fix to snapATAC2 is as follows:

Make _harmony() orient by n_obs rather than unconditionally transposing, so it's robust across harmonypy versions:

out = harmony_out.Z_corr
return out if out.shape[0] == data_matrix.shape[0] else out.T

Let me know if there's any additional information I can provide that would be helpful