Skip to content

feat: Allow using accessors in sc.get.aggregate#4199

Open
flying-sheep wants to merge 17 commits into
mainfrom
aggregate-dim
Open

feat: Allow using accessors in sc.get.aggregate#4199
flying-sheep wants to merge 17 commits into
mainfrom
aggregate-dim

Conversation

@flying-sheep

Copy link
Copy Markdown
Member
  • Closes #
  • Tests included or not required because:

@codecov

codecov Bot commented Jul 6, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 94.69027% with 6 lines in your changes missing coverage. Please review.
✅ Project coverage is 80.02%. Comparing base (7df9ad6) to head (90311d3).
✅ All tests successful. No failed tests found.

Files with missing lines Patch % Lines
src/scanpy/get/get.py 73.91% 6 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main    #4199      +/-   ##
==========================================
+ Coverage   79.91%   80.02%   +0.11%     
==========================================
  Files         121      121              
  Lines       12949    13012      +63     
==========================================
+ Hits        10348    10413      +65     
+ Misses       2601     2599       -2     
Flag Coverage Δ
hatch-test.low-vers 78.90% <67.25%> (-0.14%) ⬇️
hatch-test.pre 79.94% <90.26%> (+0.07%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines Coverage Δ
src/scanpy/_settings/presets.py 90.97% <100.00%> (+0.12%) ⬆️
src/scanpy/get/_aggregated.py 97.07% <100.00%> (+3.16%) ⬆️
src/scanpy/get/get.py 90.70% <73.91%> (-2.15%) ⬇️

... and 1 file with indirect coverage changes

@flying-sheep flying-sheep changed the title WIP aggregate AdRef Aggregate AdRef Jul 7, 2026
@flying-sheep flying-sheep requested a review from ilan-gold July 7, 2026 06:37
@flying-sheep flying-sheep marked this pull request as ready for review July 7, 2026 06:37
@flying-sheep flying-sheep changed the title Aggregate AdRef feat: Allow using accessors in sc.get.aggregate Jul 7, 2026
raise TypeError(msg)
data = adata[acc]
dim_df = pd.DataFrame({
ref.idx if isinstance(ref.idx, str) else str(ref): adata[ref]

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Idk how to do this better:

  • only simplify A.obs[k] to string? If so, should we allow to mix strings and AdRefs in by=[…]?
  • simplify unless there are duplicate .idxs?
  • don’t stringify at all and use AdRefs as df.columns?

@ilan-gold ilan-gold Jul 7, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is the question here about the isinstance check? Like, what to do about people who pass in the index as a by column?

This line of code seems inoffensive other than I am not sure why ref.idx would be anything other than a str

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the question is about “we now accept AdRefs, what to use as dataframe columns when they’re used”

I am not sure why ref.idx would be anything other than a str

>>> type(ad.acc.A.obsm["y"][:, 0].idx)
int

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the question is about “we now accept AdRefs, what to use as dataframe columns when they’re used”

But these columns should really just be ref.idx since ad.acc.A.obsm["y"][:, 0] isn't a valid by unless I'm missing something.

@flying-sheep flying-sheep Jul 7, 2026

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why would it not be?

import numpy as np
import scanpy as sc
from anndata import acc, AnnData

adata = AnnData(np.random.rand(3, 4), obsm=dict(thing=np.array(["x", "y", "y"])))
sc.get.aggregate(adata, acc.A.obsm["thing"][:, 0], ())
# AnnData object with n_obs × n_vars = 2 × 4
#    obs: "A.obsm['thing'][:, 0]", 'n_obs_aggregated'

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To clarify, if you leave it as-is, I think that's fine. If you revert to the old behavior, I think that's also perfectly reasonable (and would remove this question)

@ilan-gold ilan-gold Jul 7, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm leaving towards what you have now, at least in terms of functionality. Being able to do one-versus-rest type aggregations, perhaps over a neighborhood graph, could be cool, if it's supported i.e., vec of 1s and 0s (neighbors and not neighbors). Not sure if people will really do this though, but just a thought.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

what do you think of the third option from my initial comment in this thread?

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh missed this

don’t stringify at all and use AdRefs as df.columns?

Is this serializable? What if someone tries to write their results?

@flying-sheep flying-sheep Jul 9, 2026

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm, the problem is that stringifying it introduces one more format that isn’t really supported. We have

  1. the actual AdRef object
  2. the simple string version (supported via A.resolve, e.g. "obs.some_col". Can only represent a subset of possible refs)
  3. the repr (not supported as conversion format anywhere)

We could fix that by doing something like

  • here, if passing a collection of AdRefs, serialize to repr always (instead of the conditional .idx we have now)
  • in A.resolve, also support the repr format (if input.startswith('A.'): …)

then people can just use A.resolve to get back the input AdRefs from the obs column names.

Or we serialize/deserialize AdRefs in AnnData

@flying-sheep flying-sheep added this to the 1.13.0 milestone Jul 7, 2026

@ilan-gold ilan-gold left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't see any tests for the added behavior i.e., around by=A.obsm["foo"][:, 1]

Comment thread tests/test_aggregated.py Outdated
Comment thread src/scanpy/get/_aggregated.py Outdated
@flying-sheep flying-sheep requested a review from ilan-gold July 7, 2026 13:11
Comment thread tests/test_aggregated.py
Comment on lines +688 to +689
result = sc.get.aggregate(adata, by=A.obsm["thing"][:, 0], func=["sum", "mean"])
expected = sc.get.aggregate(adata, by="blobs", func=["sum", "mean"])

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does it make sense to check the names of the result i.e., result.obs_names?

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also the obs of result should contain a certain column derived from by, right?

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yeah but: #4199 (comment)

@flying-sheep flying-sheep requested review from ilan-gold and removed request for ilan-gold July 9, 2026 07:05
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants