Skip to content

Read-only (covariant) list parameter annotations#1745

Merged
Dr-Irv merged 5 commits into
pandas-dev:mainfrom
hamdanal:covariant-list
May 17, 2026
Merged

Read-only (covariant) list parameter annotations#1745
Dr-Irv merged 5 commits into
pandas-dev:mainfrom
hamdanal:covariant-list

Conversation

@hamdanal
Copy link
Copy Markdown
Contributor

Some pandas functions expect a list as argument but not any sequence. Because list is invariant, parameters annotated with list[float] will not accept list[int], for examlpe, even though we know that the list is read-only and will not be modified. To fix this, I propose the following trick using the CovariantList protocol. This protocol can replace list annotations in parameter positions (never use as a return type, instead keep using list there).

Note that this problem has manifested before and one trick you use today is to parametrize the list with a type variable that has an appropriate upper bound (for example you use list[HashableT] where you want a list of hashable objects). This is cumbersome because for every type of list element, you'll need to define a corresponding type variable. Also, there are some places where a list of a very wide type like a union is still used (for example list[AggFuncTypeBase] in DataFrame.aggregate). All of these can be replaced with the CovariantList protocol.

In this PR I only introduce the protocol and change one method to use it to demonstrate its value. I think it would be better to incrementally use this protocol in follow up PR(s) for easier work and review. For example a PR could replace all list[HashableT] and list[Hashable] in parameter positions by CovariantList[Hashable] then other uses could be audited and changed in separate follow-up PRs.

@hamdanal hamdanal changed the title Covariant list Read-only (covariant) list parameter annotations May 17, 2026
Copy link
Copy Markdown
Collaborator

@Dr-Irv Dr-Irv left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thanks. This looks like an interesting idea!

Comment thread tests/test_typing.py Outdated
Comment thread pandas-stubs/_typing.pyi
Copy link
Copy Markdown
Collaborator

@Dr-Irv Dr-Irv left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thanks @hamdanal

@Dr-Irv Dr-Irv merged commit 5c8669d into pandas-dev:main May 17, 2026
14 checks passed
@hamdanal hamdanal deleted the covariant-list branch May 18, 2026 19:03
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

BUG: set_index rejects List[str]

2 participants