Support string indices for operators with `by` option

[bentsherman]

Spawned from the language improvements mega-discussion (#3107). Operators that combine or group channels based on an index (i.e. by option) currently only support integer indices. They should also support string indices so that they can be used with maps.

To my knowledge, these operators include:

• groupTuple
• join
• combine

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[Midnighter]

Would love this feature! I started working on a plugin just for adding these kinds of operators but if nextflow can do it natively, that'd be so much better.

I've seen plenty of issues in nf-core where such channel operations are done using the entire map as the key, hopefully string keys would quickly resolve them.

[Midnighter]

I think the cross operator also qualifies.

Just wanted to say, I found a "fun" example where grouping happens on two maps at different positions 😬 In the below example from the taxprofiler pipeline, position 0 and 2 are Maps. Should one be able to specify different keys for different positions?

.map {
    meta, reads, db_meta, db ->
        [[id: db_meta.db_name, single_end: meta.single_end], reads, db_meta, db]
}
.groupTuple(by: [0,2,3])

[bentsherman]

@Midnighter , I went ahead and pushed a draft PR, currently only groupMap is implemented.

I don't think cross needs to be changed because it just does an outer product regardless of the item type. See the example in the Nextflow docs:. It already supports lists and maps alike.

Also, your issue #3175 is about using a map as a join key, whereas this issue is about using string keys. So your use case should already work, as long as you aren't modifying your maps in place. In others, tuples can already be joined on a map element, but maps cannot be joined because it requires string indices.

[Midnighter]

I see, I think I misunderstood the intent of this issue then. You are talking about joining channels that have single map elements only?

Yes, I'm mostly interested in joining channels where a map is an element of a tuple. I thought you were working on a solution that would allow setting string keys also in that case. Indeed I think that joining on the entire map is bad practice and joining on chosen keys from that map is much more explicit and will prevent many accidental errors.

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[Midnighter]

I vaguely remember you talking about allowing closures for the by argument, which would solve this elegantly. I can't remember where, though.