fix #2760 - provide parser override via new Argument.CaptureRemainingTokens property

[jonsequitur]

This fixes #2760 by adding a new property currently named Argument.CaptureRemainingTokens.

I'm not sure this is the right name but the behavior is that this argument will capture everything following it regardless of whether it happens to match some well-known symbol. A few notes on this implementation:

• It's intended to be used with enumerable types but this is not enforced.
• It stops capturing when it reaches the arity limit for the argument. For this reason, a name like Greedy might be more accurate. Can we assume this term is intuitive enough?
• The -- token has no special meaning when captured by an Argument with this property set to true. It gets captured as one of the tokens.

@baronfel @tmds Thoughts?

[KalleOlaviNiemitalo]

The -- token has no special meaning when captured by an Argument with this property set to true. It gets captured as one of the tokens.

That is what I would expect.

[KalleOlaviNiemitalo]

It's intended to be used with enumerable types but this is not enforced.

I imagine a custom parser could very well transform this token sequence to a non-enumerable type.

[jeffhandley]

Since this is an implicit --, I started entertaining if we could take a different approach where we encapsulate the current double-dash implementation in a special-purpose Argument<string[]> that always captures all remaining tokens. That argument could be explicitly added to the CLI definition (as the last argument) to make the -- implicit, and if it's not already added to the CLI then we would add it with the "--" name.

An approach like that could prevent us from having 2 different mechanism for capturing the remaining tokens, and we could disallow that special argument from having a max capacity that can get filled up.

BUT after some feasibility research, I don't think it would come in very cleanly for a few reasons:

1. Trying to encapsulate -- as an argument or option would affect tokenizing logic and how the remaining tokens are treated/exposed
2. It would be challenging to make that change in a backwards-compatible manner for the existing UnmatchedTokens
3. Would it be an Option or an Argument? Kind of both. Kind of neither.

I'm not convinced on the CaptureRemainingTokens name either, and I'm anxious about the complexity of filling to capacity and needing to spill over and that being potentially unintuitive. But this does plug in pretty cleanly overall.

I'll come back to this PR after thinking about it some more.

[jonsequitur]

Yeah, the name CaptureRemainingTokens doesn't entirely convey one implication of the design, which is that an argument with this property set to true can still be limited by its arity. Another name I was weighing was Greedy which I think is reasonable as well. It conveys the sense that while other parsing rules (e.g. the names of options, or the -- token) might apply, this argument will override them.

[jonsequitur]

Since this is an implicit --, I started entertaining if we could take a different approach where we encapsulate the current double-dash implementation in a special-purpose Argument<string[]> that always captures all remaining tokens.

We considered this design when we were changing the way -- worked. If I recall the conversation correctly, we decided that the -- is an escape mechanism and a feature of the grammar, and not a symbol in and of itself. Like POSIX bundling, it's there to meet end users' expectations and provide ecosystem consistency, not as an abstraction for developers.

It is appealing as a way to reduce concept count though.