Release 3.3.0 incoming

[azmeuk]

Hi everyone,

I worked on all the opened issues and PR that don't required overhauling the library. I released version 3.3.0b1 and will release version 3.3.0 final in the coming days or weeks.
Please give feedback here if you notice things that deserve changes before 3.3.0 final. We can still change the API. I will try to reach a few maintainers of projects depending on wtforms before cutting the release.

The full changelog is here. Most changes are non breaking but those points may require attention:

• the (re)introduction of the SelectChoice dataclass might break fields inheriting SelectField and expecting iter_choices to return a list of tuples, now this is a list of SelectChoice.
• SelectField no longer has has_groups and iter_groups
• URL no longer inherits Regexp, and use urllib internally to parse urls. It accepts only http and https by default, other schemes must be whitelisted with schemes=
• FloatField default widget is NumberInput and no longer TextInput

After this, and I won't promise exactly when, I will work on the remaining tickets for wtforms v4. This version will have breaking changes as I want to rework the form processing part so it is more consistent, the field binding part etc. It should bring interesting features too, such as type annotations #618, multiple fields #799, maybe async #839.

There will maybe be an intermediate 3.4 release containing compatibility helpers for v4 and deprecation warnings.

/cc @PanderMusubi @Daverball @ThiefMaster @davidism @TurnrDev

[Daverball]

I'm a little bit concerned about the breaking change SelectChoice introduces, especially because the current design feels a little sloppy with the internal _selected field that's used to smuggle that value through iter_choices.

I think I would like it better if greater care was taken to keep things somewhat more backwards compatible for custom SelectField subclasses, especially since it's quite common for choices to be populated after the field is constructed, if the choices are dynamic. You took care to keep the static path compatible, but the dynamic path will break still.

Here's a few suggestions for an alternate design:

1. Have separate types for the choices that are stored on the field, and the ones that are returned by iter_choices, i.e. the selected field only exists on the type returned by iter_choices.
2. Make those types NamedTuple objects, so they can still be used in the same way as the old raw tuples, while retaining the ergonomic benefits of giving names to each component
3. To better match the old behavior, when only supplying a value for a choice, the constructor should copy the value to the label automatically. It might also be worth populating render_kw with an empty dictionary. That way you're not introducing new None values, so code can work the same way as before.
4. I would suggest keeping the old has_groups and iter_groups methods intact for compatibility until 4.0.0
5. iter_choices and iter_groups should still yield tuples of the same shape, it's fine if they're NamedTuple but I don't think it's worth breaking subclasses, just to get rid of those methods.
6. choices should become a settable property, that way the coercion to the new objects can happen when assigning to field.choices, not just in __init__.

I think normalizing choices is the best part of your change, so it's worth making, it was always kind of awkward in static typing not knowing what shape choices were, since so many different forms were accepted. So that part is a big improvement. But we can get that improvement without paying such a big cost in backward incompatibility.

That being said, even with this design there's still potential for subtle breakages, consider the following code:

class Foo(Form):
    bar = SelectField()

form = Foo()
form.bar.choices = choices = some_dynamic_choice_generator()
choices.insert(0, ('', ''))  # insert an empty choice at the start

This code would keep running, but the extra empty choice would now be missing from the field, since choices were coerced to the new format, so updating the list, has no effect on the choices on the field. With a different order of operations this would be unproblematic.

It's definitely worth thinking about, whether the benefits of this new design, outweigh the costs of introducing bugs like that. In that sense, a breaking change might be better, since people will know that they have to change their code to keep it working, but I would probably hold off until 4.0.0 for that change. You can still provide the new choice objects for passing into the choices parameter, while the rest of the API stays the same (this works because a NamedTuple would provide exactly the same interface as the raw tuples that were already accepted), this also would still allow you to emit deprecation warnings when you're given choices in formats that won't be supported in the future.

[coleifer]

I haven’t reviewed the changes closely but we do a lot with select fields in peewee integration. Any jankiness or kludges there are likely to compound in a way that makes things much harder to maintain longer term. I’d be careful messing with that contract in particular.

Ref https://github.com/coleifer/wtf-peewee/blob/master/wtfpeewee/fields.py

[Daverball]

As a side note. I think it would be nice if choices accepted a callback in the constructor with the same parameters as a widget (i.e. form and field using the _form parameter in __init__ and self), that would eliminate most of the need for code that manipulates choices after field construction. I think that would be a higher value change in the short term, than introducing new objects.

[ElLorans]

What are the advantages of moving to a dataclass rather than moving to a named tuple? Wouldn't the latter the best of both worlds?

[azmeuk]

What are the advantages of moving to a dataclass rather than moving to a named tuple? Wouldn't the latter the best of both worlds?

This is a point @Daverball raised earlier, this is fixed in #923.

#923 restores the backward compatibility for SelectField and adds deprecation warnings, (confirmed by launching the downstream unit test suites for wtf-peewee, wtforms-alchemy, starlette-wtf, flask-appbuilder).