WIP: initial poc of switching to holidays python library#829
WIP: initial poc of switching to holidays python library#829eyJhb wants to merge 8 commits intothunderbird:masterfrom
Conversation
janbrasna
left a comment
There was a problem hiding this comment.
FYI I did generate the actual .ics too, janbrasna@fe68215 — in case we want to show folks the new data and check how viable it is for their regions…
One thing to add to TODO: no need to require the API key if using this new provider.
One nit spotted:
| python-dateutil~=2.8.2 | ||
| packaging~=23.1 | ||
| setuptools==78.1.1 | ||
| hollidays>=0.73 |
There was a problem hiding this comment.
That doesn't seem right, probably:
| hollidays>=0.73 | |
| holidays>=0.73 |
|
i've tried to clean up the code a bit (not pushed yet), but most of it ended up in the model instead. I still think it looks cleaner in the end, but I'll remove a bunch of more stuff. I'll try to get it pushed tomorrow at some point :) It shouldn't change the output, EXCEPT that I'm now skipping any calendars with NO events. Otherwise we would produce empty calendars that would error out the parser AFAIK. But this should maybe be changed :) |
|
I've pushed the changes now, and there are quite some. I also tried to update the I've moved most of the logic into the I've tried to not touch too much of the |
|
Hello! Apologies I've been busy on other projects. Will hopefully get to this review by EOD today (my time) or early next week. Thanks again! |
No worries, take your time! :) Thus far I'm really impressed with the engagement from the team. That's always nice to see :) |
|
Throwing in a tiny bump here :) |
janbrasna
left a comment
There was a problem hiding this comment.
This is looking really good — I've generated the calendars a few times as the library updates progressed, and shared with people who were concerned with their locale caldata quality to evaluate. And the TLDR is: anything is better than the current state;D
| python-dateutil~=2.8.2 | ||
| packaging~=23.1 | ||
| setuptools==78.1.1 | ||
| holidays>=0.73 |
There was a problem hiding this comment.
Let's bump to:
| holidays>=0.73 | |
| holidays>=0.83 |
that has all the required regions we need.
|
This PR has currently a conflict with master that needs rebasing, but hopefully it's just some easy readme or config. While I think removing the old model and provider is called for, as there's not really a reason to go back, there are tests relying on some of its specifics. If this approach is (pre-)approved by the maintainers, the tests would also need updating for the new data. |
|
QQ: … and I've also seen it in tests — is it expected to have the holidays sets as |
MelissaAutumn
left a comment
There was a problem hiding this comment.
Looking good, I see calendar events and afaik they look correct...besides some French where it doesn't belong 😄 If you have some time and are still willing to work on this, could you rebase and update tests as well?
Apologies for the long delay, we've been quite busy over here!
| subdivs = cal.subdivisions | ||
|
|
||
| # get the names for all the subdivisions | ||
| subdiv_to_name = {v: k for k, v in cal.subdivisions_aliases.items()} |
There was a problem hiding this comment.
I think you can probably drop retrieving the aliases.
It doesn't look like we could reverse this dict as at least in Canada aliases include the French names for Provinces but no information on what locale the name might be from. (https://holidays.readthedocs.io/en/latest/auto_gen_docs/canada/#holidays.countries.canada.Canada.subdivisions)
| subdiv=subdiv, | ||
| categories=cal.supported_categories, | ||
| ) | ||
| for d, n in tcal.items(): |
There was a problem hiding this comment.
nit: I think we can write out date and name, it's only a few more letters 😄
| # this should really be localised, but uncertain how to do this best | ||
| subdivs_except_text = "" | ||
| if subdivs_except: | ||
| subdivs_except_text = "All except " |
There was a problem hiding this comment.
I think this would be okay to localize ourselves, so you can probably pull in gettext_object from translate.py and pass in the locale. It should return an initialized gettext instance so you can use the return value like:
regions = ', '.join(subdivs)
translator.gettext('All except %(list_of_regions)s' % {'list_of_regions': regions})
Which might produce: All Except BC, ON, QC
Unfortunately we use old-style python string formats for l10n. The localizers will see this as just All except %(list_of_regions)s so they can place it as needed.
| ``` | ||
|
|
||
| If you are an MZLA employee you can find the api key within the infra vault in 1password. | ||
| Please ensure that you are using the latest release of the hollidays library. |
There was a problem hiding this comment.
| Please ensure that you are using the latest release of the hollidays library. | |
| Please ensure that you are using the latest release of the holidays library. |
| categories=cal.supported_categories, | ||
| ) | ||
|
|
||
| map_subdiv_to_name = {v: k for k, v in cal.subdivisions_aliases.items()} |
There was a problem hiding this comment.
This doesn't seem to work for Canadian holidays, as our aliases include the french name for it as well. However not all provinces have an official french name like Manitoba.
('British Columbia', 'BC'), ('Colombie-Britannique', 'BC')
leading to the BC => Colombie-Britannique overriding the english name
which ends up with SUMMARY:British Columbia Day (Colombie-Britannique).
I'm not sure if there's a good fix here, might be an issue we would need to raise with the library.
|
^ I forgot I had half a review ready from May 🤦 |
| # insert the excluded text here | ||
| # it should be done in a better way than this... | ||
| excluded_text = "All except " |
There was a problem hiding this comment.
Ah, I was surprised where the "Año Nuevo Judío (Rosh Hashana) (All except Tucumán)" comes from in those instances;)
QQ: If that is wrapped in gettext… does the calendar generation actually run in any l10n context, to apply anything besides just the en-US defaults for such strings?
EDIT: Now I see the comment above, where Mel describes how to add an initialized instance — nice!
This is just an initial POC of how it could look like, if we switched to the Holidays library in Python. It works fairly well, and it seems to be well maintained, with an active userbase.
The library in question https://github.com/vacanza/holidays
This PR needs a lot of cleanup. I just threw it together, to get some idea of how everything works together. But this should be fairly doable? However, the calendar will look vastly different from the old one, and there are a lot more types in this calendar (bank, government, school, etc. etc.).
This would close #753 , if it gets implemented fully.
TODO:
holiday-calendar.mdto reflect how to run this codeAny maintainer of this project are 100% welcome to take the steps I've made, and build upon it so it can be pushed and start getting used :)