Skip to content

WIP: initial poc of switching to holidays python library#829

Draft
eyJhb wants to merge 8 commits intothunderbird:masterfrom
eyJhb:foss-holidays-provider
Draft

WIP: initial poc of switching to holidays python library#829
eyJhb wants to merge 8 commits intothunderbird:masterfrom
eyJhb:foss-holidays-provider

Conversation

@eyJhb
Copy link
Copy Markdown

@eyJhb eyJhb commented May 23, 2025

This is just an initial POC of how it could look like, if we switched to the Holidays library in Python. It works fairly well, and it seems to be well maintained, with an active userbase.

The library in question https://github.com/vacanza/holidays

This PR needs a lot of cleanup. I just threw it together, to get some idea of how everything works together. But this should be fairly doable? However, the calendar will look vastly different from the old one, and there are a lot more types in this calendar (bank, government, school, etc. etc.).

This would close #753 , if it gets implemented fully.

TODO:

  • Update holiday-calendar.md to reflect how to run this code
    • Ensure holidays is UPDATED (pip)
    • How to contribute/fix data
  • Sanity check if this is good enough?
  • No need to require the API key if using this new provider

Any maintainer of this project are 100% welcome to take the steps I've made, and build upon it so it can be pushed and start getting used :)

@eyJhb eyJhb marked this pull request as draft May 23, 2025 20:49
Copy link
Copy Markdown
Contributor

@janbrasna janbrasna left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

FYI I did generate the actual .ics too, janbrasna@fe68215 — in case we want to show folks the new data and check how viable it is for their regions…

One thing to add to TODO: no need to require the API key if using this new provider.

One nit spotted:

Comment thread requirements-dev.txt Outdated
python-dateutil~=2.8.2
packaging~=23.1
setuptools==78.1.1
hollidays>=0.73
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That doesn't seem right, probably:

Suggested change
hollidays>=0.73
holidays>=0.73

@eyJhb
Copy link
Copy Markdown
Author

eyJhb commented May 29, 2025

i've tried to clean up the code a bit (not pushed yet), but most of it ended up in the model instead. I still think it looks cleaner in the end, but I'll remove a bunch of more stuff.

I'll try to get it pushed tomorrow at some point :) It shouldn't change the output, EXCEPT that I'm now skipping any calendars with NO events. Otherwise we would produce empty calendars that would error out the parser AFAIK. But this should maybe be changed :)

@eyJhb
Copy link
Copy Markdown
Author

eyJhb commented May 30, 2025

I've pushed the changes now, and there are quite some. I also tried to update the holidays-calendars.md file a bit, or at least remove calendarific references.

I've moved most of the logic into the VacanzaHoliday model instead, to make it more simple and clear. It's not really that pretty, any of the code, but it's hopefully not impossible to read.

I've tried to not touch too much of the Calendar things, e.g. from_api.

@MelissaAutumn
Copy link
Copy Markdown
Member

Hello! Apologies I've been busy on other projects. Will hopefully get to this review by EOD today (my time) or early next week.

Thanks again!

@eyJhb
Copy link
Copy Markdown
Author

eyJhb commented May 30, 2025

Hello! Apologies I've been busy on other projects. Will hopefully get to this review by EOD today (my time) or early next week.

Thanks again!

No worries, take your time! :) Thus far I'm really impressed with the engagement from the team. That's always nice to see :)
Let me know if there is anything else I can do, and do not be afraid to ping me, I don't mind :)

@eyJhb
Copy link
Copy Markdown
Author

eyJhb commented Jun 26, 2025

Throwing in a tiny bump here :)

Copy link
Copy Markdown
Contributor

@janbrasna janbrasna left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is looking really good — I've generated the calendars a few times as the library updates progressed, and shared with people who were concerned with their locale caldata quality to evaluate. And the TLDR is: anything is better than the current state;D

Comment thread requirements-dev.txt
python-dateutil~=2.8.2
packaging~=23.1
setuptools==78.1.1
holidays>=0.73
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's bump to:

Suggested change
holidays>=0.73
holidays>=0.83

that has all the required regions we need.

@janbrasna
Copy link
Copy Markdown
Contributor

This PR has currently a conflict with master that needs rebasing, but hopefully it's just some easy readme or config.

While I think removing the old model and provider is called for, as there's not really a reason to go back, there are tests relying on some of its specifics. If this approach is (pre-)approved by the maintainers, the tests would also need updating for the new data.

@janbrasna
Copy link
Copy Markdown
Contributor

QQ: … and I've also seen it in tests — is it expected to have the holidays sets as OPAQUE i.e. as "busy" by default? Or that's a feature, for the public/national/bank holidays per se? (And in case there would be other types of holiday events added/configured like observance or religious, those would conversely be configured as TRANSPARENT explicitly for the difference?)

Copy link
Copy Markdown
Member

@MelissaAutumn MelissaAutumn left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looking good, I see calendar events and afaik they look correct...besides some French where it doesn't belong 😄 If you have some time and are still willing to work on this, could you rebase and update tests as well?

Apologies for the long delay, we've been quite busy over here!

subdivs = cal.subdivisions

# get the names for all the subdivisions
subdiv_to_name = {v: k for k, v in cal.subdivisions_aliases.items()}
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think you can probably drop retrieving the aliases.

It doesn't look like we could reverse this dict as at least in Canada aliases include the French names for Provinces but no information on what locale the name might be from. (https://holidays.readthedocs.io/en/latest/auto_gen_docs/canada/#holidays.countries.canada.Canada.subdivisions)

subdiv=subdiv,
categories=cal.supported_categories,
)
for d, n in tcal.items():
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: I think we can write out date and name, it's only a few more letters 😄

Comment thread calgen/models/VacanzaHolidays.py Outdated
# this should really be localised, but uncertain how to do this best
subdivs_except_text = ""
if subdivs_except:
subdivs_except_text = "All except "
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this would be okay to localize ourselves, so you can probably pull in gettext_object from translate.py and pass in the locale. It should return an initialized gettext instance so you can use the return value like:

regions = ', '.join(subdivs)
translator.gettext('All except %(list_of_regions)s' % {'list_of_regions': regions})

Which might produce: All Except BC, ON, QC

Unfortunately we use old-style python string formats for l10n. The localizers will see this as just All except %(list_of_regions)s so they can place it as needed.

Comment thread docs/holiday-calendars.md
```

If you are an MZLA employee you can find the api key within the infra vault in 1password.
Please ensure that you are using the latest release of the hollidays library.
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
Please ensure that you are using the latest release of the hollidays library.
Please ensure that you are using the latest release of the holidays library.

categories=cal.supported_categories,
)

map_subdiv_to_name = {v: k for k, v in cal.subdivisions_aliases.items()}
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This doesn't seem to work for Canadian holidays, as our aliases include the french name for it as well. However not all provinces have an official french name like Manitoba.

('British Columbia', 'BC'), ('Colombie-Britannique', 'BC')

leading to the BC => Colombie-Britannique overriding the english name

which ends up with SUMMARY:British Columbia Day (Colombie-Britannique).

I'm not sure if there's a good fix here, might be an issue we would need to raise with the library.

@MelissaAutumn
Copy link
Copy Markdown
Member

^ I forgot I had half a review ready from May 🤦

Comment on lines +62 to +64
# insert the excluded text here
# it should be done in a better way than this...
excluded_text = "All except "
Copy link
Copy Markdown
Contributor

@janbrasna janbrasna Oct 29, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah, I was surprised where the "Año Nuevo Judío (Rosh Hashana) (All except Tucumán)" comes from in those instances;)

QQ: If that is wrapped in gettext… does the calendar generation actually run in any l10n context, to apply anything besides just the en-US defaults for such strings?

EDIT: Now I see the comment above, where Mel describes how to add an initialized instance — nice!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

use free software to generate public calendars

3 participants