Skip to content

Split runtime from static configuration and use it for request context.#467

Merged
Martin-Rehr merged 1 commit intonextfrom
add/runtime-configuration-and-assistants
Apr 23, 2026
Merged

Split runtime from static configuration and use it for request context.#467
Martin-Rehr merged 1 commit intonextfrom
add/runtime-configuration-and-assistants

Conversation

@albu-diku
Copy link
Copy Markdown
Contributor

@albu-diku albu-diku commented Feb 20, 2026

The purpose of this change is twofold: first, firm up the notion of a runtime configuration as distinct from the static configuration loaded on disk and second, with this separation enforced, to prepare for further uses of the runtime side.

Note that the need to do this originated with the needs of templates - consider that a user may need a specific locale to be reflected in the ui for example. An observation was made that we already attach runtime properties to configuration (see mig_server_id, the logger) and must be additional work in various places to sidestep those additions.

Opt to do this by making an object which internally keeps a reference
to an underlying configuration object, proxies attribute access to and can thus be used as a substitute in any call stack of logic. Name this RuntimeConfiguration.

Use the split to add a the notion of a context. This idea was independently arrived at by myself and others on the team. With the support in hand, rework email sending to be context aware - that is, one can simply make use of send_email as before. Internally it will check whether sending was previously done (by consulting the context) and skip any construction steps if so.

As a result of this small change, we become able to centrally intercept email sending without threading arguments through multiple levels of the call stack of logic code purely to override email sending and we can do useful general enforcement.

Add a means for tests to check the the number of emails sent and whether an email was sent to a particular recipient. By default, enforce disallowing unexpected outgoing emails. This has the effect of nudging test writers to account for emails as a covered side effect of the logic being exercised. For cases where the email send is not relevant t, there is an escape hatch.

@albu-diku
Copy link
Copy Markdown
Contributor Author

@rasmunk this is what came of our discussion of the concept the other day :)

@albu-diku albu-diku changed the title Add/runtime configuration and assistants Split runtime from static configuration and use the former for assistants. Feb 20, 2026
@albu-diku albu-diku force-pushed the add/runtime-configuration-and-assistants branch 2 times, most recently from 20a5865 to 7b0193b Compare February 20, 2026 15:51
@albu-diku albu-diku force-pushed the add/runtime-configuration-and-assistants branch 3 times, most recently from d668a66 to 497886e Compare March 4, 2026 15:23
@albu-diku albu-diku changed the title Split runtime from static configuration and use the former for assistants. Split runtime from static configuration and use it for request context. Mar 5, 2026
Comment thread mig/shared/conf.py Outdated
Comment thread mig/shared/notification.py Outdated
@albu-diku albu-diku marked this pull request as ready for review March 9, 2026 10:58
@albu-diku albu-diku force-pushed the add/runtime-configuration-and-assistants branch from 5fc0d43 to 58f2dd8 Compare March 11, 2026 14:09
@rasmunk
Copy link
Copy Markdown
Contributor

rasmunk commented Mar 25, 2026

Looks good with the added changes. Since it changes a core piece that almost everything touches I think it is prudent that you checkout and test the branch on either dev or test ERDA before we merge it into next

@albu-diku albu-diku force-pushed the add/runtime-configuration-and-assistants branch 4 times, most recently from 1898891 to 6e35a47 Compare April 13, 2026 17:17
@albu-diku albu-diku force-pushed the add/runtime-configuration-and-assistants branch from 6e35a47 to be373c8 Compare April 19, 2026 18:29
@albu-diku
Copy link
Copy Markdown
Contributor Author

albu-diku commented Apr 19, 2026

Looks good with the added changes. Since it changes a core piece that almost everything touches I think it is prudent that you checkout and test the branch on either dev or test ERDA before we merge it into next

This contents of the branch have been running on dev-erda as part of the ui branch for the last couple of months and I've not seen any regressions in that time.

At the start of the coming week I'll make a point of using the other functionality, and in particular try to get some emails being sent - but otherwise I think this has now baked long enough.

Copy link
Copy Markdown
Contributor

@Martin-Rehr Martin-Rehr left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this email change commit deserves a PR of it's own.

Comment thread tests/test_mig_shared_conf.py Outdated
# --- BEGIN_HEADER ---
#
# test_mig_shared_configuration - unit test of configuration
# Copyright (C) 2003-2025 The MiG Project by the Science HPC Center at UCPH
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We are in 2026 by now :)

Copy link
Copy Markdown
Contributor

@Martin-Rehr Martin-Rehr left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approved the RuntimeConfiguration part, please split the email change out to it's own PR.

@albu-diku
Copy link
Copy Markdown
Contributor Author

albu-diku commented Apr 20, 2026

Approved the RuntimeConfiguration part, please split the email change out to it's own PR.

Will do. Two quick notes:

  • we do make use of that part in the larger work, specifically for some of the paths we fiddle with peers which would otherwise fail when they reach trying to mail send.
  • there was a previous note by you to see if we can try harder to get the dummy switch expressed in the configuration. Just to say I’ve not forgotten about that and if still relevant have an idea about how to achieve it.

@albu-diku albu-diku force-pushed the add/runtime-configuration-and-assistants branch from be373c8 to 09bee5a Compare April 20, 2026 11:55
Comment thread mig/shared/conf.py Outdated
Comment thread mig/shared/conf.py Outdated
This change paves the way adding further runtime specific data to
configuration objects. An instance of a configuration already contains
such data, such as the mig_server_id property when in use by daemons
and a reference to the logger. With needs arising for further runtime
data, make a clear separation between runtime attributes and the
static configuration data clear.

Opt to do this by making an object which internally keeps a reference
to an underlying configuration object, proxies attribute access to it
and can then be used as a drop-in substitute in ay call stack of logic.
Name this RuntimeConfiguration.

Stop short of moving existing runtime properties as of this commit.
@albu-diku albu-diku force-pushed the add/runtime-configuration-and-assistants branch from ce87442 to c73c542 Compare April 22, 2026 12:11
@albu-diku
Copy link
Copy Markdown
Contributor Author

I've corrected the typos. I do not have permission to land changes, eve with approval, so if you are happy with this it requires someone with merge privileges to move forward.

@Martin-Rehr
Copy link
Copy Markdown
Contributor

I've corrected the typos. I do not have permission to land changes, eve with approval, so if you are happy with this it requires someone with merge privileges to move forward.

It's pretty hard to review the changes when they are force pushed along with a bunch of other changes.

I've corrected the typos. I do not have permission to land changes, eve with approval, so if you are happy with this it requires someone with merge privileges to move forward.

Just a few notes:

  1. __setattr__ modifies the original (static) object, don't know if that's on purpose ?
  2. It's a bit hard to verify requested changes when they are force pushed along with a bunch of other stuff, please address the changes in simple commits during review

@albu-diku
Copy link
Copy Markdown
Contributor Author

albu-diku commented Apr 23, 2026

I've corrected the typos. I do not have permission to land changes, eve with approval, so if you are happy with this it requires someone with merge privileges to move forward.

It's pretty hard to review the changes when they are force pushed along with a bunch of other changes.

I've corrected the typos. I do not have permission to land changes, eve with approval, so if you are happy with this it requires someone with merge privileges to move forward.

Just a few notes:

1. `__setattr__` modifies the original (static) object, don't know if that's on purpose ?

2. It's a bit hard to verify requested changes when they are force pushed along with a bunch of other stuff, please address the changes in simple commits during review

I'm sorry for the force-push, was certainly not trying to complicate life. I'll avoid doing that while a review is ongoing from this point forward.

Regarding __setattr__, yes unfortunately it is. There is code in various places that dynamically sets properties on the configuration. In order to allow RuntimeConfiguration to be drop in for the original, it mimics that behaviour until such time as we are able to track all of that down. It's actually the same problem as was hit in the tests when ting to establish FakeConfiguration.

@Martin-Rehr
Copy link
Copy Markdown
Contributor

I've corrected the typos. I do not have permission to land changes, eve with approval, so if you are happy with this it requires someone with merge privileges to move forward.

It's pretty hard to review the changes when they are force pushed along with a bunch of other changes.

I've corrected the typos. I do not have permission to land changes, eve with approval, so if you are happy with this it requires someone with merge privileges to move forward.

Just a few notes:

1. `__setattr__` modifies the original (static) object, don't know if that's on purpose ?

2. It's a bit hard to verify requested changes when they are force pushed along with a bunch of other stuff, please address the changes in simple commits during review

I'm sorry for the force-push, was certainly not trying to complicate life. I'll avoid doing that while a review is ongoing from this point forward.

Regarding setattr, yes unfortunately it is. There is code in various places that dynamically sets properties on the configuration. In order to allow RuntimeConfiguration to be drop in for the original, it mimics that behaviour until such time as we are able to track all of that down. It's actually the same problem as was hit in the tests when ting to establish FakeConfiguration.

All good, thanks for the clarification, I'll merge it.

@Martin-Rehr Martin-Rehr merged commit ae8c3d8 into next Apr 23, 2026
11 checks passed
@Martin-Rehr Martin-Rehr deleted the add/runtime-configuration-and-assistants branch April 23, 2026 15:21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants