Skip to content

Deployment Data Background Refetching#747

Open
jjoderis wants to merge 16 commits into
mainfrom
deployment-data-background-refetching
Open

Deployment Data Background Refetching#747
jjoderis wants to merge 16 commits into
mainfrom
deployment-data-background-refetching

Conversation

@jjoderis
Copy link
Copy Markdown
Contributor

Summary

Instead of always fetching the latest deployment information from all available machines every time a user tries to access it deployment and instance data is now served from the database. To ensure that the user has access to up to date information this PR adds a refetch loop, that runs as long as a user is connected to the MS, which fetches the latest information for all deployments known to the MS from all available engines and updates the deployment and instance data in the database.

Details

  • added logic that fetches the newest data for all known deployments (if possible)
    • if some user has the ui open in their browser their frontend will request the server to run the refech logic in a configurable interval (we need the logic to be triggered by a user to use caching logic that will be implemented in a future PR)
    • the backend will execute the refetch logic only once in the configured interval and skip all additional requests by other users
    • the refetch function can be awaited to ensure that the last fetch was at most on interval timer ago so that the data fetched from the database is not too old
  • added handling of deployment removal for deployments on engines that are not reachable when the user wants to delete the deployment
    • a flag is set in the deployment entry in the database and when the engine becomes reachable again the refetch function will remove the deployment from the engine
  • changed all components and tools that got their deployment data from the engines fetch it from the database now instead

jjoderis added 15 commits May 21, 2026 16:45
…on from the engines and updates the db if a deployment has been removed from the engine it is expected to be on
…he database instead of having to fetch it from the reachable engine

This should speed up loading the data since we don't have to aggregate
the data over all available engines.
It is now also possible to see deployed processes where the engine is
currently not available and to filter out deployments the user should
not be able to see.
…ts instances from the database instead of fetching it from the available engines
PROCEED_PUBLIC_DEPLOYMENT_REFETCHING_INTERVAL controls how frequently
thhe interval run
If PROCEED_PUBLIC_PROCESS_AUTOMATION_ACTIVE is set to false the loop is
disabled
…er for its conclusion or for the conclusion of the last fetch in the refetch timeout
…atabase with optional fetching if the data has not been fetched inside the configured timeout
…se with optional refetching if the data is older than the configured refetch timeout
…the database with optional refetching if the data is older than the configured refetch timeout
…ployment list will count instances and versions of deployments that have been deleted
@jjoderis jjoderis requested a review from Zayn-Javed May 27, 2026 17:59
@github-actions

This comment has been minimized.

@jjoderis jjoderis deployed to Staging May 27, 2026 18:08 — with GitHub Actions Active
@github-actions
Copy link
Copy Markdown

CLOUDRUN ACTIONS

✅ Successfully created Preview Deployment.

https://pr-747---ms-server-staging-c4f6qdpj7q-ew.a.run.app

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant