enhance notification system

[mwildt]

Current Behavior

We use the application with many projects, some of which are based on a similar technology stack (microservices). Each service is mapped as a separate project - sometimes with different versions - in dependec track. To this end, we have implemented a dependency for the integration of jira via webhook notifcations and jira automations. Now we have the problem that the notifcations arrive quasi parallel in jira and thus the depulization does not take effect.

Proposed Behavior

Please extend the system so that the webhook notifications are processed in a queue to prevent race conditions in the integration of target systems Like jira.

Alternatively, a corresponding plugin API, which enables the implementation of plugins by third parties, would be useful.

Checklist

• I have read and understand the contributing guidelines
• I have checked the existing issues for whether this enhancement was already requested

[nscuro]

Please extend the system so that the webhook notifications are processed in a queue

They are already processed via an in-memory queue, but the queue has 4 consumers that compete for work. The number of consumers is currently hard-coded to 4, but we could make it configurable so you could set it to 1. It would help, but also significantly reduce throughput.

Achieving ordering is tricky because the ordering requirements can differ widely. Do you want total ordering, as in all notifications to Jira should be processed serially? Or do you want ordering by project UUID, or ordering by Jira project key?

If this is an immediate need, you could look into putting a Bento instance between DT and Jira. There are a variety of options to buffer and deliver messages while taking your concurrency requirements into consideration.

Alternatively, a corresponding plugin API, which enables the implementation of plugins by third parties, would be useful.

What part specifically would you like to be pluggable? A plugin system will come in version 5, but in order for something to be pluggable, we of course need to define extension points first.

[mwildt]

Hi Niklas,

thanks for your reply. In fact, I hadn't looked as far as the alpine library, and overlooked the multithreaded queue mechanism.

If the number of notification sender threads could be configured, that would actually be enough for us. I think throughput should not be the decisive issue (it's only about the notifications).

I would appreciate it if you could create a corresponding configuration option...