Feat observer esque alerts

[Poofjunior]

Here's an alternate proposal for sending alerts.

Currently, we're appending Brainslosher functionality inside a one-liner stream callback. Specifically, in brainslosher_main.py, we call

self.add_stream("check_job_status", 1, self.check_job_status)

But self.check_job_status also sends emails in addition to sending the job status to the subscriber. (Streams should just be calling "getter" functions.)

This change makes it so we can add an arbitrary function that gets called any time self.job_status changes.

The results:

• we no longer add functionality in a zmq stream.
• we can remove the Lock around changing self.job_status

There's still a bit of a question from my end as to where we should be storing the email if this alert functionality is truly standalone, but this works for now.

Heads-up: we should test this on real hardware before merging.

[patricklatimer]

So I've been experimenting (literally just today) with this very cool library/tool called apprise that really seems to be the ideal notification tool. You can use it as a python library, CLI, or even set up a docker with a REST API to receive notification content and distribute it. It's super configurable and comes with integrations for all sorts of notification platforms.
https://github.com/AllenNeuralDynamics/SIPE-Admin/issues/159

Anyway, brainslosher should use this, then we can just configure the notification destinations from a yaml file (and actually apprise supports loading config from http so we can easily use Jessy's ficus config api)

[Poofjunior]

Anyway, brainslosher should use this, then we can just configure the notification destinations from a yaml file (and actually apprise supports loading config from http so we can easily use Jessy's ficus config api)

Ok sounds good! It should only be a matter of changing the implementation of the current function that sends emails.