Features to add for more production robustness

[vsoch]

When we have this deployed in a production setting, I want:

• A configurable timeout variable for ask_secretary, the idea being that if a cluster is asked and somehow the entire thing goes down (likely the cluster itself would not go down but the server might be stopped) we do not wait indefinitely. the asynio gather function will wait.