Disaster recovery tests

[jchristgit]

To ensure the DevOps team is prepared for the worst, we should carry out
(scheduled) disaster recovery tests.

We don't necessarily have to disable live services for this. The idea is more to
practice situations like the following and document what happens:

• our LDAP server was nuked by accident, how do we restore it?
• we lost the infractions table on the PostgreSQL database
• netcup accidentally deleted lovelace
• [please suggest more]

This issue just stands to plan this and discuss it.

Discussion so far

• Joe mentioned that we have backups from Blackbox stored in Linode. The admins
  team has access to this.