DRAFT 1.1 Migration Guide

This page covers migrating from CollectOSS v1.0 to CollectOSS v1.1. Since this release contains several changes that may affect the configuration of existing instances, we are providing this migration guide with suggestions for how to handle some of the changes.

If you would like maintainer assistance with any of these steps, or if you have questions that are not addressed here, you can use the CHAOSS Slack workspace in the #wg-collectoss-8knot channel to contact the maintainers.

Schema names

v1.1 has renamed the core database schemas. If you have queries that are hardcoded to relying on the schema names, please rename them as follows:

• augur_data -> data
• augur_operations -> operations

Unfortunately, it is not technically possible to configure CollectOSS to preserve the functioning of both names in order to provide a transition period.

If your deployment has downstream applications relying on hardcoded schemas, we encourage you to update your downstream applications to utilize the postgres schema search path. We recommend setting the search path so that it contains the string data,augur_data, ensuring that any unqualified table queries will first hit data (the new schema name) if it is present, falling back to augur_data (the old name) if it isn't. This also makes the transition easier since it reduces the need to coordinate and deploy updates to multiple applications at the same time.

Note

The data and operations schemas are considered internal to CollectOSS and may be subject to breaking changes. In a future version, we will be introducing a stable schema that will be tested and will become part of our public/stable API commitment.

Environment Variables

CollectOSS v1.1 introduces a new environment variable subsystem that allows the application to read from environment variables that are present with either the new COLLECTOSS_ or the old AUGUR_ prefixes. We recommend you migrate to the new names at your convenience.

Any environment variables that don't have a prefix, such as CONFIG_DATADIR can remain the same.

Config File Naming

CollectOSS v1.1 also changed the name of the primary config file from augur.json to collectoss.json. The config manager will first look for the new file name, falling back to the old one if it cannot be found. This config file, if used, should be located in the directory specified by the CONFIG_DATADIR environment variable.

Docker Entrypoints

RabbitMQ

The custom RabbitMQ image that Augur shipped with was known to require the RabbitMQ credentials to be provided at build time (which were then baked into the image). CollectOSS v1.1 removes this problem by eliminating redundant or moot configuration files/lines, and relies on environment variables and path mounts, along with the upstream rabbitmq:4.1-management-alpine container image to provide the RabbitMQ service for the data collection portions of CollectOSS.

Optional: Database rename

This part concerns renaming your database. This is a breaking change for anything that is connected to your database and is completely optional. It is recommended, but not required to perform this step after you have already migrated to collectoss 1.1.

To rename your database:

1. Ensure you have an adequate backup. These instructions will not alter the contents of your CollectOSS database, but this process involves potentially creating and deleting a temporary database. Mistyping these commands may lead to data loss.
2. Stop CollectOSS but keep your database online (using a command like docker stop collectoss-redis-1 collectoss-core-1 collectoss-rabbitmq-1 collectoss-flower-1 collectoss-keyman-1 collectoss-core-1 if your docker compose file was started from a folder called "collectoss")
3. Ensure that all connections to the database are disconnected. Note that some may still be present from other applications
4. If you have another database on your PostgresQL instance, disconnect and reconnect using that database. You may need to create a new temporary database for this purpose. The following step will not work if you are connecting to the database that is being renamed.
5. Run ALTER DATABASE augur RENAME TO collectoss; (or choose your own new name)
6. Ensure that the new database name is updated in your docker container configuration for CollectOSS (via environment variables, specifically for the database, core, and flower services).
7. If you created a temporary database for an earlier step, you can now connect to the newly renamed database and drop the test database.
8. Start CollectOSS again (such as with docker start collectoss-redis-1 collectoss-core-1 collectoss-rabbitmq-1 collectoss-flower-1 collectoss-keyman-1 collectoss-core-1) to ensure that everything comes up the way you expect.
9. Ensure other downstream applications that connect to your database are updated with the new database name. Examples of these downstreams may include:
   • The collectoss-monitoring grafana dashboards (if in use)
   • 8Knot
   • any other downstream tools connecting to your database for analysis work (scripts, Jupyter notebooks, etc).