Overhaul stats: Persist all torrents downloads metric

[josecelano]

A new metric was added to the tracker-core package.

It must be persisted so the counter is initialised with the previous value when the tracker is restarted.

Proposal 1

Allow persistence/load of metric samples from the database.

This would solve the problem for any case where we need to persist a metric.

However, there are a lot of open questions:

• Should we persist only the latest sample? all samples?
• How should we represent the labels? One single column with a compact JSON? Independent table for labels?
• Should we represent "counter" and "gauge" types with different column types?
• Should we only provide methods to save/load or methods to increase values too?

Given that we only have one persited metric (number of downloads) I'm going to keep it simple for now. See proposal 2.

Proposal 2

Currently, we have a table:

CREATE TABLE
    IF NOT EXISTS torrents (
        id integer PRIMARY KEY AUTO_INCREMENT,
        info_hash VARCHAR(40) NOT NULL UNIQUE,
        completed INTEGER DEFAULT 0 NOT NULL
    );

To store the number of downloads per torrent.

The type DatabasePersistentTorrentRepository is a repository that handles those metrics.

I also defined these two aliases:

pub type PersistentTorrent = u32;
pub type PersistentTorrents = BTreeMap<InfoHash, PersistentTorrent>;

The reason I called PersistentTorrent is that we could need to persist more info related to the torrent. In the end, that table only contains metrics. My proposal is to add a new table for aggregate metrics and also refactor the current types to reflect what they are doing right now.

# mysql

# The current table with the download counter per torrent
CREATE TABLE
    # This should be renamed to `torrent_metrics` in the future
    IF NOT EXISTS torrents (
        id integer PRIMARY KEY AUTO_INCREMENT,
        info_hash VARCHAR(40) NOT NULL UNIQUE,
        completed INTEGER DEFAULT 0 NOT NULL
    );

# The new table with the global (all torrents) download counter
CREATE TABLE
    IF NOT EXISTS torrent_aggregate_metrics (
        id integer PRIMARY KEY AUTO_INCREMENT,
        key VARCHAR(50) NOT NULL UNIQUE,
        value INTEGER DEFAULT 0 NOT NULL
    );

• We can insert a new record: 1, "completed", 100 (id, key, value)
• We can provide a method to directly increase the value like DatabasePersistentTorrentRepository::increase_number_of_downloads

NOTE: I will keep AUTO_INCREMENT to follow the current design. However, I think we don't need it.

I will rename the types and methods. For example:

• DatabasePersistentTorrentRepository to DatabaseDownloadsMetricRepository
• PersistentTorrent to NumberOfDownloads (I might introduce a new type for this)
• Etc

I think this solution is much simpler and enough for now. Besides, if some users want to persist extra metrics, they can build their listeners.

On the other hand, it also allows adding more persisted metrics by adding new columns to the torrents metrics.

In the end, the current solution is an example of how you can build your metrics views from events.

If you think there are other metrics that we should (optionally) persist, let me know. I don't remember any feature request to persist more metrics.

cc @da2ce7