POC: New API endpoint with extendable metrics

[josecelano]

Relates to:

• Overhaul stats: Segregated metrics for each tracker running on a different socket address #1263 (comment)
• Wrong statistics imported from the tracker (global stats) torrust-index#809

This is only a Proof of Concept.

It adds a new REST API endpoint with a new extendable format for metrics:

URL: http://0.0.0.0:1212/api/v1/metrics?token=MyAccessToken

Sample response:

{
  "labeled_metrics": [
    {
      "metric": {
        "name": "announce_requests_received_total",
        "kind": "counter",
        "value": 1
      },
      "labels": {
        "ip_version": "ipv4",
        "protocol": "http",
        "url": "http://0.0.0.0:7070"
      }
    },
    {
      "metric": {
        "name": "scrape_requests_received_total",
        "kind": "counter",
        "value": 1
      },
      "labels": {
        "ip_version": "ipv4",
        "protocol": "http",
        "url": "http://0.0.0.0:7070"
      }
    }
  ]
}

It only supports JSON format for now. It doesn't support Prometheus format like the current stats endpoint:

URL: http://0.0.0.0:1212/api/v1/stats?token=MyAccessToken&format=prometheus

The Prometheus format response should be like the following:

announce_requests_received_total{ ip_version="ipv4", protocol="http", url="http://0.0.0.0:7070"} 1

scrape_requests_received_total{ ip_version="ipv4", protocol="http", url="http://0.0.0.0:7070"} 1

TODO

• Replace primitive types in the new labeled metrics with new tpyes to enforce constraints. For example, metrics names, labels, metric types, etc.
• Inject the rigth URL scheme. It's hardcoded now.
• Implement for UDP metrics. It's only implemented for HTTP tracker metrics. This would require merging labeled metrics.
• Use a f64 for metric values instead of u64.
• The metric name and label set pair must be unique in the array of labeled metrics. Enforce constraint.
• Implement versioning. A given API version must contain a set of labeled metrics. Clients expects some labeled metrics to be included in a API version. We need to initialize the array of metrics with all the expected labeled metrics with the initial value.
• Review names for fields in the API response. For example rename url to server_url, ...
• Review what labels we include per metric. For example the "protocol" label can be derived from the URL but it requires parsing the URL and it makes harder to build graphs in Grafana. I guess we should include labels for the type of aggregate date we want to get.

IMPORTANT

• Each metric+label set values combination is a different time series in Prometheus. We can't use labels with high cardinality. See https://prometheus.io/docs/practices/naming/#labels. For example, we can't use client's IPs.

@da2ce7 it would be nice to have feedback before continuing working on this approach.

[josecelano]

I have continued with the implementation of the POC here.