How to access stats & bin requests

[Azathothas]

Hi, I would like to access the stats & bin requests i.e I want to know what crates people are installing (requesting) the most.

The stats site linked @https://github.com/cargo-bins/cargo-binstall?tab=readme-ov-file#telemetry-collection, no longer seems active: https://warehouse-clerk-tmp.vercel.app/api/stats

And the plain text that used to exist @popular-crates.txt, also seems to be no longer available.

Per #268 (comment), it seems you have moved to fly.io but where...

[NobodyXu]

Hello, this is the python file we use for getting statistics

https://github.com/cargo-bins/cargo-quickinstall/blob/main/cronjob_scripts/stats.py

[Azathothas]

But that seems to be locked behind a token: https://github.com/cargo-bins/cargo-quickinstall/blob/main/.github/workflows/cronjob.yml#L40
Is it not publicly accessible?

[NobodyXu]

cc @alsuren do we have public access for stats?

[alsuren]

I did have a public grafana dash somewhere. I'm currently away from my computer. I have put a task in for Tuesday when I am back. I will dig it up and also try making another read-only influxdb token to share.

[NobodyXu]

Thanks alsuren! Sorry I totally forgot how to access it 😅

[alsuren]

@Azathothas it turns out I did write some stuff down: #279 - https://alsuren.grafana.net/public-dashboards/12d4ec3edf2548a1850a813e00592b53 . Take a look and tell me if this is enough for you.

If you need access to the raw data then I will make a new token for you (if you promise not to blast through our rate limits) or even add you to the influx cloud account. The tokens UI in influxdb cloud is a bit confusing (I can't find the tokens that are used for the stats server or the grafana dash).

@NobodyXu your influx cloud invite expired without being accepted. I have resent it.

[Azathothas]

Hi @alsuren, that dashboard certainly offers part of what I was looking for.
But I think publishing raw data would be better.
If rate limits are an issue, what about a daily dump as a GitHub release, or on a new branch, right here on the repo, once a day?
This would allow me to have the data without blasting your actual stat server.

[alsuren]

@Azathothas do you think you can find a way to publish the stats so they can be easily imported by datasette lite? Can we publish a stream of parquet files somewhere? I feel like there must be a standard way of doing this.

Have a think about a few possible ways of doing this and I'll give you a token on Tuesday to play about with (can I have your email address to send the token?)

[Azathothas]

Hi, unfortunately I am not at all familiar with any of these.
So I am not sure how I can help you.

I wanted the raw data so I could just grep it....