Feature request: better tools for controlling spend on the org level, especially checkpoint storage spend

[sasha-apollo]

We have an org in Tinker with multiple people in it.

Every person can see what they themselves spend on tokens and checkpoints but AFAICT there is no way currently to get an overview of what the whole org spends the money on. At org level I just see how much credits we bought and that's it. There is no way to figure out who is spending money and on what.

We have a lot of checkpoints and I suspect we are spending a bunch of money just on storing them. I wanted to build a tool that would clean them up but apparently the only way to do that is to ask every person to create and give me an API key, so that I'm able to see their checkpoints?

Is there an easier solution that I'm missing?
Are any API and Web UI improvements in this area planned?

[dphuang2]

Are any API and Web UI improvements in this area planned?

We have Web UI improvements on the roadmap. So please stay tuned.

Also are you just looking for observability or also limit controls?

[sasha-apollo]

Mostly observability and also some solution to checkpoints accumulating uncontrollably because people will forget to remove them.

I was thinking of writing a cron job that:

1. Sets TTL=30d for all checkpoints that don't already have a TTL.
2. Warns people about checkpoints that are about to be deleted.

People can then opt out by setting their own TTL for checkpoints that they want to keep.

It seems doable already. The inconvenient things are:

1. Everyone on the team has to give me their Tinker API key.
2. If I sum the spend by every user who gave me their key and it doesn't match the total org spend, I don't know who is responsible for the unexplained spend.

If something like this gets implemented on Tinker side then maybe that's even better.