Ontologies table not loading

[anitacaron]

I've checked the network tab in Inspection, and I see an error when fetching dashboard-results.json, then I asked AI what the issue is.

This request attempted to retrieve a JSON file from GitHub, but it failed with a 429 Too Many Requests status code. This indicates that the client has sent too many requests in a given amount of time and has been rate-limited by the server (GitHub).

The low timing durations suggest the request was quickly rejected by the server due to the rate limit, rather than experiencing network latency or large response size issues.

The problem is that your application is making requests to this GitHub resource too frequently, exceeding the server's allowed rate. To fix this, you'll need to:

Reduce the frequency of requests to raw.githubusercontent.com.
Implement a rate limiting strategy on your client-side to avoid hitting this limit.
Consider caching the data if it doesn't need to be fetched on every page load or user interaction.
If possible, explore if there's a more appropriate API or method to get this data from OBOFoundry, as directly fetching from raw.githubusercontent.com might not be designed for high-volume programmatic access.

[matentzn]

Yikes!

[jsstevenson]

Thanks for catching this @anitacaron -- this is a little annoying but not unsurprising. A few action items

1. the dashboard itself should be resilient to this problem - I'm guessing we don't properly catch the 429 response here, but there's no reason why the rest of the table should be unable to render. I can do something for that this week

2. I suspect this is a pretty rare occurrence, so I'm not sure whether it's worth it to do much re-engineering beyond this. If a complete fix is necessary, though, there are probably two days to go about it

a) manual but annoying -- import the dashboard repo as a submodule here, and then make new commits to update the submodule whenever a new dashboard release comes out

b) automatic but more complicated -- the above might be achievable largely as a github action, although IIRC it is kind of tricky to get github actions to make commits without somebody providing/updating an authorization token

Either way I'll make a note about this in the OFOC call agenda

[jsstevenson]

Started seeing this today, on a fresh page load, and it didn't go away after reloading a few times.

[nlharris]

I'm seeing this as well. Cloudflare was down earlier today, and it broke a lot of things - could that be the cause?

[jsstevenson]

Maybe? I would think things are back to functional again, but it would also kind of make sense to see more stringent request throttling behavior in the wake of a CDN outage. Worth looking into either way

[nlharris]

Sounds like it's because GitHub is down (or at least having issues): https://www.githubstatus.com/

[jsstevenson]

General notes here -- the code in assets/js/custom.js is written such that the table never renders if the original HTTP request fails. It uses a chain of .then() calls. I'm revamping this to set a default for the dasbhoard results values of "unknown", and then the request updates those defaults if it's successful, and leaves them as ??? in the rare case that it fails.

[nlharris]

Is this related to the Cloudflare stuff @kltm was discussing in email?

[jsstevenson]

@nlharris unlikely -- I wouldn't expect that to be returning a 429

[nlharris]

@jsstevenson is this something you're looking into?

[jsstevenson]

@nlharris (sigh) yes and no -- the rewrite I'd like to do to address this is a few hours of work, I've had a few false starts but it's been tough to find a good block of time to knock it out