test: use go-vcr to mock osv.dev bulk query calls

[G-Rath]

This is an attempt to make our CI easier to work with day-to-day by introducing go-vcr as a proxy to record and replay certain HTTP requests, with our "update snapshots" workflow being updated to also update these records.

The idea is this will let us continue effectively testing against real external APIs but let us decide when those external APIs change by way of updating the HTTP records (terms cassettes, in keeping with the VCR metaphor).

For now I've just focused on the osv.dev client, setting up go-vcr to record interactions with any endpoint aside from /v1/vulns/<id> since those requests have huge responses making the cassettes easily balloon beyond 5mb per file - while these do somewhat frequently change, I'm hoping that their day-to-day changes won't be as noisy and especially considering its the response of the bulk query endpoint that leads to the specific vuln lookups so since they're being recorded things should be a lot more predictable.

I've structured things so that a recording client is created per top-level test so that a single cassette is used for any subtests, and a X-Test-Name header is included in each request via a second http.Client with a custom RoundTripper that gets created in each subtest wrapping the recording client, which allows us to sort the interactions afterwards giving us a stable diff when regenerating.

I have also included a custom hook to remove unneeded headers to both further reduce the noise in diffs and the size of the cassettes overall; the only header I'm on the fence about is the Date header since that relates to caching and its technically possible for code to check that for determining if data should be re-fetched, but I think it's best to omit it for now given its present in every single request (ideally I'd like to have it be updated only when something else changes, but that doesn't seem possible without a lot more work).

Finally, one notable downside for us to keep an eye on is with keeping the cassettes up to date: the nature of this functionality means its inherently hard for such a library to determine if a particular record is obsolete or needs to be generated as (among other things) that would most likely require making the request in the first place e.g. if we change what headers are excluded but don't explicitly regenerate everything, I don't think CI will complain - thankfully, the worse case is that this should get caught by our "update snapshots" workflow

[G-Rath]

hmm so (unsurprisingly, in hindsight) this really moves the problem rather than solve it outright as now our CI can fail due to other external conditions, and it doesn't present itself super obviously.

We have these failing tests:

• TestCommand_OCIImage_JSONFormat/Scanning_python_image_with_some_packages
• TestCommand_OCIImage_JSONFormat/Scanning_python_image_with_some_packages
• TestCommand_LockfileWithExplicitParseAs/"apk-installed"_is_supported
• TestCommand_LockfileWithExplicitParseAs/"dpkg-status"_is_supported
• TestCommand/requirements.txt_can_have_all_kinds_of_names

The first two are really the same failure because its the same underlying image just with a different output format, and the issue is that a new version of certifi has been published

This is addressed by running scripts/build_test_images.sh --force --no-cache, and something we already knew about.

The next two I'm guessing are a result of the underlying OS extractors pulling information from the local environment which for me is Ubuntu 22.04 but CI is e.g. Ubuntu 24.04

And then the last one I'm guessing is the result of typing-extensions having a new version, and that that is a transient dependency of one of our fixtures since its not mentioned anywhere in our codebase

[codecov-commenter]

Codecov Report

❌ Patch coverage is 59.09091% with 63 lines in your changes missing coverage. Please review.
✅ Project coverage is 67.68%. Comparing base (fe8ffc5) to head (3a3e2ca).

Files with missing lines	Patch %	Lines
cmd/osv-scanner/internal/testcmd/vcr.go	49.58%	48 Missing and 13 partials ⚠️
cmd/osv-scanner/main.go	0.00%	1 Missing ⚠️
cmd/osv-scanner/mcp/command.go	0.00%	1 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #2200      +/-   ##
==========================================
- Coverage   67.83%   67.68%   -0.16%     
==========================================
  Files         171      172       +1     
  Lines       13019    13147     +128     
==========================================
+ Hits         8831     8898      +67     
- Misses       3503     3551      +48     
- Partials      685      698      +13     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[another-rex]

It works!

@G-Rath PTAL, and I think we might need to move update script to a bot or similar so we can update the snapshots without having to run windows.

[G-Rath]

(I can't approve this since it's my own PR)

looks good, lets get this landed before the snapshots change again 😅 (which I think they already have...)

[another-rex]

One thing I want to do before merging is to do the formatting similar to how it's done on osv.dev. Right now it's really hard to see the diff when updating cassettes.