Better Logs for Hearing Scraper#2170
Merged
Merged
Conversation
…ling the error ourselves has been masking issues where we'd want to investigate even if we don't want the fallback to blow up the whole function execution (particularly in batch scrape jobs). This commit handles the scrapeHearings function where we saw the issue, but there will be follow-ups shortly to handle other functions we have seen this for
Mephistic
requested review from
alexjball,
kiminkim724,
mertbagt,
mvictor55,
nesanders,
sashamaryl and
timblais
as code owners
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
Collaborator
Author
|
For reference, this worked as expected - I'm going to go through and replace all of the catch/console.errors with functions.logger.errors |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
We've run into a recent problem with the hearing scraper where we hit our bill spend limit on AssemblyAI, but did not notice because nothing was emitted to Error Reporting. I believe this is happening because we're catching and handling the error instead of letting it percolate to the Firebase Functions error reporting handler - which is why we see some errors correctly reported, but not all.
This will hit us especially hard on our batch scrapers, because we swallow errors with individual resources to avoid crashing the whole job - which means we can miss widespread issues..
This PR is a test of a fix for this (that can only be confirmed in a deployed environment) - by using the firebase functions error logger directly (instead of indirectly via the
consoledecorators), it should make it simpler for Google Cloud Error Reporting to detect and report these errors.This is likely related to #2120 - a similar error reporting-related issue we found in our Bill Scraper.
If this works as expected, I'll follow up by instrumenting all of our caught errors in Firebase Functions with the same fix.