feat(bigquery): tag job IDs with bruin_<source> prefix#2030
Open
sabrikaragonen wants to merge 2 commits into
Open
feat(bigquery): tag job IDs with bruin_<source> prefix#2030sabrikaragonen wants to merge 2 commits into
sabrikaragonen wants to merge 2 commits into
Conversation
Replaces the bare 27-char random job ID generated by the BigQuery Go SDK with a recognizable "bruin_<source>-<random>" form so users can identify the origin of a job in the BQ console and INFORMATION_SCHEMA.JOBS. The 27-char random suffix is preserved (AddJobIDSuffix=true) so polling sensors and multi-statement materializations don't 409 Conflict. Sources tagged: main (asset run), column/custom checks, sensor, query/dryrun (bruin query), diff (data-diff), import, patch, ping, schema (internal browser), enhance. Wired via a context label in pkg/query so non-BQ backends ignore it without coupling. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Contributor
Prompt To Fix All With AIFix the following 1 code review issue. Work through them one at a time, proposing concise fixes.
---
### Issue 1 of 1
pkg/query/logging.go:27-29
`QueryTypeFetch` is defined but never referenced anywhere in the codebase. The PR description's prefix table doesn't list a `bruin_fetch` source either — the `cmd/fetch.go` file's `Query()` command uses `QueryTypeQuery` and `QueryTypeDryRun` instead. Leaving an exported, unused constant creates confusion about what operation it is supposed to label.
```suggestion
QueryTypePatch = "patch"
QueryTypeDryRun = "dryrun"
```
Reviews (1): Last reviewed commit: "feat(bigquery): tag job IDs with bruin_<..." | Re-trigger Greptile |
- copyloopvar: drop tc := tc rebinding (unneeded since Go 1.22) - testifylint: use assert.Empty instead of assert.Equal(t, "", x) - remove unused QueryTypeFetch constant (greptile review): the bruin query command tags as QueryTypeQuery / QueryTypeDryRun rather than fetch. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Replaces the bare 27-char random BigQuery job ID with a recognizable
bruin_<source>-<random>form. Users can now spot — and group/filter — Bruin-originated jobs in the BQ console andINFORMATION_SCHEMA.JOBS.Before:
Z1zrM70lUZw8pqTCNzwhGOstBKMAfter:
bruin_main-Z1zrM70lUZw8pqTCNzwhGOstBKMThe random 27-char suffix is preserved (
AddJobIDSuffix=true) so polling sensors and multi-statement materializations don't collide on(project, location, jobId).Sources tagged
bruin_mainBasicOperator)bruin_columnbruin_custombruin_sensorbruin_querybruin query(live)bruin_dryrunbruin query --dry-runand lint asset/hook/custom-check dry-runsbruin_diffbruin data-diffbruin_importbruin import(db + scheduled queries)bruin_patchbruin patchcolumn fillbruin_pingbruin connections pingbruin_schemainternal connectionsschema browser (GetDatabaseSummary/GetDatabases/GetTables/GetColumns)bruin_enhancebruin enhancecolumn samplingImplementation
pkg/query/logging.go— addsWithQueryType(ctx, type)/QueryTypeFromContext(ctx)plus stable string constants.pkg/bigquery/db.go— new privateapplyJobIDPrefix(ctx, *bigquery.Query)called at all fiveclient.Query(...)sites (IsValid,RunQueryWithoutResult,Select,SelectWithSchema,QueryDryRun). Reads the context label and setsJobID = "bruin_" + labelwithAddJobIDSuffix = true.BasicOperator,ColumnCheckOperator,CustomCheckOperator,QuerySensor,TableSensor) and CLI entry points wrap the context with the appropriate label.Context-based on purpose: non-BQ backends ignore the label without any coupling, and we don't have to touch every backend's mocks.
Test plan
make formatclean (Python ruff, go vet, gci, gofumpt)make test— full unit suite passespkg/query/logging_test.go::TestQueryType— round-trip, empty, overridepkg/bigquery/jobid_test.go::TestApplyJobIDPrefix— no-op when unset; all 12 type → prefix mappingsbruin_main-...in the BQ consolebruin query,bruin data-diff,bruin connections ping, lint with dry-run — confirm each produces the expected prefix inINFORMATION_SCHEMA.JOBS🤖 Generated with Claude Code