Skip to content

feat(bigquery): tag job IDs with bruin_<source> prefix#2030

Open
sabrikaragonen wants to merge 2 commits into
mainfrom
sabrikaragonen/fix-bq-job-id
Open

feat(bigquery): tag job IDs with bruin_<source> prefix#2030
sabrikaragonen wants to merge 2 commits into
mainfrom
sabrikaragonen/fix-bq-job-id

Conversation

@sabrikaragonen
Copy link
Copy Markdown
Contributor

Summary

Replaces the bare 27-char random BigQuery job ID with a recognizable bruin_<source>-<random> form. Users can now spot — and group/filter — Bruin-originated jobs in the BQ console and INFORMATION_SCHEMA.JOBS.

Before: Z1zrM70lUZw8pqTCNzwhGOstBKM
After: bruin_main-Z1zrM70lUZw8pqTCNzwhGOstBKM

The random 27-char suffix is preserved (AddJobIDSuffix=true) so polling sensors and multi-statement materializations don't collide on (project, location, jobId).

Sources tagged

Prefix Origin
bruin_main Asset materialization (BasicOperator)
bruin_column Column checks
bruin_custom Custom checks
bruin_sensor Query / table sensors
bruin_query bruin query (live)
bruin_dryrun bruin query --dry-run and lint asset/hook/custom-check dry-runs
bruin_diff bruin data-diff
bruin_import bruin import (db + scheduled queries)
bruin_patch bruin patch column fill
bruin_ping bruin connections ping
bruin_schema internal connections schema browser (GetDatabaseSummary/GetDatabases/GetTables/GetColumns)
bruin_enhance bruin enhance column sampling

Implementation

  • pkg/query/logging.go — adds WithQueryType(ctx, type) / QueryTypeFromContext(ctx) plus stable string constants.
  • pkg/bigquery/db.go — new private applyJobIDPrefix(ctx, *bigquery.Query) called at all five client.Query(...) sites (IsValid, RunQueryWithoutResult, Select, SelectWithSchema, QueryDryRun). Reads the context label and sets JobID = "bruin_" + label with AddJobIDSuffix = true.
  • Operators (BasicOperator, ColumnCheckOperator, CustomCheckOperator, QuerySensor, TableSensor) and CLI entry points wrap the context with the appropriate label.

Context-based on purpose: non-BQ backends ignore the label without any coupling, and we don't have to touch every backend's mocks.

Test plan

  • make format clean (Python ruff, go vet, gci, gofumpt)
  • make test — full unit suite passes
  • New unit tests:
    • pkg/query/logging_test.go::TestQueryType — round-trip, empty, override
    • pkg/bigquery/jobid_test.go::TestApplyJobIDPrefix — no-op when unset; all 12 type → prefix mappings
  • Manual: run an asset against BigQuery and confirm the job appears as bruin_main-... in the BQ console
  • Manual: trigger a sensor and confirm repeated polls don't 409 (random suffix preserved)
  • Manual: bruin query, bruin data-diff, bruin connections ping, lint with dry-run — confirm each produces the expected prefix in INFORMATION_SCHEMA.JOBS

🤖 Generated with Claude Code

Replaces the bare 27-char random job ID generated by the BigQuery Go SDK
with a recognizable "bruin_<source>-<random>" form so users can identify
the origin of a job in the BQ console and INFORMATION_SCHEMA.JOBS. The
27-char random suffix is preserved (AddJobIDSuffix=true) so polling
sensors and multi-statement materializations don't 409 Conflict.

Sources tagged: main (asset run), column/custom checks, sensor,
query/dryrun (bruin query), diff (data-diff), import, patch, ping,
schema (internal browser), enhance.

Wired via a context label in pkg/query so non-BQ backends ignore it
without coupling.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
@greptile-apps
Copy link
Copy Markdown
Contributor

greptile-apps Bot commented May 9, 2026

Prompt To Fix All With AI
Fix the following 1 code review issue. Work through them one at a time, proposing concise fixes.

---

### Issue 1 of 1
pkg/query/logging.go:27-29
`QueryTypeFetch` is defined but never referenced anywhere in the codebase. The PR description's prefix table doesn't list a `bruin_fetch` source either — the `cmd/fetch.go` file's `Query()` command uses `QueryTypeQuery` and `QueryTypeDryRun` instead. Leaving an exported, unused constant creates confusion about what operation it is supposed to label.

```suggestion
	QueryTypePatch   = "patch"
	QueryTypeDryRun  = "dryrun"
```

Reviews (1): Last reviewed commit: "feat(bigquery): tag job IDs with bruin_<..." | Re-trigger Greptile

Comment thread pkg/query/logging.go
- copyloopvar: drop tc := tc rebinding (unneeded since Go 1.22)
- testifylint: use assert.Empty instead of assert.Equal(t, "", x)
- remove unused QueryTypeFetch constant (greptile review): the
  bruin query command tags as QueryTypeQuery / QueryTypeDryRun
  rather than fetch.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant