feat: add secret redaction feature to replace sensitive values by Ubisoft-potato · Pull Request #14 · bucketeer-io/code-refs

Ubisoft-potato · 2026-07-07T02:50:06Z

What

Redacts secret-looking values from the code lines sent to Bucketeer, replacing them with [REDACTED] so credentials near a flag reference don't leak into the dashboard or CSV output.

How

On by default (--redactSecrets, default true); disable with --redactSecrets=false or BUCKETEER_REDACT_SECRETS=false.
Detection is powered by the gitleaks ruleset (github.com/zricethezav/gitleaks/v8, MIT), covering hundreds of vendor credential formats with entropy checks.
A generic layer on top redacts Authorization header values, passwords in URLs (postgres://user:pass@host), and quoted or unquoted assignments to secret-keyword variables (apikey, secret, token, passwd, password, credential, auth). Unquoted values must be 12+ chars mixing letters and digits, so code like apiKey := opts.APIKey stays untouched; this also covers ~/.aws/credentials-style key pairs.
Detection runs on the whole hunk, so multi-line PEM private key blocks are fully redacted, including blocks cut off by the hunk boundary at either end.
Extensible via .bucketeer/coderefs.yaml: redactPatterns (extra regexes) and redactKeywords (extra variable-name keywords). Invalid patterns fail fast at startup.
Redaction happens at hunk creation, keeping the API payload, CSV output, and ContentHash consistent; flag matching runs on the original lines.

Also included

Fix: copy hunk lines so truncation doesn't mutate the file's cached lines.
CI: golangci-lint upgraded to v2.12.2; config migrated to v2 format.
go directive bumped to 1.25.0; vendor tree updated.

Example

awsKey := "AKIA..."                      // → awsKey := "[REDACTED]"
password := 'hunter2hunter2'             // → password := '[REDACTED]'
client.BoolVariation("someFlag", ...)    // → unchanged, flag still detected

Docs: Secret redaction.

Copilot

Pull request overview

Adds an opt-out-by-default secret redaction layer so code hunks sent to Bucketeer don’t leak credential-like values, while keeping flag matching based on the original (unredacted) source lines.

Changes:

Add a configurable redactor (built-in patterns + YAML extensibility) and apply it during hunk creation.
Wire redaction into matcher construction via a new --redactSecrets option (default true) plus YAML fields for extra patterns/keywords.
Update docs/tests and include .ldignore in ignore-file processing.

Reviewed changes

Copilot reviewed 10 out of 10 changed files in this pull request and generated 1 comment.

Show a summary per file

File	Description
search/search.go	Copies hunk lines before mutation and applies optional redaction before truncation/hash generation.
search/search_test.go	Updates expectations for hunks when context lines are disabled.
search/redact.go	Implements the redaction engine and built-in credential patterns.
search/redact_test.go	Adds unit/integration tests for redaction behavior and hunk redaction.
search/matcher.go	Instantiates the redactor based on options and attaches it to `Matcher`.
search/files.go	Adds `.ldignore` to the list of ignore files.
options/options.go	Adds redaction-related options and validates user-provided regex patterns.
options/flags.go	Introduces the `--redactSecrets` CLI flag (default true).
docs/CONFIGURATION.md	Documents secret redaction configuration and YAML extensions.
README.md	Adds a documentation link to the secret redaction section.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

cre8ivejp · 2026-07-08T03:11:42Z

@Ubisoft-potato, it will be safer and more stable to use OSS, which is focused on scanning sensitive info.

Reference:
https://github.com/gitleaks/gitleaks
https://github.com/praetorian-inc/titus
https://github.com/trufflesecurity/trufflehog

While GitLeaks is more popular, Titus covers almost 3 times as much as GitLeaks, but it is still a bit new.
Either one would be better than implementing ourselves.
WDYT?

cre8ivejp · 2026-07-08T03:21:12Z

This one is from the same author of GitLeaks. It's more accurate and faster.
https://github.com/betterleaks/betterleaks

Repository	Gitleaks	Betterleaks	Speedup
Rails	24.5s	5.8s ✅	4.2x faster
Ruby	55.2s	10.3s ✅	5.4x faster
GitLab	11m28s	2m13s ✅	5.2x faster

Is also developed in Go. We can try it!

…[REDACTED]"

The previous golangci-lint v1.62.2 was built with Go 1.23 and fails on the go 1.25.0 directive required by betterleaks. Migrate .golangci.yml to the v2 config format (dead linters dropped by the migration tool), keep gating parity by disabling linters introduced after v1.62 and restricting staticcheck to SA checks, and fix the one new prealloc finding.

Copilot

Pull request overview

Copilot reviewed 14 out of 15 changed files in this pull request and generated 1 comment.

Copilot

Pull request overview

Copilot reviewed 14 out of 15 changed files in this pull request and generated 3 comments.

The keyword-assignment regexes matched any [:=]+ run, so comparisons like `if authType == "OAuth2.0"` or `token == expectedToken123` were redacted. Spell the separator out as :=, =, or : and forbid a leading = in unquoted values so the tail of == never matches.

godoc was split out of x/tools into its own module and frozen as v0.1.0-deprecated. Inline its small IsText check instead.

Copilot

Pull request overview

Copilot reviewed 14 out of 15 changed files in this pull request and generated 3 comments.

An empty (or empty-matching, e.g. "a*") pattern inserts [REDACTED] between every character of every scanned line. Reject such patterns in both options validation and newRedactor, and document the 6-character floor for quoted keyword assignments.

Copilot

Pull request overview

Copilot reviewed 14 out of 15 changed files in this pull request and generated 1 comment.

Copilot

Pull request overview

Copilot reviewed 14 out of 15 changed files in this pull request and generated 6 comments.

Copilot

Pull request overview

Copilot reviewed 14 out of 15 changed files in this pull request and generated 1 comment.

Copilot

Pull request overview

Copilot reviewed 14 out of 15 changed files in this pull request and generated 1 comment.

Copilot

Pull request overview

Copilot reviewed 16 out of 17 changed files in this pull request and generated 1 comment.

Copilot

Pull request overview

Copilot reviewed 16 out of 17 changed files in this pull request and generated 3 comments.

cre8ivejp

Thank you!

Ubisoft-potato marked this pull request as ready for review July 8, 2026 00:31

Ubisoft-potato requested a review from cre8ivejp July 8, 2026 00:31

cre8ivejp requested a review from Copilot July 8, 2026 00:51

Copilot started reviewing on behalf of cre8ivejp July 8, 2026 00:51 View session

Copilot AI reviewed Jul 8, 2026

View reviewed changes

Comment thread search/redact.go Outdated

Ubisoft-potato force-pushed the feat/redact-secrets branch 3 times, most recently from 21c653b to dbf5485 Compare July 8, 2026 05:46

Ubisoft-potato added 9 commits July 8, 2026 23:40

feat: add secret redaction feature to replace sensitive values with "…

9a7dc4d

…[REDACTED]"

docs: note that redactKeywords match literally, add snake_case example

cc87829

refactor: use betterleaks ruleset for secret detection

0397f6e

feat: redact passwords embedded in URLs

be19a5b

feat: switch secret detection engine from betterleaks to gitleaks

3caf6d2

fix: redact PEM blocks cut off at the top of a hunk

8ce464b

docs: keyword assignments cover unquoted values

2a70db4

chore: remove vendor dir

db01487

Ubisoft-potato force-pushed the feat/redact-secrets branch from 62b269a to db01487 Compare July 9, 2026 06:43

cre8ivejp reviewed Jul 9, 2026

View reviewed changes

Comment thread search/redact.go

cre8ivejp requested a review from Copilot July 10, 2026 08:06

Copilot started reviewing on behalf of cre8ivejp July 10, 2026 08:06 View session

Copilot AI reviewed Jul 10, 2026

View reviewed changes

Comment thread search/redact.go

fix: preserve redactHunk length invariant for empty hunks

9e62a73

cre8ivejp requested a review from Copilot July 13, 2026 03:00

Copilot started reviewing on behalf of cre8ivejp July 13, 2026 03:00 View session

Copilot AI reviewed Jul 13, 2026

View reviewed changes

Comment thread go.mod

Comment thread search/redact.go Outdated

Comment thread search/redact.go Outdated

Ubisoft-potato added 2 commits July 12, 2026 21:58

chore: drop deprecated golang.org/x/tools/godoc dependency

86df5c7

godoc was split out of x/tools into its own module and frozen as v0.1.0-deprecated. Inline its small IsText check instead.

Copilot AI reviewed Jul 13, 2026

View reviewed changes

Comment thread options/options.go

Comment thread search/redact.go

Comment thread docs/CONFIGURATION.md Outdated

Ubisoft-potato requested a review from cre8ivejp July 13, 2026 15:58

cre8ivejp requested a review from Copilot July 14, 2026 02:13

Copilot started reviewing on behalf of cre8ivejp July 14, 2026 02:14 View session

Copilot AI reviewed Jul 14, 2026

View reviewed changes

Comment thread search/search.go Outdated

perf: redact hunks once after merging overlapping windows

8fa05e5

cre8ivejp requested a review from Copilot July 14, 2026 03:22

Copilot started reviewing on behalf of cre8ivejp July 14, 2026 03:23 View session

Copilot AI reviewed Jul 14, 2026

View reviewed changes

Comment thread search/redact_test.go Outdated

Comment thread search/redact_test.go Outdated

Comment thread search/redact_test.go Outdated

Comment thread search/redact_test.go Outdated

Comment thread search/redact_test.go Outdated

Comment thread search/redact_test.go Outdated

test: assemble fake secret fixtures at runtime to avoid scanner alerts

8e3ef21

cre8ivejp requested a review from Copilot July 14, 2026 07:27

Copilot started reviewing on behalf of cre8ivejp July 14, 2026 07:27 View session

Copilot AI reviewed Jul 14, 2026

View reviewed changes

Comment thread search/matcher.go

perf: skip building the redactor when contextLines < 0 sends no code

489794b

cre8ivejp requested a review from Copilot July 14, 2026 08:04

Copilot started reviewing on behalf of cre8ivejp July 14, 2026 08:04 View session

Copilot AI reviewed Jul 14, 2026

View reviewed changes

Comment thread search/search.go

fix: reconcile code references by flag, path, and content hash

bc0496a

cre8ivejp requested a review from Copilot July 15, 2026 02:12

Copilot started reviewing on behalf of cre8ivejp July 15, 2026 02:13 View session

Copilot AI reviewed Jul 15, 2026

View reviewed changes

Comment thread search/files.go

cre8ivejp requested a review from Copilot July 15, 2026 05:47

Copilot started reviewing on behalf of cre8ivejp July 15, 2026 05:47 View session

Copilot AI reviewed Jul 15, 2026

View reviewed changes

Comment thread search/redact_test.go

Comment thread search/redact_test.go

Comment thread search/redact_test.go

test: split private key markers across literals to avoid scanner flags

4aa4d42

cre8ivejp approved these changes Jul 15, 2026

View reviewed changes

cre8ivejp merged commit efa4995 into main Jul 15, 2026
3 checks passed

cre8ivejp deleted the feat/redact-secrets branch July 15, 2026 07:35

bucketeer-bot mentioned this pull request Jul 9, 2026

chore: release v0.0.4 #13

Merged

Uh oh!

Conversation

Ubisoft-potato commented Jul 7, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What

How

Also included

Example

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

cre8ivejp commented Jul 8, 2026

Uh oh!

cre8ivejp commented Jul 8, 2026

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Uh oh!

Uh oh!

cre8ivejp left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Ubisoft-potato commented Jul 7, 2026 •

edited

Loading