Skip to content

feat: reject replacement references to undefined capture groups - #346

Open
vlasky wants to merge 1 commit into
chmln:masterfrom
vlasky:feat/validate-undefined-captures
Open

feat: reject replacement references to undefined capture groups#346
vlasky wants to merge 1 commit into
chmln:masterfrom
vlasky:feat/validate-undefined-captures

Conversation

@vlasky

@vlasky vlasky commented Jul 29, 2026

Copy link
Copy Markdown

Problem

A capture reference in the replacement that the pattern never defines expands to the empty string, with exit status 0. The most damaging form is a shell variable that arrived unexpanded because it was single-quoted:

$ cat script.sh
rm -f "$dir/x"
$ sd 'rm -f' 'rm_temp "$path"' script.sh
$ echo $?
0
$ cat script.sh
rm_temp "" "$dir/x"

$path is read as a capture reference, no such group exists, and it expands to nothing. Nothing in the output or the exit status distinguishes this from a correct edit, so it survives review and shows up later as a file that quietly lost text. The same applies to an out-of-range number ($5 against a two-group pattern) and to a misspelt named group ($wrod for $word).

validate_replace already walked the capture references in the replacement and rejected ambiguous ones like $1bad. It just never checked whether the referenced group exists.

Change

Validate capture references against the built regex. Replacer::new now builds the regex first so validation can see it, with regex construction moved into a build_regex helper.

$ sd 'rm -f' 'rm_temp "$path"' script.sh
error: The capture group `$path` in the replacement text does not exist in the pattern.
hint: The pattern has no named capture groups. If you meant a literal `$` (a shell variable, for instance), write `$$path`, or use -F to treat both args as literal strings.
rm_temp "$path"
          ^^^^

The hint is chosen from what the pattern actually offers:

$ echo abc | sd '(a)(b)' '$1$5'
error: The capture group `$5` in the replacement text does not exist in the pattern.
hint: The pattern has 2 capture group(s), so only $0 to $2 are valid.

$ echo abc | sd '(?<word>a)' '$wrod'
error: The capture group `$wrod` in the replacement text does not exist in the pattern.
hint: Available named groups: `$word`. For a literal `$`, write `$$wrod`.

Validation runs before anything is written, so the file is left untouched.

Scope and limits

This catches a reference to a group that does not exist. It cannot catch sd 'a' 'x$1' where group 1 does exist but a literal was meant, since that is genuinely ambiguous. Every case I have hit in practice was a reference to a nonexistent group.

-F/--fixed-strings is unaffected: the replacement is already literal there, so no validation happens.

Related issues

Compatibility

A replacement that referenced a nonexistent group used to "work", expanding to nothing, and is now an error. That is the point of the change, but it is a behavioural break for anyone relying on it to delete text, for which '' is the correct spelling.

Testing

Existing behaviour is preserved: $2 $1, ${1}x, $verb on a named group, $0, $01, optional groups like a(b)?, named-group alternation, -F, stdin, and the -f w word-boundary path all still work.

New: 6 unit tests in sd/src/replacer/validate.rs and 6 integration tests in sd-cli/tests/cli.rs, including three insta snapshots of the error output, a check that the file is untouched on error, and a check that $$ is still not treated as a capture. The existing proptests were pointed at validate_unambiguous, extracted for the purpose, since pattern-independent ambiguity is what they actually test.

60 tests pass on the workspace. cargo fmt --check and cargo clippy --workspace --all-targets -- -D warnings are clean.

I have left CHANGELOG.md alone, since its history looks like maintainer-written release prep. Happy to add an entry if you would prefer it in the PR.

A `$name` or out-of-range `$1`-style reference in the replacement expanded
to the empty string, so text the user meant to keep was silently deleted.
The usual cause is a single-quoted shell variable arriving unexpanded:

    sd 'rm -f' 'rm_temp "$path"'    ->    rm_temp ""

Nothing signalled the problem. The replacement is valid syntax, the pattern
matched, and the exit code was 0, so scripts carried on with a mangled file.

validate_replace already walked every capture reference to reject ambiguous
ones like `$1x`, so this extends it to also check each reference against the
built regex via captures_len/capture_names. The regex is therefore built
before the replacement is validated, which splits Replacer::new's regex
construction into build_regex.

The error names the offending group and points at the `$$` escape, since
that is the actual fix. Where the pattern has named groups it lists them,
and for a numbered reference it gives the valid range.

The pattern-independent ambiguity check moves to validate_unambiguous, which
is what the existing proptests exercise; they have no pattern to validate
against and keep their current meaning.

This is a breaking change: a command relying on an undefined capture
expanding to nothing must now write `$$` or use -F. Literal mode is
unaffected, as it never interpolates.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant