feat: reject replacement references to undefined capture groups - #346
Open
vlasky wants to merge 1 commit into
Open
feat: reject replacement references to undefined capture groups#346vlasky wants to merge 1 commit into
vlasky wants to merge 1 commit into
Conversation
A `$name` or out-of-range `$1`-style reference in the replacement expanded
to the empty string, so text the user meant to keep was silently deleted.
The usual cause is a single-quoted shell variable arriving unexpanded:
sd 'rm -f' 'rm_temp "$path"' -> rm_temp ""
Nothing signalled the problem. The replacement is valid syntax, the pattern
matched, and the exit code was 0, so scripts carried on with a mangled file.
validate_replace already walked every capture reference to reject ambiguous
ones like `$1x`, so this extends it to also check each reference against the
built regex via captures_len/capture_names. The regex is therefore built
before the replacement is validated, which splits Replacer::new's regex
construction into build_regex.
The error names the offending group and points at the `$$` escape, since
that is the actual fix. Where the pattern has named groups it lists them,
and for a numbered reference it gives the valid range.
The pattern-independent ambiguity check moves to validate_unambiguous, which
is what the existing proptests exercise; they have no pattern to validate
against and keep their current meaning.
This is a breaking change: a command relying on an undefined capture
expanding to nothing must now write `$$` or use -F. Literal mode is
unaffected, as it never interpolates.
This was referenced Jul 29, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
A capture reference in the replacement that the pattern never defines expands to the empty string, with exit status 0. The most damaging form is a shell variable that arrived unexpanded because it was single-quoted:
$pathis read as a capture reference, no such group exists, and it expands to nothing. Nothing in the output or the exit status distinguishes this from a correct edit, so it survives review and shows up later as a file that quietly lost text. The same applies to an out-of-range number ($5against a two-group pattern) and to a misspelt named group ($wrodfor$word).validate_replacealready walked the capture references in the replacement and rejected ambiguous ones like$1bad. It just never checked whether the referenced group exists.Change
Validate capture references against the built regex.
Replacer::newnow builds the regex first so validation can see it, with regex construction moved into abuild_regexhelper.The hint is chosen from what the pattern actually offers:
Validation runs before anything is written, so the file is left untouched.
Scope and limits
This catches a reference to a group that does not exist. It cannot catch
sd 'a' 'x$1'where group 1 does exist but a literal was meant, since that is genuinely ambiguous. Every case I have hit in practice was a reference to a nonexistent group.-F/--fixed-stringsis unaffected: the replacement is already literal there, so no validation happens.Related issues
$) becomes largely moot, since the error explains the escape at the point of failure.$for capture group prefix #288 asks for a capture prefix other than$because it collides with shell variables. This addresses the failure mode without a breaking syntax change.Compatibility
A replacement that referenced a nonexistent group used to "work", expanding to nothing, and is now an error. That is the point of the change, but it is a behavioural break for anyone relying on it to delete text, for which
''is the correct spelling.Testing
Existing behaviour is preserved:
$2 $1,${1}x,$verbon a named group,$0,$01, optional groups likea(b)?, named-group alternation,-F, stdin, and the-f wword-boundary path all still work.New: 6 unit tests in
sd/src/replacer/validate.rsand 6 integration tests insd-cli/tests/cli.rs, including three insta snapshots of the error output, a check that the file is untouched on error, and a check that$$is still not treated as a capture. The existing proptests were pointed atvalidate_unambiguous, extracted for the purpose, since pattern-independent ambiguity is what they actually test.60 tests pass on the workspace.
cargo fmt --checkandcargo clippy --workspace --all-targets -- -D warningsare clean.I have left
CHANGELOG.mdalone, since its history looks like maintainer-written release prep. Happy to add an entry if you would prefer it in the PR.