|
| 1 | +# Complex Value Support for Replace and Upsert |
| 2 | + |
| 3 | +**Date:** 2026-05-15 |
| 4 | +**Issue:** #18 |
| 5 | + |
| 6 | +## Problem |
| 7 | + |
| 8 | +`Document.replace()` and `Document.upsert()` only accept scalar values. Passing a dict or list raises `PatchError: Patch failed: input is not valid YAML` because yamlpatch's `apply_value_replacement` serializes the value inline after the key colon, producing invalid YAML when the serialized value is multi-line. |
| 9 | + |
| 10 | +`Op.add` and `Op.append` already handle complex values correctly. Only `Op.replace` is broken. |
| 11 | + |
| 12 | +## Approach |
| 13 | + |
| 14 | +Intercept `Replace` operations with complex values (Mapping or Sequence) in yamltrip's Rust layer before they reach yamlpatch. Perform direct string surgery on the document source. Scalar `Replace` and all other operation types pass through to yamlpatch unchanged. |
| 15 | + |
| 16 | +## Change Location |
| 17 | + |
| 18 | +All Rust changes are in `src/document.rs`. No Python API changes. No yamlpatch changes. No new files. |
| 19 | + |
| 20 | +A new function `apply_complex_replace` handles the complex case, called from `PyDocument::apply_patches`. |
| 21 | + |
| 22 | +## Algorithm: `apply_complex_replace` |
| 23 | + |
| 24 | +Input: a `yamlpath::Document`, a `yamlpath::Route`, and a `serde_yaml::Value` (Mapping or Sequence). |
| 25 | + |
| 26 | +1. **Locate the feature** — `document.query_pretty(&route)` to get the byte span with surrounding key context. |
| 27 | + |
| 28 | +2. **Extract content** — `extract_with_leading_whitespace(feature)` to get the full `key: value` text. Split at the first `: ` (colon-space) or `:\n` (colon-newline) into `key_part` (through the colon) and `value_part` (after the colon). |
| 29 | + |
| 30 | +3. **Detect inline comment** — Scan the first line of `value_part` for a trailing `# comment`. Heuristic: after the value content on the first line, find `\s+#`. Save the comment text if found. |
| 31 | + |
| 32 | +4. **Compute indentation** — Derive base indent from the number of leading spaces on the feature's line. Value content indentation = base indent + 2 spaces. |
| 33 | + |
| 34 | +5. **Serialize the new value** — `serde_yaml::to_string(&value)`, strip trailing newline, re-indent each line to the computed value indentation. |
| 35 | + |
| 36 | +6. **Assemble replacement text**: |
| 37 | + - With comment: `key: # comment\n indented_value` |
| 38 | + - Without comment: `key:\n indented_value` |
| 39 | + |
| 40 | +7. **String surgery** — Replace the byte range from the feature's span (adjusted for leading whitespace) in the document source. |
| 41 | + |
| 42 | +8. **Re-parse** — `yamlpath::Document::new(patched_source)` to validate. |
| 43 | + |
| 44 | +### Root-level replace |
| 45 | + |
| 46 | +If the route is empty, skip key extraction. Replace the entire document content with the serialized value. This matches yamlpatch's existing root-replace behavior. |
| 47 | + |
| 48 | +## Patch Routing in `apply_patches` |
| 49 | + |
| 50 | +In `PyDocument::apply_patches`, before building the `Vec<yamlpatch::Patch>`: |
| 51 | + |
| 52 | +``` |
| 53 | +for each patch: |
| 54 | + if patch.operation is Replace(value) AND value is Mapping or Sequence: |
| 55 | + apply_complex_replace(document, route, value) |
| 56 | + re-parse document for subsequent patches |
| 57 | + else: |
| 58 | + collect into yamlpatch batch |
| 59 | +``` |
| 60 | + |
| 61 | +Patches are applied sequentially. If a complex replace is encountered mid-batch, the preceding yamlpatch batch is flushed first, then the complex replace is applied, then the next batch starts from the updated document. |
| 62 | + |
| 63 | +## How Upsert Benefits |
| 64 | + |
| 65 | +`Document.upsert()` delegates to `replace()` when the path exists, and to `_create_at()` (which uses `Op.add`/`Op.merge_into`) when it doesn't. Since `Op.add` and `Op.merge_into` already handle complex values, the only broken path is the `replace` delegation — fixed by this change. No changes to `document.py`. |
| 66 | + |
| 67 | +## Comment Relocation |
| 68 | + |
| 69 | +When replacing a scalar that has an inline comment (e.g. `repos: [] # managed by tool`) with a multi-line block value: |
| 70 | + |
| 71 | +- The comment is preserved on the key line: `repos: # managed by tool` |
| 72 | +- The block value starts on the next line, indented |
| 73 | + |
| 74 | +Detection heuristic: after stripping the YAML value from the first line of `value_part`, look for `\s+#` at the end. |
| 75 | + |
| 76 | +**Limitation:** Comments inside quoted strings that happen to contain `#` could be misdetected. This is unlikely for the scalar values being replaced and is documented as a known limitation. |
| 77 | + |
| 78 | +## Serialization Style |
| 79 | + |
| 80 | +No changes to `Op.add` behavior. Complex values in `Replace` use `serde_yaml::to_string()` which produces block style by default (block mappings, block sequences). This matches the pre-commit config use case where block style is expected. |
| 81 | + |
| 82 | +## Testing |
| 83 | + |
| 84 | +New tests in `tests/test_document.py`: |
| 85 | + |
| 86 | +- `replace()` with a dict value |
| 87 | +- `replace()` with a list value |
| 88 | +- `replace()` with nested dict-in-list-in-dict |
| 89 | +- `upsert()` with a dict value (path exists → goes through replace) |
| 90 | +- `upsert()` with a dict value (path doesn't exist → goes through add, already works) |
| 91 | +- Comment preservation when replacing scalar with complex value |
| 92 | +- Replacing a complex value with another complex value |
| 93 | +- Root-level replace with complex value |
| 94 | +- Indentation correctness at various nesting depths (0, 2, 4 spaces) |
| 95 | + |
| 96 | +## Scope Boundaries |
| 97 | + |
| 98 | +**In scope:** |
| 99 | +- `Replace` with Mapping/Sequence values |
| 100 | +- Inline comment relocation |
| 101 | +- Indentation handling |
| 102 | + |
| 103 | +**Out of scope:** |
| 104 | +- Changing `Add`/`Append` serialization style |
| 105 | +- Multi-document YAML support |
| 106 | +- Tagged values |
| 107 | +- Flow-style output preference (uses serde_yaml default block style) |
0 commit comments