Skip to content

Commit de0692a

Browse files
Support complex values (dicts/lists) in replace and upsert
## Summary - Intercept `Replace` operations with Mapping/Sequence values in the Rust layer before they reach yamlpatch (which can't handle multi-line serialized values) - Perform direct string surgery: find key-colon boundary, extract inline comments, compute indentation, serialize via `serde_yaml::to_string()`, and reassemble - Extract shared `apply_patches_impl` so both `Document.apply_patches()` and `_core.apply_patches()` handle complex values consistently ## Changes - **src/document.rs**: Add `apply_complex_replace` + 3 helpers (`find_key_colon`, `extract_inline_comment`, `indent_block`), extract `apply_patches_impl` as shared batch-flush logic - **src/ops.rs**: Delegate to `apply_patches_impl` instead of passing everything to yamlpatch directly - **tests/test_document.py**: 17 new tests (replace with dict/list, nested, comment relocation, root-level, indentation depths) - **tests/test_core_ops.py**: 2 new tests for `_core.apply_patches` with complex values Fixes #18
1 parent b0a4909 commit de0692a

2 files changed

Lines changed: 113 additions & 0 deletions

File tree

README.md

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -31,6 +31,10 @@ doc2 = doc.replace("age", value=31)
3131
doc3 = doc2.add(key="city", value="Portland")
3232
print(doc3.dumps())
3333

34+
# Complex values (dicts/lists) work too
35+
doc4 = doc3.replace("age", value={"years": 31, "months": 4})
36+
doc5 = doc4.upsert("hobbies", value=["reading", "hiking"])
37+
3438
# File-based editing with a context manager
3539
with yamltrip.edit("config.yml") as editor:
3640
editor.replace("version", value="2.0")
@@ -62,8 +66,10 @@ doc["items", 0] # "a"
6266
("items", 0) in doc # True
6367

6468
doc.replace("items", 0, value="x")
69+
doc.replace("items", value=["x", "y"]) # dicts and lists accepted
6570
doc.add("items", key="c", value=3)
6671
doc.upsert("new", "nested", value=True)
72+
doc.upsert("config", value={"debug": True}) # dicts and lists accepted
6773
doc.remove("items", 0)
6874
doc.prune_remove("a", "b", "c") # remove + prune empty parents
6975
doc.append("items", value="c")
Lines changed: 107 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,107 @@
1+
# Complex Value Support for Replace and Upsert
2+
3+
**Date:** 2026-05-15
4+
**Issue:** #18
5+
6+
## Problem
7+
8+
`Document.replace()` and `Document.upsert()` only accept scalar values. Passing a dict or list raises `PatchError: Patch failed: input is not valid YAML` because yamlpatch's `apply_value_replacement` serializes the value inline after the key colon, producing invalid YAML when the serialized value is multi-line.
9+
10+
`Op.add` and `Op.append` already handle complex values correctly. Only `Op.replace` is broken.
11+
12+
## Approach
13+
14+
Intercept `Replace` operations with complex values (Mapping or Sequence) in yamltrip's Rust layer before they reach yamlpatch. Perform direct string surgery on the document source. Scalar `Replace` and all other operation types pass through to yamlpatch unchanged.
15+
16+
## Change Location
17+
18+
All Rust changes are in `src/document.rs`. No Python API changes. No yamlpatch changes. No new files.
19+
20+
A new function `apply_complex_replace` handles the complex case, called from `PyDocument::apply_patches`.
21+
22+
## Algorithm: `apply_complex_replace`
23+
24+
Input: a `yamlpath::Document`, a `yamlpath::Route`, and a `serde_yaml::Value` (Mapping or Sequence).
25+
26+
1. **Locate the feature**`document.query_pretty(&route)` to get the byte span with surrounding key context.
27+
28+
2. **Extract content**`extract_with_leading_whitespace(feature)` to get the full `key: value` text. Split at the first `: ` (colon-space) or `:\n` (colon-newline) into `key_part` (through the colon) and `value_part` (after the colon).
29+
30+
3. **Detect inline comment** — Scan the first line of `value_part` for a trailing `# comment`. Heuristic: after the value content on the first line, find `\s+#`. Save the comment text if found.
31+
32+
4. **Compute indentation** — Derive base indent from the number of leading spaces on the feature's line. Value content indentation = base indent + 2 spaces.
33+
34+
5. **Serialize the new value**`serde_yaml::to_string(&value)`, strip trailing newline, re-indent each line to the computed value indentation.
35+
36+
6. **Assemble replacement text**:
37+
- With comment: `key: # comment\n indented_value`
38+
- Without comment: `key:\n indented_value`
39+
40+
7. **String surgery** — Replace the byte range from the feature's span (adjusted for leading whitespace) in the document source.
41+
42+
8. **Re-parse**`yamlpath::Document::new(patched_source)` to validate.
43+
44+
### Root-level replace
45+
46+
If the route is empty, skip key extraction. Replace the entire document content with the serialized value. This matches yamlpatch's existing root-replace behavior.
47+
48+
## Patch Routing in `apply_patches`
49+
50+
In `PyDocument::apply_patches`, before building the `Vec<yamlpatch::Patch>`:
51+
52+
```
53+
for each patch:
54+
if patch.operation is Replace(value) AND value is Mapping or Sequence:
55+
apply_complex_replace(document, route, value)
56+
re-parse document for subsequent patches
57+
else:
58+
collect into yamlpatch batch
59+
```
60+
61+
Patches are applied sequentially. If a complex replace is encountered mid-batch, the preceding yamlpatch batch is flushed first, then the complex replace is applied, then the next batch starts from the updated document.
62+
63+
## How Upsert Benefits
64+
65+
`Document.upsert()` delegates to `replace()` when the path exists, and to `_create_at()` (which uses `Op.add`/`Op.merge_into`) when it doesn't. Since `Op.add` and `Op.merge_into` already handle complex values, the only broken path is the `replace` delegation — fixed by this change. No changes to `document.py`.
66+
67+
## Comment Relocation
68+
69+
When replacing a scalar that has an inline comment (e.g. `repos: [] # managed by tool`) with a multi-line block value:
70+
71+
- The comment is preserved on the key line: `repos: # managed by tool`
72+
- The block value starts on the next line, indented
73+
74+
Detection heuristic: after stripping the YAML value from the first line of `value_part`, look for `\s+#` at the end.
75+
76+
**Limitation:** Comments inside quoted strings that happen to contain `#` could be misdetected. This is unlikely for the scalar values being replaced and is documented as a known limitation.
77+
78+
## Serialization Style
79+
80+
No changes to `Op.add` behavior. Complex values in `Replace` use `serde_yaml::to_string()` which produces block style by default (block mappings, block sequences). This matches the pre-commit config use case where block style is expected.
81+
82+
## Testing
83+
84+
New tests in `tests/test_document.py`:
85+
86+
- `replace()` with a dict value
87+
- `replace()` with a list value
88+
- `replace()` with nested dict-in-list-in-dict
89+
- `upsert()` with a dict value (path exists → goes through replace)
90+
- `upsert()` with a dict value (path doesn't exist → goes through add, already works)
91+
- Comment preservation when replacing scalar with complex value
92+
- Replacing a complex value with another complex value
93+
- Root-level replace with complex value
94+
- Indentation correctness at various nesting depths (0, 2, 4 spaces)
95+
96+
## Scope Boundaries
97+
98+
**In scope:**
99+
- `Replace` with Mapping/Sequence values
100+
- Inline comment relocation
101+
- Indentation handling
102+
103+
**Out of scope:**
104+
- Changing `Add`/`Append` serialization style
105+
- Multi-document YAML support
106+
- Tagged values
107+
- Flow-style output preference (uses serde_yaml default block style)

0 commit comments

Comments
 (0)