Fix referenceText to not emit invalid MQL for unsafe field names (fixes #709)

[hanhan761]

Summary

Stops toFieldCompletionItems() from emitting invalid MQL aggregation references for unsafe field names (containing dashes, spaces, quotes, $-prefixed names, etc.).

Previously, referenceText was always $ + field path, which produces broken MQL for fields like order-items, my field, or say"hi".

Changes in src/utils/json/data-api/autocomplete/toFieldCompletionItems.ts

• Added AGGREGATION_SAFE_SEGMENT_PATTERN regex -- stricter than the JS identifier pattern, disallows leading/embedded $ which would create ambiguous variable references
• Added escapeFieldName() helper -- escapes backslashes and double-quotes for use in $getField expressions
• Added buildAggregationReference() helper -- emits $field for fully safe paths, { $getField: "name" } for unsafe top-level fields, and { $getField: { field: "<leaf>", input: <parent-ref> } } for nested paths with unsafe segments
• Removed the TODO comment from the referenceText field doc and replaced it with accurate documentation

Examples

Field name	Before	After
age	$age	$age (unchanged)
address.city	$address.city	$address.city (unchanged)
order-items	$order-items (invalid)	{ $getField: "order-items" }
a.order-items	$a.order-items (invalid)	{ $getField: { field: "order-items", input: "$a" } }
my field	$my field (invalid)	{ $getField: "my field" }
$price	$$price (misread as variable)	{ $getField: "$price" }
say"hi"	$say"hi" (invalid)	{ $getField: "say\"hi\"" }

Documentation files updated

• src/utils/json/data-api/autocomplete/future-work.md -- item 2 marked as resolved, Option B ($getField) documented
• docs/ai-and-plans/PRs/717-referenceText-unsafe-field-names.md -- new PR tracking doc added

Testing

• All 47 autocomplete tests pass (4 test suites)
• New tests for unsafe field name referenceText generation including nested paths and $-prefixed names
• TypeScript compilation passes with no errors

Issue

Fixes #709

[Copilot]

Pull request overview

Note

Copilot was unable to run its full agentic suite in this review.

Updates autocomplete field completion output to generate valid aggregation field references for “unsafe” field names, and enhances the DocumentDB Indexes tree node to show a live index count.

Changes:

• Generate referenceText using $field.path only when each dot-separated segment is identifier-safe; otherwise emit a $getField expression.
• Add unit tests covering $getField usage and escaping for embedded quotes.
• Cache and display index counts in the Indexes tree item, refreshing the node when counts change.

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 3 comments.

File	Description
src/utils/json/data-api/autocomplete/toFieldCompletionItems.ts	Adds “safe reference” detection and emits $getField form for unsafe field paths.
src/utils/json/data-api/autocomplete/toFieldCompletionItems.test.ts	Adds tests for $getField reference generation and escaping.
src/tree/documentdb/IndexesItem.ts	Caches index count and updates tree item description with localized count.

[tnaum-ms]

Maintainer update — rebased + correctness fixes

Thanks for the contribution! I rebased this branch and extended the fix so it produces valid MQL in all the cases the issue implied. Summary of what changed since the last push:

Scope cleanup (rebase)

• The branch was stacked on the #659 index-count work, which is already merged to main — that was the source of the merge conflict. Rebased onto main, dropping the stale #659 commits. The PR now touches only the two autocomplete files (+ a PR note).

Correctness fixes on top of the original $getField fallback

• Nested paths now use { $getField: { field, input } }. The single-string form { $getField: "a.order-items" } was still wrong — it looks up a top-level field literally named a.order-items instead of the nested field.
• $-containing names (e.g. $price) now route through $getField. The old identifier check allowed $, so it emitted $$price, which MQL reads as the variable $$price, not the field.

Where it applies

referenceText is the aggregation expression form ($field), so this affects pipelines plus $expr / pipeline-style updates — not plain field-name key positions (those use insertText, unchanged). Note: no production code reads referenceText yet, so there's no user-facing behavior change today; this makes the data correct for the upcoming aggregation completion provider.

Before vs. after (what would be inserted)

field picked        BEFORE (always "$"+path)        AFTER
──────────────────  ──────────────────────────────  ─────────────────────────────────────────────────
age                 $age                      ✅     $age                                          ✅ (unchanged)
address.city        $address.city             ✅     $address.city                                 ✅ (unchanged)

order-items         $order-items              ❌     { $getField: "order-items" }                  ✅
                    └ parsed as  $order MINUS items

a.order-items       $a.order-items            ❌     { $getField: { field: "order-items",          ✅
                    └ "$" form breaks on dash                      input: "$a" } }

$price              $$price                   ❌     { $getField: "$price" }                       ✅
                    └ "$$" = a VARIABLE, not a field

The nested case (the core fix)

{ $getField: "a.order-items" }                         { $getField: { field: "order-items", input: "$a" } }
┌───────────── ROOT ──────────────┐                    ┌───────────── ROOT ──────────────┐
│ key literally named              │                    │ input: $a  ──►  { order-items: 5}│
│ "a.order-items"  →  not found    │  ✗                 │ field: "order-items" ──► 5       │  ✓
└──────────────────────────────────┘                    └──────────────────────────────────┘

Tests added

Nested unsafe leaf, safe-prefix collapse (a.b.c-d → { $getField: { field: "c-d", input: "$a.b" } }), unsafe-then-safe chaining, nested embedded quotes, $-in-name, literal-dot ambiguity (a.b → $a.b, documented limitation), and a defensive empty/degenerate-segment case.

future-work.md item 2 is marked resolved. All checks pass locally: prettier, l10n, lint, full Jest suite (2529), and tsc build.

[Copilot]

Pull request overview

Copilot reviewed 4 out of 4 changed files in this pull request and generated 1 comment.