Skip to content

perf: improve Java LSP tool result handoff#1031

Open
wenytang-ms wants to merge 10 commits into
mainfrom
experiment/lsp-tool-contract-tuning
Open

perf: improve Java LSP tool result handoff#1031
wenytang-ms wants to merge 10 commits into
mainfrom
experiment/lsp-tool-contract-tuning

Conversation

@wenytang-ms

@wenytang-ms wenytang-ms commented Jun 9, 2026

Copy link
Copy Markdown
Contributor

Summary

This draft updates the Java LSP Copilot tool contract so successful symbol and file-outline results are easier to consume by downstream tools, especially VS Code Copilot's built-in read_file.

Recent telemetry analysis showed that raw findSymbol -> findSymbol is not a reliable problem metric by itself because most adjacent pairs are likely same-round/batch tool calls. The PR therefore avoids prescriptive next-step hints and focuses on factual, read-file-compatible result fields.

Changes

  • lsp_java_findSymbol now returns structured location fields plus a direct read_file adapter:
    • file
    • startLine
    • endLine
    • range
    • readFileInput: { filePath, offset, limit }
  • Removed the ambiguous location: path:line field from findSymbol; file is the canonical handoff field for lsp_java_getFileStructure.
  • lsp_java_getFileStructure now returns a top-level file and per-symbol read ranges:
    • startLine
    • endLine
    • range
    • readFileRange: { offset, limit }
  • Added limit to lsp_java_getFileStructure to control payload size:
    • default: 20
    • max: 60
    • returns truncated: true when capped
  • Improved multi-root relative path handling when asRelativePath includes the workspace folder name.
  • Updated LM tool metadata, skill docs, and Java chat instructions to describe the structured handoff.

Expected experiment signal

This PR is not expected to eliminate all findSymbol -> findSymbol transitions. Same-round/batch symbol lookups can still be valid.

Expected improvements:

  • Higher findSymbol -> readFile when source is needed.
  • Higher findSymbol -> readFile/getFileStructure within the next few tool calls.
  • Lower delayed findSymbol -> findSymbol repeat rate, especially pairs with delta > 1s or delta > 5s.
  • Lower generic search fallback after successful small findSymbol results.
  • Smaller getFileStructure payloads while preserving getFileStructure -> readFile consumption.

Metrics that should not be treated as primary success criteria:

  • Raw immediate findSymbol -> findSymbol, because it is inflated by batch/parallel tool calls.
  • Forcing findSymbol to always be followed by read_file; lsp_java_getFileStructure, additional symbol lookup, or generic search can still be valid depending on the task.

Intentional non-goals

  • No nextStep field in tool results; next-tool selection remains with the model.
  • No outlineInput; file is the single path handoff field.
  • No speculative path:line parsing in lsp_java_getFileStructure; findSymbol no longer returns a path:line field that would encourage this misuse.

Validation

  • npm run compile
  • npm run tslint
  • npm test currently fails in existing Library Controller tests because toReferencedLibraryPath / toReferencedLibraryExcludePath are not exported from extension_bundle_1; unrelated to this LSP tool change.

Return structured file and line fields from findSymbol so the model can consume precise ranges without repeating symbol search. Accept line-suffixed Java locations in file structure resolution and document outlineInput handoff.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@wenytang-ms wenytang-ms changed the title Improve Java LSP tool result handoff perf: improve Java LSP tool result handoff Jun 9, 2026
@wenytang-ms wenytang-ms requested a review from Copilot June 9, 2026 07:59

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR improves the Java LSP Copilot tools’ result “handoff” so downstream steps (e.g., read_file / lsp_java_getFileStructure) can consume lsp_java_findSymbol output more directly, and so file resolution is more tolerant of Java-style path:line inputs and multi-root workspaces.

Changes:

  • Enhanced lsp_java_findSymbol payload with file, startLine, endLine, and outlineInput, and added a nextStep hint to reduce unnecessary repeat tool calls.
  • Made file URI resolution tolerate Java location suffixes (e.g., src/Foo.java:42) and improved multi-root relative-path handling when the workspace folder name is included.
  • Updated skill docs, instrument instructions, and tool metadata to prefer outlineInput/file handoff patterns.

Reviewed changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated 2 comments.

File Description
src/copilot/tools/javaContextTools.ts Adds Java path:line tolerance, improves multi-root relative resolution, and enriches findSymbol results with additional fields and guidance.
resources/skills/java-lsp-tools/SKILL.md Documents the updated findSymbol output fields and the preferred handoff flow using outlineInput/file.
resources/instruments/javaLspContext.instructions.md Updates Java chat instructions to prefer read_file with file/range and getFileStructure with outlineInput.
package.json Updates tool metadata/schema descriptions to reflect new fields and tolerated path:line inputs.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread src/copilot/tools/javaContextTools.ts
Comment thread src/copilot/tools/javaContextTools.ts Outdated
Use normalized relative paths consistently when resolving file structure inputs and classify truncated findSymbol responses before small exact-result guidance.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@wenytang-ms wenytang-ms marked this pull request as ready for review June 10, 2026 04:47
wenytang-ms and others added 5 commits June 10, 2026 13:59
Keep the PR focused on the evidence-backed findSymbol handoff contract. Do not claim getFileStructure accepts path:line inputs without telemetry proving that case.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Keep findSymbol results as structured location data and leave next-tool choice to the model. The payload still exposes file, line, range, and outlineInput for downstream consumption without embedding result-level routing guidance.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Remove the redundant outlineInput field and keep findSymbol output focused on factual location data: file, startLine, endLine, and range. getFileStructure can consume the returned file when broader file outline context is needed.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 4 out of 4 changed files in this pull request and generated 2 comments.

Comments suppressed due to low confidence (1)

src/copilot/tools/javaContextTools.ts:344

  • The PR description says lsp_java_findSymbol will add an outlineInput field and a top-level nextStep hint, but the tool output here only includes file/startLine/endLine/readFileInput and returns { results, total }. If those fields are required for handoff, they should be included in the actual payload.
            const results = symbols.slice(0, limit).map(s => {
                const file = vscode.workspace.asRelativePath(s.location.uri);
                const { startLine, endLine } = toInclusiveLineRange(s.location.range);
                return {
                    name: s.name,
                    kind: vscode.SymbolKind[s.kind],
                    container: s.containerName || undefined,
                    file,
                    startLine,
                    endLine,
                    readFileInput: toReadFileInput(file, startLine, endLine),
                    location: `${file}:${startLine}`,
                    range: `L${startLine}-${endLine}`,
                };
            });
            resultCount = results.length;
            const findSymbolPayload = { results, total: symbols.length };
            responseCharCount = getResponseCharCount(findSymbolPayload);
            return toResult(findSymbolPayload);

Comment on lines 121 to +126
let uri: vscode.Uri;
const normalizedInput = input.trim();

if (input.includes("://")) {
if (normalizedInput.includes("://")) {
// URI string (e.g. "file:///home/user/project/src/Main.java")
uri = vscode.Uri.parse(input);
uri = vscode.Uri.parse(normalizedInput);

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Addressed in 467198e by removing the ambiguous location: file:line field from lsp_java_findSymbol; the intended lsp_java_getFileStructure handoff now uses the structured file field instead.

Comment on lines 37 to 40
**findSymbol → getFileStructure → read_file (specific lines only)**

If `findSymbol` returns relevant symbols, move forward to `getFileStructure` or `read_file`; do not call `findSymbol` again with the same or similar identifier.
If `findSymbol` returns relevant symbols and source is needed, call `read_file` with the returned `readFileInput`, or call `getFileStructure` with the returned `file` when broader file context is needed.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Addressed in 467198e by using the full lsp_java_getFileStructure tool name in the workflow example and guidance.

wenytang-ms and others added 3 commits June 10, 2026 16:47
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants