Suggestion: redesign Insert_edit_into_file tool

[ElliottLester]

I love this plugin and the workflow it provides, I have been using local models via llama.cpp, I have noticed that smaller models(gemma-4-31B) seem to have trouble using the Insert_edit_into_file.

I wanted to check if this was an issue with llama.cpp or the model, so I used opencode as a comparison and was surprised to find that their apply_patch tool had much higher success rate with the same model on the same server.

While Insert_edit_into_file is simple client side, it is fragile and the tool call is complicated and lacks support for multiple simultaneous operations move/delete operations.

Standard Git patches are better but fail often because LLMs hallucinate line numbers in hunk headers (@@ -l,s +l,s @@).

So Opencode uses a custom patch format solves this by replacing arithmetic (line counts) with pattern recognition (markers and context), which seems to be less error prone.

Example Tool Call Comparison
Scenario: Change console.log("Hi") to console.log("Hello World") in src/main.ts.
Current Approach (insert_edit_into_file):

{
  "filepath": "src/main.ts",
  "explanation": "Update greeting message",
  "mode": "append",
  "dryRun": false,
  "edits": [
    {
      "oldText": "  console.log(\"Hi\")",
      "newText": "  console.log(\"Hello World\")",
      "replaceAll": false
    }
  ]
}

Proposed Approach (apply_patch):

{
  "patchText": "*** Begin Patch\n*** Update File: src/main.ts\n@@ context\n-  console.log(\"Hi\")\n+  console.log(\"Hello World\")\n*** End Patch"
}

the implementation for this tool can be found here apply_patch.ts

[ElliottLester]

I should clarify, is this a change you would be interested in accepting?
I haven't written nvim plugins, I know you had local models as "out of scope" for now so this is more of a request for comment.

[olimorris]

Hey @ElliottLester.

Firstly thanks for such a great, well-thought out and detailed post.

One of the hardest things that I've encountered since starting CodeCompanion was that of reliably editing files from within the chat buffer. It's been quite a journey. #2219 was the source of the current tool and introduced a ton of improvements compared to where we were previously.

Probably the biggest challenge of all with the tool is that it has to be LLM agnostic. For OpenAI, Anthropic, and Google models, this hasn't been a problem, but in local, lower-parameter models, it's been incredibly challenging, as you correctly pointed out.

Now Opencode's implementation is of particular interest to me because they too are faced with the same constraints that CodeCompanion is. On that basis, I think it would be very interesting to take a look at a revised tool. Probably the only criticism I have of the insert_edit_into_file tool in its current state is that we have to coach the LLMs through an excessive prompt in order to get them to behave adequately. If there's a way that we could improve on that, then I think that would be very compelling.

Be more than happy to collaborate with you on this as a PR. It's not something that I expect we would sort quickly, but many hands make light work.

[olimorris]

I recall it may have been so an LLM can understand if its edits are going to work as expected. If they fail, it can refactor and try again. #2219 probably has the answer somewhere...but it's a monster PR.

[bassamsdata]

Hello! Just to add a bit of context here:

• dry_run was originally added mainly for testing. At the time, it was pretty hard to reliably test across different models, especially since most open-source ones weren’t great at tool calling yet.
• explanation was meant to give some lightweight context about why an edit happened, useful for both users and the agent, especially toward the end of a long context window.

That said, I agree these probably aren’t as necessary anymore. With better models and more capable agent workflows now, they might just add overhead (especially explanation, since it consumes tokens).

And yes the tool prompt for the apply patch tool is much shorter. :)

Yeah, if I remember correctly, there used to be two edit tools in opencode (one being edit.ts, though not sure if that’s still around).

There’s definitely room to simplify things, and that would likely reduce the prompt size quite a bit. The main challenge back then was handling edge cases, which is why things ended up a bit more complex.

Happy to help if needed!

[ElliottLester]

I have a proof of concept at
https://github.com/ElliottLester/codecompanion.nvim/tree/apply_patch_tool

• We are currently editing files directly, is there a place I can see the buffer editing and how we call the diff tool, other than inside the insert_edit_at_line tool which is very complicated?

• Is there a way to verify the buffer edits behavior in the test environment?

• I think this tool should replace the tools insert_edit_at_line, create_file & delete_file.

[olimorris]

Make a draft pull request and we can collaborate.

We are currently editing files directly, is there a place I can see the buffer editing and how we call the diff tool, other than inside the insert_edit_at_line tool which is very complicated?

All of that is contained within /tools/builtin/insert_edit_into_file/* and the diff logic sits in diff.lua and is called in init.lua. My preference would be we conform to how the existing tool creates the diff, so we're minimizing the amount of line changes in the PR.

Is there a way to verify the buffer edits behavior in the test environment?

This is what your test does, no?

I think this tool should replace the tools insert_edit_at_line, create_file & delete_file.

I like the idea of having this a separate tool for now. That gives me flexibility in terms of keeping insert_edit_into_file around for a bit longer and allowing people to test it out, without having to issue a breaking change and bump to v20.0.0. I think create_file and delete_file are too distinct operations and so should remain that way. From an approvals perspective, giving the greenlight to an LLM to apply patches across your project and also delete files would feel dangerous.

[ElliottLester]

opened as #3045