Semantic Tokens Support

[UndeadCJ]

Description

While developing an Elixir plugin for IntelliJ IDEA using elixir-ls, I have not found any explicit mention of Semantic Tokens support in the current implementation, and I'm seeking clarification on its status and potential implementation.

Specific Questions

1. Does elixir-ls currently support semantic tokens?
2. If supported, what is the current implementation status?
3. Are there plans to improve semantic token support?

Technical Details

• Elixir Version: 1.18
• elixir-ls Version: 0.27.1

Additional Context

• Reference LSP Specification: LSP Semantic Tokens

(ps, sorry for my english xd)

[lukaszsamson]

It's not currently supported. I did a PoC implementation once that emitted some semantic tokens but I never got to finishing it. I never felt it introduced much value. There are a few problems with that provider

• It requires access to token level info and in elixir tokenizer is a private API that should not be used in other projects. ElixirLS is guilty of using it anyway
• With the current architecture of elixir parser and tokenizer it's not trivial to combine AST level info with token level info
• Code highlighting themes have to support semantic tokens. Last time I checked none of the widely used in VSCode had any elixir specific semantic token support
• There is no consensus on what tokens should have what meaning. e.g for this code

defmodule Foo do
  @attr 123
  @doc "bar fun"
  def bar(baz) do
    # some comment
    Some.fun(baz + @attr)
  end
end

which tokens should be marked with which semantic tokens with which modifiers from https://microsoft.github.io/language-server-protocol/specifications/lsp/3.18/specification/#textDocument_semanticTokens