Syntax highlighting design

[cgay]

Hey, I've been reading through the TextMate docs and playing around with dylan.tmLanguage.json and I wonder if it's possible to impose a little bit of structure on the grammar....

What if we separate the grammar into (at least) three top level rules:

1. A rule to match any valid header (Module:, Synopsis:, etc) down to the next blank line.
2. A rule to match from ^define to the following ^end.
3. A rule to match define constant|variable.

Each rule would contain subrules ("patterns") as appropriate. We might find that we need separate top-level rules for define macro and define class; I don't know yet.

I will play around with this idea and update this bug as I go.

[cgay]

The regex for a Dylan NAME is +(([a-zA-Z]|[0-9][a-zA-Z][a-zA-Z]|[!&*<>|^$%@_][a-zA-Z])[-+~?/=!&*<>|^$%@_a-zA-Z0-9]*)). I was hoping it would be possible to define this in the registry and then re-use it throughout the configuration, but there doesn't appear to be a way to include something from the registry within a regex string.

[cgay]

My preference for syntax highlighting is to have minimal colorization. Most themes (and certainly all the built-in VS Code themes I've tried) have a color for every token and for me it ceases to convey much useful information.

What I would like is

• Highlight bindings where they are introduced
  • Different color for global and local.
  • Class name in define class
  • Function name in define function
  • foo in let foo = ...
  • Parameter names
  • etc.
• Dylan literals (strings, symbols, numbers) should be called out somehow, probably all the same color.
• Comments
  • Somewhat brighter than surrounding code because presumably they were written for an important reason and are not just reiterating what the code does.

Here are some screenshots of TypeScript's TextMate scope names. I've customized my user settings as follows to make the variables stand out more (i.e., magenta):

    "editor.tokenColorCustomizations": {
        "variables": "#ff00f2"
    }

If we can provide the same scope names for the relevant Dylan tokens I should be able to create the theme I want. But...I notice that the semantic token type (provided by the TypeScript LSP I think) is "variable", so that's what's probably picking up my user setting. AND, they are providing that same "variable" type both for the binding and for the uses of the variable, which is definitely not what I want.

Anyway, I'm dropping these clues here for reference...