Does the plugin configuration support using JSON or YAML TextMate grammars?

[loradd]

Does the plugin support using TextMate grammars defined as JSON or YAML? It seems that only XML definitions are supported and used to build a grammar definition - what if we want to re-use the JSON/YAML grammar from an existing Visual Studio Code extension?

[martin-azpillaga]

I tried using a TextMate grammar in JSON format, but no success.
There is probably a way to support them
In the meantime, we can use this extension to convert between formats:
https://marketplace.visualstudio.com/items?itemName=Togusa09.tmlanguage
as explained in the end of its documentation

[martin-azpillaga]

I faced this challenge again recently, and I think I know how to support JSON formatted Textmate grammars.

Vscode knows about the grammar thanks to the contribution point in package.json, which is currently generated like:

{"language": "${configuration.language}", "scopeName": "${configuration.textmateGrammarScope}", "path": "./grammar.tmLanguage"}

Importantly, the path is hardcoded to grammar.tmLanguage.

From my experimentation, vscode uses file extensions to determine the format of the Textmate grammar file before parsing and using it.
In particular, .tmLanguage files are expected to be in XML format.
To use grammars in JSON format, the grammar file needs to end with .json.

I have tested referencing the same grammar in JSON format from a file ending in .json and another file with the same content but ending in .tmLanguage and only the .json one works.

Knowing this, I suggest preserving the filename of the file set as textmate grammar in the configuration. Both while copying the file:

kolasu-languageserver-library/plugin/src/main/kotlin/com/strumenta/kolasu/languageserver/plugin/LanguageServerPlugin.kt

Lines 302 to 308 in 0597e0c

	if (Files.exists(configuration.textmateGrammarPath)) {
	Files.copy(
	configuration.textmateGrammarPath,
	Paths.get(configuration.outputPath.toString(), "grammar.tmLanguage"),
	StandardCopyOption.REPLACE_EXISTING
	)
	}

and also while referencing it:

kolasu-languageserver-library/plugin/src/main/kotlin/com/strumenta/kolasu/languageserver/plugin/LanguageServerPlugin.kt

Lines 186 to 196 in 0597e0c

	var grammars = ""
	if (Files.exists(configuration.textmateGrammarPath)) {
	grammars =
	"""
	,
	"grammars":
	[
	{"language": "${configuration.language}", "scopeName": "${configuration.textmateGrammarScope}", "path": "./grammar.tmLanguage"}
	]
	""".trimIndent()
	}

[martin-azpillaga]

As for YAML, Vscode does not support YAML grammars natively, they must be converted to JSON or XML.
The documentation suggests using js-yaml for converting the grammars with a CLI:

npx js-yaml abc.tmLanguage.yaml > abc.tmLanguage.json

[martin-azpillaga]

Refer to the Kuki language server implementation for a concrete example of how to support JSON formatted grammars:
JSON grammar and custom package.json