-
Notifications
You must be signed in to change notification settings - Fork 450
Add support for generating LuaCATS definitions for the Lua API #3755
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
kalimag
wants to merge
35
commits into
TASEmulators:master
Choose a base branch
from
kalimag:pr/generate-lua-api-definitions
base: master
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Changes from all commits
Commits
Show all changes
35 commits
Select commit
Hold shift + click to select a range
79a891a
wip
vadosnaprimer aedd2a3
Add support for generating LuaCATS definitions for the Lua API
kalimag e605a18
Add missing numeric types
kalimag fabb10c
Throw error when definition file is executed as a script
kalimag 554d0cf
Add support for bytes-as-string parameters
kalimag 4301a88
Apply suggestions from code review
vadosnaprimer fa6748b
add examples and hawk info
vadosnaprimer 6d9f4ee
line endings
vadosnaprimer 535c79f
move editor/ide integration to its own menu
vadosnaprimer fd7e68b
does "Braces XOR single-line" mean this?
vadosnaprimer d729148
braces
vadosnaprimer 9a33894
name the definitions file according to Lua Language Server spec
vadosnaprimer 687b481
proper Program Files folder search
vadosnaprimer f11aa9c
Refactor example formatting
kalimag ae30e6b
Preamble, line-ending stuff
kalimag 9951d79
Make `LuaCanvas` local
kalimag 3c4d8c2
Refactor `GetLuaType(ParameterInfo)`
kalimag cb486da
Enumerate valid drawingSurface values
kalimag d8f7719
It's free braces
kalimag 03bbaab
Make class static
kalimag 928d55a
Make class static Part II
kalimag f772809
split into individual files?
vadosnaprimer 6cb96a2
Remove return statement from definition files
kalimag 6667a7b
Include default parameter values
kalimag db4ce34
Fix code style
kalimag eb210ae
Rename color aliases
kalimag 3b79e1f
Fix name collision with `userdata`
kalimag 12da1a4
Mark zero-indexed return values
kalimag 29b153e
Handle zero-indexed parameters
kalimag 3350d27
Support Nullable Reference Type annotations
kalimag 1db4234
Revert "wip" - unrelated submodule change
kalimag 06bdc0a
Switch to NRT extensions
kalimag f1cf53f
Adjust annotation whitespace and comment
kalimag a48d075
Sort color union
kalimag 4c08946
Annotate more specific `createcanvas` return type
kalimag File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
294 changes: 294 additions & 0 deletions
294
src/BizHawk.Client.Common/lua/Documentation/LuaCatsGenerator.cs
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,294 @@ | ||
| #nullable enable | ||
|
|
||
| #pragma warning disable MA0136 // Raw String contains an implicit end of line character, line endings will be normalized | ||
|
|
||
| using System.Collections.Generic; | ||
| using System.IO; | ||
| using System.Linq; | ||
| using System.Reflection; | ||
| using System.Text; | ||
| using System.Text.RegularExpressions; | ||
| using BizHawk.Common; | ||
| using BizHawk.Common.ReflectionExtensions; | ||
| using NLua; | ||
|
|
||
| namespace BizHawk.Client.Common; | ||
|
|
||
| /// <summary> | ||
| /// Generates API definitions in the LuaCATS format. | ||
| /// </summary> | ||
| /// <remarks> | ||
| /// See https://luals.github.io/wiki/annotations | ||
| /// </remarks> | ||
| internal static class LuaCatsGenerator | ||
| { | ||
| private static readonly Dictionary<Type, string> TypeConversions = new() | ||
| { | ||
| [typeof(object)] = "any", | ||
| [typeof(byte)] = "integer", | ||
| [typeof(sbyte)] = "integer", | ||
| [typeof(int)] = "integer", | ||
| [typeof(uint)] = "integer", | ||
| [typeof(short)] = "integer", | ||
| [typeof(ushort)] = "integer", | ||
| [typeof(long)] = "integer", | ||
| [typeof(ulong)] = "integer", | ||
| [typeof(float)] = "number", | ||
| [typeof(double)] = "number", | ||
| [typeof(decimal)] = "number", | ||
| [typeof(string)] = "string", | ||
| [typeof(bool)] = "boolean", | ||
| [typeof(byte[])] = "string", | ||
| [typeof(Memory<byte>)] = "string", | ||
| [typeof(ReadOnlyMemory<byte>)] = "string", | ||
| [typeof(LuaFunction)] = "function", | ||
| [typeof(LuaTable)] = "table", | ||
| [typeof(System.Drawing.Color)] = "dotnetcolor", | ||
| }; | ||
|
|
||
| private const string Classes = """ | ||
| ---@class dotnetcolor : userdata | ||
|
|
||
| ---A color in one of the following formats: | ||
| --- - Number in the format `0xAARRGGBB` | ||
| --- - String in the format `"#RRGGBB"` or `"#AARRGGBB"` | ||
| --- - A CSS3/X11 color name e.g. `"blue"`, `"palegoldenrod"` | ||
| --- - Color created with `forms.createcolor` | ||
| ---@alias color dotnetcolor | integer | string | ||
|
|
||
| ---@alias surface | ||
| ---| "emucore" # Draw on the emulated screen. Resolution depends on emulated system and game. Drawing is scaled with the rest of the display. | ||
| ---| "client" # Draw on the BizHawk window. Resolution depends on the window size. Drawing is not scaled. | ||
| """; | ||
|
|
||
| private const string Preamble = """ | ||
| -- https://tasvideos.org/Bizhawk | ||
|
|
||
| error("This is a definition file for Lua Language Server and not a usable script") | ||
|
|
||
| ---@meta _ | ||
| """; | ||
|
|
||
| private static string? GetHardcodedType(ParameterInfo parameter) | ||
| { | ||
| // Technically any string parameter can be passed a number in BizHawk's Lua API, but let's just focus on the ones where it's commonly used | ||
| // like `gui.text` and `forms.settext` instead of polluting the entire API surface | ||
| if (parameter.Name is "message" or "caption" && parameter.ParameterType == typeof(string)) | ||
| { | ||
| return "string|number"; | ||
| } | ||
|
|
||
| if (parameter.Member.DeclaringType.Name == "GuiLuaLibrary" && parameter.Name == "surfaceName" && parameter.ParameterType == typeof(string)) | ||
| { | ||
| return "surface"; | ||
|
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. We should introduce new attributes if this grows to any more enums. |
||
| } | ||
|
|
||
| return null; | ||
| } | ||
|
|
||
| public static void Generate(LuaDocumentation docs, string path) | ||
| { | ||
| var sb0 = new StringBuilder(); | ||
|
|
||
| sb0.AppendLine($"-- Lua functions available in EmuHawk {VersionInfo.MainVersion}"); | ||
| sb0.AppendLine(Preamble); | ||
| sb0.AppendLine(); | ||
| sb0.AppendLine(Classes); | ||
| sb0.AppendLine(); | ||
|
|
||
| File.WriteAllText(Path.Combine(path, "classes.d.lua"), sb0.ToString().ReplaceLineEndings()); | ||
|
|
||
| foreach (var libraryGroup in docs.GroupBy(func => func.Library).OrderBy(group => group.Key)) | ||
|
YoshiRulz marked this conversation as resolved.
|
||
| { | ||
| string library = libraryGroup.Key; | ||
| string libraryDescription = libraryGroup.First().LibraryDescription; | ||
| var libraryType = libraryGroup.First().Method.DeclaringType; | ||
| var filePath = Path.Combine(path, library + ".d.lua"); | ||
| var sb = new StringBuilder(); | ||
|
|
||
| sb.AppendLine($"-- Lua functions available in EmuHawk {VersionInfo.MainVersion}"); | ||
| sb.AppendLine(Preamble); | ||
| sb.AppendLine(); | ||
|
|
||
| if (!string.IsNullOrEmpty(libraryDescription)) | ||
| { | ||
| sb.AppendLine(FormatMarkdown(libraryDescription)); | ||
| } | ||
|
|
||
| sb.AppendLine($"---@class {SafeLibraryTypeName(library)}"); | ||
| if (!typeof(LuaLibraryBase).IsAssignableFrom(libraryType)) sb.Append("local "); // don't make LuaCanvas global | ||
| sb.AppendLine($"{library} = {{}}"); | ||
| sb.AppendLine(); | ||
|
|
||
| foreach (var func in libraryGroup.OrderBy(func => func.Name)) | ||
| { | ||
| if (!string.IsNullOrEmpty(func.Description)) | ||
| { | ||
| sb.AppendLine(FormatMarkdown(func.Description)); | ||
| } | ||
|
|
||
| if (func.Example != null) | ||
| { | ||
| sb.AppendLine("---"); | ||
| sb.AppendLine("---Example:"); | ||
| sb.AppendLine("---"); | ||
| sb.AppendLine(FormatMarkdown(func.Example, "---\t")); | ||
| } | ||
|
|
||
| if (func.IsDeprecated) | ||
| { | ||
| sb.AppendLine("---@deprecated"); | ||
| } | ||
|
|
||
| foreach (var parameter in func.Method.GetParameters()) | ||
| { | ||
| if (IsParams(parameter)) | ||
| { | ||
| sb.Append("---@vararg"); | ||
| } | ||
| else | ||
| { | ||
| sb.Append($"---@param {parameter.Name}"); | ||
| if (parameter.HasDefaultValue || (parameter.IsNRTOrNullableT() ?? true)) | ||
| { | ||
| sb.Append('?'); | ||
| } | ||
| } | ||
|
|
||
| sb.Append(' '); | ||
| sb.Append(GetLuaType(parameter)); | ||
| if (IsZeroIndexed(parameter)) | ||
| { | ||
| sb.Append(" Zero-indexed array."); | ||
| } | ||
| if (parameter.HasDefaultValue && parameter.DefaultValue is not null and not "") | ||
| { | ||
| sb.Append($" Defaults to `{FormatValue(parameter.DefaultValue)}`"); | ||
| } | ||
| sb.AppendLine(); | ||
| } | ||
|
|
||
| if (func.Method.ReturnType != typeof(void)) | ||
| { | ||
| sb.Append("---@return "); | ||
| var luaType = GetLuaType(func.Method.ReturnParameter); | ||
| var nilable = func.Method.ReturnParameter.IsNRTOrNullableT() ?? true; | ||
| var wrapType = nilable && luaType.IndexOfAny([ ':', '|' ]) != -1; // ? is ambiguous on complex types like `string|int` or `fun(): string` | ||
| if (wrapType) sb.Append('('); | ||
| sb.Append(luaType); | ||
| if (wrapType) sb.Append(')'); | ||
| if (nilable) sb.Append('?'); | ||
| if (IsZeroIndexed(func.Method.ReturnParameter)) | ||
| { | ||
| sb.Append(" # Zero-indexed array."); | ||
| } | ||
| sb.AppendLine(); | ||
| } | ||
|
|
||
| sb.Append($"function {library}.{func.Name}("); | ||
|
|
||
| foreach (var parameter in func.Method.GetParameters()) | ||
| { | ||
| if (parameter.Position > 0) | ||
| { | ||
| sb.Append(", "); | ||
| } | ||
| sb.Append(IsParams(parameter) ? "..." : parameter.Name); | ||
| } | ||
|
|
||
| sb.AppendLine(") end"); | ||
| sb.AppendLine(); | ||
| } | ||
| File.WriteAllText(filePath, sb.ToString().ReplaceLineEndings()); | ||
| } | ||
| } | ||
|
|
||
| private static string FormatMarkdown(string value, string prefix = "---") | ||
| { | ||
| // prefix every line | ||
| value = Regex.Replace(value, "^", prefix, RegexOptions.Multiline); | ||
| // replace {{wiki markup}} with `markdown` | ||
| value = Regex.Replace(value, "{{(.+?)}}", "`$1`"); | ||
| // replace wiki image markup with markdown | ||
| value = Regex.Replace(value, @"\[(?<url>.+?)\|alt=(?<alt>.+?)\]", ""); | ||
| return value; | ||
| } | ||
|
|
||
| private static string FormatValue(object value) => value switch | ||
| { | ||
| string str => $"\"{str}\"", | ||
| true => "true", | ||
| false => "false", | ||
| null => "nil", | ||
| _ => value.ToString(), | ||
| }; | ||
|
|
||
| /// <summary> | ||
| /// Avoid name collisions with existing Lua types. | ||
| /// Only for the <c>@class</c> annotation, not the name of the global. | ||
| /// <see href="https://luals.github.io/wiki/annotations/#documenting-types" /> | ||
| /// </summary> | ||
| private static string SafeLibraryTypeName(string name) => name switch | ||
| { | ||
| "userdata" => $"biz{name}", | ||
| _ => name, | ||
| }; | ||
|
|
||
| private static string GetLuaType(ParameterInfo parameter) | ||
| { | ||
| if (GetOverrideType(parameter) is string overrideType) | ||
| { | ||
| return overrideType; | ||
| } | ||
|
|
||
| if (GetHardcodedType(parameter) is string hardcodedType) | ||
| { | ||
| return hardcodedType; | ||
| } | ||
|
|
||
| if (parameter.GetCustomAttribute<LuaColorParamAttribute>() is not null) | ||
| { | ||
| return "color"; // see Preamble | ||
| } | ||
|
|
||
| if (parameter.ParameterType.IsArray && IsParams(parameter)) | ||
| { | ||
| // no [] array modifier for varargs | ||
| return GetLuaType(parameter.ParameterType.GetElementType()); | ||
| } | ||
|
|
||
| return GetLuaType(parameter.ParameterType); | ||
| } | ||
|
|
||
| private static string GetLuaType(Type type) | ||
| { | ||
| // try this twice, before and after extracting the array/nullable type | ||
| if (TypeConversions.TryGetValue(type, out string luaType)) | ||
| { | ||
| return luaType; | ||
| } | ||
|
|
||
| if (type.IsArray) | ||
| { | ||
| return GetLuaType(type.GetElementType()) + "[]"; | ||
| } | ||
|
|
||
| if (Nullable.GetUnderlyingType(type) is Type underlyingType) | ||
| { | ||
| type = underlyingType; | ||
| } | ||
|
|
||
| if (TypeConversions.TryGetValue(type, out luaType)) | ||
| { | ||
| return luaType; | ||
| } | ||
|
|
||
| throw new NotSupportedException($"Unknown type {type.FullName} used in API. Generator must be updated to handle this."); | ||
| } | ||
|
|
||
| private static bool IsParams(ParameterInfo parameter) => parameter.GetCustomAttribute<ParamArrayAttribute>() is not null; | ||
|
|
||
| private static bool IsZeroIndexed(ParameterInfo parameter) => parameter.GetCustomAttribute<LuaZeroIndexedAttribute>() is not null; | ||
|
|
||
| private static string? GetOverrideType(ParameterInfo parameter) => parameter.GetCustomAttribute<LuaCatsTypeAttribute>()?.Type; | ||
| } | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We don't want to encourage this. There are enough footguns in calling
gui.*already with Lua not having named arguments.There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Removing this will likely cause a lot of unnecessary warnings
Uh oh!
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think type coercion warnings are good, but then I also consider dynamic languages like Lua to always be inferior to statically-typed languages. Is passing numbers for strings idiomatic in general, or only in concatenation?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Well, number-to-string conversion is specifically a feature in BizHawk's Lua bindings, not a feature of the Lua runtime. That's why LLS warns about it unless the parameter definition allows numbers.
But reading numbers from memory and passing them to gui functions is a pretty common pattern in BizHawk scripts, so I added this special case for those functions to avoid warnings in existing, working scripts. I have a test directory with the bundled Lua scripts and some from the Lua scripts catalog, and removing this code causes 73 additional warnings, 16 of which are in BizHawk's own scripts (for example).
I have no strong opinions on deprecating this behavior, but currently it's valid so the definitions say it's valid.