Rearrange documentation and document language server

Author: Sander Toonen

README.md

@@ -0,0 +1,233 @@
+# TypeScript Port Enhancements
+
+This is a modern TypeScript port of the expr-eval library, completely rewritten with contemporary build tools and development practices. Originally based on [expr-eval 2.0.2](http://silentmatt.com/javascript-expression-evaluator/), this version has been restructured with a modular architecture, TypeScript support, and comprehensive testing using Vitest. The library almost maintains backward compatibility while providing enhanced features and improved maintainability.
+
+This port adds the following enhancements over the original:
+
+## Support for json() function
+
+This will return a JSON string:
+
+```js
+json([1, 2, 3])
+```
+
+## Support for undefined
+
+The concept of JavaScript's undefined has been added to the parser.
+
+### undefined keyword
+
+The undefined keyword has been added to the parser allowing it to be used in expressions.
+
+```js
+x > 3 ? undefined : x
+x == undefined ? 1 : 2
+```
+
+### Setting expression variables to undefined
+
+If you set a local variable to undefined, expr-eval would generate an error saying that your variable was unrecognized.
+
+For example:
+
+```js
+/* myCustomFn() returns undefined */
+x = myCustomFn(); x > 3
+/* Error: unrecognized variable: x */
+```
+
+This has been fixed, you can now set expression variables to undefined and they will resolve correctly.
+
+### Operators/functions gracefully support undefined
+
+All operators and built-in functions have been extended to gracefully support undefined. Generally speaking if one of the input values is undefined then the operator/function returns undefined. So `2 + undefined` is `undefined`, `max(0, 1, undefined)` is `undefined`, etc.
+
+Logical operators act just like JavaScript, so `3 > undefined` is `false`.
+
+## Coalesce Operator
+
+The coalesce operator `??` has been added; `x ?? y` will evaluate to y if x is:
+
+* `undefined`
+* `null`
+* Infinity (divide by zero)
+* NaN
+
+Examples:
+
+```js
+var parser = new Parser();
+var obj = { x: undefined, y: 10, z: 0 };
+parser.evaluate('x ?? 0', obj); // 0
+parser.evaluate('y ?? 0', obj); // 10
+parser.evaluate('x ?? 1 * 3', obj); // (undefined ?? 1) * 3 = 3
+parser.evaluate('y ?? 1 * 3', obj); // (10 ?? 1) * 3 = 30
+parser.evaluate('10 / z', obj); // Infinity
+parser.evaluate('10 / z ?? 0', obj); // 0
+parser.evaluate('sqrt -1'); // NaN
+parser.evaluate('sqrt -1 ?? 0'); // 0
+```
+
+## Not In Operator
+
+The `not in` operator has been added.
+
+`"a" not in ["a", "b", "c"]`
+
+is equivalent to
+
+`not ("a" in ["a", "b", "c"])`
+
+## Optional Chaining for Property Access
+
+Structure/array property references now act like `?.`, meaning if the entire property chain does not exist then instead of throwing an error the value of the property is undefined.
+
+For example:
+
+```js
+var parser = new Parser();
+var obj = { thingy: { array: [{ value: 10 }] } };
+parser.evaluate('thingy.array[0].value', obj); // 10
+parser.evaluate('thingy.array[1].value', obj); // undefined
+parser.evaluate('thingy.doesNotExist[0].childArray[1].notHere.alsoNotHere', obj); // undefined
+parser.evaluate('thingy.array[0].value.doesNotExist', obj); // undefined
+```
+
+This can be combined with the coalesce operator to gracefully fall back on a default value if some part of a long property reference is `undefined`.
+
+```js
+var parser = new Parser();
+var obj = { thingy: { array: [{ value: 10 }] } };
+parser.evaluate('thingy.array[1].value ?? 0', obj); // 0
+```
+
+## String Concatenation Using +
+
+The + operator can now be used to concatenate strings.
+
+```js
+var parser = new Parser();
+var obj = { thingy: { array: [{ value: 10 }] } };
+parser.evaluate('"abc" + "def" + "ghi"', obj); // 'abcdefghi'
+```
+
+## Support for Promises in Custom Functions
+
+Custom functions can return promises. When this happens evaluate will return a promise that when resolved contains the expression value.
+
+```js
+const parser = new Parser();
+
+parser.functions.doIt = value => value + value;
+parser.evaluate('doIt(2) + 3'); // 7
+
+parser.functions.doIt = value =>
+    new Promise((resolve) => setTimeout(() => resolve(value + value), 100));
+await parser.evaluate('doIt(2) + 3'); // 7
+```
+
+## Support for Custom Variable Name Resolution
+
+Custom logic can be provided to resolve unrecognized variable names. The parser has a resolve callback that will be called any time a variable name is not recognized. This can return an object that either indicates that the variable name is an alias for another variable or it can return the variable value.
+
+```js
+const parser = new Parser();
+const obj = { variables: { a: 5, b: 1 } };
+parser.resolve = token => token === '$v' ? { alias: 'variables' } : undefined;
+parser.evaluate('$v.a + variables.b', obj); // 6
+
+parser.resolve = token =>
+    token.startsWith('$') ? { value: obj.variables[token.substring(1)] } : undefined;
+assert.strictEqual(parser.evaluate('$a + $b'), 6);
+```
+
+## SQL Style Case Blocks
+
+> **NOTE:** `toJSFunction()` is not supported for expressions that use case blocks.
+
+SQL style case blocks are now supported, for both cases which evaluate a value against other values (a switch style case) and cases which test for the first truthy when (if/else/if style cases).
+
+### Switch-style case
+
+```js
+const parser = new Parser();
+const expr = `
+    case x
+        when 1 then 'one'
+        when 1+1 then 'two'
+        when 1+1+1 then 'three'
+        else 'too-big'
+    end
+`;
+parser.evaluate(expr, { x: 1 }); // 'one'
+parser.evaluate(expr, { x: 2 }); // 'two'
+parser.evaluate(expr, { x: 3 }); // 'three'
+parser.evaluate(expr, { x: 4 }); // 'too-big'
+```
+
+### If/else-style case
+
+```js
+const parser = new Parser();
+const expr = `
+    case
+        when x == 1 then 'one'
+        when x == 1+1 then 'two'
+        when x == 1+1+1 then 'three'
+        else 'too-big'
+    end
+`;
+parser.evaluate(expr, { x: 1 }); // 'one'
+parser.evaluate(expr, { x: 2 }); // 'two'
+parser.evaluate(expr, { x: 3 }); // 'three'
+parser.evaluate(expr, { x: 4 }); // 'too-big'
+```
+
+## Object Construction
+
+Objects can be created using JavaScript syntax. This allows for expressions that return object values and for object arguments to be passed to custom functions.
+
+```js
+const parser = new Parser();
+const expr = `{
+    a: x * 3,
+    b: {
+        /*this x is a property and not the x on the input object*/
+        x: "first" + "_" + "second",
+        y: min(x, 0),
+    },
+    c: [0, 1, 2, x],
+}`;
+parser.evaluate(expr, { x: 3 });
+/*
+{
+    a: 15,
+    b: {
+        x: 'first_second',
+        z: 0
+    },
+    c: [0, 1, 2, 3]
+}
+*/
+```
+
+## As Operator (Type Conversion)
+
+An as operator has been added to support type conversion. **This operator is disabled by default and must be explicitly enabled by setting `operators.conversion` to true in the options.** It can be used to perform value conversion. By default is of limited value; it only supports converting values to numbers, int/integer (by rounding the number), and boolean. The intent is to allow integration of more sophisticated value conversion packages such as numeral.js and moment for conversion of other values.
+
+```js
+const parser = new Parser({ operators: { conversion: true } });
+parser.evaluate('"1.6" as "number"'); // 1.6
+parser.evaluate('"1.6" as "int"'); // 2
+parser.evaluate('"1.6" as "integer"'); // 2
+parser.evaluate('"1.6" as "boolean"'); // true
+```
+
+The default `as` implementation can be overridden by replacing `parser.binaryOps.as`.
+
+```js
+const parser = new Parser({ operators: { conversion: true } });
+parser.binaryOps.as = (a, _b) => a + '_suffix';
+parser.evaluate('"abc" as "suffix"'); // 'abc_suffix'
+```

docs/enhancements.md

@@ -0,0 +1,93 @@
+# Expression
+
+`Parser.parse(str)` returns an `Expression` object. `Expression`s are similar to JavaScript functions, i.e. they can be "called" with variables bound to passed-in values. In fact, they can even be converted into JavaScript functions.
+
+## evaluate(variables?: object)
+
+Evaluate the expression, with variables bound to the values in `{variables}`. Each variable in the expression is bound to the corresponding member of the `variables` object. If there are unbound variables, `evaluate` will throw an exception.
+
+```js
+js> expr = Parser.parse("2 ^ x");
+(2^x)
+js> expr.evaluate({ x: 3 });
+8
+```
+
+## substitute(variable: string, expression: Expression | string | number)
+
+Create a new `Expression` with the specified variable replaced with another expression. This is similar to function composition. If `expression` is a string or number, it will be parsed into an `Expression`.
+
+```js
+js> expr = Parser.parse("2 * x + 1");
+((2*x)+1)
+js> expr.substitute("x", "4 * x");
+((2*(4*x))+1)
+js> expr2.evaluate({ x: 3 });
+25
+```
+
+## simplify(variables: object)
+
+Simplify constant sub-expressions and replace variable references with literal values. This is basically a partial evaluation, that does as much of the calculation as it can with the provided variables. Function calls are not evaluated (except the built-in operator functions), since they may not be deterministic.
+
+Simplify is pretty simple. For example, it doesn't know that addition and multiplication are associative, so `((2*(4*x))+1)` from the previous example cannot be simplified unless you provide a value for x. `2*4*x+1` can however, because it's parsed as `(((2*4)*x)+1)`, so the `(2*4)` sub-expression will be replaced with "8", resulting in `((8*x)+1)`.
+
+```js
+js> expr = Parser.parse("x * (y * atan(1))").simplify({ y: 4 });
+(x*3.141592653589793)
+js> expr.evaluate({ x: 2 });
+6.283185307179586
+```
+
+## variables(options?: object)
+
+Get an array of the unbound variables in the expression.
+
+```js
+js> expr = Parser.parse("x * (y * atan(1))");
+(x*(y*atan(1)))
+js> expr.variables();
+x,y
+js> expr.simplify({ y: 4 }).variables();
+x
+```
+
+By default, `variables` will return "top-level" objects, so for example, `Parser.parse(x.y.z).variables()` returns `['x']`. If you want to get the whole chain of object members, you can call it with `{ withMembers: true }`. So `Parser.parse(x.y.z).variables({ withMembers: true })` would return `['x.y.z']`.
+
+## symbols(options?: object)
+
+Get an array of variables, including any built-in functions used in the expression.
+
+```js
+js> expr = Parser.parse("min(x, y, z)");
+(min(x, y, z))
+js> expr.symbols();
+min,x,y,z
+js> expr.simplify({ y: 4, z: 5 }).symbols();
+min,x
+```
+
+Like `variables`, `symbols` accepts an option argument `{ withMembers: true }` to include object members.
+
+## toString()
+
+Convert the expression to a string. `toString()` surrounds every sub-expression with parentheses (except literal values, variables, and function calls), so it's useful for debugging precedence errors.
+
+## toJSFunction(parameters: array | string, variables?: object)
+
+Convert an `Expression` object into a callable JavaScript function. `parameters` is an array of parameter names, or a string, with the names separated by commas.
+
+If the optional `variables` argument is provided, the expression will be simplified with variables bound to the supplied values.
+
+```js
+js> expr = Parser.parse("x + y + z");
+((x + y) + z)
+js> f = expr.toJSFunction("x,y,z");
+[Function] // function (x, y, z) { return x + y + z; };
+js> f(1, 2, 3)
+6
+js> f = expr.toJSFunction("y,z", { x: 100 });
+[Function] // function (y, z) { return 100 + y + z; };
+js> f(2, 3)
+105
+```

docs/expression.md

@@ -0,0 +1,60 @@
+# Language Service
+
+The library includes a built-in language service that provides IDE-like features for expr-eval expressions. This is useful for integrating expr-eval into code editors like Monaco Editor (used by VS Code).
+
+## Features
+
+- **Code Completions** - Autocomplete for functions, operators, keywords, and user-defined variables
+- **Hover Information** - Documentation tooltips when hovering over functions and variables
+- **Syntax Highlighting** - Token-based highlighting for numbers, strings, keywords, operators, etc.
+
+## Basic Usage
+
+```js
+import { createLanguageService } from '@pro-fa/expr-eval';
+
+const ls = createLanguageService();
+
+// Define variables available in your expressions
+const variables = { x: 42, user: { name: 'Ada' }, flag: true };
+
+// Get completions at a position
+const completions = ls.getCompletions({
+    textDocument: doc,  // LSP-compatible text document
+    position: { line: 0, character: 5 },
+    variables
+});
+
+// Get hover information
+const hover = ls.getHover({
+    textDocument: doc,
+    position: { line: 0, character: 3 },
+    variables
+});
+
+// Get syntax highlighting tokens
+const tokens = ls.getHighlighting(doc);
+```
+
+## Monaco Editor Integration Sample
+
+A complete working example of Monaco Editor integration is included in the repository. To run it:
+
+```bash
+# Build the UMD bundle and start the sample server
+npm run monaco-sample:serve
+```
+
+Then open http://localhost:8080 in your browser. The sample demonstrates:
+
+- Autocompletion for built-in functions (`sum`, `max`, `min`, etc.) and user variables
+- Hover documentation for functions and variables
+- Live syntax highlighting
+- Real-time expression evaluation
+
+The sample code is located in `samples/language-service-sample/` and shows how to:
+
+1. Register a custom language with Monaco
+2. Connect the language service to Monaco's completion and hover providers
+3. Apply syntax highlighting using decorations
+4. Create an LSP-compatible text document wrapper for Monaco models