Usage

Prelude

use reqlang_expr::prelude::*;

The prelude provides everything needed to lex, parse, compile, disassemble, and interpret expressions.

Error Handling: ExprError & ExprResult

ExprResult<T> (type alias for Result<T, Vec<ExprErrorS>>) is the result type used to handle errors throughout the process of expression evaluation.

Multiple errors are returned at a time
Errors have span information

See: errors.rs, span.rs

Lexer

The lexer takes an input string and returns a stream of tokens.

Token	Pattern	Description
`LParan`	`(`	Opening parantheses for a call expression
`RParan`	`)`	Closing parantheses for a call expression
`Comma`	`,`	Separator between arguments in `Fn` types
`LAngle`	`<`	Generic type delimitor
`RAngle`	`>`	Generic type delimitor
`Arrow`	`->`	Separator between `Fn` arg parans and return type
`Identifier`	`[!?:]?[a-zA-Z][a-zA-Z0-9_]*`	Identifer referencing a builtin function, variable, prompt, or secret
`String`	`[^`]*`	A literal string of text delimited by backticks
`True`	`true`	A literal boolean value of `true`
`False`	`false`	A literal boolean value of `true`

Usage

let source = "(noop)";
let tokens: lex(&source);

See: lexer.rs

Parser

The parser takes a stream of tokens from the lexer and constructs an AST (Abstract Syntax Tree) in the form of a tree of Expr nodes.

Usage

let source = "(eq (type `Hello`) (type `World`))";

let ast: Expr = parse(&source)?;

Expr

All values in the language are parsed in to an expression: Expr.

pub enum Expr {
    Bool(Box<ExprBool>),
    Identifier(Box<ExprIdentifier>),
    Call(Box<ExprCall>),
    String(Box<ExprString>),
    Error,
}

Error

Expressions that can't be parsed are represented in the AST as an Expr::Error.

Span Information

Expressions that contain sub expressions (e.g. ExprCall) store those subexpressions and their location in the source as (Expr, Range<usize>).

The expression (and true false) would parse to this.

Expr::Call(ExprCall {
    callee: (Expr::identifier("and"), 1..4).into(),
    args: vec![
        (Expr::bool(true), 5..9),
        (Expr::bool(false), 10..15)
    ]
}.into());

Span information 1..4, 5..9, 10..15 is stored next to the subexpressions and, true, and false.

ExprCall

A call to a builtin (referenced by identifier) with N expressions passed as arguments.

(contains `foo`, `foobar`)
(eq `foo` (trim ` foo `))

pub struct ExprCall {
    pub callee: Box<ExprS>,
    pub args: Vec<ExprS>,
}

ExprIdentifier

An identifier referencing a builtin, type, variable, prompt, secret, or client value.

builtin_name
TypeName, TypeName<Type>, Fn(Type, ...Type) -> Type
:var_name
?prompt_name
!secret_name
@client_value

pub struct ExprIdentifier(pub String, pub IdentifierKind, pub Option<Type>);

pub enum IdentifierKind {
    Builtin,
    Var,
    Prompt,
    Secret,
    Client,
    Type,
}

let ident = ExprIdentifier::new(":foo");
let expr = Expr::Identifier(Box::new(ident));
assert_eq!(IdentifierKind::Var, *ident.identifier_kind());

Full Name

The full name of the identifier, including the sigil e.g. :, ?, !, @ (for non builtins).

let ident = ExprIdentifier::new(":foo");
assert_eq!(":foo", ident.full_name());

Look Up Name

The full name of the identifier, minus the sigil. This is used when looking up identifiers in the compilation and runtime phase.

let ident = ExprIdentifier::new(":foo");
assert_eq!("foo", ident.lookup_name());

Identifier Type

The type of the identifier is added in two passes:

Variables, prompts, secrets, and client values during the parsing phase
Builtins during the compilation phase

ExprString

A string literal of text.

`Hello World!`

pub struct ExprString(pub String);

ExprBool

A boolean literal.

true
false

pub struct ExprBool(pub bool);

See: parser.rs, grammar.lalrpop, ast.rs

Types

pub enum Type {
    Value,
    String,
    Fn {
        args: Vec<Type>,
        variadic_arg: Option<Box<Type>>,
        returns: Box<Type>,
    },
    Bool,
    Type(Box<Type>),
    Unknown,
}

See: types.rs

Compiler

The compiler produces bytecode from an AST and a compile time environment.

Bytecode

The compiler produces bytecode from an AST in the form of vector of op codes.

Version Byte Prefix

Valid bytecode will always begin with 4 bytes representing the current language version. This means op codes start at ip/index 4.

Op Codes

Op Code	Op Byte	Args	Description
`CALL`	0	$INDEX, $ARG_COUNT	Call builtin `$INDEX` with `$ARG_COUNT` arguments
`GET`	1	$LOOKUP, $INDEX	Get a builtin/variable/prompt/secret from the env by `$INDEX`
`CONSTANT`	2	$CONST_INDEX	Get constant value by `$CONST_INDEX`
`TRUE`	3		Push a `true` value on to the stack
`FALSE`	4		Push a `false` value on to the stack

Lookup Types

Type	Lookup Index	Description
`BUILTIN`	0	Builtin function
`VAR`	1	Variable (identifier prefixed with `:`)
`PROMPT`	2	Prompt (identifier prefixed with `?`)
`SECRET`	3	Secret (identifier prefixed with `!`)
`USER_BUILTIN`	4	User provided builtin function
`CLIENT_CTX`	5	Client provided value (identifier prefixed with `@`)
`TYPE`	6	A type stored in `ExprByteCode`

Compile Time Environment

The compiler's environment contains a lists of names for builtin functions, variables, prompts, and secrets.

pub struct CompileTimeEnv {
    builtins: Vec<BuiltinFn<'static>>,
    user_builtins: Vec<BuiltinFn<'static>>,
    vars: Vec<String>,
    prompts: Vec<String>,
    secrets: Vec<String>,
    client_context: Vec<String>,
}

Builtin Functions

Builtins are functions provided by the compiler/VM and the only functions available.

pub struct BuiltinFn<'a> {
    /// Needs to follow identifier naming rules
    pub name: &'static str,
    /// Arguments the function expects
    pub args: &'a [FnArg],
    /// Type returned by the function
    pub return_type: Type,
    /// Function used at runtime
    pub func: fn(Vec<Value>) -> ExprResult<Value>,
}

pub struct FnArg {
    name: String,
    ty: Type,
    varadic: bool,
}

See: builtins.rs, types.rs, value.rs

ExprByteCode

The result of an expression compilation is ExprByteCode.

pub struct ExprByteCode {
    version: [u8; 4],
    codes: Vec<u8>,
    strings: Vec<String>,
    types: Vec<Type>,
}

Version

The version of the compiler encoded as bytes.

Codes

The actual bytecode values.

Strings

An indexed list of strings encountered during compilation. These string indexes are referenced by the opcode::CONSTANT opcode.

Type

An indexed list of types encountered during compilation. These type indexes are referenced by the opcode::GET opcode and lookup::TYPE lookup.

Usage

let source = "(noop)";

let ast: Expr = parse(&source)?;

let var_names = vec![];
let prompt_names = vec![];
let secret_names = vec![];
let client_context_names = vec![];

let mut env = CompileTimeEnv::new(var_names, prompt_names, secret_names, client_context_namess);

let bytecode = compile(&ast, &env)?;

See: compiler.rs

Virtual Machine

The virtual machine (VM) takes in a runtime environment, evaluates a stream of bytecode and produces a value.

Values

Value represent expression values during runtime.

pub enum Value {
    String(String),
    Fn(Rc<BuiltinFn>),
    Bool(bool),
    Type(Box<Type>),
}

See: value.rs, builtins.rs

Convert Values To Types

A Value's Type can be retrieved using get_type():

let value = Value::String("Hello World".to_string());
let value_type: Type = value.get_type();

This also works:

let value = Value::String("Hello World".to_string());
let value_type: Type = value.into();

Runtime Environment

The VM's runtime environment contains a lists of values for variables, prompts, and secrets.

pub struct RuntimeEnv {
    pub vars: Vec<String>,
    pub prompts: Vec<String>,
    pub secrets: Vec<String>,
    pub client_context: Vec<Value>,
}

See: vm.rs, value.rs

Usage

let source = "(noop)";

let ast: Expr = parse(&source)?;

let var_names = vec![];
let prompt_names = vec![];
let secret_names = vec![];
let client_context_names = vec![];

let mut env = CompileTimeEnv::new(var_names, prompt_names, secret_names, client_context_names);

let bytecode = compile(&ast, &env)?;

let mut vm = Vm::new();

let var_values = vec![];
let prompt_values = vec![];
let secret_values = vec![];
let client_context_values = vec![];

let runtime_env: RuntimeEnv = RuntimeEnv {
    vars: var_values,
    prompts: prompt_values,
    secrets: secret_values,
    client_context_values
};

let value = vm.interpret(bytecode.into(), &env, &runtime_env)?;

See: vm.rs, value.rs

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Usage

Prelude

Error Handling: ExprError & ExprResult

Lexer

Usage

Parser

Usage

Expr

Error

Span Information

ExprCall

ExprIdentifier

Full Name

Look Up Name

Identifier Type

ExprString

ExprBool

Types

Compiler

Bytecode

Version Byte Prefix

Op Codes

Lookup Types

Compile Time Environment

Builtin Functions

ExprByteCode

Version

Codes

Strings

Type

Usage

Virtual Machine

Values

Convert Values To Types

Runtime Environment

Usage

FilesExpand file tree

USAGE.md

Latest commit

History

USAGE.md

File metadata and controls

Usage

Prelude

Error Handling: ExprError & ExprResult

Lexer

Usage

Parser

Usage

Expr

Error

Span Information

ExprCall

ExprIdentifier

Full Name

Look Up Name

Identifier Type

ExprString

ExprBool

Types

Compiler

Bytecode

Version Byte Prefix

Op Codes

Lookup Types

Compile Time Environment

Builtin Functions

ExprByteCode

Version

Codes

Strings

Type

Usage

Virtual Machine

Values

Convert Values To Types

Runtime Environment

Usage