Commit dc3f1b3
Claude/wokelang ebnf grammar qoe vv (#20)
* Implement WokeLang interpreter in Rust
Complete implementation of WokeLang with:
Lexer (src/lexer/):
- Token definitions for all keywords, operators, literals
- Uses logos for fast tokenization
- Handles comments, strings with escapes, numbers
Parser (src/parser/):
- Recursive descent parser
- Full expression parsing with correct precedence
- All statement types: remember, when/otherwise, repeat, attempt safely
- Pattern matching with decide based on
- Emote tags, consent blocks, gratitude declarations
AST (src/ast/):
- Complete type definitions for all language constructs
- Spanned nodes for error reporting
Interpreter (src/interpreter/):
- Tree-walking interpreter
- Scoped environments for variables
- Built-in functions: print, len, toString, toInt
- Consent system with interactive prompts
- Pattern matching evaluation
- All operators (arithmetic, comparison, logical)
CLI (src/main.rs):
- woke <file.woke> - run program
- woke --tokenize <file> - show tokens
- woke --parse <file> - show AST
Examples:
- examples/hello.woke - feature showcase
- examples/demo.woke - runnable demo
* Add REPL, WASM compilation, and Zig FFI
REPL (src/repl.rs):
- Interactive command-line interface with rustyline
- Commands: :help, :quit, :clear, :reset, :load, :ast
- Expression evaluation with automatic result printing
- Start with `woke` or `woke --repl`
WASM Compilation (src/codegen/):
- Compile WokeLang to WebAssembly binary format
- Uses wasm-encoder for proper WASM generation
- Supports functions, expressions, loops, conditionals
- Pattern matching compilation
- CLI: `woke -c input.woke` outputs input.wasm
Zig FFI (src/ffi/, zig/, include/):
- C-compatible API for embedding WokeLang
- Interpreter lifecycle: woke_interpreter_new/free
- Code execution: woke_exec, woke_eval
- Value operations: type checking, conversion, creation
- Static library (libwokelang.a) and shared library (.so)
- C header (include/wokelang.h)
- Zig bindings (zig/wokelang.zig) with idiomatic wrapper
- Example Zig program and build.zig
Other:
- examples/math.woke - math functions for WASM demo
- Cargo.toml updated for cdylib/staticlib targets
* Add comprehensive documentation and wiki
Includes:
- Project roadmap (ROADMAP.md) with version timeline through v1.0
- Wiki home page with table of contents
- Getting Started guides: Installation, Hello World, Basic Syntax, REPL
- Language Guide: Functions, Control Flow, Error Handling, Variables/Types
- Core Concepts: Consent System, Gratitude, Emote Tags
- Reference: CLI, Built-in Functions, Keywords, Operators, Language Spec
- Internals: Architecture, Lexer, Parser, Interpreter, WASM, FFI
- Tutorial: Building a CLI app
* Implement Phase 2 Language Completeness features
- Add Result types (Okay/Oops) with pattern matching support
- Implement error propagation operator (?) for Result types
- Add pattern matching destructuring with guard clauses
- Implement module system with use/share keywords
- Add new builtins: isOkay, isOops, getOkay, getOops
- Update WASM compiler and FFI for new AST variants
- Add example programs for Result types and modules
* Add static type inference with Hindley-Milner style checker
- Implement TypeChecker module with type inference
- Support for basic types: Int, Float, String, Bool, Unit
- Support for compound types: Array, Result, Maybe, Function
- Implement unification algorithm for type constraints
- Add built-in function type handling
- Include comprehensive test suite for type checking
* Implement Phase 3 Worker System with message passing
- Add async worker module with thread-based execution
- Implement WorkerHandle, WorkerContext, WorkerPool
- Add message passing: WorkerMessage enum with Value, Stop, Ping, etc.
- Add cancellation tokens for task control
- New lexer tokens: send, receive, channel, await, cancel, from
- New AST nodes: SendMessage, ReceiveMessage, AwaitWorker, CancelWorker
- Update parser for new worker syntax
- Update interpreter with worker statement handling
- Update WASM codegen and typechecker for new statements
- Add worker tests (4 new tests)
- Add workers.woke example demonstrating worker system
- Update ROADMAP.md with Phase 2 completion status
* Implement Phase 3 Security: capability-based security and persistent consent storage
- Add CapabilityRegistry with support for FileRead, FileWrite, Execute, Network,
Environment, Process, SystemInfo, Crypto, Clipboard, Notify capabilities
- Implement wildcard capability matching (e.g., file:read:* grants all file reads)
- Add audit logging for all capability requests, grants, denials, and revocations
- Create ConsentStore for persistent storage of consent decisions
- Support consent durations: Session, Day, Week, Forever, Once
- Add security.woke example demonstrating superpower declarations
* Implement Phase 4 Standard Library for WokeLang
Add comprehensive standard library with the following modules:
- std.math: Mathematical functions (abs, sqrt, pow, sin, cos, tan, floor, ceil,
round, min, max, random, pi, e)
- std.io: File I/O with consent-based access control (readFile, writeFile,
appendFile, exists, delete, listDir, createDir, readLine)
- std.json: JSON parsing and generation (parse, stringify, get, set)
- std.time: Date/time handling (now, timestamp, format, parse, sleep, elapsed)
- std.net: HTTP networking with consent (httpGet, httpPost, download)
Key features:
- StdlibRegistry for function registration and lookup
- Capability-based security integration for I/O and network operations
- Simple JSON parser/serializer without external dependencies
- Basic HTTP client using TCP sockets (no TLS for now)
- Elapsed timer with thread-local storage
- Add Record variant to Value for JSON objects
Includes stdlib_demo.woke example demonstrating all modules.
* Implement Phase 5: Bytecode Compiler and VM with Optimizations
Add a complete bytecode virtual machine for WokeLang with:
Bytecode Infrastructure:
- Stack-based OpCode instruction set (arithmetic, control flow, functions)
- CompiledFunction with constant pool and local variable slots
- CompiledProgram with function registry and entry point detection
Bytecode Compiler:
- Compiles AST to bytecode instructions
- Handles all statement types (VarDecl, Assignment, Conditional, Loop, etc.)
- Pattern matching compilation for decide statements
- Function call and closure support
- Result type (Okay/Oops) compilation
Virtual Machine:
- Stack-based execution with call frames
- Local and global variable support
- Full arithmetic and comparison operations
- Control flow (jumps, conditionals, loops)
- Array and record operations
- Result type handling with error propagation
- Safety limits (max stack size, max call depth)
Optimizer:
- Constant folding (evaluates constant expressions at compile time)
- Dead code elimination (removes unreachable instructions)
- Peephole optimizations (double negation, jump-to-next removal)
- Nop removal with jump target patching
Performance Benchmarks:
- Comparative benchmarks for interpreter vs VM execution
- Shows 6-11x speedup with precompiled bytecode
* Add EBNF grammar and standard library structure
EBNF Grammar (docs/grammar.ebnf):
- Complete lexical grammar (literals, identifiers, operators)
- Full syntactic grammar for all language constructs
- Program structure, functions, statements, expressions
- Pattern matching, concurrency, modules, types
- Consent blocks, gratitude declarations, pragmas
- Emote tags and emotional annotations
- Operator precedence documentation
Standard Library Structure:
lib/common/ - Language-agnostic utilities:
- prelude.woke: Core utilities (identity, boolean, numeric, string, array, result)
- collections.woke: Higher-order functions (map, filter, reduce, zip, partition)
- async.woke: Concurrent patterns (workers, retry, timeout, channels)
lib/wokelang/ - WokeLang-specific features:
- consent.woke: Consent and permission management with safe operations
- emotes.woke: Emotional annotations, sentiment-aware code, code health
- gratitude.woke: Attribution management, contributor recognition
Includes README documenting library structure and usage patterns.
---------
Co-authored-by: Claude <noreply@anthropic.com>1 parent 7fe5027 commit dc3f1b3
14 files changed
Lines changed: 4416 additions & 0 deletions
File tree
- benches
- docs
- lib
- common
- wokelang
- src/vm
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
| 10 | + | |
| 11 | + | |
| 12 | + | |
| 13 | + | |
| 14 | + | |
| 15 | + | |
| 16 | + | |
| 17 | + | |
| 18 | + | |
| 19 | + | |
| 20 | + | |
| 21 | + | |
| 22 | + | |
| 23 | + | |
| 24 | + | |
| 25 | + | |
| 26 | + | |
| 27 | + | |
| 28 | + | |
| 29 | + | |
| 30 | + | |
| 31 | + | |
| 32 | + | |
| 33 | + | |
| 34 | + | |
| 35 | + | |
| 36 | + | |
| 37 | + | |
| 38 | + | |
| 39 | + | |
| 40 | + | |
| 41 | + | |
| 42 | + | |
| 43 | + | |
| 44 | + | |
| 45 | + | |
| 46 | + | |
| 47 | + | |
| 48 | + | |
| 49 | + | |
| 50 | + | |
| 51 | + | |
| 52 | + | |
| 53 | + | |
| 54 | + | |
| 55 | + | |
| 56 | + | |
| 57 | + | |
| 58 | + | |
| 59 | + | |
| 60 | + | |
| 61 | + | |
| 62 | + | |
| 63 | + | |
| 64 | + | |
| 65 | + | |
| 66 | + | |
| 67 | + | |
| 68 | + | |
| 69 | + | |
| 70 | + | |
| 71 | + | |
| 72 | + | |
| 73 | + | |
| 74 | + | |
| 75 | + | |
| 76 | + | |
| 77 | + | |
| 78 | + | |
| 79 | + | |
| 80 | + | |
| 81 | + | |
| 82 | + | |
| 83 | + | |
| 84 | + | |
| 85 | + | |
| 86 | + | |
| 87 | + | |
| 88 | + | |
| 89 | + | |
| 90 | + | |
| 91 | + | |
| 92 | + | |
| 93 | + | |
| 94 | + | |
| 95 | + | |
| 96 | + | |
| 97 | + | |
| 98 | + | |
| 99 | + | |
| 100 | + | |
| 101 | + | |
| 102 | + | |
| 103 | + | |
| 104 | + | |
| 105 | + | |
| 106 | + | |
| 107 | + | |
| 108 | + | |
| 109 | + | |
| 110 | + | |
| 111 | + | |
| 112 | + | |
| 113 | + | |
| 114 | + | |
| 115 | + | |
| 116 | + | |
| 117 | + | |
| 118 | + | |
| 119 | + | |
| 120 | + | |
| 121 | + | |
| 122 | + | |
| 123 | + | |
| 124 | + | |
| 125 | + | |
| 126 | + | |
| 127 | + | |
| 128 | + | |
| 129 | + | |
| 130 | + | |
| 131 | + | |
| 132 | + | |
| 133 | + | |
| 134 | + | |
| 135 | + | |
| 136 | + | |
| 137 | + | |
| 138 | + | |
| 139 | + | |
| 140 | + | |
| 141 | + | |
| 142 | + | |
| 143 | + | |
| 144 | + | |
| 145 | + | |
| 146 | + | |
| 147 | + | |
| 148 | + | |
| 149 | + | |
| 150 | + | |
| 151 | + | |
| 152 | + | |
| 153 | + | |
| 154 | + | |
| 155 | + | |
| 156 | + | |
| 157 | + | |
| 158 | + | |
0 commit comments