Skip to content

Commit 501634e

Browse files
Claude/wokelang ebnf grammar qoe vv (#12)
* Implement WokeLang interpreter in Rust Complete implementation of WokeLang with: Lexer (src/lexer/): - Token definitions for all keywords, operators, literals - Uses logos for fast tokenization - Handles comments, strings with escapes, numbers Parser (src/parser/): - Recursive descent parser - Full expression parsing with correct precedence - All statement types: remember, when/otherwise, repeat, attempt safely - Pattern matching with decide based on - Emote tags, consent blocks, gratitude declarations AST (src/ast/): - Complete type definitions for all language constructs - Spanned nodes for error reporting Interpreter (src/interpreter/): - Tree-walking interpreter - Scoped environments for variables - Built-in functions: print, len, toString, toInt - Consent system with interactive prompts - Pattern matching evaluation - All operators (arithmetic, comparison, logical) CLI (src/main.rs): - woke <file.woke> - run program - woke --tokenize <file> - show tokens - woke --parse <file> - show AST Examples: - examples/hello.woke - feature showcase - examples/demo.woke - runnable demo * Add REPL, WASM compilation, and Zig FFI REPL (src/repl.rs): - Interactive command-line interface with rustyline - Commands: :help, :quit, :clear, :reset, :load, :ast - Expression evaluation with automatic result printing - Start with `woke` or `woke --repl` WASM Compilation (src/codegen/): - Compile WokeLang to WebAssembly binary format - Uses wasm-encoder for proper WASM generation - Supports functions, expressions, loops, conditionals - Pattern matching compilation - CLI: `woke -c input.woke` outputs input.wasm Zig FFI (src/ffi/, zig/, include/): - C-compatible API for embedding WokeLang - Interpreter lifecycle: woke_interpreter_new/free - Code execution: woke_exec, woke_eval - Value operations: type checking, conversion, creation - Static library (libwokelang.a) and shared library (.so) - C header (include/wokelang.h) - Zig bindings (zig/wokelang.zig) with idiomatic wrapper - Example Zig program and build.zig Other: - examples/math.woke - math functions for WASM demo - Cargo.toml updated for cdylib/staticlib targets * Add comprehensive documentation and wiki Includes: - Project roadmap (ROADMAP.md) with version timeline through v1.0 - Wiki home page with table of contents - Getting Started guides: Installation, Hello World, Basic Syntax, REPL - Language Guide: Functions, Control Flow, Error Handling, Variables/Types - Core Concepts: Consent System, Gratitude, Emote Tags - Reference: CLI, Built-in Functions, Keywords, Operators, Language Spec - Internals: Architecture, Lexer, Parser, Interpreter, WASM, FFI - Tutorial: Building a CLI app * Implement Phase 2 Language Completeness features - Add Result types (Okay/Oops) with pattern matching support - Implement error propagation operator (?) for Result types - Add pattern matching destructuring with guard clauses - Implement module system with use/share keywords - Add new builtins: isOkay, isOops, getOkay, getOops - Update WASM compiler and FFI for new AST variants - Add example programs for Result types and modules * Add static type inference with Hindley-Milner style checker - Implement TypeChecker module with type inference - Support for basic types: Int, Float, String, Bool, Unit - Support for compound types: Array, Result, Maybe, Function - Implement unification algorithm for type constraints - Add built-in function type handling - Include comprehensive test suite for type checking * Implement Phase 3 Worker System with message passing - Add async worker module with thread-based execution - Implement WorkerHandle, WorkerContext, WorkerPool - Add message passing: WorkerMessage enum with Value, Stop, Ping, etc. - Add cancellation tokens for task control - New lexer tokens: send, receive, channel, await, cancel, from - New AST nodes: SendMessage, ReceiveMessage, AwaitWorker, CancelWorker - Update parser for new worker syntax - Update interpreter with worker statement handling - Update WASM codegen and typechecker for new statements - Add worker tests (4 new tests) - Add workers.woke example demonstrating worker system - Update ROADMAP.md with Phase 2 completion status * Implement Phase 3 Security: capability-based security and persistent consent storage - Add CapabilityRegistry with support for FileRead, FileWrite, Execute, Network, Environment, Process, SystemInfo, Crypto, Clipboard, Notify capabilities - Implement wildcard capability matching (e.g., file:read:* grants all file reads) - Add audit logging for all capability requests, grants, denials, and revocations - Create ConsentStore for persistent storage of consent decisions - Support consent durations: Session, Day, Week, Forever, Once - Add security.woke example demonstrating superpower declarations * Implement Phase 4 Standard Library for WokeLang Add comprehensive standard library with the following modules: - std.math: Mathematical functions (abs, sqrt, pow, sin, cos, tan, floor, ceil, round, min, max, random, pi, e) - std.io: File I/O with consent-based access control (readFile, writeFile, appendFile, exists, delete, listDir, createDir, readLine) - std.json: JSON parsing and generation (parse, stringify, get, set) - std.time: Date/time handling (now, timestamp, format, parse, sleep, elapsed) - std.net: HTTP networking with consent (httpGet, httpPost, download) Key features: - StdlibRegistry for function registration and lookup - Capability-based security integration for I/O and network operations - Simple JSON parser/serializer without external dependencies - Basic HTTP client using TCP sockets (no TLS for now) - Elapsed timer with thread-local storage - Add Record variant to Value for JSON objects Includes stdlib_demo.woke example demonstrating all modules. --------- Co-authored-by: Claude <noreply@anthropic.com>
1 parent 7cc1410 commit 501634e

7 files changed

Lines changed: 2117 additions & 0 deletions

File tree

examples/stdlib_demo.woke

Lines changed: 91 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,91 @@
1+
// Example: WokeLang Standard Library Demo
2+
// Demonstrates usage of std.math, std.time, std.json, std.io, and std.net
3+
4+
thanks to {
5+
"WokeLang Standard Library" -> "For comprehensive utilities";
6+
}
7+
8+
to main() {
9+
hello "Standard Library Demo";
10+
11+
print("=== Math Functions ===");
12+
// Math doesn't require capabilities
13+
print("abs(-42) = " + toString(std.math.abs(-42)));
14+
print("sqrt(16) = " + toString(std.math.sqrt(16)));
15+
print("pow(2, 8) = " + toString(std.math.pow(2, 8)));
16+
print("floor(3.7) = " + toString(std.math.floor(3.7)));
17+
print("ceil(3.2) = " + toString(std.math.ceil(3.2)));
18+
print("min(10, 5) = " + toString(std.math.min(10, 5)));
19+
print("max(10, 5) = " + toString(std.math.max(10, 5)));
20+
print("PI = " + toString(std.math.pi()));
21+
print("E = " + toString(std.math.e()));
22+
23+
print("");
24+
print("=== Time Functions ===");
25+
remember now_ms = std.time.now();
26+
print("Current time (ms): " + toString(now_ms));
27+
28+
remember formatted = std.time.format(now_ms, "%Y-%m-%d %H:%M:%S");
29+
print("Formatted: " + formatted);
30+
31+
// Elapsed timer example
32+
std.time.elapsed("start", "demo_timer");
33+
// Do some work...
34+
std.time.sleep(100);
35+
remember elapsed_ms = std.time.elapsed("stop", "demo_timer");
36+
print("Elapsed: " + toString(elapsed_ms) + "ms");
37+
38+
print("");
39+
print("=== JSON Functions ===");
40+
// Parse JSON
41+
remember json_str = "{\"name\": \"WokeLang\", \"version\": 1, \"features\": [\"consent\", \"safety\"]}";
42+
remember parsed = std.json.parse(json_str);
43+
print("Parsed JSON: " + toString(parsed));
44+
45+
// Get nested value
46+
remember name = std.json.get(parsed, "name");
47+
print("Name: " + toString(name));
48+
49+
// Stringify
50+
remember back_to_string = std.json.stringify(parsed);
51+
print("Stringified: " + back_to_string);
52+
53+
print("");
54+
print("=== File I/O (requires consent) ===");
55+
// These operations require file:read and file:write capabilities
56+
// In a real program, the user would be prompted for consent
57+
58+
only if okay {
59+
superpower file:write;
60+
61+
remember path = "/tmp/wokelang_demo.txt";
62+
std.io.writeFile(path, "Hello from WokeLang!");
63+
print("Wrote to: " + path);
64+
65+
remember content = std.io.readFile(path);
66+
print("Read back: " + content);
67+
68+
remember exists = std.io.exists(path);
69+
print("File exists: " + toString(exists));
70+
71+
std.io.delete(path);
72+
print("File deleted");
73+
}
74+
75+
print("");
76+
print("=== Network (requires consent) ===");
77+
// Network operations require network:* capability
78+
// Note: HTTPS requires TLS library
79+
80+
only if okay {
81+
superpower network:*;
82+
83+
// HTTP GET (would work with a real HTTP server)
84+
// remember response = std.net.httpGet("http://example.com");
85+
// print("Response: " + response);
86+
87+
print("Network functions available but require actual server");
88+
}
89+
90+
goodbye "Demo complete!";
91+
}

src/stdlib/io.rs

Lines changed: 287 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,287 @@
1+
//! WokeLang Standard Library - I/O Module
2+
//!
3+
//! File I/O operations that require explicit consent through capabilities.
4+
5+
use crate::interpreter::Value;
6+
use crate::security::{Capability, CapabilityRegistry};
7+
use super::{check_arity, check_arity_range, expect_string, StdlibError};
8+
use std::fs;
9+
use std::io::{self, BufRead, Write};
10+
use std::path::PathBuf;
11+
12+
/// Helper to require file read capability
13+
fn require_read(path: &str, caps: &mut CapabilityRegistry) -> Result<(), StdlibError> {
14+
let cap = Capability::FileRead(Some(PathBuf::from(path)));
15+
if caps.request("stdlib", &cap).is_err() {
16+
Err(StdlibError::PermissionDenied(format!(
17+
"File read access denied: {}",
18+
path
19+
)))
20+
} else {
21+
Ok(())
22+
}
23+
}
24+
25+
/// Helper to require file write capability
26+
fn require_write(path: &str, caps: &mut CapabilityRegistry) -> Result<(), StdlibError> {
27+
let cap = Capability::FileWrite(Some(PathBuf::from(path)));
28+
if caps.request("stdlib", &cap).is_err() {
29+
Err(StdlibError::PermissionDenied(format!(
30+
"File write access denied: {}",
31+
path
32+
)))
33+
} else {
34+
Ok(())
35+
}
36+
}
37+
38+
/// Read entire file contents as a string
39+
pub fn read_file(args: &[Value], caps: &mut CapabilityRegistry) -> Result<Value, StdlibError> {
40+
check_arity(args, 1)?;
41+
let path = expect_string(&args[0], "path")?;
42+
43+
require_read(&path, caps)?;
44+
45+
match fs::read_to_string(&path) {
46+
Ok(contents) => Ok(Value::String(contents)),
47+
Err(e) => Err(StdlibError::IoError(e.to_string())),
48+
}
49+
}
50+
51+
/// Write string contents to a file
52+
pub fn write_file(args: &[Value], caps: &mut CapabilityRegistry) -> Result<Value, StdlibError> {
53+
check_arity(args, 2)?;
54+
let path = expect_string(&args[0], "path")?;
55+
let contents = expect_string(&args[1], "contents")?;
56+
57+
require_write(&path, caps)?;
58+
59+
match fs::write(&path, &contents) {
60+
Ok(()) => Ok(Value::Bool(true)),
61+
Err(e) => Err(StdlibError::IoError(e.to_string())),
62+
}
63+
}
64+
65+
/// Append string contents to a file
66+
pub fn append_file(args: &[Value], caps: &mut CapabilityRegistry) -> Result<Value, StdlibError> {
67+
check_arity(args, 2)?;
68+
let path = expect_string(&args[0], "path")?;
69+
let contents = expect_string(&args[1], "contents")?;
70+
71+
require_write(&path, caps)?;
72+
73+
use std::fs::OpenOptions;
74+
match OpenOptions::new().create(true).append(true).open(&path) {
75+
Ok(mut file) => match file.write_all(contents.as_bytes()) {
76+
Ok(()) => Ok(Value::Bool(true)),
77+
Err(e) => Err(StdlibError::IoError(e.to_string())),
78+
},
79+
Err(e) => Err(StdlibError::IoError(e.to_string())),
80+
}
81+
}
82+
83+
/// Check if a file or directory exists
84+
pub fn exists(args: &[Value], caps: &mut CapabilityRegistry) -> Result<Value, StdlibError> {
85+
check_arity(args, 1)?;
86+
let path = expect_string(&args[0], "path")?;
87+
88+
// exists only needs read capability to check
89+
require_read(&path, caps)?;
90+
91+
Ok(Value::Bool(std::path::Path::new(&path).exists()))
92+
}
93+
94+
/// Delete a file
95+
pub fn delete(args: &[Value], caps: &mut CapabilityRegistry) -> Result<Value, StdlibError> {
96+
check_arity(args, 1)?;
97+
let path = expect_string(&args[0], "path")?;
98+
99+
require_write(&path, caps)?;
100+
101+
match fs::remove_file(&path) {
102+
Ok(()) => Ok(Value::Bool(true)),
103+
Err(e) => Err(StdlibError::IoError(e.to_string())),
104+
}
105+
}
106+
107+
/// List directory contents
108+
pub fn list_dir(args: &[Value], caps: &mut CapabilityRegistry) -> Result<Value, StdlibError> {
109+
check_arity(args, 1)?;
110+
let path = expect_string(&args[0], "path")?;
111+
112+
require_read(&path, caps)?;
113+
114+
match fs::read_dir(&path) {
115+
Ok(entries) => {
116+
let files: Vec<Value> = entries
117+
.filter_map(|e| e.ok())
118+
.map(|e| Value::String(e.file_name().to_string_lossy().to_string()))
119+
.collect();
120+
Ok(Value::Array(files))
121+
}
122+
Err(e) => Err(StdlibError::IoError(e.to_string())),
123+
}
124+
}
125+
126+
/// Create a directory (and parents if needed)
127+
pub fn create_dir(args: &[Value], caps: &mut CapabilityRegistry) -> Result<Value, StdlibError> {
128+
check_arity(args, 1)?;
129+
let path = expect_string(&args[0], "path")?;
130+
131+
require_write(&path, caps)?;
132+
133+
match fs::create_dir_all(&path) {
134+
Ok(()) => Ok(Value::Bool(true)),
135+
Err(e) => Err(StdlibError::IoError(e.to_string())),
136+
}
137+
}
138+
139+
/// Read a line from stdin (interactive)
140+
pub fn read_line(args: &[Value], caps: &mut CapabilityRegistry) -> Result<Value, StdlibError> {
141+
check_arity_range(args, 0, 1)?;
142+
143+
// Print optional prompt
144+
if let Some(prompt) = args.first() {
145+
let prompt_str = expect_string(prompt, "prompt")?;
146+
print!("{}", prompt_str);
147+
io::stdout().flush().ok();
148+
}
149+
150+
let stdin = io::stdin();
151+
let mut line = String::new();
152+
match stdin.lock().read_line(&mut line) {
153+
Ok(_) => Ok(Value::String(line.trim_end_matches('\n').to_string())),
154+
Err(e) => Err(StdlibError::IoError(e.to_string())),
155+
}
156+
}
157+
158+
#[cfg(test)]
159+
mod tests {
160+
use super::*;
161+
use std::env;
162+
163+
fn test_caps() -> CapabilityRegistry {
164+
CapabilityRegistry::permissive()
165+
}
166+
167+
fn temp_file(name: &str) -> String {
168+
env::temp_dir()
169+
.join(format!("wokelang_test_{}", name))
170+
.to_string_lossy()
171+
.to_string()
172+
}
173+
174+
#[test]
175+
fn test_write_and_read() {
176+
let mut caps = test_caps();
177+
let path = temp_file("io_test.txt");
178+
179+
// Write
180+
let write_result = write_file(
181+
&[Value::String(path.clone()), Value::String("Hello, WokeLang!".to_string())],
182+
&mut caps,
183+
);
184+
assert!(write_result.is_ok());
185+
186+
// Read
187+
let read_result = read_file(&[Value::String(path.clone())], &mut caps);
188+
assert_eq!(
189+
read_result.unwrap(),
190+
Value::String("Hello, WokeLang!".to_string())
191+
);
192+
193+
// Cleanup
194+
let _ = fs::remove_file(&path);
195+
}
196+
197+
#[test]
198+
fn test_exists() {
199+
let mut caps = test_caps();
200+
let path = temp_file("exists_test.txt");
201+
202+
// Should not exist yet
203+
let result = exists(&[Value::String(path.clone())], &mut caps);
204+
assert_eq!(result.unwrap(), Value::Bool(false));
205+
206+
// Create file
207+
fs::write(&path, "test").unwrap();
208+
209+
// Should exist now
210+
let result = exists(&[Value::String(path.clone())], &mut caps);
211+
assert_eq!(result.unwrap(), Value::Bool(true));
212+
213+
// Cleanup
214+
let _ = fs::remove_file(&path);
215+
}
216+
217+
#[test]
218+
fn test_append() {
219+
let mut caps = test_caps();
220+
let path = temp_file("append_test.txt");
221+
222+
// Write initial content
223+
write_file(
224+
&[Value::String(path.clone()), Value::String("Hello".to_string())],
225+
&mut caps,
226+
)
227+
.unwrap();
228+
229+
// Append
230+
append_file(
231+
&[Value::String(path.clone()), Value::String(", World!".to_string())],
232+
&mut caps,
233+
)
234+
.unwrap();
235+
236+
// Read and verify
237+
let result = read_file(&[Value::String(path.clone())], &mut caps);
238+
assert_eq!(
239+
result.unwrap(),
240+
Value::String("Hello, World!".to_string())
241+
);
242+
243+
// Cleanup
244+
let _ = fs::remove_file(&path);
245+
}
246+
247+
#[test]
248+
fn test_delete() {
249+
let mut caps = test_caps();
250+
let path = temp_file("delete_test.txt");
251+
252+
// Create file
253+
fs::write(&path, "test").unwrap();
254+
assert!(std::path::Path::new(&path).exists());
255+
256+
// Delete
257+
let result = delete(&[Value::String(path.clone())], &mut caps);
258+
assert!(result.is_ok());
259+
assert!(!std::path::Path::new(&path).exists());
260+
}
261+
262+
#[test]
263+
fn test_create_dir_and_list() {
264+
let mut caps = test_caps();
265+
let dir_path = temp_file("test_dir");
266+
267+
// Create directory
268+
create_dir(&[Value::String(dir_path.clone())], &mut caps).unwrap();
269+
assert!(std::path::Path::new(&dir_path).is_dir());
270+
271+
// Create a file in the directory
272+
let file_path = format!("{}/test.txt", dir_path);
273+
fs::write(&file_path, "test").unwrap();
274+
275+
// List directory
276+
let result = list_dir(&[Value::String(dir_path.clone())], &mut caps);
277+
match result.unwrap() {
278+
Value::Array(files) => {
279+
assert!(files.contains(&Value::String("test.txt".to_string())));
280+
}
281+
_ => panic!("Expected array"),
282+
}
283+
284+
// Cleanup
285+
let _ = fs::remove_dir_all(&dir_path);
286+
}
287+
}

0 commit comments

Comments
 (0)