Add depth limits

Author: mitchpaulus

CHANGELOG.md

@@ -17,6 +17,9 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
   existing `?` unwrap for a die-loud form, e.g.
   `"pkgs.json" parseJson tryAs Manifest ? :packages? (:name?) map`.
   A `tryAs` whose source type could never be the target is a type error.
+  Validation depth is limited to 1024 nested levels; exceeding it (a cyclic
+  value, a cyclic `type` declaration, or absurdly deep data) fails the script
+  with a clear error rather than producing `none`.
 
 - CLI completions for `cargo`: subcommands (including installed third-party
   ones via `cargo --list`), per-subcommand options, and dynamic values for

ai/tryas_review.md

@@ -0,0 +1,119 @@
+# tryAs review — outstanding items
+
+Critical review of the `try-as` branch (`7dce9f5 Implement tryAs`), 2026-07-09.
+The unbounded-recursion crashes found in this review are already **fixed** on the
+branch (depth budget of 1024 in `validateObj`, see `maxTryAsValidateDepth` in
+`mshell/TypeExpr.go`, with regression tests in `tests/fail/tryas_cyclic_type.msh`,
+`tests/fail/tryas_cyclic_value.msh`, `tests/fail/tryas_depth_exceeded.msh`, and
+`tests/success/tryas_depth.msh`). Everything below is still open and needs a
+decision.
+
+## 1. The recursive-type contradiction (design decision needed)
+
+The runtime and the checker disagree about whether recursive named types exist.
+
+- Runtime: `tryAs Name` resolves names late through `state.typeDefs`, so
+  `type Tree = {value: int, kids?: [Tree]}` validates finite values correctly.
+  Verified: `type A = [B]` / `type B = [A]` works at runtime.
+- Checker: `CheckProgram` pre-pass 1 resolves each `type` body in file order
+  *before* declaring the name, so any self- or forward-reference is
+  "unknown type" — recursive types are statically unrepresentable.
+
+Consequence: recursive types (the natural way to type JSON trees, which is the
+flagship `tryAs` use case) work only when you *don't* pass `--check-types`.
+The two halves should agree. Options:
+
+- **(a)** Register all type names before resolving bodies so the checker admits
+  recursion. The arena/TypeId side needs a story for the cycle (a `TidNamed`
+  indirection or iterative resolution). Runtime is already safe now that the
+  depth budget exists. This is the option that makes the flagship use case work.
+- **(b)** Make the runtime reject what the checker rejects: resolve/validate the
+  type AST eagerly at `type` declaration time and fail on unknown names. Loses
+  recursive types entirely.
+
+## 2. JSON integer footgun passes the checker silently (high)
+
+`parseJson` produces a `float` for every JSON number, so any `int` inside a
+`tryAs` target is **always-none** against JSON data — and `--check-types` says
+nothing. Verified:
+
+```mshell
+"[1, 2]" parseJson tryAs [int]    # always none; checker silent
+```
+
+The docs warn about this (good), but the branch already added a checker error
+for statically-impossible casts (`5 tryAs str`); this is the same bug class and
+the one users will actually hit, since `{count: int}` is what everyone writes
+first. Options:
+
+- Checker warning when the cast source is the JSON output union and the target
+  contains `int` anywhere.
+- Accept integral floats for `int` in `validateObj` — rejected in the branch's
+  own comments because validation deliberately mirrors `match` type-arm
+  semantics; changing it here without changing `match` creates a new
+  inconsistency.
+- Make `parseJson` produce `int` for integral numbers — biggest change, fixes
+  the root cause, affects existing scripts that assume float.
+
+## 3. Wildcard shapes validate nothing at runtime (medium)
+
+`TypeShapeExpr.validateObj` never reads `a.Wildcard`. Verified inconsistency:
+
+```mshell
+{"a": 1, "b": "x"} tryAs {a: float, *: float}   # just  — wildcard ignored
+{"a": 1, "b": "x"} tryAs {*: float}             # none  — dict form checks values
+```
+
+For static `as` dropping the wildcard was harmless (width subtyping already
+allows extra keys — comment at the `TypeShapeExpr` declaration in
+`mshell/TypeExpr.go`), but `tryAs` makes runtime claims: a user who writes
+`*: float` is asking for enforcement they silently don't get. Options:
+
+- Validate non-field keys against the wildcard type in
+  `TypeShapeExpr.validateObj` (my preference: it's what the syntax says).
+- Reject wildcard shapes as `tryAs` targets with a clear error (fail closed).
+
+Either is better than the current silent half-validation.
+
+## 4. Quotation targets are half-open (medium)
+
+`tryAs (int -- int)` checks only that the value is a quotation
+(`TypeQuoteExpr.validateObj`) — signatures are not verified, so
+`("hi" wl) tryAs (int -- int)` produces `just`, and the checker afterward
+trusts `Maybe[(int -- int)]` for a value with a different signature. That is a
+static-soundness hole shaped like a checked cast. It is documented, but
+consider failing closed instead: reject quote types in `tryAs` targets with an
+error ("quotation signatures cannot be verified at runtime"), rather than
+issuing a certificate the runtime can't back. If kind-only checking is kept,
+it stays a documented sharp edge.
+
+## 5. Union arm errors are value-dependent (low)
+
+`1 tryAs int | Bogus` never reports the unknown type `Bogus` (first arm
+matches, short-circuit); a value that matches no arm hits the error instead.
+So whether a malformed type expression is diagnosed depends on the data. A
+one-time eager walk of the type AST at cast time (validating all names/arity)
+would make the errors deterministic. Cheap to do now; confusing bug reports
+later if skipped.
+
+## 6. Notes, no action required
+
+- `validateObj`'s `case Maybe:` (value, non-pointer) branch is dead code — the
+  codebase only ever pushes `*Maybe`. Harmless.
+- The `just` result retains the original mutable object; a post-validation
+  mutation through an alias (`append` on a still-reachable inner list) can
+  invalidate the certified type. Inherent to the language's aliasing
+  semantics, not specific to this branch; maybe worth one sentence in the
+  docs next to "parse, don't validate".
+- Lexer edges verified good: `tryas`, `tryAsX`, `tryA`, `trying` all stay
+  LITERAL; bare `tryAs` and `Maybe`-without-argument produce clean parse
+  errors.
+- `builtinNamedTypes` table (single source for checker resolution + runtime
+  check) and the `TypeExpression` interface forcing `validateObj` on every
+  node kind are both good structure — keep that pattern for future node kinds.
+- Test-coverage suggestions once decisions above land: wildcard-shape
+  behavior (whichever way it goes), a quotation-target case, nested
+  `Maybe[Maybe[T]]`, and a `tryAs` inside a quotation in `tests/success/`
+  (works today — verified `(tryAs int ...) map` — but only top-level uses are
+  covered, and quotations dispatch through the separate `evaluateItems` path
+  in `mshell/Evaluator.go`).

doc/mshell.md

@@ -590,6 +590,7 @@ Shape validation allows extra keys, requires all non-optional fields, and valida
 A `none` conforms to `Maybe[T]` for every `T`; quotation types are checked by kind only (signatures are not verified at runtime).
 Named types must be declared before the cast runs.
 Note that `parseJson` produces a `float` for every JSON number, so validate JSON numbers as `float`, not `int`.
+Validation depth is limited to 1024 nested levels; exceeding it (a cyclic value, a cyclic `type` declaration, or absurdly deep data) fails the script with a clear error rather than producing `none`.
 
 Dictionary types are split into homogeneous dictionaries and shapes.
 A homogeneous dictionary is for dynamic keys where every value has the same type.

doc/type_system.inc.html

@@ -189,6 +189,7 @@ <h2 id="type-system-tryas">Checked casts with <code>tryAs</code> <a class="secti
 A <code>none</code> conforms to <code>Maybe[T]</code> for every <code>T</code>; a <code>just</code> validates its inner value.
 Quotation types are checked by kind only — signatures are not verified at runtime.
 A named type must have been declared before the cast runs.
+Validation depth is limited to 1024 nested levels (structural nesting, named-type expansion, and union dispatch each consume one); exceeding the limit fails the script with a clear error rather than producing <code>none</code>, which turns a cyclic value or a cyclic <code>type</code> declaration into a diagnosable failure.
 </p>
 
 <h1 id="type-system-dictionaries">Dictionaries <a class="section-link" href="#type-system-dictionaries" aria-label="Permalink">§</a> <a class="back-to-top" href="#type-system-top">Back to top</a></h1>

mshell/Evaluator.go

@@ -947,7 +947,7 @@ func (state *EvalState) processTryAsCast(cast *MShellTryAsCast, stack *MShellSta
 	if err != nil {
 		return state.FailWithMessage(fmt.Sprintf("%d:%d: 'tryAs' requires a value on the stack.\n", cast.TryAsToken.Line, cast.TryAsToken.Column))
 	}
-	conforms, verr := cast.Target.validateObj(state, obj)
+	conforms, verr := cast.Target.validateObj(state, obj, maxTryAsValidateDepth)
 	if verr != nil {
 		return state.FailWithMessage(fmt.Sprintf("%d:%d: %s\n", cast.TryAsToken.Line, cast.TryAsToken.Column, verr.Error()))
 	}

mshell/TypeExpr.go

@@ -75,11 +75,32 @@ func (parser *MShellParser) skipToShapeSync() {
 // introduced without implementing runtime validation — the program will
 // not compile. validateObj is the runtime half of `tryAs`: it reports
 // whether a value conforms to the type. Its error return is reserved for
-// malformed casts (an unknown type name), which fail the script rather
-// than producing a `none`.
+// malformed casts (an unknown type name, or a blown depth budget), which
+// fail the script rather than producing a `none`.
+//
+// depth is the remaining recursion budget. Every node checks it on entry
+// and passes depth-1 to child validations, so the walk is bounded even
+// against a cyclic value (a list appended to itself) or a runtime type
+// environment with a cyclic declaration (`type Loop = Loop`), both of
+// which would otherwise overflow the Go stack and kill the process.
+// Entry points start at maxTryAsValidateDepth.
 type TypeExpression interface {
 	MShellParseItem
-	validateObj(state *EvalState, obj MShellObject) (bool, error)
+	validateObj(state *EvalState, obj MShellObject, depth int) (bool, error)
+}
+
+// maxTryAsValidateDepth bounds validateObj recursion. Structural nesting,
+// named-type expansion, and union dispatch each consume one level, so a
+// recursive union type like `type T = [float | T]` spends about four
+// levels per level of value nesting — 1024 still admits values nested
+// ~250 deep, well past any real boundary data (serde, for comparison,
+// caps JSON at 128), while turning cyclic values and cyclic type
+// declarations into a loud, catchable script failure instead of a fatal
+// stack overflow.
+const maxTryAsValidateDepth = 1024
+
+func errTryAsDepth() error {
+	return fmt.Errorf("tryAs validation exceeded the maximum depth of %d; the value or a 'type' declaration is likely cyclic", maxTryAsValidateDepth)
 }
 
 // TypePrim is a primitive type keyword (int, float, bool, str).
@@ -855,7 +876,10 @@ var builtinNamedTypes = map[string]builtinNamedType{
 // validation allows extra keys (width subtyping), requires all
 // non-optional fields, and validates optional fields only when present.
 
-func (a *TypePrim) validateObj(state *EvalState, obj MShellObject) (bool, error) {
+func (a *TypePrim) validateObj(state *EvalState, obj MShellObject, depth int) (bool, error) {
+	if depth <= 0 {
+		return false, errTryAsDepth()
+	}
 	switch a.Tid {
 	case TidInt:
 		_, ok := obj.(MShellInt)
@@ -876,7 +900,10 @@ func (a *TypePrim) validateObj(state *EvalState, obj MShellObject) (bool, error)
 	return false, fmt.Errorf("tryAs cannot check the type '%s'", a.Tok.Lexeme)
 }
 
-func (a *TypeNamed) validateObj(state *EvalState, obj MShellObject) (bool, error) {
+func (a *TypeNamed) validateObj(state *EvalState, obj MShellObject, depth int) (bool, error) {
+	if depth <= 0 {
+		return false, errTryAsDepth()
+	}
 	if bt, ok := builtinNamedTypes[a.Name]; ok {
 		return bt.runtimeCheck(obj), nil
 	}
@@ -897,44 +924,53 @@ func (a *TypeNamed) validateObj(state *EvalState, obj MShellObject) (bool, error
 		if len(a.Args) != 1 {
 			return false, fmt.Errorf("Maybe requires a type argument: Maybe[T]")
 		}
-		return a.Args[0].validateObj(state, inner)
+		return a.Args[0].validateObj(state, inner, depth-1)
 	}
 	body, ok := state.typeDefs[a.Name]
 	if !ok {
 		return false, fmt.Errorf("unknown type '%s' in tryAs; a `type %s = ...` declaration must run before the cast", a.Name, a.Name)
 	}
-	return body.validateObj(state, obj)
+	return body.validateObj(state, obj, depth-1)
 }
 
-func (a *TypeListExpr) validateObj(state *EvalState, obj MShellObject) (bool, error) {
+func (a *TypeListExpr) validateObj(state *EvalState, obj MShellObject, depth int) (bool, error) {
+	if depth <= 0 {
+		return false, errTryAsDepth()
+	}
 	list, ok := obj.(*MShellList)
 	if !ok {
 		return false, nil
 	}
 	for _, item := range list.Items {
-		itemOk, err := a.Elem.validateObj(state, item)
+		itemOk, err := a.Elem.validateObj(state, item, depth-1)
 		if err != nil || !itemOk {
 			return itemOk, err
 		}
 	}
 	return true, nil
 }
 
-func (a *TypeDictExpr) validateObj(state *EvalState, obj MShellObject) (bool, error) {
+func (a *TypeDictExpr) validateObj(state *EvalState, obj MShellObject, depth int) (bool, error) {
+	if depth <= 0 {
+		return false, errTryAsDepth()
+	}
 	dict, ok := obj.(*MShellDict)
 	if !ok {
 		return false, nil
 	}
 	for _, val := range dict.Items {
-		valOk, err := a.Value.validateObj(state, val)
+		valOk, err := a.Value.validateObj(state, val, depth-1)
 		if err != nil || !valOk {
 			return valOk, err
 		}
 	}
 	return true, nil
 }
 
-func (a *TypeShapeExpr) validateObj(state *EvalState, obj MShellObject) (bool, error) {
+func (a *TypeShapeExpr) validateObj(state *EvalState, obj MShellObject, depth int) (bool, error) {
+	if depth <= 0 {
+		return false, errTryAsDepth()
+	}
 	dict, ok := obj.(*MShellDict)
 	if !ok {
 		return false, nil
@@ -947,22 +983,28 @@ func (a *TypeShapeExpr) validateObj(state *EvalState, obj MShellObject) (bool, e
 			}
 			return false, nil
 		}
-		fieldOk, err := f.Type.validateObj(state, val)
+		fieldOk, err := f.Type.validateObj(state, val, depth-1)
 		if err != nil || !fieldOk {
 			return fieldOk, err
 		}
 	}
 	return true, nil
 }
 
-func (a *TypeQuoteExpr) validateObj(state *EvalState, obj MShellObject) (bool, error) {
+func (a *TypeQuoteExpr) validateObj(state *EvalState, obj MShellObject, depth int) (bool, error) {
+	if depth <= 0 {
+		return false, errTryAsDepth()
+	}
 	_, ok := obj.(*MShellQuotation)
 	return ok, nil
 }
 
-func (a *TypeUnionExpr) validateObj(state *EvalState, obj MShellObject) (bool, error) {
+func (a *TypeUnionExpr) validateObj(state *EvalState, obj MShellObject, depth int) (bool, error) {
+	if depth <= 0 {
+		return false, errTryAsDepth()
+	}
 	for _, arm := range a.Arms {
-		armOk, err := arm.validateObj(state, obj)
+		armOk, err := arm.validateObj(state, obj, depth-1)
 		if err != nil {
 			return false, err
 		}

tests/fail/tryas_cyclic_type.msh

@@ -0,0 +1,4 @@
+# A cyclic runtime type declaration must exhaust the tryAs depth
+# budget and fail loudly instead of overflowing the Go stack.
+type Loop = Loop
+1 tryAs Loop

tests/fail/tryas_cyclic_type.msh.stderr

@@ -0,0 +1 @@
+4:3: tryAs validation exceeded the maximum depth of 1024; the value or a 'type' declaration is likely cyclic

tests/fail/tryas_cyclic_value.msh

@@ -0,0 +1,5 @@
+# A cyclic value (a list containing itself) reached through a
+# recursive type must exhaust the tryAs depth budget and fail
+# loudly instead of overflowing the Go stack.
+type T = [int | T]
+[1] l! @l @l append tryAs T

tests/fail/tryas_cyclic_value.msh.stderr

@@ -0,0 +1 @@
+5:21: tryAs validation exceeded the maximum depth of 1024; the value or a 'type' declaration is likely cyclic