You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: README.md
+8-6Lines changed: 8 additions & 6 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -15,7 +15,9 @@
15
15
16
16
**Install:**`pip install labtrust-gym[env,plots]` then `labtrust --version` and `labtrust quick-eval`.
17
17
18
-
> **Research and simulation only** — This project is for research, benchmarks, and regression testing. It is not production medical or laboratory software. See [SECURITY.md](SECURITY.md) for scope and reporting issues.
18
+
**Documentation** — The published site at [https://fraware.github.io/LabTrust-Gym/](https://fraware.github.io/LabTrust-Gym/) covers getting started, benchmarks, coordination, PCS, and operations.
19
+
20
+
> **Research and simulation only** — This project targets research, benchmarks, and regression testing inside a simulated hospital lab environment. It excludes production medical or laboratory software. See [SECURITY.md](SECURITY.md) for scope and reporting issues.
19
21
20
22
## What is LabTrust-Gym?
21
23
@@ -29,14 +31,14 @@
29
31
30
32
**Principles**
31
33
32
-
-**Golden scenarios drive development** — Correctness is defined as passing the golden suite; the suite is the specification for regression. It does not cover all failure modes; gaps imply gaps in assured behavior.
34
+
-**Golden scenarios drive development** — Correctness is defined as passing the golden suite, which acts as the specification for regression. The suite covers the scenarios encoded in policy; uncovered failure modes remain outside assured behavior until added to golden policy.
33
35
-**Policy is data** — Invariants, tokens, reason codes, catalogue, zones live in versioned files under `policy/`.
34
-
-**No silent failure** — Missing hooks or invalid data fail loudly with reason codes.
36
+
-**Explicit failures** — Missing hooks or invalid data raise reason codes instead of continuing silently.
35
37
-**Evidence over claims** — Security and safety are evidenced by the attack suite, coordination security pack, and risk register; required_bench cells must be covered or explicitly waived.
36
38
37
-
System and threat model:[Systems and threat model](docs/architecture/systems_and_threat_model.md).
39
+
The system and threat model are described in[Systems and threat model](docs/architecture/systems_and_threat_model.md).
38
40
39
-
> **Limitation** — Passing all sim tests and gates does **not** imply production safety. Production adds distribution shift, real adversaries, key/ops failures, and environment drift. Use sim for development and regression; production assurance is the integrator's responsibility.
41
+
> **Limitation** — Passing every simulation test and gate demonstrates behavior inside the benchmark harness. Production deployments still require the integrator to account for distribution shift, live adversaries, key and operations failures, and environment drift. Use the simulation for development and regression; production assurance remains the integrator's responsibility.
40
42
41
43
---
42
44
@@ -235,7 +237,7 @@ Put CLI outputs in `labtrust_runs/` or `--out`. Exit codes, minimal smoke args,
Copy file name to clipboardExpand all lines: docs/agents/extension_development.md
+3-3Lines changed: 3 additions & 3 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -4,7 +4,7 @@ This guide describes how to extend LabTrust-Gym with your own domains, coordinat
4
4
5
5
## Integration pattern (Option B)
6
6
7
-
Installing `labtrust-gym` and shipping your own pip package (using `register_*` or entry_points, `--profile`, and `extension_packages`) is the **integration pattern** for using the stack as a platform rather than a fixed app. One minimal worked example—e.g. a package that adds a single task or coordination method—demonstrates that the stack is reusable.
7
+
Installing `labtrust-gym` and shipping your own pip package (using `register_*` or entry_points, `--profile`, and `extension_packages`) is the **integration pattern** for using the stack as an extensible platform. One minimal worked example, such as a package that adds a single task or coordination method, demonstrates that the stack is reusable.
8
8
9
9
**Minimal worked example (in-doc):** Create a package that declares one task or one coordination method via entry_point. In `pyproject.toml`:
The CLI loads your plugin and your task (or coordination method) is available. This pattern shows the stack is used as a platform, not only as a fixed application.
22
+
The CLI loads your plugin and your task (or coordination method) is available. This pattern treats the stack as an extensible platform beyond the fixed in-repo application.
The profile `example` is defined in `policy/lab_profiles/example.v0.1.yaml` and lists `extension_packages: ["example-plugin"]`. Copy `examples/extension_example/` to start your own plugin; the only supported contract for new tasks is `BenchmarkTask` and for coordination methods is `CoordinationMethod` (see Contracts below). The extension example is installed and run in CI to verify the plugin mechanism on every commit; see [CI](../operations/ci.md).
34
34
35
-
A second minimal example, **`examples/coord_method_example/`**, registers one coordination method (`example_noop_coord`) via the `labtrust_gym.coordination_methods` entry point. It does not add a new task; use an existing task (e.g. `coord_scale`) with`--coord-method example_noop_coord`:
35
+
A second minimal example, **`examples/coord_method_example/`**, registers one coordination method (`example_noop_coord`) via the `labtrust_gym.coordination_methods` entry point. Pair it with an existing task (e.g. `coord_scale`) via`--coord-method example_noop_coord`:
Copy file name to clipboardExpand all lines: docs/agents/llm_baselines.md
+6-6Lines changed: 6 additions & 6 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -4,7 +4,7 @@ Offline-safe, **constrained and reproducible by default** LLM agent interface fo
4
4
5
5
## Relationship to the environment
6
6
7
-
The LLM agent is a **policy**: it receives observations and returns an action (with optional action_info). In **run-benchmark** (and coordination studies), those observations come from the **PettingZoo env** (LabTrustParallelEnv); the benchmark runner feeds obs to the agent and calls `env.step` with the returned action. The agent never creates or steps the env. In the **security suite** (scenario_ref, llm_attacker), the same agent is tested with **synthetic observations** and the env is not run. See [Simulation, LLMs, and agentic systems](../architecture/simulation_llm_agentic.md).
7
+
The LLM agent is a **policy** that receives observations and returns an action (with optional action_info). In **run-benchmark** (and coordination studies), those observations come from the **PettingZoo env** (LabTrustParallelEnv); the benchmark runner feeds obs to the agent and calls `env.step` with the returned action. The agent never creates or steps the env. In the **security suite** (scenario_ref, llm_attacker), the same agent is tested with **synthetic observations** and the env stays idle. See [Simulation, LLMs, and agentic systems](../architecture/simulation_llm_agentic.md).
8
8
9
9
## Design
10
10
@@ -54,7 +54,7 @@ When the engine runs with **strict_signatures**, mutating actions must be signed
54
54
-**ProviderBackend** (protocol in `baselines.llm.provider`): live backends return **ActionProposal** dicts via `propose_action(context)`. No optional deps; engine logic depends only on this interface.
55
55
-**Capability flags**: `supports_structured_outputs` (bool), `supports_tool_calls` (bool). Backends set them; best quality when structured outputs are supported (e.g. OpenAI with `response_format`).
56
56
-**Per-provider code** is behind optional extras: **llm_openai** (OpenAILiveBackend), **llm_anthropic** (later). **OllamaLiveBackend** (no extra) for local Ollama. Add new providers without touching engine logic.
57
-
-**Fallback path**: If a provider does not support strict JSON schema, the agent uses **parse_utils.extract_first_json_object** to extract a JSON object from raw text, then validate + NOOP on failure. Audit field **used_structured_outputs** in LLM_DECISION records whether the backend natively returned schema-conforming output (preferred) or fallback was used.
57
+
-**Fallback path.** When a provider lacks strict JSON schema support, the agent uses **parse_utils.extract_first_json_object** to extract a JSON object from raw text, then validate + NOOP on failure. Audit field **used_structured_outputs** in LLM_DECISION records whether the backend natively returned schema-conforming output or the fallback path ran.
58
58
59
59
## Backends
60
60
@@ -85,7 +85,7 @@ When the engine runs with **strict_signatures**, mutating actions must be signed
-**Optional** backend for llm_offline only; not required for CI. Chooses from **allowed_actions** using a fixed **preference order** (e.g. START_RUN, QUEUE_RUN, MOVE, TICK, NOOP); seeded RNG breaks ties.
88
+
-**Optional** backend for llm_offline only; CI runs without it. Chooses from **allowed_actions** using a fixed **preference order** (e.g. START_RUN, QUEUE_RUN, MOVE, TICK, NOOP); seeded RNG breaks ties.
89
89
- Same interface as DeterministicConstrainedBackend. Use when you want a slightly more "productive" deterministic baseline without fixtures or live LLM. Enable via CLI: `labtrust run-benchmark --llm-backend deterministic_policy_v1 --pipeline-mode llm_offline`.
@@ -112,20 +112,20 @@ When the engine runs with **strict_signatures**, mutating actions must be signed
112
112
-**Constrained decoder** (`src/labtrust_gym/baselines/llm/decoder.py`): At **decode time** (before env step), validates schema, **requires rationale**, restricts **action_type** to `allowed_actions`, and optionally checks zone/device. Refuses impossible actions (RBAC/devices/zones) so the agent cannot propose them without being rejected at decode time.
113
113
-**Shield** (`src/labtrust_gym/baselines/llm/shield.py`): After decode, filters through **RBAC** (context) and **signature required** (when `strict_signatures`). Token validity is left to the engine.
114
114
- If blocked (decode or shield): returns safe NOOP, `_shield_filtered=True`, and **reason_code** (e.g. `MISSING_RATIONALE`, `RBAC_ACTION_DENY`, `SIG_MISSING`). Step output records **LLM_ACTION_FILTERED** in emits and `blocked_reason_code`.
115
-
-**Uncertainty-aware refusal**: When the policy enables `refuse_below_confidence` (via the shield’s `build_policy_summary(..., refuse_below_confidence=True, refusal_confidence_threshold=0.6)`), the **decoder** compares the candidate’s `confidence` (ActionProposal) to the threshold before the env step. If `confidence < refusal_confidence_threshold`, the decoder returns NOOP with **reason_code**`RC_LLM_LOW_CONFIDENCE_REFUSAL` and does not execute the proposed action. This is opt-in; with refusal disabled, behaviour is unchanged. The threshold is configurable so that “act only when P(correct) > theta” can be enforced without code change.
115
+
-**Uncertainty-aware refusal.** When the policy enables `refuse_below_confidence` (via the shield’s `build_policy_summary(..., refuse_below_confidence=True, refusal_confidence_threshold=0.6)`), the **decoder** compares the candidate’s `confidence` (ActionProposal) to the threshold before the env step. Below `refusal_confidence_threshold`, the decoder returns NOOP with **reason_code**`RC_LLM_LOW_CONFIDENCE_REFUSAL` and skips the proposed action. Refusal is opt-in; with refusal disabled, behaviour is unchanged. The threshold is configurable so that “act only when P(correct) > theta” can be enforced without code change.
116
116
-**multi_site_stat and insider_key_misuse** run with **llm_safe_v1** deterministically: `run_benchmark(..., use_llm_safe_v1_ops=True)` uses `--llm-backend deterministic_constrained` by default (seeded RNG). Use `--llm-backend deterministic` for FixtureBackend (requires recorded fixtures). insider_key_misuse demonstrates signature/RBAC attack containment with the LLM baseline.
117
117
118
118
## Deterministic vs non-deterministic
119
119
120
120
-**Deterministic**: **FixtureBackend** (offline lookup from fixtures; run **record-llm-fixtures** with network to populate) or **DeterministicConstrainedBackend(seed)** / **MockDeterministicBackendV2(canned=...)**. Same inputs ⇒ same actions. Required for CI; deterministic CI never performs network calls.
121
-
-**Non-deterministic**:**OpenAIHostedBackend**, **OpenAILiveBackend**, or local Ollama. Use `--llm-backend openai_hosted` or `openai_live` with `--allow-network` for live runs. Do not compare metrics across runs without fixing seed/temperature on the provider side.
121
+
-**Non-deterministic.****OpenAIHostedBackend**, **OpenAILiveBackend**, or local Ollama. Use `--llm-backend openai_hosted` or `openai_live` with `--allow-network` for live runs. Fix seed and temperature on the provider side before you compare metrics across runs.
122
122
123
123
## llm_offline in practice
124
124
125
125
What **llm_offline** gives you in practice (no overselling):
126
126
127
127
-**FixtureBackend:** Agent path only. Keyed by SHA-256 digest of the messages passed to `generate(messages)`. Coverage is one task (or a few) recorded manually via `record-llm-fixtures`. Coordination fixtures are optional: use `record-coordination-fixtures` to capture proposal/bid responses, then pass `coord_fixtures_path` to replay in llm_offline (see Coordination fixtures below).
128
-
-**DeterministicConstrainedBackend:**Purpose is schema/CI and pipeline correctness, not policy quality. It picks a random allowed action (seeded RNG) with a fixed rationale. Use it when you need the LLM path to run in CI without fixtures or network; do not use it to evaluate "how good" the LLM is at the task.
128
+
-**DeterministicConstrainedBackend.**Targets schema/CI and pipeline correctness. It picks a random allowed action (seeded RNG) with a fixed rationale. Use it when you need the LLM path to run in CI without fixtures or network; reserve live backends for policy-quality evaluation.
129
129
-**Fault injection:** Agent and coordination (proposal/bid) paths support the same fault model as repair (policy/llm/llm_fault_model.v0.1.yaml) in llm_offline. Replay-from-live: capture a live run into agent and (optionally) coordination fixtures, then replay with fixture backends and same seeds for regression and audit.
0 commit comments