Skip to content

Commit e52fd01

Browse files
docs: add README and LICENSE
1 parent 9c6ed33 commit e52fd01

2 files changed

Lines changed: 156 additions & 0 deletions

File tree

LICENSE

Lines changed: 21 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,21 @@
1+
MIT License
2+
3+
Copyright (c) 2025 Practical Mind
4+
5+
Permission is hereby granted, free of charge, to any person obtaining a copy
6+
of this software and associated documentation files (the "Software"), to deal
7+
in the Software without restriction, including without limitation the rights
8+
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
9+
copies of the Software, and to permit persons to whom the Software is
10+
furnished to do so, subject to the following conditions:
11+
12+
The above copyright notice and this permission notice shall be included in all
13+
copies or substantial portions of the Software.
14+
15+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
16+
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
17+
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
18+
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
19+
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
20+
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
21+
SOFTWARE.

README.md

Lines changed: 135 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,135 @@
1+
# Gateframe
2+
3+
Behavioral validation for LLM outputs. Not schema validation - tools like Instructor and Pydantic AI already handle that. gateframe validates whether an LLM output *behaves correctly* given the context it was generated in, stays within decision boundaries, and fails in ways your system can recover from.
4+
5+
```python
6+
from pydantic import BaseModel
7+
from gateframe.core.contract import ValidationContract
8+
from gateframe.core.failure import FailureMode
9+
from gateframe.rules.structural import StructuralRule
10+
11+
class TriageOutput(BaseModel):
12+
severity: str
13+
recommendation: str
14+
confidence: float
15+
16+
contract = ValidationContract(
17+
name="triage_check",
18+
rules=[
19+
StructuralRule(schema=TriageOutput),
20+
],
21+
)
22+
23+
result = contract.validate({
24+
"severity": "high",
25+
"recommendation": "Escalate to on-call engineer",
26+
"confidence": 0.92,
27+
})
28+
29+
assert result.passed
30+
```
31+
32+
When validation fails, you get structured failure records - not just a boolean:
33+
34+
```python
35+
result = contract.validate({"severity": "high"})
36+
37+
assert not result.passed
38+
for failure in result.failures:
39+
print(failure.message)
40+
# structural:TriageOutput failed: recommendation: Field required; confidence: Field required.
41+
```
42+
43+
## Failure modes
44+
45+
gateframe distinguishes four failure modes instead of binary pass/fail:
46+
47+
**HARD_FAIL** - Stop execution, escalate. The output is wrong in a way that cannot be auto-recovered.
48+
49+
```python
50+
# Missing required fields in a medical triage output
51+
StructuralRule(schema=TriageOutput, failure_mode=FailureMode.HARD_FAIL)
52+
```
53+
54+
**SOFT_FAIL** - Flag the output and continue with degraded confidence. Something is off but not critical.
55+
56+
```python
57+
# Output has all fields but confidence is below threshold
58+
StructuralRule(schema=TriageOutput, failure_mode=FailureMode.SOFT_FAIL)
59+
```
60+
61+
**RETRY** - Re-prompt with failure context. The output is fixable by trying again.
62+
63+
```python
64+
# Malformed JSON that might parse on a second attempt
65+
StructuralRule(schema=TriageOutput, failure_mode=FailureMode.RETRY)
66+
```
67+
68+
**SILENT_FAIL** - The most dangerous kind. Output looks valid but violates semantic or boundary rules. gateframe makes these visible instead of letting them pass through.
69+
70+
## When to use gateframe
71+
72+
Use it when:
73+
- You need to validate LLM outputs beyond schema checks - behavioral contracts, decision boundaries, cross-step propagation
74+
- You need structured failure records for debugging production incidents
75+
- You want to distinguish between "retry this", "flag this", and "stop everything"
76+
77+
Don't use it when:
78+
- You only need schema extraction from LLM outputs (use Instructor or Pydantic AI)
79+
- You need offline model evaluation or benchmarking (use DeepEval or RAGAS)
80+
- You need content safety filtering (use a dedicated guardrails tool)
81+
82+
## Installation
83+
84+
```bash
85+
pip install gateframe
86+
```
87+
88+
For development:
89+
90+
```bash
91+
git clone https://github.com/practicalmind-ai/gateframe.git
92+
cd gateframe
93+
pip install -e ".[dev]"
94+
```
95+
96+
## Quickstart
97+
98+
```python
99+
from pydantic import BaseModel
100+
from gateframe.core.contract import ValidationContract
101+
from gateframe.rules.structural import StructuralRule
102+
from gateframe.audit.log import AuditLog
103+
104+
# Define your expected output schema
105+
class AgentResponse(BaseModel):
106+
action: str
107+
reasoning: str
108+
109+
# Create a contract with rules
110+
contract = ValidationContract(
111+
name="agent_response_check",
112+
rules=[StructuralRule(schema=AgentResponse)],
113+
)
114+
115+
# Validate LLM output
116+
result = contract.validate({
117+
"action": "send_email",
118+
"reasoning": "User requested a follow-up email to the client.",
119+
})
120+
121+
# Log the validation event
122+
audit = AuditLog()
123+
audit.record(result)
124+
```
125+
126+
## Running tests
127+
128+
```bash
129+
pip install -e ".[dev]"
130+
pytest -q
131+
```
132+
133+
## License
134+
135+
This project is licensed under the MIT License - see the [LICENSE](LICENSE) file for details.

0 commit comments

Comments
 (0)