Skip to content

Commit 121c958

Browse files
frankkilcomminsfrankkilcommins
committed
feat(docs): auto-publish framework to jentic docs site
1 parent bf124d4 commit 121c958

6 files changed

Lines changed: 483 additions & 2 deletions

File tree

‎.github/workflows/publish-spec-to-docs.yaml‎

Lines changed: 29 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,29 @@
1+
name: Publish Specification to Docs
2+
3+
on:
4+
push:
5+
branches: [main]
6+
paths:
7+
- 'docs/specification/spec.md'
8+
workflow_dispatch:
9+
10+
jobs:
11+
publish:
12+
runs-on: ubuntu-latest
13+
14+
steps:
15+
- name: Checkout repository
16+
uses: actions/checkout@v4
17+
18+
- name: Publish to jentic-docs
19+
env:
20+
GH_TOKEN: ${{ secrets.JENTIC_DOCS_TOKEN }}
21+
run: |
22+
bash scripts/publish-to-docs.sh ${{ github.sha }} ${{ secrets.JENTIC_DOCS_TOKEN }}
23+
24+
- name: Summary
25+
if: success()
26+
run: |
27+
echo "✅ Specification published successfully" >> $GITHUB_STEP_SUMMARY
28+
echo "" >> $GITHUB_STEP_SUMMARY
29+
echo "Check the created PR in jentic-docs repository." >> $GITHUB_STEP_SUMMARY

‎.gitignore‎

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -8,4 +8,5 @@ node_modules/
88
deploy/
99
deploy-preview/
1010
history
11-
coverage/
11+
coverage/
12+
.preview/

‎README.md‎

Lines changed: 9 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -23,13 +23,21 @@ JAIRF is published as a Markdown specification designed for adoption, extension,
2323

2424
Head over to [Jentic API AI-Readiness Scoring](https://jentic.com/api-score-preview) page, to see AI-readiness scoring in action.
2525

26+
## Publishing
27+
28+
Changes to `docs/specification/spec.md` on the `main` branch automatically publish to [jentic-docs](https://github.com/jentic/jentic-docs) via GitHub Actions. The workflow creates a PR with the updated specification.
29+
30+
**Requirements**: Repository secret `JENTIC_DOCS_TOKEN` (GitHub PAT with `repo` scope).
31+
32+
**Local testing**: `./scripts/publish-to-docs.sh --dry-run test-sha dummy-token`
33+
2634
## Maintainers
2735

2836
See [MAINTAINERS.md](./MAINTAINERS.md).
2937

3038
## Contributing
3139

32-
A `CONTRIBUTING.md` guide will be added shortly.
40+
A `CONTRIBUTING.md` guide will be added shortly.
3341
Issues, discussions, and pull requests are welcome.
3442

3543
## License

‎docs/publishing/overview-template.md‎

Lines changed: 112 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,112 @@
1+
# API AI-Readiness Framework
2+
3+
**Current Version**: {{VERSION}}
4+
**Last Updated**: {{TIMESTAMP}}
5+
6+
[View Full Specification](specification.md){ .md-button .md-button--primary }
7+
8+
## What is the API AI-Readiness Framework?
9+
10+
The Jentic API AI-Readiness Framework provides a standardized methodology for evaluating how well an API supports consumption by AI agents, LLMs, and automated systems.
11+
12+
Most APIs today are designed for human developers. They rely on documentation, implicit conventions, and tribal knowledge. AI agents need something different: **machine-readable semantics, predictable structures, and explicit operational contracts**.
13+
14+
This framework defines **six scored dimensions** that measure an API's ability to be:
15+
16+
- **Interpretable** — easily understood by reasoning systems
17+
- **Operable** — safe and predictable to execute
18+
- **Discoverable** — findable, indexable, and contextually exposed
19+
- **Governable** — aligned with secure and trustworthy practices
20+
21+
The scoring model produces an overall AI-Readiness Index between **0 and 100**, mapped to five readiness levels that guide improvement efforts.
22+
23+
## The Six Dimensions
24+
25+
The framework evaluates APIs across six dimensions, grouped into three pillars:
26+
27+
### Foundational & Developer Experience (FDX)
28+
29+
**1. Foundational Compliance (FC)**
30+
Structural validity, standards-conformance, and parsability. Can modern API tooling reliably interpret this specification?
31+
32+
**2. Developer Experience & Tooling Compatibility (DXJ)**
33+
Documentation clarity, example coverage, response completeness, and ingestion health. Is this API pleasant and predictable for humans and tools?
34+
35+
### AI-Readiness & Usability (AIRU)
36+
37+
**3. AI-Readiness & Agent Experience (ARAX)**
38+
Semantic clarity, intent expression, datatype specificity, and error standardization. Can AI systems infer purpose and constraints?
39+
40+
**4. Agent Usability (AU)**
41+
Operational composability, complexity comfort, navigation affordances, and safety patterns. Can agents operate this API reliably and efficiently?
42+
43+
### Trust, Safety, & Discoverability (TSD)
44+
45+
**5. Security (SEC)**
46+
Authentication strength, transport security, secret hygiene, and OWASP risk posture. Can this API be safely exposed to AI systems?
47+
48+
**6. AI Discoverability (AID)**
49+
Descriptive richness, intent phrasing, workflow context, and registry signals. Can AI systems locate and classify this API?
50+
51+
## Readiness Levels
52+
53+
The final AI-Readiness score maps to one of five levels:
54+
55+
| Level | Score Range | Name | Meaning |
56+
|-------|-------------|------|---------|
57+
| **Level 0** | < 40 | **Not Ready** | Fundamentally unsuitable for AI or agents |
58+
| **Level 1** | 40–60 | **Foundational** | Developer-ready, partially AI-usable |
59+
| **Level 2** | 60–75 | **AI-Aware** | Semantically interpretable, safe for guided use |
60+
| **Level 3** | 75–90 | **AI-Ready** | Structurally rich, semantically clear, agent-friendly |
61+
| **Level 4** | 90+ | **Agent-Optimized** | Highly composable, predictable, automation-ready |
62+
63+
## When to Use This Framework
64+
65+
Apply this framework when you need to:
66+
67+
- **Validate API designs** for AI-readiness during development
68+
- **Assess existing APIs** for compatibility with LLM-driven systems
69+
- **Establish governance controls** with measurable quality gates
70+
- **Improve discoverability** for AI agents searching API catalogs
71+
- **Benchmark API portfolios** across enterprise or ecosystem-wide collections
72+
- **Guide modernization** efforts with prioritized, actionable recommendations
73+
74+
The framework works with OpenAPI 3.x and 2.x specifications. It's implementation-agnostic, vendor-neutral, and designed for both design-time validation and runtime assessment.
75+
76+
## How It Works
77+
78+
1. **Signal Extraction** — Measurable properties are extracted from the API specification (e.g., authentication coverage, example density, operation complexity)
79+
2. **Normalization** — Raw measurements are converted to a 0–1 scale using defined normalization rules
80+
3. **Dimension Scoring** — Normalized signals are aggregated into six dimension scores (0–100)
81+
4. **Weighted Harmonic Aggregation** — Dimension scores are combined using a weighted harmonic mean to prevent weak dimensions from being masked by strong ones
82+
5. **Gating Rules** — Safety constraints (e.g., hardcoded credentials, missing auth on sensitive operations) cap scores where appropriate
83+
6. **Readiness Classification** — The final score is mapped to a readiness level with actionable guidance
84+
85+
## Key Principles
86+
87+
**Validity ≠ Usability**: A syntactically valid OpenAPI specification doesn't guarantee an API is interpretable by AI systems. This framework measures **semantic clarity and operational safety**, not just schema correctness.
88+
89+
**Harmonic Penalization**: The scoring model uses a weighted harmonic mean rather than arithmetic averaging. Weak dimensions pull down the overall score more aggressively, ensuring balanced improvement across all areas.
90+
91+
**Context-Aware Security**: Security scores are adjusted based on API sensitivity (low/moderate/high) and exposure (internal/partner/public). A public API with sensitive operations faces stricter requirements than an internal API.
92+
93+
**No Silver Bullets**: There's no single fix that makes an API AI-ready. The framework identifies **prioritized improvements** across structural, semantic, operational, and security domains.
94+
95+
## Related Resources
96+
97+
- **[Full Specification](specification.md)** — Complete technical specification with normative requirements
98+
- **[Jentic Public APIs](https://github.com/jentic/jentic-public-apis)** — Collection of scored API specifications demonstrating the framework in practice
99+
- **Blog Posts**:
100+
- [Why Most APIs Fail in AI Systems and How to Fix It](https://thenewstack.io/why-most-apis-fail-in-ai-systems-and-how-to-fix-it/) (The New Stack)
101+
- [Is Your OpenAPI AI-Ready?](https://jentic.com/blog/is-your-open-api-ai-ready) (Jentic)
102+
- [Scoring APIs for AI](https://jentic.com/blog/scoring-apis-for-ai) (Jentic)
103+
104+
## Specification Metadata
105+
106+
- **Version**: {{VERSION}}
107+
- **Last Published**: {{TIMESTAMP}}
108+
- **License**: [Apache License 2.0](https://www.apache.org/licenses/LICENSE-2.0.html)
109+
110+
---
111+
112+
**Questions or feedback?** Reach out to the [Jentic team](https://jentic.com).

‎docs/publishing/scoring-engine-status-template.md‎

Lines changed: 87 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,87 @@
1+
# Scoring Engine Implementation Status
2+
3+
**Version**: {{VERSION}}
4+
**Last Updated**: {{TIMESTAMP}}
5+
6+
This page tracks the implementation status of signals in the Jentic API AI-Readiness scoring engine.
7+
8+
## Implementation Status Key
9+
10+
| Symbol | Status | Description |
11+
|--------|--------|-------------|
12+
| ✅ | Implemented | Signal is fully implemented and tested |
13+
| 🚧 | In Progress | Signal implementation is underway |
14+
| 📋 | Planned | Signal is specified but not yet implemented |
15+
| ⚠️ | Partial | Signal is partially implemented or has known limitations |
16+
17+
---
18+
19+
## Foundational Compliance (FC)
20+
21+
| Signal | Status | Notes |
22+
|--------|--------|-------|
23+
| `spec_validity` | ✅ | Binary check for valid OpenAPI |
24+
| `resolution_completeness` | ✅ | $ref resolution tracking |
25+
| `lint_results` | ✅ | Spectral/Redocly integration |
26+
| `structural_integrity` | ✅ | Schema coherence validation |
27+
28+
## Developer Experience & Tooling Compatibility (DXJ)
29+
30+
| Signal | Status | Notes |
31+
|--------|--------|-------|
32+
| `example_density` | ✅ | Coverage across eligible locations |
33+
| `example_validity` | ✅ | Schema conformance checking |
34+
| `doc_clarity` | 📋 | Readability scoring |
35+
| `response_coverage` | ✅ | Success and error response presence |
36+
| `tooling_readiness` | ✅ | Jentic ingestion health |
37+
38+
## AI-Readiness & Agent Experience (ARAX)
39+
40+
| Signal | Status | Notes |
41+
|--------|--------|-------|
42+
| `summary_coverage` | ✅ | Summary field presence |
43+
| `description_coverage` | ✅ | Description field presence |
44+
| `type_specificity` | 📋 | Datatype richness |
45+
| `policy_presence` | 📋 | SLA/rate-limit metadata |
46+
| `error_standardization` | ✅ | RFC 9457 usage |
47+
| `opid_quality` | ✅ | operationId coverage, uniqueness, casing |
48+
| `ai_semantic_surface` | 📋 | AI-oriented metadata bonus |
49+
50+
## Agent Usability (AU)
51+
52+
| Signal | Status | Notes |
53+
|--------|--------|-------|
54+
| `complexity_comfort` | ✅ | Endpoint count + schema depth |
55+
| `distinctiveness` | 📋 | Semantic similarity analysis |
56+
| `pagination` | 📋 | Paginated GET detection |
57+
| `hypermedia_support` | 📋 | HATEOAS/HAL/JSON:API |
58+
| `intent_legibility` | 📋 | Verb-object alignment |
59+
| `safety` | 📋 | Idempotency + sensitive op protection |
60+
| `tool_calling_alignment` | 📋 | LLM tool-calling compatibility |
61+
| `navigation` | 📋 | Composite of pagination + hypermedia |
62+
63+
## Security (SEC)
64+
65+
| Signal | Status | Notes |
66+
|--------|--------|-------|
67+
| `auth_coverage` | 📋 | Protected sensitive operations |
68+
| `auth_strength` | ✅ | Security scheme strength scoring |
69+
| `transport_security` | 📋 | HTTPS requirement |
70+
| `secret_hygiene` | 📋 | Hardcoded credential detection |
71+
| `sensitive_handling` | 📋 | PII field protection |
72+
| `owasp_posture` | 📋 | OWASP risk assessment |
73+
74+
## AI Discoverability (AID)
75+
76+
| Signal | Status | Notes |
77+
|--------|--------|-------|
78+
| `descriptive_richness` | ✅ | Semantic description quality |
79+
| `intent_phrasing` | 📋 | Verb-object clarity |
80+
| `workflow_context` | 📋 | Arazzo/workflow references |
81+
| `registry_signals` | 📋 | llms.txt, APIs.json, MCP metadata |
82+
| `domain_tagging` | 📋 | Domain/taxonomy classification |
83+
84+
---
85+
86+
**Last Updated**: {{TIMESTAMP}}
87+
[View Full Specification](specification.md) | [Back to Overview](overview.md)

0 commit comments

Comments
 (0)