RFC-0005 and RFC-0006: Skill Registry and Harness Integration#10
Conversation
Propose a governed, metadata-first registry for AI agent skills in MLflow with typed source pointers, lifecycle management, security scan tracking, skill groups, and federated discovery. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
There was a problem hiding this comment.
Pull request overview
This PR adds RFC-0005 proposing a new governed, metadata-first Skill Registry for MLflow, including versioning, lifecycle/publish-state governance, typed source pointers (Git/OCI/ZIP), and first-class skill groups.
Changes:
- Introduces the Skill Registry conceptual model (skills, skill versions, tags, aliases) and publish-state lifecycle.
- Defines skill groups with versioned membership snapshots, plus associated tags/aliases.
- Specifies proposed DB schema (12 tables) and API surfaces (REST, Python SDK, CLI, UI).
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| published skills, not the process of writing them. | ||
| - **Skill format specification.** The registry is format-agnostic. It | ||
| does not define or enforce what a skill looks like (SKILL.md, plugin | ||
| manifests, etc.). |
There was a problem hiding this comment.
Could we consider MLflow artifact storage as a first-class source type here? That would give us a natural UI upload flow and keeps the door open for more MLflow-native packaging/optimization later, even if Git and OCI stay the main paths.
There was a problem hiding this comment.
Good idea. We've added mlflow as an explicit future source type in the "Source type extensibility" section — it would allow storing skill content directly in MLflow artifact storage, providing a natural UI upload flow. It's deferred from the initial implementation to keep the registry metadata-first, but can be added as a follow-up without breaking changes. The adoption strategy's "Follow-up" section also calls this out.
— This comment was posted by Claude Code under the supervision of Bill Murdock.
| RETIRED = "retired" | ||
|
|
||
|
|
||
| class SkillSourceType(StrEnum): |
There was a problem hiding this comment.
Can we line this lifecycle up more closely with the MCP registry RFC in PR 12 (#12)? Right now this introduces draft, published, deprecated, and retired plus a separate skill-level status, which makes the cross-registry pattern feel less consistent unless there’s a strong skill-specific reason to diverge.
There was a problem hiding this comment.
Fully aligned — replaced the 4-state publish lifecycle (draft/published/deprecated/retired) with RFC-0004's 3-state model using status (active/deprecated/deleted). SkillStatus is now the only lifecycle enum. Parent entity status (Skill.status, SkillGroup.status) is derived from the latest version rather than set independently.
— This comment was posted by Claude Code under the supervision of Bill Murdock.
| does not perform scans. Scanning tools are separate. | ||
| - **Agent harness integration.** How a specific agent harness (Claude | ||
| Code, Codex, Cursor, etc.) installs or loads skills from the registry | ||
| is outside this RFC. The registry provides the metadata; harness |
There was a problem hiding this comment.
Do we want one short paragraph on external skill conventions we expect to interoperate with? Even if the registry stays format-agnostic, it would help to say whether we’re aligning with existing Claude/Cursor/GitHub-style skill packaging or deliberately avoiding standardization for now.
There was a problem hiding this comment.
Done — fully aligned with RFC-0004. Version lifecycle now uses status with three states (active, deprecated, deleted), matching MCPStatus. New versions default to active. Skill.status and SkillGroup.status are now read-only, derived from the latest version's status. Also added latest_version_alias on both Skill and SkillGroup, following RFC-0004's pattern for deterministic "latest" resolution. Store interface switched from AbstractSkillRegistryStore with @abstractmethod to SkillRegistryMixin with raise NotImplementedError, matching MCPServerRegistryMixin.
(Comment from Claude Code under the supervision of Bill Murdock.)
| #### SkillVersion | ||
|
|
||
| A versioned record containing a typed source pointer, publish state, | ||
| and tags. |
There was a problem hiding this comment.
I initially expected aliases to live on SkillVersion, but after checking MLflow model registry, the existing pattern is that aliases are owned by the top-level entity and point to a version. If the goal is consistency with MLflow and PR 12 (#12), I think this design is fine. It may just be worth making that pattern more explicit in the text.
There was a problem hiding this comment.
Aligned — set_skill_alias takes (name, alias, version) and sets the alias on the Skill entity, pointing to a specific version. Same pattern for set_skill_group_alias. This matches the MCP RFC's approach where aliases are owned by the parent entity rather than the version.
— This comment was posted by Claude Code under the supervision of Bill Murdock.
| ) | ||
| ``` | ||
|
|
||
| ## Create a skill group with a versioned membership snapshot |
There was a problem hiding this comment.
What's the justification for skill groups? Why not just using tags on skills
There was a problem hiding this comment.
Added a "Why groups instead of tags?" section to the RFC. The key reasons: (1) groups pin specific member versions for reproducibility, (2) groups have their own lifecycle and aliases independent of member status, (3) groups can reference capabilities across registries (skill registry + MCP registry), (4) groups map naturally to the "plugin" concept in agent harnesses, and (5) tags are flat key-value pairs that can't express ordered, versioned membership.
— This comment was posted by Claude Code under the supervision of Bill Murdock.
| trace data back to a governed skill record to understand adoption | ||
| across an organization. | ||
|
|
||
| ### Use cases |
There was a problem hiding this comment.
Could we add explicit JTBDs per persona (developer, admin etc.)? It would make it easier to evaluate which parts of the data model are required
There was a problem hiding this comment.
Done — replaced the abstract use case bullets with three end-to-end persona flows: a platform administrator (registers, scans, groups, aliases, deprecates), a developer (searches, resolves alias, pulls), and a security engineer (queries scan tags, deprecates failed versions, tracks compliance). These ground the data model in concrete user workflows.
— This comment was posted by Claude Code under the supervision of Bill Murdock.
- Switch to YAML frontmatter to match repo template - Make ER diagram a mermaid code block - Rename source_url → source (MLflow consistency) - Rename content_hash → content_digest (OCI alignment) - Use SkillSourceType enum consistently for alias/membership types - Fix Skill.aliases type to list[SkillAlias] (was dict, missing source_type) - Fix search_skills example to use search_skill_versions (publish_state is on version) - Clarify alias resolve returns both version and source_type - Specify valid filter fields per search endpoint - Fix delete semantics: skill delete blocked when versions referenced by groups - Drop Databricks dogfooding anecdote - Remove "Relationship to other AI asset registries" section - Remove "Impact on existing MLflow components" table - Condense SDK/CLI sections (defer to store interface + examples) - Tighten drawbacks and trim weaker alternatives Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Remove redundant group_name/group_version/workspace fields from the entity — parent identity is provided by the enclosing SkillGroupVersion. The DB schema retains those columns as FKs. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Update: Review Feedback Round 1Pushed changes addressing the straightforward review feedback: YAML frontmatter, mermaid ER diagram, field renames ( Still openStructural:
Scope:
Content:
(Comment from Claude Code under the supervision of Bill Murdock.) |
Version uniqueness is now (name, version) instead of (name, version, source_type). source_type and source are optional fields on SkillVersion. Cascaded this change through entities, DB schema, store interface, REST API paths, examples, and CLI. Added RFC-0006 covering harness-specific installation with adapters for Claude Code, Codex CLI, Cursor, and Antigravity. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Status update (2026-04-27)Changes since last updateRFC-0005 (Skill Registry):
RFC-0006 (Skill Registry Harness Integration) — new:
Resolved review comments (newly replied)
Still open
(Comment from Claude Code under the supervision of Bill Murdock.) |
Move marketplace.json from Alternatives into Detailed Design with full endpoint spec, response format, configuration, and limitations. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
jwm4
changed the title
|
Also, I've converted this to draft given all the open issues listed above. |
- Lifecycle: publish_state → status, 3 states (active/deprecated/ deleted), derived parent status. Aligns with RFC-0004. - Add latest_version_alias to Skill and SkillGroup. - Store: AbstractSkillRegistryStore → SkillRegistryMixin with NotImplementedError. Add order_by to search methods. - Cross-registry membership: rename skill_name/skill_version to member_name/member_version, add registry field (skill/mcp). - Conditional FK for MCP refs → application-layer enforcement. - Pull clarified as client-side, removed from store mixin and REST API. - Dual MCP registration: MCP registry is default, kind=mcp-server reserved for embedded configs only. - SDK namespace: mlflow.skills → mlflow.genai.skills. - Add external skill conventions paragraph. - Add skill groups justification (why not just tags). Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Document `mlflow` as a deferred source type in the extensibility section and adoption strategy follow-up, per review feedback. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Status update (2026-04-29)Changes since last updateRFC-0005 (Skill Registry):
RFC-0006 (Harness Integration):
Still open (deferred to future pass)
(Comment from Claude Code under the supervision of Bill Murdock.) |
- Replace abstract use case bullets with end-to-end persona flows
(platform admin, developer, security engineer)
- Expand security scan tracking with structured tag namespace convention
(scan.{type}.{field}) and documented fields/examples
- Add permissions section mapping operations to MLflow's READ/EDIT/MANAGE
levels, with status transitions and alias management requiring MANAGE
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Status update (2026-04-29, part 2)The four items listed as "still open" in the previous update are now all addressed:
All review comments from mprahl, serena-ruan, and Copilot now have replies. I don't have further edits planned at this time. The RFC is ready for another round of review. (Comment from Claude Code under the supervision of Bill Murdock.) |
| rfc_pr: https://github.com/mlflow/rfcs/pull/10 | ||
| --- | ||
|
|
||
| # RFC: Skill Registry |
There was a problem hiding this comment.
@Bill Murdock — Reading through the RFC and the proposed API, one thing struck me: given that the registry already supports kind=[skill, agent, mcp-server, hook], should we consider a more generic name than "skills" / "skill groups"?
The data model is already kind-agnostic — a Skill entity with kind=agent is conceptually an asset that happens to be an agent, not a skill. Calling the registry an Asset Registry (or Capability Registry) and using create_asset() / create_asset_group() instead of create_skill() / create_skill_group() would better reflect what the API actually does and avoid the naming tension where "skill" is both a specific kind and the umbrella term for all kinds.
The REST paths and CLI would follow naturally: mlflow assets create, mlflow asset-groups create-version, etc.
There was a problem hiding this comment.
+1
I'd like to also propose bundles. It satisfies the generic 'kind-agnostic' semantics of assets, while also implying plurality even in the singular term (skill-group and bundle both imply grouping of some asset).
I also agree that having an independent name (assets/bundles/capabilities) greatly improves function discoverability in the SDK
There was a problem hiding this comment.
Alternatively, if none of the names appeal, I'd suggest at least in the CLI dropping skill-groups in favor of skills with a --group flag. Something like:
mlflow skills pull --group pr-workflow --alias production
There was a problem hiding this comment.
+1 to @etirelli here, the umbrella term should not be a skill. Currently in the RFC the skill that is a kind/skill is more consistent with the industry's usage of the term, but the top level usage of the word skill would be very confusing.
My preference is asset registry or capability registry, or something equally broader.
| Add a Skill Registry to MLflow: a governed, metadata-first registry for | ||
| AI agent capabilities. The registry stores metadata and typed source | ||
| pointers (to Git repos, OCI registries, ZIP archives, etc.) rather | ||
| than artifacts directly. It provides enterprise governance on top of |
There was a problem hiding this comment.
Could we support both approaches? The user would either set pointers to external storage (Git, OCI, ZIP) or store assets directly in MLflow's internal artifact store.
The reason: source pointers are the right default for customers who already manage assets in Git or other systems. But for customers who don't have that infrastructure in place, or who already use MLflow to store models and want their agent capabilities alongside them, or who operate in airgapped or restricted environments where reaching external sources isn't practical — the ability to store assets directly in the registry would be a better fit.
There was a problem hiding this comment.
Agreed. The latest revision adds source_type="mlflow" as a first-class source type in the initial implementation. Content is stored as a directory tree of individual files in MLflow's artifact store, consistent with how model artifacts are stored. Users can choose external source pointers (Git, OCI, ZIP) or direct artifact storage depending on their environment. This covers the airgapped and "capabilities alongside models" use cases you described.
-- This comment was posted by Claude Code under the supervision of Bill Murdock.
|
|
||
| ### Out of scope | ||
|
|
||
| - **Artifact storage.** The registry stores metadata and source |
There was a problem hiding this comment.
given that mlflow already has an "artifact store" to store models and traces, I think the proposal could just embrace it and support it as an option, as per my previous comment.
There was a problem hiding this comment.
Addressed in the same change. See reply to your earlier comment.
-- This comment was posted by Claude Code under the supervision of Bill Murdock.
|
|
||
| The list view shows skills and skill groups in a card-based or table | ||
| layout, with name, description, latest version, status, and tags. Users | ||
| can filter by status, source type, and search by name or description. A |
There was a problem hiding this comment.
It would be useful to track install counts and surface them in the MLflow UI. That would help enterprises understand which capabilities are actually being adopted, and it would also let end users sort or rank skills / skill groups by popularity.
🤖 Generated with Claude
There was a problem hiding this comment.
Good idea. We added install count tracking and surfacing in the UI to the adoption strategy section, enabling users to sort or rank capabilities by adoption.
-- Bill Murdock (with assistance from Claude Code)
|
|
||
| ```python | ||
| @dataclass(frozen=True) | ||
| class SkillGroupVersionMembership: |
There was a problem hiding this comment.
Nit: SkillGroupVersionMember might be more intuitive than SkillGroupVersionMembership.
🤖 Generated with Claude
There was a problem hiding this comment.
Done. Renamed to SkillGroupVersionMember throughout.
-- Bill Murdock (with assistance from Claude Code)
| The list view shows skills and skill groups in a card-based or table | ||
| layout, with name, description, latest version, status, and tags. Users | ||
| can filter by status, source type, and search by name or description. A | ||
| toggle switches between individual skills and skill groups. |
There was a problem hiding this comment.
From a user perspective, I don't think they should have to distinguish between skills and skill groups up front when searching. A unified discovery experience with clear type badges / filters seems more intuitive than separate search flows, since users usually start with "I want something that helps me do X" rather than "I want a single skill vs. a bundle."
🤖 Generated with Claude
There was a problem hiding this comment.
Agreed that users shouldn't have to distinguish up front. We softened the UI section to avoid prescribing a specific interaction pattern, leaving room for the design team to determine the best discovery experience. The key principle, that skills and groups should be discoverable together with appropriate filtering, is captured.
-- Bill Murdock (with assistance from Claude Code)
| @dataclass | ||
| class Skill: | ||
| name: str | ||
| kind: SkillKind = SkillKind.SKILL |
There was a problem hiding this comment.
Skill(kind=SKILL) feels a little awkward semantically since this entity can also represent agents, hooks, and MCP servers. Would a more general name like AgentCapability make the model clearer?
🤖 Generated with Claude
There was a problem hiding this comment.
The naming question is still open. Proposals include AgentCapability, Asset, Bundle, and others. I agree the Skill(kind=SKILL) redundancy is awkward. I'll update the entity names once the naming question is resolved.
-- Bill Murdock (with assistance from Claude Code)
| schema changes since the column stores a string value. `kind` is | ||
| immutable after creation. | ||
|
|
||
| **MCP servers: two registration paths.** The MCP server registry |
There was a problem hiding this comment.
MCP servers: two registration paths seems off to me. RFC-0004 is about governing canonical MCP server definitions via server.json, while this RFC appears to be talking about embedded harness / plugin MCP configuration. Those feel like different asset types, so calling them two registration paths for MCP servers is confusing. Should this section instead frame skill groups as referencing MCP registry entries, with harness-specific .mcp.json / mcp.json generated at install time?
🤖 Generated with Claude
There was a problem hiding this comment.
Agreed. I removed kind=mcp-server from the skill registry entirely and replaced that section in the previous commit, in response to a similar point from Khaled. MCP servers now belong exclusively in the MCP Registry (RFC-0004). Skill groups reference MCP registry entries via registry="mcp" in their membership, and harness-specific .mcp.json configs are generated at install time by RFC-0006's harness adapters. This is essentially what you suggested.
-- Bill Murdock (with assistance from Claude Code)
| |---|---|---| | ||
| | `version` | `str` | Publisher-supplied version string. Semver recommended but not enforced | | ||
| | `source_type` | `SkillSourceType` | Optional distribution mechanism: `git`, `oci`, `zip` | | ||
| | `source` | `str` | Optional pointer to the content in the source system. Required for standalone pull; omit when content is only available via a group-level source | |
There was a problem hiding this comment.
Should we consider adding a subpath field so source points to the Git repo or OCI artifact, while subpath identifies the path within it that contains the actual skill / agent / config to use? That would make the model clearer by separating "what to download" from "where inside the downloaded content the relevant asset lives," and it could help with cases where multiple capabilities are stored in a shared repo or artifact.
🤖 Generated with Claude
There was a problem hiding this comment.
Thanks for the suggestion. We added subpath as an optional field on SkillVersion and SkillGroupVersion.
The field separates "what to download" from "where inside the downloaded content the asset lives." Its applicability varies by source type:
- OCI and ZIP:
subpathis meaningful and expected when multiple skills share a single artifact. For example, a skill group might point toghcr.io/acme/agent-plugin:v1.0.0as its group-level source, while individual member skills usesubpath(e.g.,skills/code-review) to identify their location within that image. This is the primary use case. - Git: Not used. Git tree URLs already encode the repository, ref, and path in a single
sourcestring (e.g.,https://github.com/acme/skills/tree/v1.0.0/code-review). Decomposing that intosource+subpathwould be awkward because Git sources actually have three dimensions (repo, ref, path), which don't map cleanly to two fields. - MLflow artifacts: Not used. The artifact path is already scoped to the specific skill version at upload time.
See the "Subpath usage by source type" table in the updated RFC for the full breakdown.
This comment is from Bill Murdock with assistance from Claude Code.
| version of a capability; `source_type` and `source` describe where to | ||
| find it but are not part of its identity. If the same content is | ||
| available from multiple distribution mechanisms (e.g., Git and OCI), | ||
| register separate versions or use a group-level source. |
There was a problem hiding this comment.
I don't like the UX of having multiple SkillVersion records just because the same logical capability is available from multiple download locations. From a user perspective, version should mean "the contents changed," not "there is now another distribution source." Should the model instead support one logical version with multiple attached sources / mirrors?
🤖 Generated with Claude
There was a problem hiding this comment.
Agreed that version should mean "the contents changed." The PK was already (name, version) with source_type not part of the identity, but the text still suggested registering separate versions for different distribution mechanisms. I've removed that guidance. If someone does need to track the same content across multiple sources, separate versions are a reasonable workaround, but it's not the recommended pattern.
-- Bill Murdock (with assistance from Claude Code)
| (single OCI image or Git repo) and an assembled plugin (members from | ||
| different sources). | ||
|
|
||
| If `content_digest` is set, `pull` verifies the fetched content |
There was a problem hiding this comment.
For source_type="mlflow", should the server validate content_digest, or is integrity checking still purely client-side? I'm thinking about cases where the underlying artifact store contents change out of band (for example, the backing S3 objects are modified after registration). It would be helpful to clarify where that validation responsibility lives.
🤖 Generated with Claude
There was a problem hiding this comment.
Added a "Content integrity" section that specifies this. For source_type="mlflow", the server computes content_digest at upload time; on pull, the client recomputes and rejects on mismatch. For external source types, content_digest is client-supplied and validation is the consumer's responsibility.
-- Bill Murdock (with assistance from Claude Code)
| ``` | ||
|
|
||
| The `source` field contains the MLflow artifact URI (e.g., | ||
| `mlflow-artifacts:/skills/code-review/1.0.0/`). Pull downloads the |
There was a problem hiding this comment.
The source_type="mlflow" wording here seems a little misleading. mlflow-artifacts:/... is specific to artifact proxying / --serve-artifacts, but MLflow can also use direct artifact-store URIs when it is not serving artifacts. Can this section clarify whether source_type="mlflow" means "stored in MLflow-managed artifact storage" more generally, rather than specifically implying the mlflow-artifacts: scheme or guaranteed UI file browsing?
🤖 Generated with Claude
There was a problem hiding this comment.
Clarified. The text now says source_type="mlflow" means "stored in MLflow-managed artifact storage," not a specific URI scheme. The source field contains whatever artifact URI MLflow resolves, whether that's mlflow-artifacts:/... via the artifact proxy or a direct artifact-store URI.
-- Bill Murdock (with assistance from Claude Code)
| within a workspace. | ||
|
|
||
| **Group-level source.** A group version can optionally have its own | ||
| `source_type`, `source`, and `content_digest`, pointing to a single |
There was a problem hiding this comment.
I'm not sure the group-level source concept is worth it. Once a SkillGroup has its own source artifact, version, lifecycle, and pull/install behavior, it starts to look like a first-class capability rather than just a grouping construct. At that point, why isn't it just a normal skill, or a separate bundle / plugin asset type?
🤖 Generated with Claude
There was a problem hiding this comment.
I agree this adds complexity, but I think it's necessary to work with the way skills are actually developed and deployed. It's not reasonable to expect someone developing a plugin with a set of skills, subagents, hooks, etc. to make separate OCI images for each of these. The group-level source lets them package and distribute the plugin as a single artifact, which is how most harness plugin ecosystems work in practice.
-- Bill Murdock (with assistance from Claude Code)
|
|
||
| ### Pull semantics | ||
|
|
||
| `pull` is a client-side operation. The SDK reads the source pointer |
There was a problem hiding this comment.
Should there be a lock-file concept so the common workflow can be something like just running mlflow skills pull, similar to npm install? It seems useful to have a checked-in manifest / lock file that records the exact resolved skills, versions, and maybe sources, so a project can reproduce the same local setup without requiring users to manually pass names and aliases each time.
🤖 Generated with Claude
There was a problem hiding this comment.
Agreed. We added a lock file section to RFC-0006 (Harness Integration). The mlflow-skills.lock file records the exact resolved skills, versions, sources, and harness so that mlflow skills install with no arguments reproduces the same local setup, analogous to package-lock.json. The workflow is --lock to write it, bare mlflow skills install to reproduce from it, and --lock --update to re-resolve and refresh.
-- Bill Murdock (with assistance from Claude Code)
| harness-specific manifests or place files in harness-specific | ||
| directories. Harness-specific installation is covered in RFC-0006. | ||
|
|
||
| ### Error handling |
There was a problem hiding this comment.
The Error handling section feels too verbose for an RFC and reads more like implementation detail / API spec material. I think it could be removed or drastically shortened so the RFC stays focused on the higher-level design.
🤖 Generated with Claude
There was a problem hiding this comment.
Removed the error handling section. RFC-0004 (MCP Registry) has no equivalent section, and this level of detail is better suited to the implementation.
This comment is from Bill Murdock with assistance from Claude Code.
| list, aliases, and tags. Each group version shows its status and the | ||
| pinned member versions it contains. | ||
|
|
||
| ### Security scan tracking |
There was a problem hiding this comment.
For Security scan tracking, I think we should either make it a small first-class structured field on SkillVersion / SkillGroupVersion, or remove it from the RFC. If MLflow expects to display scan state in the UI and support reliable search/filtering, the data needs to be in a consistent format. Tags don't feel like the right place for that.
🤖 Generated with Claude
|
|
||
| **Querying.** Scan results are queryable using the existing filter | ||
| syntax: `tags.scan.prompt-injection.status = 'pass'` or | ||
| `tags.scan.code-vulnerability.date < '2026-01-01'`. |
There was a problem hiding this comment.
I don't think the RFC's query example for scan metadata is actually supported by MLflow's existing filter syntax. Tag filters support equality / like-style comparisons, but not numeric or date comparisons like tags.scan.code-vulnerability.date < '2026-01-01'. That seems like another sign that scan metadata should be modeled as structured fields rather than tags if the design expects reliable querying.
🤖 Generated with Claude
| | Delete MCP server version referenced by a group version | `INVALID_PARAMETER_VALUE` | 400 | | ||
| | Delete skill or group with no group references | Cascading delete (succeeds) | 200 | | ||
|
|
||
| ### Workspace scoping |
There was a problem hiding this comment.
I think both Workspace scoping and Adoption strategy could be trimmed quite a bit. Workspace scoping seems like it can just say this registry follows MLflow's existing workspace-aware registry patterns and leave cross-workspace sharing out of scope. Adoption strategy also feels more detailed than necessary for an RFC and could be shortened to a compact phased summary.
🤖 Generated with Claude
There was a problem hiding this comment.
Agreed on both. Workspace scoping now just references MLflow's existing workspace-aware registry patterns instead of enumerating implementation details. Adoption strategy is condensed to a three-phase summary (this RFC, RFC-0006, follow-up items).
This comment is from Bill Murdock with assistance from Claude Code.
| Harnesses without marketplace support (Cursor, Antigravity, OpenClaw) | ||
| use the adapter-based `mlflow skills install` command instead. | ||
|
|
||
| ### Store interface |
There was a problem hiding this comment.
Could you make this a separate store interface so this can be plugin based through Python entrypoints like other plugins? That way MLflow doesn't have to embed all installation types but allows someone to pip install implementations from vendors.
There was a problem hiding this comment.
Yes. RFC-0006 now specifies that adapters are registered via Python entrypoints (group mlflow.skill_harness_adapters), so third-party adapters can be installed via pip install without modifying MLflow core. The initial release includes builtin adapters for Claude Code, Codex CLI, and Cursor, with additional harnesses added as the community contributes them.
-- Bill Murdock (with assistance from Claude Code)
|
|
||
| This is error-prone and discourages adoption. | ||
|
|
||
| ### The cross-harness landscape |
There was a problem hiding this comment.
Could you clarify which ones are in scope for being builtin to MLflow?
There was a problem hiding this comment.
Claude Code, Codex CLI, and Cursor ship as builtin adapters in the initial release. Additional harnesses are community-contributed via Python entrypoints (see the updated "Other harness adapters" section). The cross-harness landscape table documents the broader ecosystem for reference, but only the three initial adapters are in scope for now.
-- Bill Murdock (with assistance from Claude Code)
| | `POST` | `/ajax-api/3.0/mlflow/skill-groups/{name}/install` | Install a skill group for a harness | | ||
| | `GET` | `/ajax-api/3.0/mlflow/skill-groups/marketplace.json` | Generate marketplace catalog for a harness | | ||
|
|
||
| ### CLI |
There was a problem hiding this comment.
Should we add support in the MLflow CLI for importing from an existing Git repository? That seems like a useful onboarding path for teams that already have a repo following Claude-style skill conventions, since it would let them bring existing content into MLflow without manually re-registering everything from scratch.
🤖 Generated with Claude
There was a problem hiding this comment.
Interesting idea. Git import could be a useful onboarding path, but I'd like to keep it out of scope for now and see how the core registry workflow lands first. What do you think?
-- Bill Murdock (with assistance from Claude Code)
…, unify pull RFC-0005: - Add register_skill() SDK convenience function matching MCP RFC's register_mcp_server() pattern, with content_path for artifact upload - Remove kind=mcp-server from SkillKind (MCP servers belong in MCP registry; embedded configs are artifact content for harness adapters) - Unify pull/pull-group into single pull command with --group flag - Add shared base extraction note to adoption strategy - Update basic examples to use register_skill() RFC-0006: - Drop install POST endpoints (install is client-side) - Keep marketplace.json GET endpoint - Update SDK example to use unified install() Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
|
Status update (2026-05-17) Replied to all of Khaled's review comments. Key decisions:
Next steps: I need to review Matt's comments and the upcoming MCP Registry changes in PR #18. Once I've worked through both, I may retract some of the decisions made for this round in favor of further alignment. -- Bill Murdock (with assistance from Claude Code) |
…k file RFC-0005 changes: - Clarify summary: artifact storage is supported but metadata-first - Clarify source_type="mlflow" means MLflow-managed storage, not a specific URI scheme - Merge content integrity into one section with server-side validation for mlflow sources and client-side for external sources - Remove guidance to register separate versions for different sources - Rename SkillGroupVersionMembership to SkillGroupVersionMember - Unified UI list view showing skills and groups together - Add install count tracking to follow-up roadmap RFC-0006 changes: - Add lock file section for reproducible installs (mlflow-skills.lock) - Add Python entrypoint support for third-party harness adapters Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
|
Status update on RFC-0005 / RFC-0006 We reviewed the new MCP Registry ADR (PR #18) and confirmed it has no impact on these RFCs. We've addressed many of Matt's latest review comments, including:
Still need to work through several of Matt's other comments. Also, we need to keep working on naming. -- Bill Murdock (with assistance from Claude Code) |
| rfc_pr: https://github.com/mlflow/rfcs/pull/10 | ||
| --- | ||
|
|
||
| # RFC: Skill Registry |
There was a problem hiding this comment.
+1 to @etirelli here, the umbrella term should not be a skill. Currently in the RFC the skill that is a kind/skill is more consistent with the industry's usage of the term, but the top level usage of the word skill would be very confusing.
My preference is asset registry or capability registry, or something equally broader.
| The registry is format-agnostic but is designed to interoperate with | ||
| the conventions gaining adoption across agent harnesses: | ||
|
|
||
| - **SKILL.md** — a markdown file with structured instructions for the |
There was a problem hiding this comment.
Since sub agent is a kind in this RFC, maybe a sentence mentioning sub agents would be useful
| SkillGroupVersionMember( | ||
| member_name="github-mcp", member_version="2.0.0", | ||
| registry="mcp", | ||
| ), |
There was a problem hiding this comment.
Nit: I feel like it's a bit awkward that a user needs to mention the type of registry. Maybe we could also just allow users to pass the specific version object and the client can infer the registry based on type. So for example:
my_skill_version = mlflow.genai.skills.register_skill(
name="code-review",
version="1.0.0",
....
)
my_mcp_server_version = mlflow.genai.register_mcp_server(
server_json={
"name": "io.github.anthropic/brave-search",
"title": "Brave Search",
...
)
create_skill_group_version(
name="my_group",
version="1.0.0",
members=[
my_skill_version,
my_mcp_server_version,
],
)
)Internally this would translate to SkillGroupVersionMember(..) or users could use SkillGroupVersionMember() directly as fallback if they wanted.
|
|
||
| ```python | ||
| @dataclass(frozen=True) | ||
| class SkillAliasHistory: |
There was a problem hiding this comment.
We don't have an Alias history in the MCP registry, I don't see this as a requirement for the MVP, can we drop this and associated apis? I think if we are to consider this we should consider it hollistically for all assets types, mcp, skills, prompts to ensure consistency.
Same with SkillGroupAliasHistory
| class SkillGroupVersionMember: | ||
| member_name: str | ||
| member_version: str | ||
| registry: str = "skill" # "skill" or "mcp" |
There was a problem hiding this comment.
Should registry be an enum similar to SkillStatus, SkillKind, etc?
| schema changes since the column stores a string value. `kind` is | ||
| immutable after creation. | ||
|
|
||
| **MCP servers.** MCP servers are registered in the MCP Server Registry |
There was a problem hiding this comment.
Have you thought about the prompt registry and where that fits in here? I feel like prompts are a real part of an agent package so some consideration to them in this RFC would be warranted. If you agree but feel it doesn't need to be part of the MVP, I think that is fine but we should list those as out of scope.
edit: Looked more into this and I see that it's just skills not prompts for most plugin bundles (is that the only intended usecase here?). Probably should add a note addressing prompts and where they fit in (or don't).
| code-review/SKILL.md # Pulled from registered source | ||
| agents/ | ||
| security-auditor.md # Pulled from registered source | ||
| .mcp.json # Generated from mcp-server members |
There was a problem hiding this comment.
nit: do you feel the usage of the word install here could be a bit misleading, since mcp registry exposes canonical server_json and packages[], which naturally reads as "here is how you actually install/run this MCP server." But really we're just generating/creating files, not installing the mcp server(s). Would mlflow skills generate or something similar be better suited?
| "version": "1.0.0", | ||
| "description": "End-to-end pull request review workflow", | ||
| "author": { "name": "Generated by MLflow Skill Registry" }, | ||
| "source": "https://mlflow.example.com/ajax-api/3.0/mlflow/skill-groups/pr-workflow/install?harness=claude-code", |
There was a problem hiding this comment.
This source looks like it's implying that Claude is downloading through the MLflow server - is that what you intended?
| --harness claude-code --lock --update | ||
| ``` | ||
|
|
||
| The lock file records enough information to reproduce the install |
There was a problem hiding this comment.
One key benefit from having the Skills registry is the governance aspect, and the ability for admins to deprecate, pull, or roll back skills/mcp/hooks/agents, so wouldn't this by pass that governance if this doesn't contact the registry?
| ) | ||
| ``` | ||
|
|
||
| ## Pull skills to a local directory |
There was a problem hiding this comment.
I don’t see the RFC specify how authorization works for external sources during pull, especially for private Git repos, private OCI registries, or authenticated ZIP URLs. Since pull is client-side and the registry server is not involved in content transfer, is the intent that auth is entirely delegated to the caller’s local Git/OCI/HTTP credentials? If so, it would help to state that explicitly, along with whether the registry ever validates source accessibility at registration time.
For OCI and ZIP source types, multiple skills may share a single artifact. The new subpath field identifies where each skill lives within the artifact. Not used for Git (tree URLs encode the path) or MLflow artifacts (path is scoped at upload time). Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The MCP Registry RFC (RFC-0004) has no error handling section. Consistent with that precedent, this detail is better left to implementation. Reference copy saved in error-handling-reference.md (not checked in). Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Workspace scoping now references MLflow's existing patterns instead of enumerating implementation details. Adoption strategy condensed to a three-phase summary. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
|
Status update (2026-05-19) Addressed three more of Matt's review comments:
Also added Still open:
This comment is from Bill Murdock with assistance from Claude Code. |
| Membership is at the version level, so a group version is a | ||
| reproducible snapshot of "these specific asset versions work together." | ||
|
|
||
| **Group-level source and embedded MCP configs.** When a group version |
There was a problem hiding this comment.
Is having a group level source/source_type a requirement for MVP? To me this feels like there are unanswered questions in this + the harness integration rfc.
- if a group has a source, does install consume that artifact directly?
- is that artifact expected to already be a harness-ready plugin bundle, or just a generic package of skills/agents/hooks/MCP config?
- do adapters still generate manifests from metadata, or trust what’s inside the group artifact?
- what happens if the artifact contents disagree with the declared members?
- presumably sources like git repo would need to respect directory structures/layouts, how would you enforce that? or ensure it's adhered to?
IMO if this is not required I would take it out, and leave it for a future enhancement.
|
|
||
| ## Drawbacks | ||
|
|
||
| - **Source pointer validity.** The registry stores source pointers but |
There was a problem hiding this comment.
We should mention how such cases are handled. For example what happens if you try to pull a resource where the source no longer exists?
| 2. Otherwise, pull each member individually from its own `source` to | ||
| a subdirectory of the destination, named by the member's skill name. | ||
| Members without a `source` are skipped with a warning. | ||
|
|
There was a problem hiding this comment.
Why is source not a required field for skills? This feels like it creates too much ambiguity.
| code-review/SKILL.md # Pulled from registered source | ||
| agents/ | ||
| security-auditor.md # Pulled from registered source | ||
| .mcp.json # Generated from mcp-server members |
There was a problem hiding this comment.
.mcp.json contains information about what servers are available, and how the client connect to them.
MCPAccessBinding can give you information about the active endpoints. But it is worth making a brief note that considers the MCP gateway as well (see the MCP registry RFC Journey 2). The adapter would also need a selection policy (what if there is more than one MCPAccessBinding for a server?)
Also note that while we can get the user most of the way there we can't configure things like credentials, certificates, authorization headers, etc.
| and hooks together. But there is no agent-neutral way to represent | ||
| these bundles for governance and discovery. | ||
|
|
||
| 5. **No usage analytics linkage.** MLflow traces can capture skill |
There was a problem hiding this comment.
Can you provide the mechanics of how we would link traces back to the skills/agents/hooks/mcp servers? Because at the moment this part is not clear to me. It seems majority of these are uniquely identified by (workspace, name, version), but how would mlflow know this information, specifically the workspace?
As an example for prompts would have to explicitly load or link the prompts via mlflow.genai.load_prompt or MlflowClient.link_prompt_versions_to_trace(...)
Can you provide an example of a trace call, and briefly explain how this would work? At the moment to me it seems like we do not have sufficient information to do this since the usecase described with harness integration seems to be to do an install and generate the files in the file system.
Summary
Related Issues
Numbering
This RFC is numbered 0005, assuming the MCP Server Registry proposal (mlflow/mlflow#22625) will be RFC-0004.
Design Highlights
mlflow.skills), and CLI (mlflow skills,mlflow skill-groups)🤖 Generated with Claude Code
This comment comes from Claude Code under the supervision of Bill Murdock.