Skip to content

Add Prose format support for pipeline templates#2

Open
terraboops wants to merge 1 commit intomainfrom
claude/integrate-prose-yaml-30Kj7
Open

Add Prose format support for pipeline templates#2
terraboops wants to merge 1 commit intomainfrom
claude/integrate-prose-yaml-30Kj7

Conversation

@terraboops
Copy link
Copy Markdown
Owner

Summary

Introduces Prose, a declarative orchestration language for defining pipeline templates alongside the existing YAML format. Both formats produce identical internal pipeline structures and are fully interchangeable.

Key Changes

  • New Prose parser (trellis/core/prose_parser.py): Lark-based parser that converts .prose files to the canonical pipeline dict. Includes:

    • Indentation-based grammar with explicit INDENT/DEDENT token preprocessing
    • parse_pipeline_prose() to load Prose files
    • emit_pipeline_prose() to serialize pipelines back to Prose format
    • Full roundtrip support (parse → emit → re-parse produces identical dicts)
  • Format abstraction layer (trellis/core/pipeline_format.py): Unified interface for loading/saving both formats:

    • detect_format() determines format from file extension
    • load_pipeline() handles both .prose and .yaml/.yml files
    • save_pipeline() writes in either format
    • find_template() searches for templates with format fallback priority
  • Web API updates (trellis/web/api/routes/pipelines.py):

    • Updated template loading to discover .prose files alongside YAML
    • Added format parameter to template creation
    • Refactored to use new format abstraction layer
  • CLI command (trellis/cli.py): New pipelines-to-prose command to batch-convert existing YAML templates to Prose format with optional dry-run mode

  • Frontend updates (trellis/web/frontend/templates/):

    • Format selector in pipeline creation UI (YAML/Prose radio buttons)
    • Format badge display in template listings
    • Format indicator in pipeline detail view
  • Documentation (docs/, README.md): Comprehensive examples of both formats with side-by-side comparisons and conversion guidance

  • Tests: Full test coverage for parser, emitter, and format detection with roundtrip validation

  • Dependencies: Added lark>=1.1 for parsing

Implementation Details

  • Prose syntax uses indentation-based structure with keywords: pipeline NAME:, session:, gate:, parallel:, description:
  • Parser converts Prose to the same canonical dict as YAML: {name, description, agents, post_ready, parallel_groups, gating}
  • Format detection is automatic based on file extension; both formats coexist in the same directory
  • Template loading prioritizes .prose over .yaml/.yml when multiple formats exist for the same name
  • All existing YAML functionality remains unchanged; this is purely additive

https://claude.ai/code/session_01VuaBTgXWpJs7GWdPCMiMRv

Add support for .prose files alongside .yaml for pipeline templates.
Both formats are co-equal — they produce the same internal pipeline dict
and can coexist in pipeline-templates/. YAML remains the default.

New modules:
- trellis/core/prose_parser.py: Lark-based parser and emitter for the
  Prose pipeline subset (pipeline/session/parallel/gate blocks)
- trellis/core/pipeline_format.py: format-aware load/save/find helpers

Changes:
- Pipeline routes use format-aware helpers instead of direct yaml calls
- Dashboard shows format badge (YAML/Prose) per template
- New template form includes format selector
- Updates preserve the existing file's format
- New CLI command: trellis pipelines-to-prose (with --dry-run)
- Reference default.prose shipped in package defaults
- Docs updated throughout (README, pipelines.md, architecture.md)

The pool scheduler and TLA+ spec are completely unaffected — they only
read the pipeline dict from status.json, never template files.

https://claude.ai/code/session_01VuaBTgXWpJs7GWdPCMiMRv
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants