Merge pull request #449 from Pipelex/release/v0.15.4

### Added
 - **Enhanced `pipelex build` Command**: Now generates a self-contained directory (e.g., `results/pipeline_01/`) containing `bundle.plx`, `inputs.json`, `run_{pipe_code}.py`, `bundle_view.html`, and `bundle_view.svg`. New CLI options: `--output-name (-o)` for custom base name, `--output-dir` for custom directory, and `--no-extras` to generate only the `.plx` file.
 - **CLI Readiness Check**: Verifies that a virtual environment is active for development installations.
 - **Model Deck Presets**: Added `llm_for_creativity` and `cheap_llm_for_creativity` model waterfalls, plus `[cogt.model_deck_config]` section in `pipelex.toml` for configuring model fallback behavior.
 - WIP: **Groq Inference Backend Support**: Integrated full support for the Groq API with configuration file (`.pipelex/inference/backends/groq.toml`), model specifications, costs, capabilities, new model aliases (`base-groq`, `fast-groq`, `vision-groq`), and routing profile (`all_groq`).

### Changed
 - **CLI Output and Visualization**: Overhauled command-line output with rich, table-based layouts for pipeline components. Final output of `pipelex run` is now pretty-printed and adapts to content type.
 - **Documentation**: Updated "Get Started" and "Build Reliable AI Workflows" to reflect new directory-based build output and CLI options.
 - **Internal Code Refactoring**: Reorganized exception hierarchy into dedicated `exceptions.py` files per module, centralized validation logic into `validation.py` modules, added `ValueError` to blueprints, and removed unused exceptions for improved maintainability.
- Updated pytest to `>=9.0.1` to support their new `pyproject.toml` config format.

### Fixed
 - Adjusted default temperature for `llm_for_testing_gen_object` preset from `0.5` to `0.1` for more deterministic structured data generation.
 - Corrected `LLM_FOR_VISUAL_DESIGN` skill in `pipe_llm_spec` to point to `cheap_llm_for_creativity` preset.
 - Standardized input variable names in `pipe_llm_vision.plx` from `imageA`/`imageB` to `image_a`/`image_b`.

### Removed
 - Deleted `pipelex/core/validation_errors.py` file as part of exception hierarchy refactoring.

Author: lchoquel

.badges/tests.json

@@ -1,7 +1,7 @@
 {
   "schemaVersion": 1,
   "label": "tests",
-  "message": "1435",
+  "message": "1526",
   "color": "blue",
   "cacheSeconds": 300
 }

.pipelex/inference/backends.toml

@@ -16,6 +16,11 @@ enabled = true
 endpoint = "https://inference.pipelex.com/v1"
 api_key = "${PIPELEX_INFERENCE_API_KEY}"
 
+[anthropic]
+enabled = true
+api_key = "${ANTHROPIC_API_KEY}"
+claude_4_tokens_limit = 8192
+
 [azure_openai]
 display_name = "Azure OpenAI"
 enabled = true
@@ -28,53 +33,54 @@ display_name = "Amazon Bedrock"
 enabled = true
 aws_region = "${AWS_REGION}"
 
-[google]
-display_name = "Google AI"
+[blackboxai]
+display_name = "BlackBox AI"
 enabled = true
-api_key = "${GOOGLE_API_KEY}"
+endpoint = "https://api.blackbox.ai/v1"
+api_key = "${BLACKBOX_API_KEY}"
 
-[vertexai]
-display_name = "Google Vertex AI"
+[fal]
+display_name = "FAL"
 enabled = true
-gcp_project_id = "${GCP_PROJECT_ID}"
-gcp_location = "${GCP_LOCATION}"
-gcp_credentials_file_path = "${GCP_CREDENTIALS_FILE_PATH}"
+api_key = "${FAL_API_KEY}"
 
-[openai]
-display_name = "OpenAI"
+[google]
+display_name = "Google AI"
 enabled = true
-api_key = "${OPENAI_API_KEY}"
+api_key = "${GOOGLE_API_KEY}"
 
-[anthropic]
+[groq]
+display_name = "Groq"
 enabled = true
-api_key = "${ANTHROPIC_API_KEY}"
-claude_4_tokens_limit = 8192
+endpoint = "https://api.groq.com/openai/v1"
+api_key = "${GROQ_API_KEY}"
 
 [mistral]
 display_name = "Mistral AI"
 enabled = true
 api_key = "${MISTRAL_API_KEY}"
 
-[xai]
-display_name = "xAI"
-enabled = true
-endpoint = "https://api.x.ai/v1"
-api_key = "${XAI_API_KEY}"
-
 [ollama]
 enabled = true
 endpoint = "http://localhost:11434/v1"
 
-[blackboxai]
-display_name = "BlackBox AI"
+[openai]
+display_name = "OpenAI"
 enabled = true
-endpoint = "https://api.blackbox.ai/v1"
-api_key = "${BLACKBOX_API_KEY}"
+api_key = "${OPENAI_API_KEY}"
 
-[fal]
-display_name = "FAL"
+[vertexai]
+display_name = "Google Vertex AI"
 enabled = true
-api_key = "${FAL_API_KEY}"
+gcp_project_id = "${GCP_PROJECT_ID}"
+gcp_location = "${GCP_LOCATION}"
+gcp_credentials_file_path = "${GCP_CREDENTIALS_FILE_PATH}"
+
+[xai]
+display_name = "xAI"
+enabled = true
+endpoint = "https://api.x.ai/v1"
+api_key = "${XAI_API_KEY}"
 
 [internal] # software-only backend, runs internally, without AI
 enabled = true

.pipelex/inference/backends/groq.toml

@@ -0,0 +1,129 @@
+################################################################################
+# Groq Backend Configuration
+################################################################################
+#
+# This file defines the model specifications for Groq models.
+# It contains model definitions for various LLM models accessible through
+# the Groq API, including text-only and vision-capable models.
+#
+# Configuration structure:
+# - Each model is defined in its own section with the model name as the header
+# - Headers with dots or slashes must be quoted (e.g., ["meta-llama/llama-4-scout"])
+# - Model costs are in USD per million tokens (input/output)
+# - Vision models support max 5 images per request, 33MP max resolution
+#
+# Documentation: https://docs.pipelex.com
+# Support: https://go.pipelex.com/discord
+#
+################################################################################
+
+################################################################################
+# MODEL DEFAULTS
+################################################################################
+
+[defaults]
+model_type = "llm"
+sdk = "groq"
+prompting_target = "groq"
+
+################################################################################
+# PRODUCTION TEXT MODELS
+################################################################################
+
+# --- Meta Llama 3.x Series ----------------------------------------------------
+["llama-3.1-8b-instant"]
+model_id = "llama-3.1-8b-instant"
+max_tokens = 131072
+inputs = ["text"]
+outputs = ["text", "structured"]
+costs = { input = 0.05, output = 0.08 }
+
+["llama-3.3-70b-versatile"]
+model_id = "llama-3.3-70b-versatile"
+max_tokens = 32768
+inputs = ["text"]
+outputs = ["text", "structured"]
+costs = { input = 0.59, output = 0.79 }
+
+# --- Meta Llama Guard ---------------------------------------------------------
+["meta-llama/llama-guard-4-12b"]
+model_id = "meta-llama/llama-guard-4-12b"
+max_tokens = 1024
+inputs = ["text"]
+outputs = ["text", "structured"]
+costs = { input = 0.20, output = 0.20 }
+
+# --- OpenAI GPT-OSS Models ----------------------------------------------------
+["openai/gpt-oss-20b"]
+model_id = "openai/gpt-oss-20b"
+max_tokens = 65536
+inputs = ["text"]
+outputs = ["text", "structured"]
+costs = { input = 0.075, output = 0.30 }
+
+["openai/gpt-oss-120b"]
+model_id = "openai/gpt-oss-120b"
+max_tokens = 65536
+inputs = ["text"]
+outputs = ["text", "structured"]
+costs = { input = 0.15, output = 0.60 }
+
+# --- Groq Compound Systems ----------------------------------------------------
+["groq/compound"]
+model_id = "groq/compound"
+max_tokens = 8192
+inputs = ["text"]
+outputs = ["text", "structured"]
+costs = { input = 0.15, output = 0.45 }
+
+["groq/compound-mini"]
+model_id = "groq/compound-mini"
+max_tokens = 8192
+inputs = ["text"]
+outputs = ["text", "structured"]
+costs = { input = 0.10, output = 0.30 }
+
+################################################################################
+# PREVIEW MODELS
+################################################################################
+
+# --- Meta Llama 4 Vision Models (Preview) -------------------------------------
+["meta-llama/llama-4-scout-17b-16e-instruct"]
+model_id = "meta-llama/llama-4-scout-17b-16e-instruct"
+max_tokens = 8192
+inputs = ["text", "images"]
+outputs = ["text", "structured"]
+max_prompt_images = 5
+costs = { input = 0.11, output = 0.34 }
+
+["meta-llama/llama-4-maverick-17b-128e-instruct"]
+model_id = "meta-llama/llama-4-maverick-17b-128e-instruct"
+max_tokens = 8192
+inputs = ["text", "images"]
+outputs = ["text", "structured"]
+max_prompt_images = 5
+costs = { input = 0.20, output = 0.60 }
+
+# --- Moonshot Kimi K2 ---------------------------------------------------------
+["moonshotai/kimi-k2-instruct-0905"]
+model_id = "moonshotai/kimi-k2-instruct-0905"
+max_tokens = 16384
+inputs = ["text"]
+outputs = ["text", "structured"]
+costs = { input = 1.00, output = 3.00 }
+
+# --- OpenAI Safety Model ------------------------------------------------------
+["openai/gpt-oss-safeguard-20b"]
+model_id = "openai/gpt-oss-safeguard-20b"
+max_tokens = 65536
+inputs = ["text"]
+outputs = ["text", "structured"]
+costs = { input = 0.075, output = 0.30 }
+
+# --- Qwen 3 -------------------------------------------------------------------
+["qwen/qwen3-32b"]
+model_id = "qwen/qwen3-32b"
+max_tokens = 40960
+inputs = ["text"]
+outputs = ["text", "structured"]
+costs = { input = 0.29, output = 0.59 }

.pipelex/inference/backends/mistral.toml

@@ -129,7 +129,7 @@ outputs = ["text", "structured"]
 costs = { input = 0.4, output = 2.0 }
 
 ################################################################################
-# OCR MODELS
+# EXTRACTION MODELS
 ################################################################################
 
 # --- OCR Models ---------------------------------------------------------------
@@ -140,4 +140,3 @@ max_tokens = 131072
 inputs = ["pdf", "image"]
 outputs = ["pages"]
 costs = { input = 0.4, output = 2.0 }
-

.pipelex/inference/deck/base_deck.toml

@@ -21,13 +21,18 @@ base-claude = "claude-4.5-sonnet"
 base-gpt = "gpt-4o"
 base-gemini = "gemini-2.5-flash"
 base-mistral = "mistral-medium"
+base-groq = "llama-3.3-70b-versatile"
 
 best-gpt = "gpt-4o"
 best-claude = "claude-4.1-opus"
 best-gemini = "gemini-2.5-pro"
 best-mistral = "mistral-medium"
 best-grok = "grok-3"
 
+# Groq-specific aliases
+fast-groq = "llama-3.1-8b-instant"
+vision-groq = "meta-llama/llama-4-scout-17b-16e-instruct"
+
 # Image generation aliases
 base-img-gen = "flux-pro/v1.1"
 best-img-gen = "flux-pro/v1.1-ultra"
@@ -39,6 +44,7 @@ fast-img-gen = "fast-lightning-sdxl"
 
 [waterfalls]
 smart_llm = [
+    "llama-3.3-70b-versatile",
     "claude-4.5-sonnet",
     "claude-4.1-opus",
     "claude-4-sonnet",
@@ -60,7 +66,7 @@ smart_llm_for_structured = [
     "claude-4-sonnet",
     "gpt-4o",            # we use gpt-4o here rather than gpt-5 until we handle reasoning parameters
 ]
-
+llm_for_creativity = ["claude-4.1-opus", "gemini-2.5-pro", "gpt-4o"]
 llm_for_large_codebase = [
     "gemini-2.5-pro",
     "claude-4.5-sonnet",
@@ -81,6 +87,12 @@ cheap_llm_for_vision = [
     "claude-3-haiku",
 ]
 cheap_llm_for_structured = ["gpt-4o-mini", "mistral-small", "claude-3-haiku"]
+cheap_llm_for_creativity = [
+    "gemini-2.5-flash",
+    "grok-3-mini",
+    "gpt-4o-mini",
+    "claude-4.5-haiku",
+]
 
 ####################################################################################################
 # LLM Presets
@@ -115,10 +127,15 @@ llm_for_table_to_text = { model = "base-claude", temperature = 0.3 }
 llm_to_prompt_img_gen = { model = "base-claude", temperature = 0.2 }
 llm_to_prompt_img_gen_cheap = { model = "gpt-4o-mini", temperature = 0.5 }
 
+# Groq-specific presets (fast inference, low cost)
+llm_groq_fast_text = { model = "fast-groq", temperature = 0.2 }
+llm_groq_balanced = { model = "base-groq", temperature = 0.5 }
+llm_groq_vision = { model = "vision-groq", temperature = 0.3 }
+
 # LLM Presets — For Testing ---------------------------------------------------------------------
 
 llm_for_testing_gen_text = { model = "cheap_llm", temperature = 0.5 }
-llm_for_testing_gen_object = { model = "cheap_llm_for_structured", temperature = 0.5 }
+llm_for_testing_gen_object = { model = "cheap_llm_for_structured", temperature = 0.1 }
 llm_for_testing_vision = { model = "cheap_llm_for_vision", temperature = 0.5 }
 llm_for_testing_vision_structured = { model = "cheap_llm_for_vision", temperature = 0.5 }
 

.pipelex/inference/routing_profiles.toml

@@ -149,3 +149,7 @@ description = "Example routing profile using specific models"
 
 "grok-3" = "pipelex_inference"
 "grok-3-mini" = "xai"
+
+[profiles.all_groq]
+description = "Use groq backend for all its supported models"
+default = "groq"

.pipelex/pipelex.toml

@@ -43,12 +43,25 @@ excluded_dirs = [
     "results",
 ]
 
+[pipelex.builder_config]
+# Uncomment to change where the generated pipelines are saved
+# default_output_dir = "."
+# default_bundle_file_name = "bundle"
+# default_directory_base_name = "pipeline"
+
 [pipelex.log_config]
 default_log_level = "INFO"
 
 [pipelex.log_config.package_log_levels]
 pipelex = "INFO"
 
+[cogt]
+[cogt.model_deck_config]
+# Uncomment to disable model fallback: it will raise errors instead of using secondary model options
+# is_model_fallback_enabled = false
+# Uncomment to change the reaction to missing presets: "raise" (default), "log" or "none"
+# missing_presets_reaction = "raise"
+
 [cogt.llm_config]
 # Uncomment any of these to enable dumping the inputs or outputs of text-genration with an LLM
 # is_dump_text_prompts_enabled = true

.vscode/launch.json

@@ -59,9 +59,9 @@
             "program": "${workspaceFolder}/.venv/bin/pipelex",
             "args": [
                 "run",
-                "results/test_1.plx",
+                "results/pipeline_01/bundle.plx",
                 "--inputs",
-                "results/inputs_2.json",
+                "results/pipeline_01/inputs.json",
             ],
             "console": "integratedTerminal",
             "justMyCode": false

CHANGELOG.md

@@ -1,5 +1,28 @@
 # Changelog
 
+## [v0.15.4] - 2025-11-12
+
+### Added
+ - **Enhanced `pipelex build` Command**: Now generates a self-contained directory (e.g., `results/pipeline_01/`) containing `bundle.plx`, `inputs.json`, `run_{pipe_code}.py`, `bundle_view.html`, and `bundle_view.svg`. New CLI options: `--output-name (-o)` for custom base name, `--output-dir` for custom directory, and `--no-extras` to generate only the `.plx` file.
+ - **CLI Readiness Check**: Verifies that a virtual environment is active for development installations.
+ - **Model Deck Presets**: Added `llm_for_creativity` and `cheap_llm_for_creativity` model waterfalls, plus `[cogt.model_deck_config]` section in `pipelex.toml` for configuring model fallback behavior.
+ - WIP: **Groq Inference Backend Support**: Integrated full support for the Groq API with configuration file (`.pipelex/inference/backends/groq.toml`), model specifications, costs, capabilities, new model aliases (`base-groq`, `fast-groq`, `vision-groq`), and routing profile (`all_groq`).
+
+### Changed
+ - **CLI Output and Visualization**: Overhauled command-line output with rich, table-based layouts for pipeline components. Final output of `pipelex run` is now pretty-printed and adapts to content type.
+ - **Documentation**: Updated "Get Started" and "Build Reliable AI Workflows" to reflect new directory-based build output and CLI options.
+ - **Internal Code Refactoring**: Reorganized exception hierarchy into dedicated `exceptions.py` files per module, centralized validation logic into `validation.py` modules, added `ValueError` to blueprints, and removed unused exceptions for improved maintainability.
+- Updated pytest to `>=9.0.1` to support their new `pyproject.toml` config format.
+
+### Fixed
+ - Adjusted default temperature for `llm_for_testing_gen_object` preset from `0.5` to `0.1` for more deterministic structured data generation.
+ - Corrected `LLM_FOR_VISUAL_DESIGN` skill in `pipe_llm_spec` to point to `cheap_llm_for_creativity` preset.
+ - Standardized input variable names in `pipe_llm_vision.plx` from `imageA`/`imageB` to `image_a`/`image_b`.
+
+### Removed
+ - Deleted `pipelex/core/validation_errors.py` file as part of exception hierarchy refactoring.
+
+
 ## [v0.15.3] - 2025-11-07
 
 ### Fixed

Makefile

@@ -222,7 +222,7 @@ codex-tests: env
 gha-tests: env
 	$(call PRINT_TITLE,"Unit testing for github actions")
 	@echo "• Running unit tests for github actions (excluding inference and gha_disabled)"
-	$(VENV_PYTEST) --exitfirst --quiet -m "(dry_runnable or not inference) and not (gha_disabled or pipelex_api)" || [ $$? = 5 ]
+	$(VENV_PYTEST) -n auto --exitfirst --quiet -m "(dry_runnable or not inference) and not (gha_disabled or pipelex_api)" || [ $$? = 5 ]
 
 run-all-tests: env
 	$(call PRINT_TITLE,"Running all unit tests")