Merge pull request #162 from cubewise-code/docs/issue-160-mode-cube-reads

docs: clarify --mode is required for optimized cube reads (#160)

Author: nicolasbisurgi

CHANGELOG.md

@@ -2,6 +2,19 @@
 
 All notable changes to RushTI are documented in this file.
 
+## Unreleased — docs: `--mode` for cube reads (#160)
+
+- **Docs fix:** corrected the long-standing claim that `--mode` is
+  deprecated/ignored. It is only auto-detected (and ignored) for **file
+  sources** (`--tasks`). A **cube read** (`--tm1-instance`) cannot infer
+  the mode — every workflow occupies the same cube measures — so it
+  defaults to `norm` and silently drops the `predecessors` measure unless
+  `--mode opt` is passed. This caused predecessors to disappear from
+  cube-read execution plans (e.g. `Sample_Optimal_Mode`). Updated the CLI
+  reference, settings reference, migration guide, TM1 integration guide
+  (new "Choosing the mode for cube reads" section), getting-started
+  task-files page, and the `rushti run --help` text.
+
 ## Unreleased — `feat/issue-154-v12-load-results`
 
 - Fix: `rushti build` now installs a TM1-version-aware `}rushti.load.results`

docs/advanced/cli-reference.md

@@ -55,7 +55,7 @@ rushti --tasks FILE [options]          # 'run' is the default command
 | `--retries` | `-r` | INT | `0` | Retry count for failed TI executions. Uses exponential backoff. |
 | `--result` | `-o` | PATH | *(empty)* | Output CSV path for execution summary. Omit to skip CSV creation. |
 | `--settings` | `-s` | PATH | auto | Path to `settings.ini`. Auto-discovered if omitted. |
-| `--mode` | `-m` | CHOICE | auto | **Deprecated.** Mode is auto-detected from file content. Ignored. |
+| `--mode` | `-m` | CHOICE | `norm` | Execution mode: `norm` or `opt`. **Ignored for file sources** (`--tasks`), where mode is auto-detected from file content. **Required for cube reads** (`--tm1-instance`) when the workflow uses explicit `predecessors` — pass `--mode opt`, otherwise the cube is read in `norm` mode and predecessors are dropped. See [TM1 integration → Choosing the mode for cube reads](../features/tm1-integration.md#choosing-the-mode-for-cube-reads). |
 | `--exclusive` | `-x` | FLAG | `false` | Enable exclusive mode. Waits for other RushTI sessions to finish. |
 | `--force` | `-f` | FLAG | `false` | Bypass exclusive mode checks and run immediately. |
 | `--optimize` | | CHOICE | *(none)* | Enable task optimization with a scheduling algorithm: `longest_first` or `shortest_first`. |
@@ -80,9 +80,12 @@ rushti run --tasks critical.json --exclusive --result results/critical.csv
 # Override workflow name for a file-based run
 rushti run --tasks daily-etl.json --workflow DailyETL --max-workers 8
 
-# Read task file from TM1 cube
+# Read task file from TM1 cube (norm mode — wait-based sequencing)
 rushti run --tm1-instance tm1srv01 --workflow DailyETL --max-workers 8
 
+# Read an optimized workflow from the cube — --mode opt is required to honour predecessors
+rushti run --tm1-instance tm1srv01 --workflow DailyETL --mode opt --max-workers 8
+
 # Optimize with shortest-first scheduling (good for shared-resource TM1 workloads)
 rushti run --tasks tasks.json --max-workers 20 --optimize shortest_first
 

docs/advanced/migration-guide.md

@@ -62,16 +62,19 @@ Logging is configured separately via `config/logging_config.ini` (Python's stand
 
 All settings have sensible defaults. You do not need to create a `settings.ini` to use v2.0 -- it works with built-in defaults.
 
-### --mode Flag: Deprecated
+### --mode Flag: Auto-Detected for Files, Still Required for Cube Reads
 
 In v1.x, the `--mode` flag (`norm` or `opt`) controlled whether RushTI used level-based (wait) or dependency-based execution.
 
-In v2.0, mode is **auto-detected** from file content:
+In v2.0, mode is **auto-detected from file content** for file sources (`--tasks`):
 
 - JSON files always use DAG execution (optimized mode).
 - TXT files use `norm` mode if they contain `wait` keywords, or `opt` mode if tasks have `id` and `predecessors` fields.
 
-The `--mode` flag is accepted for backward compatibility but ignored.
+For file sources the `--mode` flag is therefore accepted for backward compatibility but ignored.
+
+!!! warning "Cube reads still need `--mode`"
+    Auto-detection does **not** apply when reading from a TM1 cube (`--tm1-instance`). The cube stores every workflow with the same measures, so RushTI cannot tell a wait-based workflow from a predecessor-based one. Cube reads default to `norm` (wait-based) and **ignore the `predecessors` measure** unless you pass `--mode opt`. See [TM1 integration → Choosing the mode for cube reads](../features/tm1-integration.md#choosing-the-mode-for-cube-reads).
 
 ### Task File: id and predecessors in TXT Files
 
@@ -413,9 +416,9 @@ rushti run --tasks tasks.json --log-level DEBUG
 
 Check that `max_workers` is set in the correct place with the correct spelling.
 
-### Issue: Mode Deprecation Warning
+### Issue: Predecessors Missing When Running from a Cube
 
-If you see a deprecation warning about `--mode`, it is safe to ignore. Remove `--mode` from your scripts when convenient -- mode detection is automatic.
+If you run an optimized workflow from the cube (`--tm1-instance`) and the resulting plan ignores your `predecessors`, you are running in the default `norm` mode. Add `--mode opt` to the command (or set `mode = opt` in `settings.ini`). Mode auto-detection applies to file sources only — see [TM1 integration → Choosing the mode for cube reads](../features/tm1-integration.md#choosing-the-mode-for-cube-reads). For `--tasks` file sources you can safely omit `--mode` entirely.
 
 ### Issue: TXT File Not Auto-Detecting Mode
 

docs/advanced/settings-reference.md

@@ -46,9 +46,9 @@ Common execution settings that control basic RushTI behavior.
 | `max_workers` | int | `4` | Maximum number of parallel workers. Valid range: 1--100. Recommended: start at 4, increase based on TM1 server capacity. |
 | `retries` | int | `0` | Number of retry attempts for failed TI process executions. Valid range: 0--10. Retries use exponential backoff (1s, 2s, 4s, ...). |
 | `result_file` | str | `""` (empty) | CSV output path for execution summary. Empty string means no CSV is created. |
-| `mode` | str | `norm` | **Deprecated.** Execution mode is now auto-detected from file content. JSON files always use DAG execution; TXT files use the mode indicated by their content structure. Kept for backward compatibility only. |
+| `mode` | str | `norm` | Default execution mode (`norm` or `opt`) for **cube reads** (`--tm1-instance`). A cube read cannot auto-detect its mode, so this value (or the `--mode` CLI override) decides whether the `predecessors` measure is honoured (`opt`) or the `wait` measure drives sequencing (`norm`). **Ignored for file sources**, where the mode is auto-detected from file content. See [Choosing the mode for cube reads](../features/tm1-integration.md#choosing-the-mode-for-cube-reads). |
 
-**Overridable via CLI:** `--max-workers` / `-w`, `--retries` / `-r`, `--result` / `-o`, `--mode` / `-m` (deprecated)
+**Overridable via CLI:** `--max-workers` / `-w`, `--retries` / `-r`, `--result` / `-o`, `--mode` / `-m`
 
 **Overridable via JSON task file:** `max_workers`, `retries`, `result_file`
 
@@ -243,7 +243,10 @@ Copy this template to `config/settings.ini` and uncomment the settings you want
 # Default: (empty - no CSV created)
 # result_file = rushti.csv
 
-# Execution mode (deprecated - auto-detected from file content)
+# Default execution mode for cube reads (--tm1-instance): norm or opt.
+# Cube reads cannot auto-detect the mode; set opt here if most of your
+# cube-stored workflows use explicit predecessors. Ignored for file
+# sources (--tasks), where the mode is auto-detected from file content.
 # Default: norm
 # mode = norm
 

docs/features/tm1-integration.md

@@ -69,6 +69,17 @@ rushti run --tm1-instance tm1srv01 --workflow Sample_Stage_Mode --max-workers 4
 
 RushTI reads the task definitions from the cube, builds the execution plan, and runs them — exactly like running from a file.
 
+!!! warning "Optimized workflows need `--mode opt`"
+    When you run from the cube, RushTI **cannot auto-detect** whether the workflow is wait-based (`norm`) or predecessor-based (`opt`) — the cube row layout is identical either way. It therefore defaults to `norm`, which uses the `wait` measure for sequencing and **ignores the `predecessors` measure entirely**.
+
+    To run an optimized workflow (one that defines explicit `predecessors`), you must pass `--mode opt`:
+
+    ```bash
+    rushti run --tm1-instance tm1srv01 --workflow Sample_Optimal_Mode --mode opt --max-workers 4
+    ```
+
+    Without `--mode opt`, the predecessors silently disappear from the execution plan (RushTI logs a `predecessors ignored in norm mode` warning, but the run still proceeds). See [Choosing the mode for cube reads](#choosing-the-mode-for-cube-reads) below.
+
 ### View Results in the Cube
 
 Once the run completes, RushTI writes the results back to the same cube under a timestamped `rushti_run_id` element. Open the `rushti_results` subset to see execution data:
@@ -180,14 +191,41 @@ rushti run --tm1-instance tm1srv01 --workflow Sample_Normal_Mode --max-workers 4
 # Run the stage-based sample
 rushti run --tm1-instance tm1srv01 --workflow Sample_Stage_Mode --max-workers 4
 
-# Run the optimized sample (with explicit dependencies)
-rushti run --tm1-instance tm1srv01 --workflow Sample_Optimal_Mode --max-workers 4
+# Run the optimized sample (with explicit dependencies) — note --mode opt
+rushti run --tm1-instance tm1srv01 --workflow Sample_Optimal_Mode --mode opt --max-workers 4
 ```
 
+`Sample_Optimal_Mode` defines its task ordering through the `predecessors` measure, so it **requires `--mode opt`**. Run it without that flag and RushTI reads the cube in `norm` mode, drops the predecessors, and falls back to wait-based sequencing — the run "works" but does not respect the dependency graph you defined.
+
 Check the `rushti` cube afterward to confirm that results were written.
 
 ---
 
+## Choosing the Mode for Cube Reads
+
+Unlike file sources — where the mode is auto-detected from the file's structure — a cube read **cannot infer the mode**. Every workflow occupies the same set of measures in the `rushti` cube (`wait`, `predecessors`, `instance`, `process`, …), so there is no structural signal that distinguishes a wait-based workflow from a predecessor-based one. You tell RushTI which one to use with `--mode`:
+
+| `--mode` | Sequencing driver | `wait` measure | `predecessors` measure |
+|----------|-------------------|----------------|------------------------|
+| `norm` (default) | `wait` markers create sequential groups | **Used** | **Ignored** (logs a warning if populated) |
+| `opt` | Explicit `predecessors` form a DAG | Ignored | **Used** |
+
+```bash
+# Wait-based (norm) workflow — the default, no flag needed
+rushti run --tm1-instance tm1srv01 --workflow Sample_Normal_Mode --max-workers 4
+
+# Predecessor-based (opt) workflow — --mode opt is required
+rushti run --tm1-instance tm1srv01 --workflow Sample_Optimal_Mode --mode opt --max-workers 4
+```
+
+!!! tip "Make `opt` the default if most of your cube workflows use predecessors"
+    If the majority of your cube-stored workflows are predecessor-based, set `mode = opt` in the `[defaults]` section of `settings.ini` so you don't have to pass `--mode opt` on every run. The CLI `--mode` flag still overrides it per run.
+
+!!! note "`--mode` and file sources"
+    For `--tasks` (JSON/TXT files) the mode is auto-detected from content and `--mode` is ignored. The flag only changes behaviour for cube reads (`--tm1-instance`).
+
+---
+
 ## Configuration
 
 ### Settings Reference

docs/getting-started/task-files.md

@@ -72,11 +72,14 @@ Each task is defined by writing values to these measures:
 ### Run Tasks from the Cube
 
 ```bash
-rushti run --tm1-instance tm1srv01 --workflow daily-refresh --max-workers 4
+rushti run --tm1-instance tm1srv01 --workflow daily-refresh --mode opt --max-workers 4
 ```
 
 RushTI reads the task definitions from the cube, builds the DAG, and executes — exactly like running from a file.
 
+!!! warning "`--mode opt` is required for predecessor-based workflows"
+    Because the workflow above uses the `predecessors` measure, you must pass `--mode opt`. A cube read cannot auto-detect its mode (unlike a file source) and defaults to `norm`, which ignores `predecessors` and sequences tasks by the `wait` measure instead. See [TM1 integration → Choosing the mode for cube reads](../features/tm1-integration.md#choosing-the-mode-for-cube-reads).
+
 ### View Results
 
 When result pushing is enabled, RushTI writes execution results back to the same cube. Each run gets its own `rushti_run_id` element (e.g., `20260209_143022`), so you can see the full execution history:

src/rushti/cli.py

@@ -249,11 +249,14 @@ def create_argument_parser() -> argparse.ArgumentParser:
   %(prog)s -t tasks.txt -w 4 -r 2 -o results.csv
   %(prog)s --tasks tasks.json --max-workers 8 --retries 3
   %(prog)s --tm1-instance tm1srv01 --workflow Sample --max-workers 4
+  %(prog)s --tm1-instance tm1srv01 --workflow Sample_Optimal_Mode --mode opt
 
 Configuration:
   Default values can be set in settings.ini (see settings.ini.template).
   Settings precedence: CLI args > JSON task file > settings.ini > defaults
-  File format is auto-detected (--mode is deprecated for file sources).
+  File sources auto-detect the mode (--mode is ignored). Cube reads
+  (--tm1-instance) cannot, and default to norm; pass --mode opt to honour
+  the 'predecessors' measure.
         """,
     )
 
@@ -286,7 +289,7 @@ def create_argument_parser() -> argparse.ArgumentParser:
         dest="execution_mode",
         choices=["norm", "opt"],
         default=None,
-        help="[Deprecated] Execution mode is now auto-detected from file content. This option is kept for backwards compatibility but ignored.",
+        help="Execution mode: norm or opt. Auto-detected (and ignored) for file sources. For cube reads (--tm1-instance) it defaults to norm; pass 'opt' to honour the 'predecessors' measure.",
     )
 
     parser.add_argument(