Execution results are written to wrong measures

[cubewise-gng]

Bug Description

The results to TM1 are written to wrong measures
eg.

1. parameters are being written to status
2. status written to start_time
   etc, values are written incorrectly to next measure.

Steps to Reproduce

1. Ran Sample_Stage_Mode rushti
2. Check results for the run_id in TM1

Expected Behavior

Result values should sit in expected measures.

RushTI Version

2.3.1

Python Version

3.13.12

TM1 Server Version

11.8.03500.4

Additional Context

No response

[nicolasbisurgi]

Thanks for the detailed report — the "shifted by one measure" symptom is the key clue, and I've traced it to a root cause. This is a version-drift issue between the rushti Python package and the TM1 objects it deploys, not a bug in the current write logic itself.

Root cause

Results are loaded into the cube positionally. upload_results_to_tm1 produces a CSV with a fixed column order, and the }rushti.load.results TI reads it with asciiHeaderRecords=1 — meaning it skips the header row and maps each CSV column to a process variable by ordinal position, then CellPutSes each variable into an explicitly-named measure. So the CSV column order and the deployed TI's variable list must stay index-aligned.

In 2.3.0 (#156 / #157), a new chore measure was inserted at position 7, between process and parameters, in both the CSV contract and the TI variable list:

... process(6), chore(7), parameters(8), status(9), start_time(10) ...

If a cube/TI was originally built with a pre-2.3.0 rushti and the Python package was later upgraded without re-running rushti build, you end up with:

• new uploads → CSV with the chore column at position 7
• stale deployed TI → variable list without vchore

So the old TI reads one position too early from chore onward:

• old vparameters (pos 7) reads column 7 → the chore value
• old vstatus (pos 8) reads column 8 → the parameters value → "parameters written to status"
• old vstart_time (pos 9) reads column 9 → the status value → "status written to start_time"

…which is exactly the shift you described. Because the header is skipped rather than validated, the mismatch is silently scrambled instead of raising an error — which is also why it can surface at multiple sites after an in-place upgrade.

Fix / workaround

Re-run the builder against the instance:

rushti build --tm1-instance <instance>

This fully repairs a drifted deployment:

• the rushti_measure dimension is additively merged, so the missing chore element is added;
• the }rushti.load.results process is deleted and recreated with the current variable list (including vchore).

After that, positions realign and results land in the correct measures. (Existing scrambled rows from prior runs would need to be cleared/re-run, since they were written under the wrong measures.)

Proactive fix we're planning

Rebuilding after an upgrade shouldn't be something you have to discover by seeing corrupted data. We're going to make the load fail loudly instead of silently scrambling by having }rushti.load.results validate the CSV header row against the expected column order and ProcessQuit with a clear message on mismatch (today the header is simply skipped). We'll also add an explicit "re-run rushti build after upgrading" note to the changelog/upgrade docs.

Could you confirm whether this deployment's cube/TI predates 2.3.0? And once you've re-run rushti build, a fresh run should write results to the correct measures — let us know if it doesn't.

[nicolasbisurgi]

Fix is up in #170. It adds header validation to }rushti.load.results so a stale process fails loudly (with the mismatched columns named) instead of silently writing to the wrong measures, and documents the "re-run rushti build after upgrading" step. Verified end-to-end on live TM1 v11 (11.8) and v12 (12.5): all measures align on a normal load, and a drifted header now ProcessQuits.

In the meantime, the immediate remedy for this deployment is still to re-run rushti build --tm1-instance <instance> and clear/re-run any rows that were written under the wrong measures.