Skip to content

fix(toolbox_reactive_scripts_editor): sort and dedupe created series before append#138

Draft
pabloinigoblasco wants to merge 1 commit into
mainfrom
fix/reactive-script-out-of-order-timestamps
Draft

fix(toolbox_reactive_scripts_editor): sort and dedupe created series before append#138
pabloinigoblasco wants to merge 1 commit into
mainfrom
fix/reactive-script-out-of-order-timestamps

Conversation

@pabloinigoblasco

Copy link
Copy Markdown
Contributor

Summary

The reactive-script editor was leaving half-written orphan datasets behind when a user script pushed points in non-monotonic order. The datastore append path requires monotonically non-decreasing timestamps per topic, but the dialog drives the user script once per timestamp of the first loaded series (batch mode), so a script that fills a whole output series in a single call re-pushes the same points on every tick. The resulting series has timestamps that wrap back to 0, and `appendRecord` rejects them with:

failed to append record: datastore: Out-of-order timestamp: t=0 < last_timestamp=9950000000

Fix

Sort each created series by timestamp (quantized to the `int64` nanoseconds the store uses) and collapse duplicate timestamps (last value wins) before writing. This makes the write path robust to push order for any script style — the user no longer has to worry about the order in which their Lua code pushes samples.

The summary string now reports the points actually written instead of the raw push count, so duplicates collapsing don't read as a discrepancy.

Test plan

  • Manual: run a Lua script that fills an output series in a single call (`for t = 0, N do push(t, ...) end`), reload it across multiple input ticks → series imports cleanly, no `Out-of-order timestamp` error.
  • CI green on all matrix jobs.

…before append

The datastore append path requires monotonically non-decreasing timestamps
per topic. The dialog drives the user script once per timestamp of the first
loaded series (batch mode), so a script that fills a whole output series in a
single call re-pushes the same points on every tick. The resulting series has
timestamps that wrap back to 0, and appendRecord rejects them with
"Out-of-order timestamp", leaving a half-written orphan dataset behind.

Sort each created series by timestamp (quantized to the int64 nanoseconds the
store uses) and collapse duplicate timestamps (last value wins) before writing.
This makes the write path robust to push order for any script style, and the
summary now reports the points actually written instead of the raw push count.
@pabloinigoblasco pabloinigoblasco marked this pull request as draft June 26, 2026 08:38
pabloinigoblasco added a commit that referenced this pull request Jun 27, 2026
Ports PlotJuggler 3's Custom Series / Function Editor as a data-only toolbox
plugin. The UI mirrors PJ3: a Single Function tab (drag-drop source, additional
sources, function body) and a Batch Functions tab, plus a function library — an
interactive browser sub-panel and Import / Export of the library to a JSON file.

Data-only: the plugin never runs a script itself. On Save it hands the host a
self-describing Luau class (N inputs -> M outputs) via `pj.data_processors.v1`,
run live as a DerivedEngine node; the live preview is produced host-side too
(ephemeral transform + validate). There is no local Lua engine in the plugin.

Requires plotjuggler_sdk 0.14.0 — the data-processor script API, the interactive
sub-panel + table radio-column dialog additions (#136), and the save-file picker
used by the library Export (#138). The `extern/plotjuggler_core` submodule is
pinned to an SDK build carrying both.

Contents:
- transform_editor_plugin.cpp — dialog + toolbox: drag-drop, function library
  browser, Import/Export, host-side preview, name-prompt + overwrite on Save,
  createTransform on Save.
- transform_editor_dialog.ui — PJ3-style UI.
- manifest.json / CMakeLists.txt / conanfile.py — registration; SDK 0.14.0.

Pairs with the host side: PlotJuggler/PJ4#252.
pabloinigoblasco added a commit that referenced this pull request Jun 28, 2026
Ports PlotJuggler 3's Custom Series / Function Editor as a data-only toolbox
plugin. The UI mirrors PJ3: a Single Function tab (drag-drop source, additional
sources with a primary-source radio in the first column, function body), a Batch
Functions tab, and a function library — an interactive browser sub-panel and
Import / Export of the library to a JSON file.

Data-only: the plugin never runs a script itself. On Save it hands the host a
self-describing Luau class (N inputs -> M outputs) via `pj.data_processors.v1`,
run live as a DerivedEngine node; the live preview is produced host-side too
(ephemeral transform + validate). There is no local Lua engine in the plugin.

Requires plotjuggler_sdk 0.14.0 — the data-processor script API, the interactive
sub-panel + table radio-column dialog additions (#136), and the save-file picker
used by the library Export (#138). The extern/plotjuggler_core submodule is
pinned to an SDK build carrying both.

Contents:
- transform_editor_plugin.cpp — dialog + toolbox: drag-drop, primary-source radio,
  function library browser, Import/Export, host-side preview, name-prompt +
  overwrite on Save, createTransform on Save.
- transform_editor_dialog.ui — PJ3-style UI.
- manifest.json / CMakeLists.txt / conanfile.py — registration; SDK 0.14.0.

Pairs with the host side: PlotJuggler/PJ4#252.
Alvvalencia added a commit that referenced this pull request Jun 29, 2026
… contract

The "Save rule as..." picker now declares its extension via the new
setSaveFilePicker(..., "json") arg (SDK PR #138). The host appends ".json" when
the user types a name without one, so the manual extension fix-up in saveRuleTo()
is removed — the host is the single source of truth (verified E2E: "a" -> "a.json").

DEPENDENCY: the 5-arg setSaveFilePicker requires the SDK to carry #138. The
extern/plotjuggler_core gitlink bump to a #138-containing commit follows once #138
merges into the SDK feature/plot-markers branch; held local (not pushed) until then.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0122VUqQqWd1KVQQfhrG7632
Alvvalencia added a commit that referenced this pull request Jun 29, 2026
…eeds it)

The standalone root CMakeLists' find_package(GTest REQUIRED) runs for every
per-plugin build, but toolbox_anomaly_detector/conanfile.py never listed gtest
(every other plugin recipe does). The omission was masked because the Linux
per-plugin leg died earlier resolving pj_scripting_core; with that spurious
dependency gone (prev commit), Conan now succeeds and CMake configure failed on
"Could NOT find GTest". Add gtest/1.17.0 to the recipe, matching the other plugins.

Verified via the per-plugin path (./build.sh toolbox_anomaly_detector): GTest
resolves, configure succeeds, the .so links.

NOTE: this leg still fails to COMPILE against the published SDK (0.13.0) because
anomaly_detector.cpp adopts the 5-arg setSaveFilePicker(default_suffix) contract,
which lives in the not-yet-merged SDK default_suffix PR. That is the known, planned
convergence dependency — it greens once the unified SDK PR merges and SDK_VERSION
is bumped. This commit removes the gtest blocker so #138 is the single remaining one.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0122VUqQqWd1KVQQfhrG7632
Alvvalencia added a commit that referenced this pull request Jun 29, 2026
…th default_suffix)

The Anomaly Detector adopted the SDK default_suffix save-file-picker contract
(anomaly_detector.cpp:184 calls the 5-arg setSaveFilePicker), but the pinned SDK
submodule (4ed853a) predated that API — its setSaveFilePicker took 4 args, so the
plugin failed to compile on all three platforms (the last remaining CI blocker, #3).

Pablo unified the markers PR (#135) and the default_suffix PR (#138) into #135;
a3d15132 = 4ed853a + the single "feat(dialog): add save-file picker (#138)" commit,
which adds the 5-arg setSaveFilePicker (default_suffix="") and the
saveFilePickerDefaultSuffix accessor. Move the gitlink forward to it.

SDK_VERSION stays 0.13.0: cloudsmith has no prebuilt 0.13.0 (confirmed in the
acquire-core log), so CI builds the SDK from this submodule — there is no version
collision, and macOS/Windows build core from source anyway.

Verified with a clean rebuild (not the locally-patched cache): purged 0.13.0 from
the Conan cache, rebuilt the SDK from this submodule via conan create, and built
toolbox_anomaly_detector — anomaly_detector.cpp compiles and the .so links.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0122VUqQqWd1KVQQfhrG7632
pabloinigoblasco added a commit that referenced this pull request Jun 29, 2026
…158)

* feat: Transform Editor toolbox plugin — PJ3 Custom Series port

Ports PlotJuggler 3's Custom Series / Function Editor as a data-only toolbox
plugin. The UI mirrors PJ3: a Single Function tab (drag-drop source, additional
sources with a primary-source radio in the first column, function body), a Batch
Functions tab, and a function library — an interactive browser sub-panel and
Import / Export of the library to a JSON file.

Data-only: the plugin never runs a script itself. On Save it hands the host a
self-describing Luau class (N inputs -> M outputs) via `pj.data_processors.v1`,
run live as a DerivedEngine node; the live preview is produced host-side too
(ephemeral transform + validate). There is no local Lua engine in the plugin.

Requires plotjuggler_sdk 0.14.0 — the data-processor script API, the interactive
sub-panel + table radio-column dialog additions (#136), and the save-file picker
used by the library Export (#138). The extern/plotjuggler_core submodule is
pinned to an SDK build carrying both.

Contents:
- transform_editor_plugin.cpp — dialog + toolbox: drag-drop, primary-source radio,
  function library browser, Import/Export, host-side preview, name-prompt +
  overwrite on Save, createTransform on Save.
- transform_editor_dialog.ui — PJ3-style UI.
- manifest.json / CMakeLists.txt / conanfile.py — registration; SDK 0.14.0.

Pairs with the host side: PlotJuggler/PJ4#252.

* feat(toolbox_transform_editor): sync plugin to latest

Bring the Transform Editor plugin up to date: per-snippet language tagging in
the library, Python module emission for the Python backend, and the
validateScript(kind, language, script) update for the unified data_processors SDK.

* style: clang-format the transform editor plugin

Apply clang-format so pre-commit passes.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant