Add operant control of rewards based on velocity#568
Open
galenlynch wants to merge 8 commits into
Open
Conversation
galenlynch
changed the title
The probability-grid stages now run operant (is_operant=True), so abandoning a wait forfeits the reward — making the long delay a real wait-or-abandon decision (it was free water on a schedule before). Adds OperantLogic.abort_velocity_threshold (cm/s) as an ADDITIONAL abort source alongside grace distance and leaving the site: an operant choice aborts if velocity exceeds the threshold OR displacement exceeds grace_distance_threshold OR the animal leaves the reward site. It catches slow-creepers who lick while drifting forward (e.g. 860900: velocity stays <15 cm/s while engaged, ramps to ~45-55 to leave) — a pure spatial grace clips their genuine licking. Set to 15 on both grid stages; shaping stages leave it None. Optional (default None), so prior trainer states/JSON deserialize unchanged. NOT YET ENFORCED ON THE RIG (Python/curriculum side only): enforcing the threshold requires adding a velocity-abort branch ALONGSIDE the grace/leave-site sources in InstantiateSite.bonsai, plus regenerating the core JSON schema + C# (vr-foraging regenerate, needs .NET). The core schema and Generated.cs are not regenerated here. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…ity_threshold This curriculum schema was left stale on the branch — it embeds OperantLogic but was missing the new abort_velocity_threshold field that the other six curriculum schemas already carry. Regenerated so the branch is internally consistent and a rig-side regenerate.py only produces the core schema/C# diff. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…eshold Per review feedback: the velocity abort is evaluated on the same filtered velocity signal as stop detection (IsStopped = velocity < StopVelocityThreshold), so the two gates partition one velocity axis in opposite directions. Document that abort_velocity_threshold must stay >= the operative StopVelocityThreshold (else a locked stop instantly aborts), and that because StopVelocityThreshold is shaped (60->8 via the GAIN updater) while the abort is a fixed absolute, the abort may only be enabled where the stop gate is already floored below it. Add the matching rationale where the grid stages set 15 against their static stop threshold of 8. Docstring only; schema/*.json and Generated.cs must be regenerated on the rig. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Per review feedback. Drop the InstantiateSite.bonsai location pointer after the abort_velocity_threshold field (its semantics live in the field description) and condense the 13-line inline block in _probability_grid_blocks to the stage-specific values plus the stop-threshold constraint; the parameter mechanics are already in make_patch's docstring and the field description. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
galenlynch
force-pushed
the
feat-operant-abort-velocity
branch
from
3c37f8e to
f34fdd2
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Attempt to fix #566. This version includes a separate velocity threshold than the stop threshold, which is more flexible. Happy to discuss that detail, or others.