[#144] Resource reward randomization#278
Merged
Merged
Conversation
There was a problem hiding this comment.
Pull Request Overview
Adds support for per-episode randomization of resource reward weights and exposes those weights in the observation stream.
- Introduce
ResourceRewardWeightobservation and hook it into the reset lifecycle - Extend
ResourceRewardto sample weights inreset_pre_sim_initand publish them inreset_post_sim_init - Wire reset methods through
Satellite,ObservationBuilder, andActionBuilderand add corresponding tests
Reviewed Changes
Copilot reviewed 10 out of 10 changed files in this pull request and generated no comments.
Show a summary per file
| File | Description |
|---|---|
| tests/unittest/sats/test_access_satellite.py | Add mocks for builders in reset test |
| tests/unittest/obs/test_observations.py | Add unit tests for ResourceRewardWeight |
| tests/unittest/data/test_data.py | Add tests for deterministic and randomized ResourceReward |
| tests/integration/obs/test_int_observations.py | Add integration test for weight observation in multi-reward env |
| src/bsk_rl/sats/satellite.py | Call builder reset hooks in Satellite.reset_* methods |
| src/bsk_rl/obs/observations.py | Implement Resetable, add ResourceRewardWeight, builder resets |
| src/bsk_rl/data/resource_data.py | Extend ResourceReward with randomized weight sampling |
| src/bsk_rl/act/actions.py | Add reset hooks to ActionBuilder and Action |
| docs/source/release_notes.rst | Document new feature in release notes |
Comments suppressed due to low confidence (3)
tests/unittest/sats/test_access_satellite.py:315
- The test named
test_reset_pre_sim_initcallsreset_overwrite_previousinstead ofreset_pre_sim_init; update the call to match the test’s intent.
sat.reset_overwrite_previous()
tests/unittest/data/test_data.py:344
- This test computes
rewardbut does not assert its value; add an assertion to verify the calculated reward matches the expected output.
reward = dm.calculate_reward(
src/bsk_rl/obs/observations.py:513
- The signature indicates returning a float but the implementation returns a NumPy array; consider updating the return annotation to
-> np.ndarrayor similar for clarity.
def get_obs(self) -> float:
Mark2000
force-pushed
the
feature/reward-randomization
branch
from
June 12, 2025 16:26
f0062f2 to
eaf1981
Compare
LorenzzoQM
approved these changes
Jun 12, 2025
LorenzzoQM
left a comment
Contributor
There was a problem hiding this comment.
Very nice addition! Code looks good.
Mark2000
force-pushed
the
feature/reward-randomization
branch
from
June 12, 2025 19:05
eaf1981 to
448d4a5
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Allows for the resource reward weight to be randomized per-episode for multi objective cases.
Type of change
How should this pull request be reviewed?
How Has This Been Tested?
Added tests.
Future Work
Generalize to all rewarders.
Checklist