Skip to content

Render initial state in evaluate_policy (Fixes #1692)#2257

Open
midhunxavier wants to merge 1 commit into
DLR-RM:masterfrom
midhunxavier:fix/evalcallback-render-initial-state-1692
Open

Render initial state in evaluate_policy (Fixes #1692)#2257
midhunxavier wants to merge 1 commit into
DLR-RM:masterfrom
midhunxavier:fix/evalcallback-render-initial-state-1692

Conversation

@midhunxavier

Copy link
Copy Markdown

Description

evaluate_policy() (used by EvalCallback) only called env.render() after
env.step(), so the initial post-reset state of each episode was never rendered.
This adds a single env.render() after env.reset(), before the first
predict(), so the initial state is shown. The final (post-terminal) frame is
intentionally still not rendered because VecEnv auto-resets sub-environments;
this caveat is now documented in the render docstring with a workaround.

This implements the minimal approach @araffin proposed in the issue thread (render
before predict + document the final-frame limitation), not the rejected
VecEnv-subclass approach.

Motivation and Context

closes #1692

  • I have raised an issue to propose this change (required for new features and bug fixes)

Types of changes

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to change)
  • Documentation (update in the documentation)

Checklist

  • I've read the CONTRIBUTION guide (required)
  • I have updated the changelog accordingly (docs/misc/changelog.md) (required).
  • My change requires a change to the documentation.
  • I have updated the tests accordingly (required for a bug fix or a new feature).
  • I have updated the documentation accordingly.
  • I have opened an associated PR on the SB3-Contrib repository (if necessary)
  • I have opened an associated PR on the RL-Zoo3 repository (if necessary)
  • I have reformatted the code using make format (required)
  • I have checked the codestyle using make check-codestyle and make lint (required)
  • I have ensured make pytest and make type both pass. (required)
  • I have checked that the documentation builds using make doc (required)

AI-assistance disclosure (per CONTRIBUTING.md): this change was developed with
the assistance of an LLM (Claude). The approach was proposed by the maintainer
(@araffin) in #1692; I have reviewed and own the change.

evaluate_policy() only called env.render() after env.step(), so the
initial post-reset state was never rendered (a problem for envs with
dynamic/random initial states, e.g. via EvalCallback). This renders the
initial state once after reset, before the first predict(). The final
post-terminal frame is still not rendered because VecEnv auto-resets
sub-environments; this caveat is documented in the docstring and via an
inline comment, matching the maintainer's accepted minimal approach.

Adds a regression test that fails on the old behavior and passes now.

AI-assistance disclosure: this change was developed with the assistance
of an LLM (Claude), per the SB3 contributing guidelines.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug]: Rendering with EvalCallback does not render the initial or final state

1 participant