Torch Ops, Multi-ROM support, Multi-step support for macro-actions

[AdrianOrenstein]

Torch Ops, Multi-ROM support, Multi-step support for macro-actions

Variable-length actions (step / send)

step() accepts a list of per-env action arrays in addition to a flat array. Each env executes its sequence for len(seq) * frame_skip raw frames, accumulating gamma-discounted rewards. info["steps_taken"] reports how many steps each env ran. Empty sequences return the last observation with zero reward.

Note: if an empty list is sent to the environment, gamma should be set to 1 for that rom step.

Multi-ROM environments

game accepts a list of ROM names: each env runs a different game. num_envs is inferred from the list length. num_actions and action_set are per-env. The vector action_space is Tuple(Discrete(n_i) ...) with per-env bounds; single_action_space is Discrete(max(num_actions)) for compatibility.

PyTorch custom ops (env.torch())

Returns ale::send, ale::recv, and ale::send_sequences as registered torch custom ops with EffectType.ORDERED. Buffers are CPU-pinned for async GPU transfer. Compatible with torch.compile(fullgraph=True). TorchOpsWrapper wraps an env with bound op methods. XLA ops moved into _xla_ops.py.

Bug fixes

• Worker thread exceptions now propagate to the caller via recv() instead of never raising an error.
• frameskip=1 + maxpool=True now correctly pools the current frame against the previous one.
• Rewards are now float types to support discounted accumulated rewards with a per-rom gamma.

[pseudo-rnd-thoughts]

Thanks for the PR @AdrianOrenstein, I've had similar ideas however I want to merge #652 first to resolve a couple of underlying issues with the vectorizers before adding more interesting features.
Also, could these features be added in separate PRs to make review them easier

[AdrianOrenstein]

Yep. I'll branch off of the commit you mentioned.

[AdrianOrenstein]

I see the #652 is merged, i'll start on this PR

[AdrianOrenstein]

multi-rom has moved to #694 with a number of changes from this PR.