The problem of different action time scales

[battery7]

I want to use HPPO to handle mixed action space problems, but I encountered a problem where the time scales of discrete and continuous actions are different. For example, if a continuous action is executed once, the discrete action needs to be executed many times. How should I handle this issue? I think it's difficult to converge in setting the reward function. Or are there any other algorithms that support it, or am I discretizing continuous actions and then using a discrete action mask?

[PaParaZz1]

Maybe you can still keep the same logic for data collection, but add some new data sampling and reward pre-processing strategies to balance both parts.

[battery7]

Thank you for your reply. Could you please provide a more detailed explanation on how to handle it, or do you have any examples