[Bug]: Can't create the PPO model on Macbook M1

[nguyenhongson1902]

🐛 Bug

I'm using Macbook Pro M1 and running code in Jupyter Notebook. Every time I run the scripts, I get an error and the Kernel crashed. I've tried running the code in Google Colab and it went well. It seems to me that stable-baselines3 library doesn't support M1 chips. What do I need to do now to run the code locally?

To Reproduce

from stable_baselines3 import PPO
from stable_baselines3.common.env_util import make_vec_env

# Parallel environments
env = make_vec_env("CartPole-v1", n_envs=4)

model = PPO("MlpPolicy", env, verbose=1)
model.learn(total_timesteps=25000)
model.save("ppo_cartpole")

del model # remove to demonstrate saving and loading

model = PPO.load("ppo_cartpole")

obs = env.reset()
while True:
    action, _states = model.predict(obs)
    obs, rewards, dones, info = env.step(action)
    env.render()

Relevant log output / Error message

The Kernel crashed while executing code in the the current cell or a previous cell. Please review the code in the cell(s) to identify a possible cause of the failure. Click here for more info. View Jupyter log for further details.

Canceled future for execute_request message before replies were done

System Info

No response

Checklist

• I have checked that there is no similar issue in the repo
• I have read the documentation
• I have provided a minimal working example to reproduce the bug
• I've used the markdown code blocks for both code and stack traces.

[araffin]

I have checked that there is no similar issue in the repo

Duplicate of #914 #959 #1019
PR #951

EDIT: my bad, read too fast (my eyes got caught by the "M1" ^^")

[qgallouedec]

I'm not sure it is actually duplicated. Looking the code, it is supposed run on the CPU.
@nguyenhongson1902 in order for us to help you, you need to fill in the template issue completely, especially the system information section. Without this, I am not able to reproduce this bug. (I tried on my M1, everything works fine).

[nguyenhongson1902]

I'm not sure it is actually duplicated. Looking the code, it is supposed run on the CPU. @nguyenhongson1902 in order for us to help you, you need to fill in the template issue completely, especially the system information section. Without this, I am not able to reproduce this bug. (I tried on my M1, everything works fine).

@qgallouedec You're right. I've run the code on the CPU using Jupyter Notebook inside VSCode.
Here is the system information and environment I've used:
$-$ Machine: Macbook Pro M1
$-$ OS: macOS Ventura 13.0
$-$ Environment: Anaconda, Version 22.9.0
$-$ Relevant packages: pyglet 1.5.27, stable-baselines3[extra] 1.6.2, gym 0.21.0, tensorboard 2.10.1, torch 1.13.0

[qgallouedec]

Thank you. Can you make the code minimal? Therefore, the last line is the one that raises the issue.

I think it is a problem of gym rendering in the notebook.

[nguyenhongson1902]

Thank you. Can you make the code minimal? Therefore, the last line is the one that raises the issue.

I think it is a problem of gym rendering in the notebook.

Okay, let me try to walk you through step by step to reproduce the error.

import gym
from stable_baselines3 import PPO
from stable_baselines3.common.env_util import make_vec_env

$\Rightarrow$ It works fine.

# Parallel environments
env = make_vec_env("CartPole-v1", n_envs=4)

$\Rightarrow$ It works ok as well.

model = PPO("MlpPolicy", env, verbose=1)
$\Rightarrow$ Boom, the error hit.

Error info:

The Kernel crashed while executing code in the the current cell or a previous cell. Please review the code in the cell(s) to identify a possible cause of the failure. Click [here](https://aka.ms/vscodeJupyterKernelCrash) for more info. View Jupyter [log](command:jupyter.viewOutput) for further details.
Canceled future for execute_request message before replies were done

Actually, when I run env = gym.make(environment_name), there's no problem at all.

[qgallouedec]

My intuition was bad. For the moment, I fail in reproducing the bug.
Can you provide the Jupyter logs?

[nguyenhongson1902]

My intuition was bad. For the moment, I fail in reproducing the bug. Can you provide the Jupiter logs?

Sure, here you go.

Visual Studio Code (1.73.0, undefined, desktop)
Jupyter Extension Version: 2022.9.1202862440.
Python Extension Version: 2022.18.0.
Workspace folder /Users/sonnh/Coding/reinforcement-learning-tutorial
error 14:10:56.438: Exception while attempting zmq : [Error: No native build was found for platform=darwin arch=arm64 runtime=electron abi=106 uv=1 armv=8 libc=glibc node=16.14.2 electron=19.0.17
    loaded from: /Users/sonnh/.vscode/extensions/ms-toolsai.jupyter-2022.9.1202862440/out/node_modules/zeromq

	at load.path (/Users/sonnh/.vscode/extensions/ms-toolsai.jupyter-2022.9.1202862440/out/node_modules/node-gyp-build/index.js:1:3163)
	at load (/Users/sonnh/.vscode/extensions/ms-toolsai.jupyter-2022.9.1202862440/out/node_modules/node-gyp-build/index.js:1:648)
	at Object.<anonymous> (/Users/sonnh/.vscode/extensions/ms-toolsai.jupyter-2022.9.1202862440/out/node_modules/zeromq/lib/native.js:1:198)
	at u._compile (/Applications/Visual Studio Code.app/Contents/Resources/app/out/vs/loader.js:4:1313)
	at Module._extensions..js (node:internal/modules/cjs/loader:1220:10)
	at Module.load (node:internal/modules/cjs/loader:1035:32)
	at Module._load (node:internal/modules/cjs/loader:876:12)
	at c._load (node:electron/js2c/asar_bundle:5:13343)
	at a._load (/Applications/Visual Studio Code.app/Contents/Resources/app/out/vs/workbench/api/node/extensionHostProcess.js:118:14563)
	at v._load (/Applications/Visual Studio Code.app/Contents/Resources/app/out/vs/workbench/api/node/extensionHostProcess.js:113:64900)
	at w._load (/Applications/Visual Studio Code.app/Contents/Resources/app/out/vs/workbench/api/node/extensionHostProcess.js:113:64268)
	at Module.require (node:internal/modules/cjs/loader:1059:19)
	at p (/Applications/Visual Studio Code.app/Contents/Resources/app/out/vs/loader.js:4:699)
	at Object.<anonymous> (/Users/sonnh/.vscode/extensions/ms-toolsai.jupyter-2022.9.1202862440/out/node_modules/zeromq/lib/index.js:1:144)
	at u._compile (/Applications/Visual Studio Code.app/Contents/Resources/app/out/vs/loader.js:4:1313)
	at Module._extensions..js (node:internal/modules/cjs/loader:1220:10)
	at Module.load (node:internal/modules/cjs/loader:1035:32)
	at Module._load (node:internal/modules/cjs/loader:876:12)
	at c._load (node:electron/js2c/asar_bundle:5:13343)
	at a._load (/Applications/Visual Studio Code.app/Contents/Resources/app/out/vs/workbench/api/node/extensionHostProcess.js:118:14563)
	at v._load (/Applications/Visual Studio Code.app/Contents/Resources/app/out/vs/workbench/api/node/extensionHostProcess.js:113:64900)
	at w._load (/Applications/Visual Studio Code.app/Contents/Resources/app/out/vs/workbench/api/node/extensionHostProcess.js:113:64268)
	at Module.require (node:internal/modules/cjs/loader:1059:19)
	at p (/Applications/Visual Studio Code.app/Contents/Resources/app/out/vs/loader.js:4:699)
	at 92747 (/Users/sonnh/.vscode/extensions/ms-toolsai.jupyter-2022.9.1202862440/out/extension.node.js:17:593435)
	at __webpack_require__ (/Users/sonnh/.vscode/extensions/ms-toolsai.jupyter-2022.9.1202862440/out/extension.node.js:17:1480895)
	at l.zmqSupported (/Users/sonnh/.vscode/extensions/ms-toolsai.jupyter-2022.9.1202862440/out/extension.node.js:2:2220878)
	at l.isSupportedForLocalLaunch (/Users/sonnh/.vscode/extensions/ms-toolsai.jupyter-2022.9.1202862440/out/extension.node.js:2:2220621)
	at get isSupported [as isSupported] (/Users/sonnh/.vscode/extensions/ms-toolsai.jupyter-2022.9.1202862440/out/extension.node.js:2:2220531)
	at t.registerTypes (/Users/sonnh/.vscode/extensions/ms-toolsai.jupyter-2022.9.1202862440/out/extension.node.js:2:2235624)
	at /Users/sonnh/.vscode/extensions/ms-toolsai.jupyter-2022.9.1202862440/out/extension.node.js:2:1640908
	at /Users/sonnh/.vscode/extensions/ms-toolsai.jupyter-2022.9.1202862440/out/extension.node.js:2:1641663
	at /Users/sonnh/.vscode/extensions/ms-toolsai.jupyter-2022.9.1202862440/out/extension.node.js:2:1641671
	at t.activate (/Users/sonnh/.vscode/extensions/ms-toolsai.jupyter-2022.9.1202862440/out/extension.node.js:2:1641793)
	at On._callActivateOptional (/Applications/Visual Studio Code.app/Contents/Resources/app/out/vs/workbench/api/node/extensionHostProcess.js:104:17678)
	at On._callActivate (/Applications/Visual Studio Code.app/Contents/Resources/app/out/vs/workbench/api/node/extensionHostProcess.js:104:17338)
	at /Applications/Visual Studio Code.app/Contents/Resources/app/out/vs/workbench/api/node/extensionHostProcess.js:104:15133
	at process.processTicksAndRejections (node:internal/process/task_queues:96:5)
	at async w._activate (/Applications/Visual Studio Code.app/Contents/Resources/app/out/vs/workbench/api/node/extensionHostProcess.js:90:8304)
	at async w._waitForDepsThenActivate (/Applications/Visual Studio Code.app/Contents/Resources/app/out/vs/workbench/api/node/extensionHostProcess.js:90:8246)
	at async w._initialize (/Applications/Visual Studio Code.app/Contents/Resources/app/out/vs/workbench/api/node/extensionHostProcess.js:90:7610)]
User belongs to experiment group 'jupyterTestcf'
User belongs to experiment group 'jupyterEnhancedDataViewer'
info 14:10:56.640: Old kernelSpecs (created by Jupyter Extension) stored in directory /Users/sonnh/Library/Jupyter/kernels/__old_vscode_kernelspecs
info 14:10:56.645: LSP Notebooks experiment is disabled -- not in treatment group
info 14:10:56.648: Attempting to start a server because of preload conditions ...
info 14:10:56.656: Checking for server existence.
info 14:10:56.656: Checking for server usability.
info 14:10:56.962: Preferred Remote kernel for /Users/sonnh/Coding/reinforcement-learning-tutorial/tutorial.ipynb is undefined
info 14:10:57.022: Starting Jupyter Session startUsingPythonInterpreter, .jvsc74a57bd0f5b916051ec391ef3c1c0123575e59cad2c35863d294dd079abc5845c0e5babb./opt/anaconda3/envs/pytorch/python./opt/anaconda3/envs/pytorch/python.-m#ipykernel_launcher (Python Path: /opt/anaconda3/envs/pytorch, EnvType: Conda, EnvName: 'pytorch', Version: 3.9.7) for '/Users/sonnh/Coding/reinforcement-learning-tutorial/tutorial.ipynb' (disableUI=true)
info 14:10:57.046: Preferred Remote kernel for /Users/sonnh/Coding/reinforcement-learning-tutorial/tutorial.ipynb is undefined
info 14:10:57.046: Find preferred kernel for /Users/sonnh/Coding/reinforcement-learning-tutorial/tutorial.ipynb with metadata {"kernelspec":{"display_name":"Python 3.9.7 ('pytorch')","language":"python","name":"python3"},"language_info":{"codemirror_mode":{"name":"ipython","version":3},"file_extension":".py","mimetype":"text/x-python","name":"python","nbconvert_exporter":"python","pygments_lexer":"ipython3","version":"3.9.7"},"orig_nbformat":4,"vscode":{"interpreter":{"hash":"f5b916051ec391ef3c1c0123575e59cad2c35863d294dd079abc5845c0e5babb"}}} & preferred interpreter /usr/local/bin/python3
info 14:10:57.062: Preferred Remote kernel for /Users/sonnh/Coding/reinforcement-learning-tutorial/tutorial.ipynb is undefined
info 14:10:57.062: Preferred kernel .jvsc74a57bd0f5b916051ec391ef3c1c0123575e59cad2c35863d294dd079abc5845c0e5babb./opt/anaconda3/envs/pytorch/python./opt/anaconda3/envs/pytorch/python.-m#ipykernel_launcher is exact match or top match for non python kernels, (false, undefined, true, false)
info 14:10:57.062: PreferredConnection: .jvsc74a57bd0f5b916051ec391ef3c1c0123575e59cad2c35863d294dd079abc5845c0e5babb./opt/anaconda3/envs/pytorch/python./opt/anaconda3/envs/pytorch/python.-m#ipykernel_launcher found for NotebookDocument: /Users/sonnh/Coding/reinforcement-learning-tutorial/tutorial.ipynb
info 14:10:57.366: Registering dummy command feature
info 14:10:57.420: Registering dummy command feature
info 14:10:57.739: Got empty env vars with python /usr/local/bin/python3 in 693ms
info 14:10:57.740: Got empty env vars with python /usr/local/bin/python3 in 693ms
info 14:10:57.740: Got empty env vars with python /usr/local/bin/python3 in 694ms
info 14:10:57.740: Got env vars ourselves faster /usr/local/bin/python3 with env var count 39 in 695ms
info 14:10:57.740: Got env vars ourselves faster /usr/local/bin/python3 with env var count 39 in 694ms
info 14:10:57.740: Got env vars ourselves faster /usr/local/bin/python3 with env var count 39 in 694ms
info 14:10:57.741: Process Execution: > /usr/local/bin/python3 -m pip list
> /usr/local/bin/python3 -m pip list
info 14:10:57.743: Process Execution: > /usr/local/bin/python3 -c "import jupyter"
> /usr/local/bin/python3 -c "import jupyter"
info 14:10:57.743: Process Execution: > /usr/local/bin/python3 -c "import notebook"
> /usr/local/bin/python3 -c "import notebook"
info 14:10:57.795: Got env vars ourselves faster /opt/anaconda3/envs/pytorch/bin/python with env var count 42 in 774ms
info 14:10:57.800: Process Execution: > /opt/anaconda3/envs/pytorch/bin/python -m pip list
> /opt/anaconda3/envs/pytorch/bin/python -m pip list
info 14:10:57.929: Got env vars ourselves faster /usr/local/bin/python3 with env var count 39 in 1ms
info 14:10:57.929: Process Execution: > /usr/local/bin/python3 -m jupyter kernelspec --version
> /usr/local/bin/python3 -m jupyter kernelspec --version
info 14:10:58.297: Starting notebook server.
info 14:10:58.297: Connecting to server
info 14:10:58.297: Launching server
info 14:10:58.297: Starting Notebook
info 14:10:58.298: Got env vars ourselves faster /usr/local/bin/python3 with env var count 39 in 2ms
info 14:10:58.298: Got env vars ourselves faster /usr/local/bin/python3 with env var count 39 in 2ms
info 14:10:58.300: Generating custom default config at /var/folders/cz/t0wnr3c949j4t87gbzk2qkzc0000gn/T/5e30268b-d7a3-4d59-ab0e-14da044927f7/jupyter_notebook_config.py
info 14:10:58.300: Starting Jupyter Notebook
Starting Jupyter from /usr/local/bin/python3
info 14:10:58.301: Got env vars ourselves faster /usr/local/bin/python3 with env var count 39 in 1ms
info 14:10:58.301: Got env vars ourselves faster /usr/local/bin/python3 with env var count 39 in 0ms
info 14:10:58.302: Creating daemon pool for /usr/local/bin/python3 with env variables count 39
info 14:10:58.304: Process Execution: > /usr/local/bin/python3 -m vscode_datascience_helpers.daemon --daemon-module=vscode_datascience_helpers.jupyter_daemon -v --ppid 44441
> /usr/local/bin/python3 -m vscode_datascience_helpers.daemon --daemon-module=vscode_datascience_helpers.jupyter_daemon -v --ppid 44441
info 14:10:58.304: Process Execution: > /usr/local/bin/python3 -m vscode_datascience_helpers.daemon --daemon-module=vscode_datascience_helpers.jupyter_daemon -v --ppid 44441
> /usr/local/bin/python3 -m vscode_datascience_helpers.daemon --daemon-module=vscode_datascience_helpers.jupyter_daemon -v --ppid 44441
info 14:10:58.305: Process Execution: > /usr/local/bin/python3 -m vscode_datascience_helpers.daemon --daemon-module=vscode_datascience_helpers.jupyter_daemon -v --ppid 44441
> /usr/local/bin/python3 -m vscode_datascience_helpers.daemon --daemon-module=vscode_datascience_helpers.jupyter_daemon -v --ppid 44441
info 14:10:58.365: Got env vars ourselves faster /usr/local/bin/python3 with env var count 39 in 1ms
info 14:10:58.366: Process Execution: > /usr/local/bin/python3 (daemon) -m jupyter notebook --no-browser --notebook-dir="/Users/sonnh/Coding/reinforcement-learning-tutorial" --config=/var/folders/cz/t0wnr3c949j4t87gbzk2qkzc0000gn/T/5e30268b-d7a3-4d59-ab0e-14da044927f7/jupyter_notebook_config.py --NotebookApp.iopub_data_rate_limit=10000000000.0
> /usr/local/bin/python3 (daemon) -m jupyter notebook --no-browser --notebook-dir="/Users/sonnh/Coding/reinforcement-learning-tutorial" --config=/var/folders/cz/t0wnr3c949j4t87gbzk2qkzc0000gn/T/5e30268b-d7a3-4d59-ab0e-14da044927f7/jupyter_notebook_config.py --NotebookApp.iopub_data_rate_limit=10000000000.0
info 14:10:58.366: Waiting for Jupyter Notebook
/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/traitlets/traitlets.py:2392: FutureWarning: Supporting extra quotes around strings is deprecated in traitlets 5.0. You can use '/Users/sonnh/Coding/reinforcement-learning-tutorial' instead of '"/Users/sonnh/Coding/reinforcement-learning-tutorial"' if you require traitlets >=5.
  warn(
info 14:10:58.662: /Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/traitlets/traitlets.py:2392: FutureWarning: Supporting extra quotes around strings is deprecated in traitlets 5.0. You can use '/Users/sonnh/Coding/reinforcement-learning-tutorial' instead of '"/Users/sonnh/Coding/reinforcement-learning-tutorial"' if you require traitlets >=5.
  warn(

[I 14:10:58.910 NotebookApp] Serving notebooks from local directory: /Users/sonnh/Coding/reinforcement-learning-tutorial
info 14:10:58.911: /Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/traitlets/traitlets.py:2392: FutureWarning: Supporting extra quotes around strings is deprecated in traitlets 5.0. You can use '/Users/sonnh/Coding/reinforcement-learning-tutorial' instead of '"/Users/sonnh/Coding/reinforcement-learning-tutorial"' if you require traitlets >=5.
  warn(
[I 14:10:58.910 NotebookApp] Serving notebooks from local directory: /Users/sonnh/Coding/reinforcement-learning-tutorial

[I 14:10:58.910 NotebookApp] Jupyter Notebook 6.4.11 is running at:
info 14:10:58.911: /Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/traitlets/traitlets.py:2392: FutureWarning: Supporting extra quotes around strings is deprecated in traitlets 5.0. You can use '/Users/sonnh/Coding/reinforcement-learning-tutorial' instead of '"/Users/sonnh/Coding/reinforcement-learning-tutorial"' if you require traitlets >=5.
  warn(
[I 14:10:58.910 NotebookApp] Serving notebooks from local directory: /Users/sonnh/Coding/reinforcement-learning-tutorial
[I 14:10:58.910 NotebookApp] Jupyter Notebook 6.4.11 is running at:

[I 14:10:58.910 NotebookApp] http://localhost:8888/?token=b2939100aad9197e1f81a0ea08e906683e6442f421d02f76
info 14:10:58.911: /Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/traitlets/traitlets.py:2392: FutureWarning: Supporting extra quotes around strings is deprecated in traitlets 5.0. You can use '/Users/sonnh/Coding/reinforcement-learning-tutorial' instead of '"/Users/sonnh/Coding/reinforcement-learning-tutorial"' if you require traitlets >=5.
  warn(
[I 14:10:58.910 NotebookApp] Serving notebooks from local directory: /Users/sonnh/Coding/reinforcement-learning-tutorial
[I 14:10:58.910 NotebookApp] Jupyter Notebook 6.4.11 is running at:
[I 14:10:58.910 NotebookApp] http://localhost:8888/?token=b2939100aad9197e1f81a0ea08e906683e6442f421d02f76

info 14:10:58.912: Process Execution: > /usr/local/bin/python3 (daemon) ~/.vscode/extensions/ms-toolsai.jupyter-2022.9.1202862440/pythonFiles/vscode_datascience_helpers/getServerInfo.py
> /usr/local/bin/python3 (daemon) ~/.vscode/extensions/ms-toolsai.jupyter-2022.9.1202862440/pythonFiles/vscode_datascience_helpers/getServerInfo.py
info 14:10:58.912: Got env vars ourselves faster /usr/local/bin/python3 with env var count 39 in 0ms
[I 14:10:58.910 NotebookApp]  or http://127.0.0.1:8888/?token=b2939100aad9197e1f81a0ea08e906683e6442f421d02f76
info 14:10:58.914: /Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/traitlets/traitlets.py:2392: FutureWarning: Supporting extra quotes around strings is deprecated in traitlets 5.0. You can use '/Users/sonnh/Coding/reinforcement-learning-tutorial' instead of '"/Users/sonnh/Coding/reinforcement-learning-tutorial"' if you require traitlets >=5.
  warn(
[I 14:10:58.910 NotebookApp] Serving notebooks from local directory: /Users/sonnh/Coding/reinforcement-learning-tutorial
[I 14:10:58.910 NotebookApp] Jupyter Notebook 6.4.11 is running at:
[I 14:10:58.910 NotebookApp] http://localhost:8888/?token=b2939100aad9197e1f81a0ea08e906683e6442f421d02f76
[I 14:10:58.910 NotebookApp]  or http://127.0.0.1:8888/?token=b2939100aad9197e1f81a0ea08e906683e6442f421d02f76

info 14:10:58.914: Process Execution: > /usr/local/bin/python3 (daemon) ~/.vscode/extensions/ms-toolsai.jupyter-2022.9.1202862440/pythonFiles/vscode_datascience_helpers/getServerInfo.py
> /usr/local/bin/python3 (daemon) ~/.vscode/extensions/ms-toolsai.jupyter-2022.9.1202862440/pythonFiles/vscode_datascience_helpers/getServerInfo.py
info 14:10:58.914: Got env vars ourselves faster /usr/local/bin/python3 with env var count 39 in 0ms
[I 14:10:58.911 NotebookApp] Use Control-C to stop this server and shut down all kernels (twice to skip confirmation).
info 14:10:58.914: /Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/traitlets/traitlets.py:2392: FutureWarning: Supporting extra quotes around strings is deprecated in traitlets 5.0. You can use '/Users/sonnh/Coding/reinforcement-learning-tutorial' instead of '"/Users/sonnh/Coding/reinforcement-learning-tutorial"' if you require traitlets >=5.
  warn(
[I 14:10:58.910 NotebookApp] Serving notebooks from local directory: /Users/sonnh/Coding/reinforcement-learning-tutorial
[I 14:10:58.910 NotebookApp] Jupyter Notebook 6.4.11 is running at:
[I 14:10:58.910 NotebookApp] http://localhost:8888/?token=b2939100aad9197e1f81a0ea08e906683e6442f421d02f76
[I 14:10:58.910 NotebookApp]  or http://127.0.0.1:8888/?token=b2939100aad9197e1f81a0ea08e906683e6442f421d02f76
[I 14:10:58.911 NotebookApp] Use Control-C to stop this server and shut down all kernels (twice to skip confirmation).

info 14:10:58.915: Got env vars ourselves faster /usr/local/bin/python3 with env var count 39 in 0ms
[C 14:10:58.914 NotebookApp] 
    
    To access the notebook, open this file in a browser:
        file:///Users/sonnh/Library/Jupyter/runtime/nbserver-44526-open.html
    Or copy and paste one of these URLs:
        http://localhost:8888/?token=b2939100aad9197e1f81a0ea08e906683e6442f421d02f76
     or http://127.0.0.1:8888/?token=b2939100aad9197e1f81a0ea08e906683e6442f421d02f76
info 14:10:58.915: /Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/traitlets/traitlets.py:2392: FutureWarning: Supporting extra quotes around strings is deprecated in traitlets 5.0. You can use '/Users/sonnh/Coding/reinforcement-learning-tutorial' instead of '"/Users/sonnh/Coding/reinforcement-learning-tutorial"' if you require traitlets >=5.
  warn(
[I 14:10:58.910 NotebookApp] Serving notebooks from local directory: /Users/sonnh/Coding/reinforcement-learning-tutorial
[I 14:10:58.910 NotebookApp] Jupyter Notebook 6.4.11 is running at:
[I 14:10:58.910 NotebookApp] http://localhost:8888/?token=b2939100aad9197e1f81a0ea08e906683e6442f421d02f76
[I 14:10:58.910 NotebookApp]  or http://127.0.0.1:8888/?token=b2939100aad9197e1f81a0ea08e906683e6442f421d02f76
[I 14:10:58.911 NotebookApp] Use Control-C to stop this server and shut down all kernels (twice to skip confirmation).
[C 14:10:58.914 NotebookApp] 
    
    To access the notebook, open this file in a browser:
        file:///Users/sonnh/Library/Jupyter/runtime/nbserver-44526-open.html
    Or copy and paste one of these URLs:
        http://localhost:8888/?token=b2939100aad9197e1f81a0ea08e906683e6442f421d02f76
     or http://127.0.0.1:8888/?token=b2939100aad9197e1f81a0ea08e906683e6442f421d02f76

info 14:10:58.916: Got env vars ourselves faster /usr/local/bin/python3 with env var count 39 in 1ms
info 14:10:58.996: Got empty env vars with python /usr/local/bin/python3 in 1068ms
info 14:10:59.023: Got empty env vars with python /usr/local/bin/python3 in 727ms
info 14:10:59.023: Got empty env vars with python /usr/local/bin/python3 in 727ms
info 14:10:59.023: Got empty env vars with python /usr/local/bin/python3 in 723ms
info 14:10:59.029: Got empty env vars with python /usr/local/bin/python3 in 728ms
info 14:10:59.030: Got empty env vars with python /usr/local/bin/python3 in 666ms
info 14:10:59.030: Got empty env vars with python /usr/local/bin/python3 in 118ms
info 14:10:59.030: Got empty env vars with python /usr/local/bin/python3 in 116ms
info 14:10:59.031: Got empty env vars with python /usr/local/bin/python3 in 116ms
info 14:10:59.033: Got empty env vars with python /usr/local/bin/python3 in 118ms
info 14:10:59.070: Connecting to process server
info 14:10:59.070: Connecting server kernel http://localhost:8888/
info 14:10:59.071: Creating server with url : http://localhost:8888/
info 14:10:59.074: Connection complete server
info 14:10:59.075: Server started.
info 14:10:59.116: Process Execution: > /usr/local/bin/python3 (daemon) ~/.vscode/extensions/ms-toolsai.jupyter-2022.9.1202862440/pythonFiles/vscode_datascience_helpers/getServerInfo.py
> /usr/local/bin/python3 (daemon) ~/.vscode/extensions/ms-toolsai.jupyter-2022.9.1202862440/pythonFiles/vscode_datascience_helpers/getServerInfo.py
info 14:10:59.126: Process Execution: > /usr/local/bin/python3 (daemon) ~/.vscode/extensions/ms-toolsai.jupyter-2022.9.1202862440/pythonFiles/vscode_datascience_helpers/getServerInfo.py
> /usr/local/bin/python3 (daemon) ~/.vscode/extensions/ms-toolsai.jupyter-2022.9.1202862440/pythonFiles/vscode_datascience_helpers/getServerInfo.py
[I 14:10:59.189 NotebookApp] Creating new notebook in 
info 14:10:59.203: Got env vars with python /opt/anaconda3/envs/pytorch/bin/python, with env var count 43 and custom env var count 0 in 2182ms
info 14:10:59.206: installMissingDependencies /opt/anaconda3/envs/pytorch/bin/python, ui.disabled=true for resource '/Users/sonnh/Coding/reinforcement-learning-tutorial/tutorial.ipynb'
info 14:10:59.206: Got env vars with python /opt/anaconda3/envs/pytorch/bin/python, with env var count 43 and custom env var count 0 in 0ms
info 14:10:59.206: Got env vars ourselves faster /opt/anaconda3/envs/pytorch/bin/python with env var count 42 in 0ms
info 14:10:59.210: Process Execution: > /opt/anaconda3/envs/pytorch/bin/python -c "import ipykernel"
> /opt/anaconda3/envs/pytorch/bin/python -c "import ipykernel"
info 14:10:59.479: Spec argv[0] updated from '/opt/anaconda3/envs/pytorch/bin/python' to '/opt/anaconda3/envs/pytorch/bin/python'
info 14:10:59.480: Got env vars ourselves faster /opt/anaconda3/envs/pytorch/bin/python with env var count 42 in 1ms
[I 14:10:59.524 NotebookApp] Kernel started: 16fc0738-7ed0-45f2-9d6f-cf9016fde4ea, name: python397jvsc74a57bd0f5b916051ec391ef3c1c0123575e59cad2c35863d294dd079abc5845c0e5babb
[W 14:10:59.538 NotebookApp] delete /tutorial-jvsc-ca0b217c-08e4-4c22-b10d-db9ac2d6ba9d4bd2e89d-8a8c-4215-a5e1-2d6b68104d3b.ipynb
info 14:11:00.394: Got env vars with python /opt/anaconda3/envs/pytorch/bin/python, with env var count 43 and custom env var count 0 in 914ms
info 14:11:00.438: Started session for kernel startUsingPythonInterpreter:.jvsc74a57bd0f5b916051ec391ef3c1c0123575e59cad2c35863d294dd079abc5845c0e5babb./opt/anaconda3/envs/pytorch/python./opt/anaconda3/envs/pytorch/python.-m#ipykernel_launcher
info 14:11:00.439: UpdateWorkingDirectoryAndPath in Kernel
info 14:11:00.949: Got env vars with python /opt/anaconda3/envs/pytorch/bin/python, with env var count 43 and custom env var count 0 in 0ms
info 14:11:00.949: Got env vars ourselves faster /opt/anaconda3/envs/pytorch/bin/python with env var count 42 in 1ms
info 14:11:00.952: Process Execution: > /opt/anaconda3/envs/pytorch/bin/python ~/.vscode/extensions/ms-toolsai.jupyter-2022.9.1202862440/pythonFiles/printJupyterDataDir.py
> /opt/anaconda3/envs/pytorch/bin/python ~/.vscode/extensions/ms-toolsai.jupyter-2022.9.1202862440/pythonFiles/printJupyterDataDir.py
[I 14:14:11.510 NotebookApp] KernelRestarter: restarting kernel (1/5), keep random ports
error 14:14:11.516: Error in waiting for cell to complete [Error: Canceled future for execute_request message before replies were done
	at a.KernelShellFutureHandler.dispose (/Users/sonnh/.vscode/extensions/ms-toolsai.jupyter-2022.9.1202862440/out/node_modules/@jupyterlab/services.js:2:32353)
	at /Users/sonnh/.vscode/extensions/ms-toolsai.jupyter-2022.9.1202862440/out/node_modules/@jupyterlab/services.js:2:26572
	at Map.forEach (<anonymous>)
	at y._clearKernelState (/Users/sonnh/.vscode/extensions/ms-toolsai.jupyter-2022.9.1202862440/out/node_modules/@jupyterlab/services.js:2:26557)
	at /Users/sonnh/.vscode/extensions/ms-toolsai.jupyter-2022.9.1202862440/out/node_modules/@jupyterlab/services.js:2:29000
	at process.processTicksAndRejections (node:internal/process/task_queues:96:5)]
warn 14:14:11.518: Cell completed with errors {
  message: 'Canceled future for execute_request message before replies were done'
}
info 14:14:11.519: Cancel all remaining cells true || Error || undefined
[I 14:14:11.517 NotebookApp] Starting buffering for 16fc0738-7ed0-45f2-9d6f-cf9016fde4ea:49b6661c-e9f4-4511-9f29-0b82883801d7
[I 14:14:11.537 NotebookApp] Restoring connection for 16fc0738-7ed0-45f2-9d6f-cf9016fde4ea:49b6661c-e9f4-4511-9f29-0b82883801d7

[qgallouedec]

Do you agree that the minimal code would be

from stable_baselines3 import PPO
from stable_baselines3.common.env_util import make_vec_env

env = make_vec_env("CartPole-v1", n_envs=4)
PPO("MlpPolicy", env)

Does it work without multiple envs?:

from stable_baselines3 import PPO

PPO("MlpPolicy", "CartPole-v1")

[nguyenhongson1902]

Do you agree that the minimal code would be

from stable_baselines3 import PPO
from stable_baselines3.common.env_util import make_vec_env

env = make_vec_env("CartPole-v1", n_envs=4)
PPO("MlpPolicy", env)

Does it work without multiple envs?:

from stable_baselines3 import PPO

PPO("MlpPolicy", "CartPole-v1")

Without multiple envs, it still has the same error. For example:

import gym
from stable_baselines3 import PPO
from stable_baselines3.common.vec_env import DummyVecEnv

$\Rightarrow$ Works fine.

environment_name = 'CartPole-v1'
env = gym.make(environment_name)
env = DummyVecEnv([lambda: env])

$\Rightarrow$ No problems

model = PPO('MlpPolicy', env, verbose=0)
$\Rightarrow$ It got the same error as with multiple envs.
By the way, I'm using Python 3.9.7. I've included the illustration below.

[qgallouedec]

I have the same configuration as you and I do not encounter any errors.

I've noticed several times that the VSCode jupyter extension is buggy and sometimes behaving a bit strangely. I remember solving one of them by simply reinstalling the virtual environment completely and rebooting my laptop. I think it's really worth a try.

[qgallouedec]

Possibly memory-related: microsoft/vscode-jupyter#9741 (comment). It would explain why the reboot worked for me.