Add Windows CI

[ChengYen-Tang]

Description

Use Azure pipeline
10 Free parallel jobs for public projects

Motivation and Context

• I have raised an issue to propose this change (required for new features and bug fixes)
  issue [Enhancement] add windows CI #110

Types of changes

• Bug fix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds functionality)
• Breaking change (fix or feature that would cause existing functionality to change)
• Documentation (update in the documentation)

Checklist:

• I've read the CONTRIBUTION guide (required)
• I have updated the changelog accordingly (required).
• My change requires a change to the documentation.
• I have updated the tests accordingly (required for a bug fix or a new feature).
• I have updated the documentation accordingly.
• I have ensured pytest and pytype both pass.

[ChengYen-Tang]

Pytype is currently not support windows.

[ChengYen-Tang]

On Windows, the python version must be> = 3.6 to install atari-py.

[ChengYen-Tang]

Some tests fail in Windows CI.

[ChengYen-Tang]

Does anyone know how to handle this error?

________________________________ test_mpi_adam ________________________________

    def test_mpi_adam():
        """Test RunningMeanStd object for MPI"""
        return_code = subprocess.call(['mpirun', '--allow-run-as-root', '-np', '2',
>                                      'python', '-m', 'stable_baselines.common.mpi_adam'])

tests\test_mpi_adam.py:9: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
C:\hostedtoolcache\windows\Python\3.6.8\x64\lib\subprocess.py:287: in call
    with Popen(*popenargs, **kwargs) as p:
C:\hostedtoolcache\windows\Python\3.6.8\x64\lib\subprocess.py:729: in __init__
    restore_signals, start_new_session)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

self = <subprocess.Popen object at 0x0000022843618358>
args = 'mpirun --allow-run-as-root -np 2 python -m stable_baselines.common.mpi_adam'
executable = None, preexec_fn = None, close_fds = True, pass_fds = ()
cwd = None, env = None
startupinfo = <subprocess.STARTUPINFO object at 0x00000228423DD9E8>
creationflags = 0, shell = False, p2cread = -1, p2cwrite = -1, c2pread = -1
c2pwrite = -1, errread = -1, errwrite = -1, unused_restore_signals = True
unused_start_new_session = False

    def _execute_child(self, args, executable, preexec_fn, close_fds,
                       pass_fds, cwd, env,
                       startupinfo, creationflags, shell,
                       p2cread, p2cwrite,
                       c2pread, c2pwrite,
                       errread, errwrite,
                       unused_restore_signals, unused_start_new_session):
        """Execute program (MS Windows version)"""
    
        assert not pass_fds, "pass_fds not supported on Windows."
    
        if not isinstance(args, str):
            args = list2cmdline(args)
    
        # Process startup details
        if startupinfo is None:
            startupinfo = STARTUPINFO()
        if -1 not in (p2cread, c2pwrite, errwrite):
            startupinfo.dwFlags |= _winapi.STARTF_USESTDHANDLES
            startupinfo.hStdInput = p2cread
            startupinfo.hStdOutput = c2pwrite
            startupinfo.hStdError = errwrite
    
        if shell:
            startupinfo.dwFlags |= _winapi.STARTF_USESHOWWINDOW
            startupinfo.wShowWindow = _winapi.SW_HIDE
            comspec = os.environ.get("COMSPEC", "cmd.exe")
            args = '{} /c "{}"'.format (comspec, args)
    
        # Start the process
        try:
            hp, ht, pid, tid = _winapi.CreateProcess(executable, args,
                                     # no special security
                                     None, None,
                                     int(not close_fds),
                                     creationflags,
                                     env,
                                     os.fspath(cwd) if cwd is not None else None,
>                                    startupinfo)
E                                    FileNotFoundError: [WinError 2] The system cannot find the file specified

C:\hostedtoolcache\windows\Python\3.6.8\x64\lib\subprocess.py:1017: FileNotFoundError

MS MPI was successfully installed.