One ish click deploy (#197)

* dep fix?

* towncrier start

* towncrier seems to work

* Sample workflow with no real actions

* more release

* some more towncrier

* more release

* Fix borked link

* fix version blerp

https://github.com/saltstack/salt-ext-modules-vmware/runs/4988078936?check_suite_focus=true
looks like installing importlib on an earlier version is no good. Also
we're double-appending tests so... let's try this!

* Better version

This should work better - at least from what we had for docs

Author: waynew

changelog/.gitkeep

@@ -4,8 +4,8 @@ Release
 =======
 
 The purpose of this document is to specify and outline the release process for
-the Salt Extension Modules for VMware. Initially for the beta releases, many of
-these steps will be manual. Over time the process will become more automated.
+the Salt Extension Modules for VMware. Previously many of the steps were
+manual, but the process is now mostly automatic.
 
 Overview
 --------
@@ -20,89 +20,37 @@ It's possible that days go by without changes happening, but these builds will
 still be called nightly builds.
 
 The release cadence target for this module is every two weeks a release may be
-published. A release candidate (rcN) package will be published a week ahead of
-the release, allowing users time to test the upcoming packages against their
+published. A release candidate (rcN) package should be published a week ahead
+of the release, allowing users time to test the upcoming packages against their
 own environment.
 
 Automated testing will happen against release artifacts, using at least the
 current version of Salt, along with the current main development branch of
-Salt. This will ensure that any breaking changes within Salt are detected
-ahead of time, and can either be accounted for within this module, or upstream
-bugs can be filed.
+Salt. This will ensure that any breaking changes within Salt are detected ahead
+of time, and can either be accounted for within this module, or upstream bugs
+can be filed.
 
-.. _build:
+.. _dependencies:
 
-Build
------
+Dependencies
+------------
 
-To prepare for a release, the first step is to ensure you're on the latest
-commit on the main branch and build the release artifact, along with
-dependencies:
+In order to run the release script, you'll need to have gpg installed with
+gpg-agent. If you can run the following gpg command without having to type
+anything in you should be good to go:
 
 .. code::
 
-    git fetch salt
-    git stash  # if needed
-    git checkout salt/main
-    # If pip and wheel are not already installed/up-to-date
-    python -m pip install --upgrade pip wheel
-    python -m pip wheel .\[dev,tests,release\] -w dist/
+     echo 'hello' | gpg --armor --clear-sign | gpg --verify
 
-This will create a ``.whl`` file for the extension module, as well as all of
-the dependencies, in the ``dist/`` directory.
+Obviously you'll need Python installed as well, and be able to run ``git push``
+in your clone directory without `entering a
+password <https://docs.github.com/en/authentication/connecting-to-github-with-ssh>`_.
+``tar`` is also required.
 
-
-Ensure Version
---------------
-
-When it comes time to build a release, ensure that
-``src/saltext/vmware/version.py`` contains the correct version. For an release
-candidate (RC) release the format should be ``YY.M.D.PATCHrcN``. Any subsequent RC
-releases should increment ``N``. The release manager should start cutting RCs
-far enough ahead of time to be able to cut a complete release on the target
-date. For a production release, the format should be ``YY.M.D.PATCH``. For
-instance, if we were going to release on 2010-08-14, we would start with
-
-.. code::
-
-    __version__ = '10.8.14.0rc1'
-
-When the release is deemed ready, the version would be ``10.8.14.0``.
-
-If the incorrect version is present in the ``main`` branch, it should be
-updated, committed, and pushed before continuing this process.
-
-Commit ``src/saltext/vmware/version.py``, rebuild and continue testing. Do **not** merge into https://github.com/saltstack/salt-ext-modules-vmware/.
-
-Install and Test the Build
---------------------------
-
-To ensure that the virtual environment is pristine, create a new one. Then the
-package and necessary dependencies can be installed into the environment. By
-building the extension and dependencies locally, and using a pristine
-environment, this will help ensure that any missing dependencies are detected
-before release. Tests will be executed against the installed package, and not
-the local source, which helps to ensure the complete install process is tested.
-
-.. code::
-
-    deactivate  # if a venv is already activated
-    python3 -m venv /tmp/test_saltext --prompt test-vmw-ext
-    source /tmp/test_saltext/bin/activate
-    python -m pip install --upgrade pip wheel
-    python -m pip install --no-index --find-links dist/ saltext.vmware\[dev,tests,release\]
-    pytest --cov=saltext.vmware tests/
-
-This will run tests against the build artifact. If all tests pass this nightly
-build may be released.
-
-The release process be tested by using Twine with the TestPyPI_
-
-.. _TestPyPI: https://test.pypi.org/project/saltext.vmware/
-
-In order to cut a release, you must be a project maintainer on TestPyPI and
-have a token configured for the project. Your ``~/.pypirc`` should contain your
-key like the following:
+In order to cut a release, you must be a project maintainer on TestPyPI as well
+as PyPI. You'll need to have a token configured for the project. Your
+``~/.pypirc`` should contain your key like the following:
 
 .. code::
 
@@ -115,78 +63,103 @@ key like the following:
       username = __token__
       password = pypi-AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA...
 
-Then you would run:
+You *must* have ``test_saltext_vmware`` and ``saltext_vmware`` sections in your
+distutils segment as well as have tokens configured.
+
+.. _prebuild:
+
+Pre-Build
+---------
+
+Before running the script you'll want to ensure that all the tests are passing.
+With your activated venv, run the following command:
 
 .. code::
 
-    twine upload --repository test_saltext_vmware dist/saltext.vmware-VERSION-py2.py3-none-any.whl
+    python -m nox -e tests-3
 
-Merge the Version Update into the Main Repo
--------------------------------------------
+The result should be completely passing tests. If there are any failures,
+ensure that they are fixed before continuing. Because the only difference
+between this version and the actual deployed version will be the package
+version, we're banking on the fact that the tests should still pass if they
+already were here. But we *will* be testing the release artifact!
 
-Now that the build has been tested and verified, create a merge request with the new ``__version__`` to https://github.com/saltstack/salt-ext-modules-vmware/.
+Ensure Version
+--------------
 
-With the new version committed, go back and :ref:`build a new artifact<build>`.
+The first thing to do is ensure that ``src/saltext/vmware/version.py`` has the
+correct version for this release. In regards to version numbers, this project
+uses Calver_, with the ``YY.M.D.PATCH`` style. Breaking (and any other) changes
+should be communicated through the changelog_. Release candidate versions
+should be created with the **expected** release date with ``rcN`` modifier. For
+instance, if we planned to release 2010, August 14, we would set the version
+like so:
 
-Versions, Tagging, and Changelog
---------------------------------
+.. code::
 
-In order to cut a release, you must be a maintainer of this project on PyPI.
-You should have a ``[saltext_vmware]`` section in your pypirc file, similar to
-the test setting.
+    __version__ = "10.8.14rc1"
 
-In regards to version numbers, this project uses Calver_, with the
-``YY.M.D.PATCH`` style. Breaking (and any other) changes should be
-communicated through the changelog_.
+Typically there will only be one RC build - though if bugs are found,
+especially severe bugs, new RC versions will be built, tagged, and released.
 
 .. _CalVer: https://calver.org/
 .. _changelog: https://github.com/saltstack/salt-ext-modules-vmware/blob/main/CHANGELOG.md
 
-For dev/nightly builds, tags will ONLY be used if the package gets uploaded to
-the production PyPI. At that point an annotated tag should be created with the
-current changelog for that particular version.
+The changelog will be automatically updated as part of this release process,
+but if corrections need to be made it's totally fine to make updates out of
+band.
+
+Run the Release
+---------------
 
-Release candidate builds will be tagged with the **expected** release date with
-``rcN`` modifier. For instance, if we planned to release 2010, August 14, we would tag like so:
+Assuming that your environment is properly configured, all you should need to
+do now is run:
 
 .. code::
 
-   git tag 10.8.14rc1 -a
+    python tools/release.py
 
-Typically there will only be one RC build - though if bugs
-are found, especially severe bugs, new RC versions will be built, tagged, and
-released.
+The script will do several checks, ensuring that you have no outstanding
+changes in the repo, it can correctly run gpg, and that you're not running
+within a virtual environment (as it takes care of that for you).
 
-To cut a final release, the repository will be tagged as above, the changelog
-added to the tag, and then a new package will be built, installed, and tested.
-This order is required because we use setuptools_scm to generate the version
-number from the latest tag. Tagging does not produce any code changes (other
-than the version number), so the tests should continue to pass. If they fail
-for any reason other than your Internet going out, this should be considered a
-critical issue! Flaky tests are undesirable, since they are often just
-misleading. If a test scenario is that flaky, it should be performed manually,
-or not at all.
+After running the checks, which shouldn't really make any serious changes to
+your system, you will be asked:
 
-..
-    That flaky bit could be a in a different document, and linked to from here.
+.. code::
 
-Once the full test suite has passed, sign the production package with gpg and
-upload the package with twine:
+    Continue to build and test saltext.vmware version 21.12.15.0rc1? [y/N]: y
 
-.. code::
+The default is no. If you answer ``yes`` or ``y``, then a virtual environment
+will be created for testing the release, the package will be built, installed
+to the virtual environment, and then tested.
 
-    # SIGNING_KEY should be replaced with the signing key, and FINAL-VERSION
-    # with the actual version number
-    gpg --detach-sign -u SIGNING_KEY dist/saltext.vmware-FINAL-VERSION-py2.py3.none-any.whl
-    twine upload --repository saltext_vmware dist/saltext.vmware-FINAL-VERSION-py2.py3.none-any.whl
+If anything fails up to this point then the release will bail out and you'll
+need to make changes.
 
-Once the package has been uploaded to PyPI the tag should be pushed:
+If everything succeeds, and all looks OK then you will be given one last chance
+to bail out before pushing changes to PyPI and GitHub.
 
 .. code::
 
-    git push salt 10.8.14   # to use the previous example
-
-A release should also be created on GitHub, uploading both the package as well
-as the `.sig` file.
+    WARNING: This will really upload saltext.vmware version 21.12.15.0rc1 to
+    pypi. There is no going back from this point.
+    Enter "deploy" without quotes to really deploy: deploy
+
+If you enter anything besides ``deploy`` then the release will abort. Basically
+we want to be *very* sure that we're ready to release! If you are, cool - just
+type ``deploy`` and hit enter.
+
+The release will then be pushed to
+https://test.pypi.org/project/saltext.vmware/ and
+https://pypi.org/project/saltext.vmware/ after the successful completion of
+which, the tag will be created and pushed to GitHub, suitable for `drafting a
+new
+release <https://github.com/saltstack/salt-ext-modules-vmware/releases/new>`_.
+
+The finally part of the release script will create
+``/tmp/saltext.vmware-build-{version}.tar.gz`` that you can use to get the
+changelog, ``.whl``, and ``.whl.asc`` signature file to use for creating the
+GitHub release, as well as publishing notifications to social media/websites.
 
 Congrats! You've just cut a new release!

docs/release.rst

@@ -95,7 +95,7 @@ def _install_requirements(
         if install_salt:
             session.install("--progress-bar=off", SALT_REQUIREMENT, silent=PIP_INSTALL_SILENT)
 
-        if install_test_requirements:
+        if install_test_requirements and "tests" not in install_extras:
             install_extras.append("tests")
 
         if EXTRA_REQUIREMENTS_INSTALL:

noxfile.py

@@ -4,3 +4,41 @@ build-backend = "setuptools.build_meta"
 
 [tool.black]
 line-length = 100
+
+[tool.towncrier]
+    name = "saltext.vmware"
+    title_format = "{name} {version}"
+    package = "saltext.vmware"
+    filename = "CHANGELOG.md"
+    directory = "changelog/"
+    start_string = "# Changelog\n"
+
+    [[tool.towncrier.type]]
+        directory = "removed"
+        name = "Removed"
+        showcontent = true
+
+    [[tool.towncrier.type]]
+        directory = "deprecated"
+        name = "Deprecated"
+        showcontent = true
+
+    [[tool.towncrier.type]]
+        directory = "changed"
+        name = "Changed"
+        showcontent = true
+
+    [[tool.towncrier.type]]
+        directory = "fixed"
+        name = "Fixed"
+        showcontent = true
+
+    [[tool.towncrier.type]]
+        directory = "added"
+        name = "Added"
+        showcontent = true
+
+    [[tool.towncrier.type]]
+        directory = "security"
+        name = "Security"
+        showcontent = true

pyproject.toml

@@ -42,6 +42,7 @@ setup_requires =
 install_requires =
     salt>=3002
     pyvmomi==7.0.2
+    importlib_metadata; python_version < "3.8"
 
 [options.extras_require]
 tests =
@@ -50,15 +51,18 @@ tests =
   pytest-salt-factories>=0.121.1
 dev =
   nox
+  towncrier==21.9.0rc1
 release =
   twine
   wheel
+  towncrier==21.9.0rc1
 docs =
   sphinx
   furo
   sphinx-prompt
   sphinxcontrib-spelling
   importlib_metadata; python_version < "3.8"
+  towncrier==21.9.0rc1
 
 
 [options.packages.find]