Skip to content

Commit 7d37d2e

Browse files
zhongjiajiezhongjiajie
authored
feat: Use setup.cfg for dist package (#64)
Use setup.cfg to config dist package. the benefit is we can use `attr` and `file` setup.cfg for convenient setting
1 parent 129b056 commit 7d37d2e

6 files changed

Lines changed: 134 additions & 145 deletions

File tree

‎.github/workflows/ci.yaml‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -151,7 +151,7 @@ jobs:
151151
integrate-test:
152152
needs: license
153153
runs-on: ubuntu-latest
154-
if: ${{ github.event_name == 'schedule' }} || contains(github.event.head_commit.message, '[run-it]')
154+
if: ${{ github.event_name == 'schedule' || contains(toJSON(github.event.commits.*.message), '[run-it]') }}
155155
timeout-minutes: 30
156156
steps:
157157
- name: Checkout Dolphinscheduler SDK Python

‎RELEASE.md‎

Lines changed: 10 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -24,7 +24,6 @@ but we also have a [PyPi](#release-to-pypi) repository for Python package distri
2424

2525
## Prepare
2626

27-
* Change `version` in `setup.py`.
2827
* Remove `sphinx-multiversion` dependency in `setup.py`, we still can not fix this issue
2928
[Distribute tarball and wheel error with direct dependency](https://github.com/apache/dolphinscheduler/issues/12238)
3029
* Change `version_ext` about the dolphinscheduler version current support, the syntax is respect [pep-044](https://peps.python.org/pep-0440/#version-specifiers)
@@ -40,14 +39,21 @@ upload package to PyPi. You could first install and upgrade them by:
4039
# Install or upgrade dependencies
4140
python3 -m pip install --upgrade pip build twine
4241

42+
# Change version
43+
# For macOS
44+
sed -i '' "s/__version__ = \".*\"/__version__ = \"${VERSION}\"/" src/pydolphinscheduler/__init__.py
45+
# For Linux
46+
sed -i "s/__version__ = \".*\"/__version__ = \"${VERSION}\"/" src/pydolphinscheduler/__init__.py
47+
git commit -am "Release v${VERSION}"
48+
4349
# Add Tag
4450
VERSION=<VERSION> # The version of the package you want to release, e.g. 1.2.3
4551
REMOTE=<REMOTE> # The git remote name, we usually use `origin` or `remote`
4652
git tag -a "${VERSION}" -m "Release v${VERSION}"
4753
git push "${REMOTE}" --tags
4854

4955
# Build
50-
python setup.py pre_clean && python -m build
56+
python setup.py clean && python -m build
5157

5258
# Sign
5359
cd dist
@@ -210,7 +216,7 @@ Vote result should follow these:
210216
TestPyPi is a test environment of PyPi, you could release to it to test whether the package is work or not.
211217
212218
1. Create an account in [TestPyPi](https://test.pypi.org/account/register/).
213-
2. Clean unrelated files in `dist` directory, and build package `python3 setup.py pre_clean`.
219+
2. Clean unrelated files in `dist` directory, and build package `python3 setup.py clean`.
214220
3. Build package `python3 -m build`, and you will see two new files in `dist` directory, with extension
215221
`.tar.gz` and `.whl`.
216222
4. Upload to TestPyPi `python3 -m twine upload --repository testpypi dist/*`.
@@ -224,7 +230,7 @@ PyPi is the official repository of Python packages, it is highly recommended [re
224230
first to test whether the package is correct.
225231
226232
1. Create an account in [PyPI](https://pypi.org/account/register/).
227-
2. Clean unrelated files in `dist` directory, and build package `python3 setup.py pre_clean`.
233+
2. Clean unrelated files in `dist` directory, and build package `python3 setup.py clean`.
228234
3. Build package `python3 -m build`, and you will see two new files in `dist` directory, with extension
229235
`.tar.gz` and `.whl`.
230236
4. Upload to TestPyPi `python3 -m twine upload dist/*`.

‎docs/source/conf.py‎

Lines changed: 6 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -39,6 +39,9 @@
3939
.resolve()
4040
)
4141
sys.path.insert(0, str(pkg_src_dir))
42+
43+
from pydolphinscheduler import __version__ # noqa
44+
4245
# Debug to uncomment this to see the source path
4346
# print("=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=")
4447
# print(pkg_src_dir)
@@ -48,12 +51,12 @@
4851

4952
# -- Project information -----------------------------------------------------
5053

51-
project = "pydolphinscheduler"
54+
project = "apache-dolphinscheduler"
5255
copyright = "2022, apache"
53-
author = "apache dolphinscheduler contributors"
56+
author = "Apache Software Foundation"
5457

5558
# The full version, including alpha/beta/rc tags
56-
release = "0.0.1"
59+
release = __version__
5760

5861

5962
# -- General configuration ---------------------------------------------------

‎setup.cfg‎

Lines changed: 112 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -14,3 +14,115 @@
1414
# KIND, either express or implied. See the License for the
1515
# specific language governing permissions and limitations
1616
# under the License.
17+
18+
[metadata]
19+
name = apache-dolphinscheduler
20+
version = attr: pydolphinscheduler.__version__
21+
url = https://dolphinscheduler.apache.org/python/main/index.html
22+
description = pydolphinscheduler is Apache DolphinScheduler Python API.
23+
long_description = file: README.md
24+
long_description_content_type = text/markdown
25+
author = Apache Software Foundation
26+
author_email = dev@dolphinscheduler.apache.org
27+
license = Apache License 2.0
28+
license_files =
29+
file: LICENSE
30+
keywords =
31+
dolphinscheduler
32+
workflow
33+
scheduler
34+
taskflow
35+
# complete classifier list: http://pypi.python.org/pypi?%3Aaction=list_classifiers
36+
classifiers =
37+
Development Status :: 4 - Beta
38+
Environment :: Console
39+
Intended Audience :: Developers
40+
License :: OSI Approved :: Apache Software License
41+
Operating System :: Unix
42+
Operating System :: POSIX
43+
Operating System :: Microsoft :: Windows
44+
Programming Language :: Python
45+
Programming Language :: Python :: 3
46+
Programming Language :: Python :: 3.6
47+
Programming Language :: Python :: 3.7
48+
Programming Language :: Python :: 3.8
49+
Programming Language :: Python :: 3.9
50+
Programming Language :: Python :: 3.10
51+
Programming Language :: Python :: 3.11
52+
Programming Language :: Python :: Implementation :: CPython
53+
Programming Language :: Python :: Implementation :: PyPy
54+
Topic :: Software Development :: User Interfaces
55+
project_urls =
56+
Homepage = https://dolphinscheduler.apache.org/python/main/index.html
57+
Documentation = https://dolphinscheduler.apache.org/python/main/index.html
58+
Source = https://github.com/apache/dolphinscheduler-sdk-python
59+
Issue Tracker = https://github.com/apache/dolphinscheduler-sdk-python/issues
60+
Twitter = https://twitter.com/dolphinschedule
61+
62+
[options]
63+
python_requires = >=3.6
64+
include_package_data = True
65+
zip_safe = true
66+
platforms = any
67+
package_dir =
68+
=src
69+
packages =
70+
find:
71+
install_requires =
72+
boto3>=1.23.10
73+
oss2>=2.16.0
74+
python-gitlab>=2.10.1
75+
click>=8.0.0
76+
py4j~=0.10
77+
ruamel.yaml
78+
stmdency>=0.0.2
79+
80+
[options.packages.find]
81+
where=src
82+
83+
[options.package_data]
84+
pydolphinscheduler =
85+
default_config.yaml
86+
version_ext
87+
88+
[options.entry_points]
89+
console_scripts =
90+
pydolphinscheduler = pydolphinscheduler.cli.commands:cli
91+
92+
[options.extras_require]
93+
build =
94+
build
95+
setuptools>=42
96+
wheel
97+
test =
98+
pytest>=6.2
99+
freezegun>=1.1
100+
coverage>=6.1
101+
pytest-cov>=3.0
102+
docker>=5.0.3
103+
style =
104+
flake8>=4.0
105+
flake8-docstrings>=1.6
106+
flake8-black>=0.2
107+
isort>=5.10
108+
autoflake>=1.4
109+
doc =
110+
sphinx>=4.3
111+
sphinx_rtd_theme>=1.0
112+
sphinx-click>=3.0
113+
sphinx-inline-tabs
114+
sphinx-copybutton>=0.4.0
115+
# Unreleased package have a feature we want(use correct version package for API ref), so we install from
116+
# GitHub directly, see also:
117+
# https://github.com/Holzhaus/sphinx-multiversion/issues/42#issuecomment-1210539786
118+
sphinx-multiversion @ git+https://github.com/Holzhaus/sphinx-multiversion#egg=sphinx-multiversion
119+
sphinx-github-changelog
120+
dev =
121+
# build
122+
apache-dolphinscheduler[build]
123+
# test
124+
apache-dolphinscheduler[test]
125+
# style
126+
apache-dolphinscheduler[style]
127+
# doc
128+
apache-dolphinscheduler[doc]

‎setup.py‎

Lines changed: 4 additions & 134 deletions
Original file line numberDiff line numberDiff line change
@@ -20,10 +20,9 @@
2020
import os
2121
import sys
2222
from distutils.dir_util import remove_tree
23-
from os.path import dirname, join
2423
from typing import List
2524

26-
from setuptools import Command, find_packages, setup
25+
from setuptools import Command, setup
2726

2827
if sys.version_info[0] < 3:
2928
raise Exception(
@@ -32,69 +31,9 @@
3231

3332
logger = logging.getLogger(__name__)
3433

35-
version = "dev"
36-
37-
# Start package required
38-
prod = [
39-
"boto3>=1.23.10",
40-
"oss2>=2.16.0",
41-
"python-gitlab>=2.10.1",
42-
"click>=8.0.0",
43-
"py4j~=0.10",
44-
"ruamel.yaml",
45-
"stmdency>=0.0.2",
46-
]
47-
48-
build = [
49-
"build",
50-
"setuptools>=42",
51-
"wheel",
52-
]
53-
54-
doc = [
55-
"sphinx>=4.3",
56-
"sphinx_rtd_theme>=1.0",
57-
"sphinx-click>=3.0",
58-
"sphinx-inline-tabs",
59-
"sphinx-copybutton>=0.4.0",
60-
# Unreleased package have a feature we want(use correct version package for API ref), so we install from
61-
# GitHub directly, see also:
62-
# https://github.com/Holzhaus/sphinx-multiversion/issues/42#issuecomment-1210539786
63-
"sphinx-multiversion @ git+https://github.com/Holzhaus/sphinx-multiversion#egg=sphinx-multiversion",
64-
"sphinx-github-changelog",
65-
]
66-
67-
test = [
68-
"pytest>=6.2",
69-
"freezegun>=1.1",
70-
"coverage>=6.1",
71-
"pytest-cov>=3.0",
72-
"docker>=5.0.3",
73-
]
74-
75-
style = [
76-
"flake8>=4.0",
77-
"flake8-docstrings>=1.6",
78-
"flake8-black>=0.2",
79-
"isort>=5.10",
80-
"autoflake>=1.4",
81-
]
82-
83-
dev = style + test + doc + build
84-
85-
all_dep = prod + dev
86-
# End package required
87-
88-
89-
def read(*names, **kwargs):
90-
"""Read file content from given file path."""
91-
return open(
92-
join(dirname(__file__), *names), encoding=kwargs.get("encoding", "utf8")
93-
).read()
94-
9534

9635
class CleanCommand(Command):
97-
"""Command to clean up python api before setup by running `python setup.py pre_clean`."""
36+
"""Command to clean up python api before setup by running `python setup.py clean`."""
9837

9938
description = "Clean up project root"
10039
user_options: List[str] = []
@@ -121,80 +60,11 @@ def run(self) -> None:
12160
remove_tree(cl)
12261
else:
12362
os.remove(cl)
124-
logger.info("Finish pre_clean process.")
63+
logger.info("Finish clean process.")
12564

12665

12766
setup(
128-
name="apache-dolphinscheduler",
129-
version=version,
130-
license="Apache License 2.0",
131-
description="Apache DolphinScheduler Python API",
132-
long_description=read("README.md"),
133-
# Make sure pypi is expecting markdown
134-
long_description_content_type="text/markdown",
135-
author="Apache Software Foundation",
136-
author_email="dev@dolphinscheduler.apache.org",
137-
url="https://dolphinscheduler.apache.org/python/main/index.html",
138-
python_requires=">=3.6",
139-
keywords=[
140-
"dolphinscheduler",
141-
"workflow",
142-
"scheduler",
143-
"taskflow",
144-
],
145-
project_urls={
146-
"Homepage": "https://dolphinscheduler.apache.org/python/main/index.html",
147-
"Documentation": "https://dolphinscheduler.apache.org/python/main/index.html",
148-
"Source": "https://github.com/apache/dolphinscheduler-sdk-python",
149-
"Issue Tracker": "https://github.com/apache/dolphinscheduler-sdk-python/issues",
150-
"Twitter": "https://twitter.com/dolphinschedule",
151-
},
152-
packages=find_packages(where="src"),
153-
package_dir={"": "src"},
154-
include_package_data=True,
155-
package_data={
156-
"pydolphinscheduler": [
157-
"default_config.yaml",
158-
"version_ext",
159-
],
160-
},
161-
platforms=["any"],
162-
classifiers=[
163-
# complete classifier list: http://pypi.python.org/pypi?%3Aaction=list_classifiers
164-
"Development Status :: 4 - Beta",
165-
"Environment :: Console",
166-
"Intended Audience :: Developers",
167-
"License :: OSI Approved :: Apache Software License",
168-
"Operating System :: Unix",
169-
"Operating System :: POSIX",
170-
"Operating System :: Microsoft :: Windows",
171-
"Programming Language :: Python",
172-
"Programming Language :: Python :: 3",
173-
"Programming Language :: Python :: 3.6",
174-
"Programming Language :: Python :: 3.7",
175-
"Programming Language :: Python :: 3.8",
176-
"Programming Language :: Python :: 3.9",
177-
"Programming Language :: Python :: 3.10",
178-
"Programming Language :: Python :: 3.11",
179-
"Programming Language :: Python :: Implementation :: CPython",
180-
"Programming Language :: Python :: Implementation :: PyPy",
181-
"Topic :: Software Development :: User Interfaces",
182-
],
183-
install_requires=prod,
184-
extras_require={
185-
"all": all_dep,
186-
"dev": dev,
187-
"style": style,
188-
"test": test,
189-
"doc": doc,
190-
"build": build,
191-
},
19267
cmdclass={
193-
"pre_clean": CleanCommand,
194-
},
195-
entry_points={
196-
"console_scripts": [
197-
"pydolphinscheduler = pydolphinscheduler.cli.commands:cli",
198-
],
68+
"clean": CleanCommand,
19969
},
20070
)

‎src/pydolphinscheduler/__init__.py‎

Lines changed: 1 addition & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -17,6 +17,4 @@
1717

1818
"""Init root of pydolphinscheduler."""
1919

20-
from pkg_resources import get_distribution
21-
22-
__version__ = get_distribution("apache-dolphinscheduler").version
20+
__version__ = "dev"

0 commit comments

Comments
 (0)