Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
46 changes: 23 additions & 23 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -169,27 +169,27 @@ supported:
be listed here, for security reasons. See
[this issue](https://github.com/typeshed-internal/stub_uploader/issues/90)
for more information about what external dependencies are allowed.
* `extra_description` (optional): Can be used to add a custom description to
* `extra-description` (optional): Can be used to add a custom description to
the package's long description. It should be a multi-line string in
Markdown format.
* `stub_distribution` (optional): Distribution name to be uploaded to PyPI.
* `stub-distribution` (optional): Distribution name to be uploaded to PyPI.
This defaults to `types-<distribution>` and should only be set in special
cases.
* `upstream_repository` (recommended): The URL of the upstream repository.
* `obsolete_since` (optional): This field is part of our process for
* `upstream-repository` (recommended): The URL of the upstream repository.
* `obsolete-since` (optional): This field is part of our process for
[removing obsolete third-party libraries](#third-party-library-removal-policy).
It contains the first version of the corresponding library that ships
its own `py.typed` file.
* `no_longer_updated` (optional): This field is set to `true` before removing
* `no-longer-updated` (optional): This field is set to `true` before removing
stubs for other reasons than the upstream library shipping with type
information.
* `upload` (optional): This field is set to `false` to prevent automatic
uploads to PyPI. This should only be used in special cases, e.g. when the stubs
break the upload.
* `partial_stub` (optional): This field marks the type stub package as
* `partial-stub` (optional): This field marks the type stub package as
[partial](https://peps.python.org/pep-0561/#partial-stub-packages). This is for
3rd-party stubs that don't cover the entirety of the package's public API.
* `requires_python` (optional): The minimum version of Python required to install
* `requires-python` (optional): The minimum version of Python required to install
the type stub package. It must be in the form `>=3.*`. If omitted, the oldest
Python version supported by typeshed is used.

Expand All @@ -198,36 +198,36 @@ This has the following keys:
* `skip` (default: `false`): Whether stubtest should be run against this
package. Please avoid setting this to `true`, and add a comment if you have
to.
* `ignore_missing_stub`: When set to `true`, this will add the
`--ignore_missing_stub` option to the stubtest call. See
* `ignore-missing-stub`: When set to `true`, this will add the
`--ignore-missing-stub` option to the stubtest call. See
[tests/README.md](./tests/README.md) for more information. In most cases,
this field should be identical to `partial_stub`.
* `stubtest_dependencies` (default: `[]`): A list of Python packages that need
this field should be identical to `partial-stub`.
* `stubtest-dependencies` (default: `[]`): A list of Python packages that need
to be installed for stubtest to run successfully. These packages are installed
in addition to the dependencies in the `dependencies` field.
* `apt_dependencies` (default: `[]`): A list of Ubuntu APT packages
* `apt-dependencies` (default: `[]`): A list of Ubuntu APT packages
that need to be installed for stubtest to run successfully.
* `brew_dependencies` (default: `[]`): A list of MacOS Homebrew packages
* `brew-dependencies` (default: `[]`): A list of MacOS Homebrew packages
that need to be installed for stubtest to run successfully
* `choco_dependencies` (default: `[]`): A list of Windows Chocolatey packages
* `choco-dependencies` (default: `[]`): A list of Windows Chocolatey packages
that need to be installed for stubtest to run successfully
* `supported_platforms` (default: all platforms): A list of OSes on which
* `supported-platforms` (default: all platforms): A list of OSes on which
stubtest can be run. When a package is not platform-specific, this should
not be set. If the package is platform-specific, this should usually be set
to the supported platforms, unless stubtest is known to fail on a
specific platform.
* `ci_platforms` (default: `["linux"]`): A list of OSes on which to run
* `ci-platforms` (default: `["linux"]`): A list of OSes on which to run
stubtest as part of our continuous integration (CI) tests. Can contain
`win32`, `linux`, and `darwin` values. If not specified, stubtest is run
only on `linux`. Only add extra OSes to the test if there are
platform-specific branches in a stubs package.
* `mypy_plugins` (default: `[]`): A list of Python modules to use as mypy plugins
when running stubtest. For example: `mypy_plugins = ["mypy_django_plugin.main"]`
* `mypy_plugins_config` (default: `{}`): A dictionary mapping plugin names to their
* `mypy-plugins` (default: `[]`): A list of Python modules to use as mypy plugins
when running stubtest. For example: `mypy-plugins = ["mypy_django_plugin.main"]`
* `mypy-plugins-config` (default: `{}`): A dictionary mapping plugin names to their
configuration dictionaries for use by mypy plugins. For example:
`mypy_plugins_config = {"django-stubs" = {"django_settings_module" = "@tests.django_settings"}}`
`mypy-plugins-config = {"django-stubs" = {"django_settings_module" = "@tests.django_settings"}}`

`*_dependencies` are usually packages needed to `pip install` the implementation
`*-dependencies` are usually packages needed to `pip install` the implementation
distribution.

The format of all `METADATA.toml` files can be checked by running
Expand Down Expand Up @@ -436,7 +436,7 @@ following criteria is met:

Case 1: If a package ships its own `py.typed` file, please follow these steps:

1. Make sure **stubsabot** open a PR that sets the `obsolete_since` field in the
1. Make sure **stubsabot** open a PR that sets the `obsolete-since` field in the
`METADATA.toml` file to the first version of the package that shipped `py.typed`.
2. After at least six months, make sure **stubsabot** open a PR to remove the stubs.

Expand All @@ -446,7 +446,7 @@ these steps:
1. Open an issue explaining why the stubs should be removed.
2. A maintainer will add the
["stubs: removal" label](https://github.com/python/typeshed/labels/%22stubs%3A%20removal%22).
3. Open a PR that sets the `no_longer_updated` field in the `METADATA.toml`
3. Open a PR that sets the `no-longer-updated` field in the `METADATA.toml`
file to `true`.
4. When a new version of the package was automatically uploaded to PyPI (which
can take up to a day), make sure **stubsabot** open a PR to remove the stubs.
Expand Down
93 changes: 48 additions & 45 deletions lib/ts_utils/metadata.py
Original file line number Diff line number Diff line change
Expand Up @@ -58,7 +58,7 @@ def _is_nested_dict(obj: object) -> TypeGuard[dict[str, dict[str, Any]]]:
@functools.cache
def get_oldest_supported_python() -> str:
with PYPROJECT_PATH.open("rb") as config:
val = tomllib.load(config)["tool"]["typeshed"]["oldest_supported_python"]
val = tomllib.load(config)["tool"]["typeshed"]["oldest-supported-python"]
assert type(val) is str
return val

Expand Down Expand Up @@ -102,16 +102,16 @@ def read_stubtest_settings(distribution: str) -> StubtestSettings:
data: dict[str, object] = tomllib.load(f).get("tool", {}).get("stubtest", {})

skip: object = data.get("skip", False)
apt_dependencies: object = data.get("apt_dependencies", [])
brew_dependencies: object = data.get("brew_dependencies", [])
choco_dependencies: object = data.get("choco_dependencies", [])
apt_dependencies: object = data.get("apt-dependencies", [])
brew_dependencies: object = data.get("brew-dependencies", [])
choco_dependencies: object = data.get("choco-dependencies", [])
extras: object = data.get("extras", [])
ignore_missing_stub: object = data.get("ignore_missing_stub", False)
supported_platforms: object = data.get("supported_platforms")
ci_platforms: object = data.get("ci_platforms", DEFAULT_STUBTEST_PLATFORMS)
stubtest_dependencies: object = data.get("stubtest_dependencies", [])
mypy_plugins: object = data.get("mypy_plugins", [])
mypy_plugins_config: object = data.get("mypy_plugins_config", {})
ignore_missing_stub: object = data.get("ignore-missing-stub", False)
supported_platforms: object = data.get("supported-platforms")
ci_platforms: object = data.get("ci-platforms", DEFAULT_STUBTEST_PLATFORMS)
stubtest_dependencies: object = data.get("stubtest-dependencies", [])
mypy_plugins: object = data.get("mypy-plugins", [])
mypy_plugins_config: object = data.get("mypy-plugins-config", {})

assert type(skip) is bool
assert type(ignore_missing_stub) is bool
Expand All @@ -128,12 +128,12 @@ def read_stubtest_settings(distribution: str) -> StubtestSettings:
assert _is_nested_dict(mypy_plugins_config)

unrecognised_platforms = set(ci_platforms) - _STUBTEST_PLATFORM_MAPPING.keys()
assert not unrecognised_platforms, f"Unrecognised ci_platforms specified for {distribution!r}: {unrecognised_platforms}"
assert not unrecognised_platforms, f"Unrecognised ci-platforms specified for {distribution!r}: {unrecognised_platforms}"

if supported_platforms is not None:
assert set(ci_platforms).issubset(
supported_platforms
), f"ci_platforms must be a subset of supported_platforms for {distribution!r}"
), f"ci-platforms must be a subset of supported-platforms for {distribution!r}"

for platform, dep_key in _STUBTEST_PLATFORM_MAPPING.items():
if platform not in ci_platforms:
Expand Down Expand Up @@ -194,31 +194,31 @@ def is_obsolete(self) -> bool:
{
"version",
"dependencies",
"extra_description",
"stub_distribution",
"upstream_repository",
"obsolete_since",
"no_longer_updated",
"extra-description",
"stub-distribution",
"upstream-repository",
"obsolete-since",
"no-longer-updated",
"upload",
"tool",
"partial_stub",
"requires_python",
"partial-stub",
"requires-python",
"mypy-tests",
}
)
_KNOWN_METADATA_TOOL_FIELDS: Final = {
"stubtest": {
"skip",
"apt_dependencies",
"brew_dependencies",
"choco_dependencies",
"apt-dependencies",
"brew-dependencies",
"choco-dependencies",
"extras",
"ignore_missing_stub",
"supported_platforms",
"ci_platforms",
"stubtest_dependencies",
"mypy_plugins",
"mypy_plugins_config",
"ignore-missing-stub",
"supported-platforms",
"ci-platforms",
"stubtest-dependencies",
"mypy-plugins",
"mypy-plugins-config",
}
}
_DIST_NAME_RE: Final = re.compile(r"^[a-z0-9]([a-z0-9._-]*[a-z0-9])?$", re.IGNORECASE)
Expand Down Expand Up @@ -259,41 +259,41 @@ def read_metadata(distribution: str) -> StubMetadata:
assert isinstance(dependencies_s, list)
dependencies = [parse_dependencies(distribution, dep) for dep in dependencies_s]

extra_description: object = data.get("extra_description") # pyright: ignore[reportUnknownMemberType]
extra_description: object = data.get("extra-description") # pyright: ignore[reportUnknownMemberType]
assert isinstance(extra_description, (str, type(None)))

if "stub_distribution" in data:
stub_distribution = data["stub_distribution"]
if "stub-distribution" in data:
stub_distribution = data["stub-distribution"]
assert isinstance(stub_distribution, str)
assert _DIST_NAME_RE.fullmatch(stub_distribution), f"Invalid 'stub_distribution' value for {distribution!r}"
assert _DIST_NAME_RE.fullmatch(stub_distribution), f"Invalid 'stub-distribution' value for {distribution!r}"
else:
stub_distribution = f"types-{distribution}"

upstream_repository: object = data.get("upstream_repository") # pyright: ignore[reportUnknownMemberType]
upstream_repository: object = data.get("upstream-repository") # pyright: ignore[reportUnknownMemberType]
assert isinstance(upstream_repository, (str, type(None)))
if isinstance(upstream_repository, str):
parsed_url = urllib.parse.urlsplit(upstream_repository)
assert parsed_url.scheme == "https", f"{distribution}: URLs in the upstream_repository field should use https"
assert parsed_url.scheme == "https", f"{distribution}: URLs in the upstream-repository field should use https"
no_www_please = (
f"{distribution}: `World Wide Web` subdomain (`www.`) should be removed from URLs in the upstream_repository field"
f"{distribution}: `World Wide Web` subdomain (`www.`) should be removed from URLs in the upstream-repository field"
)
assert not parsed_url.netloc.startswith("www."), no_www_please
no_query_params_please = (
f"{distribution}: Query params (`?`) should be removed from URLs in the upstream_repository field"
f"{distribution}: Query params (`?`) should be removed from URLs in the upstream-repository field"
)
assert parsed_url.hostname in _QUERY_URL_ALLOWLIST or (not parsed_url.query), no_query_params_please
no_fragments_please = f"{distribution}: Fragments (`#`) should be removed from URLs in the upstream_repository field"
no_fragments_please = f"{distribution}: Fragments (`#`) should be removed from URLs in the upstream-repository field"
assert not parsed_url.fragment, no_fragments_please
if parsed_url.netloc == "github.com":
cleaned_url_path = parsed_url.path.strip("/")
num_url_path_parts = len(Path(cleaned_url_path).parts)
bad_github_url_msg = (
f"Invalid upstream_repository for {distribution!r}: "
f"Invalid upstream-repository for {distribution!r}: "
"URLs for GitHub repositories always have two parts in their paths"
)
assert num_url_path_parts == 2, bad_github_url_msg

obsolete_since: object = data.get("obsolete_since") # pyright: ignore[reportUnknownMemberType]
obsolete_since: object = data.get("obsolete-since") # pyright: ignore[reportUnknownMemberType]
assert isinstance(obsolete_since, (String, type(None)))
if obsolete_since:
comment = obsolete_since.trivia.comment
Expand All @@ -302,26 +302,26 @@ def read_metadata(distribution: str) -> StubMetadata:
obsolete = ObsoleteMetadata(since_version=obsolete_since, since_date=since_date)
else:
obsolete = None
no_longer_updated: object = data.get("no_longer_updated", False) # pyright: ignore[reportUnknownMemberType]
no_longer_updated: object = data.get("no-longer-updated", False) # pyright: ignore[reportUnknownMemberType]
assert type(no_longer_updated) is bool
uploaded_to_pypi: object = data.get("upload", True) # pyright: ignore[reportUnknownMemberType]
assert type(uploaded_to_pypi) is bool
partial_stub: object = data.get("partial_stub", True) # pyright: ignore[reportUnknownMemberType]
partial_stub: object = data.get("partial-stub", True) # pyright: ignore[reportUnknownMemberType]
assert type(partial_stub) is bool
requires_python_str: object = data.get("requires_python") # pyright: ignore[reportUnknownMemberType]
requires_python_str: object = data.get("requires-python") # pyright: ignore[reportUnknownMemberType]
oldest_supported_python = get_oldest_supported_python()
oldest_supported_python_specifier = Specifier(f">={oldest_supported_python}")
if requires_python_str is None:
requires_python = oldest_supported_python_specifier
else:
assert isinstance(requires_python_str, str)
requires_python = Specifier(requires_python_str)
assert requires_python != oldest_supported_python_specifier, f'requires_python="{requires_python}" is redundant'
assert requires_python != oldest_supported_python_specifier, f'requires-python="{requires_python}" is redundant'
# Check minimum Python version is not less than the oldest version of Python supported by typeshed
assert oldest_supported_python_specifier.contains(
requires_python.version
), f"'requires_python' contains versions lower than typeshed's oldest supported Python ({oldest_supported_python})"
assert requires_python.operator == ">=", "'requires_python' should be a minimum version specifier, use '>=3.x'"
), f"'requires-python' contains versions lower than typeshed's oldest supported Python ({oldest_supported_python})"
assert requires_python.operator == ">=", "'requires-python' should be a minimum version specifier, use '>=3.x'"

empty_tools: dict[object, object] = {}
tools_settings: object = data.get("tool", empty_tools) # pyright: ignore[reportUnknownMemberType]
Expand Down Expand Up @@ -361,6 +361,9 @@ def update_metadata(distribution: str, **new_values: object) -> tomlkit.TOMLDocu
except FileNotFoundError:
raise NoSuchStubError(f"Typeshed has no stubs for {distribution!r}!") from None
data.update(new_values) # pyright: ignore[reportUnknownMemberType] # tomlkit.TOMLDocument.update is partially typed
for key in list(data.keys()):
new_key = key.replace("_", "-") # pyright: ignore[reportUnknownMemberType] # tomlkit.TOMLDocument.keys is partially typed
data[new_key] = data.pop(key) # pyright: ignore[reportUnknownMemberType] # tomlkit.TOMLDocument.pop is partially typed
with path.open("w", encoding="UTF-8") as file:
tomlkit.dump(data, file) # pyright: ignore[reportUnknownMemberType] # tomlkit.dump has partially unknown Mapping type
return data
Expand Down
2 changes: 1 addition & 1 deletion pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -265,4 +265,4 @@ extra-standard-library = [
known-first-party = ["_utils", "ts_utils"]

[tool.typeshed]
oldest_supported_python = "3.10"
oldest-supported-python = "3.10"
4 changes: 2 additions & 2 deletions scripts/create_baseline_stubs.py
Original file line number Diff line number Diff line change
Expand Up @@ -134,11 +134,11 @@ def create_metadata(project: str, stub_dir: Path, version: str) -> None:
if upstream_repo_url is None:
warning = (
f"\nCould not find a URL pointing to the source code for {project!r}.\n"
f"Please add it as `upstream_repository` to `stubs/{project}/METADATA.toml`, if possible!\n"
f"Please add it as `upstream-repository` to `stubs/{project}/METADATA.toml`, if possible!\n"
)
print(termcolor.colored(warning, "red"))
else:
metadata += f'upstream_repository = "{upstream_repo_url}"\n'
metadata += f'upstream-repository = "{upstream_repo_url}"\n'
print(f"Writing {filename}")
filename.write_text(metadata, encoding="UTF-8")

Expand Down
2 changes: 1 addition & 1 deletion scripts/stubsabot.py
Original file line number Diff line number Diff line change
Expand Up @@ -607,7 +607,7 @@ def parse_no_longer_updated_from_archive(source: zipfile.ZipFile | tarfile.TarFi
with file as f:
toml_data: dict[str, object] = tomllib.load(f)

no_longer_updated = toml_data.get("no_longer_updated", False)
no_longer_updated = toml_data.get("no-longer-updated", False)
assert type(no_longer_updated) is bool
return bool(no_longer_updated)

Expand Down
2 changes: 1 addition & 1 deletion stubs/Authlib/METADATA.toml
Original file line number Diff line number Diff line change
@@ -1,3 +1,3 @@
version = "1.6.9"
upstream_repository = "https://github.com/authlib/authlib"
upstream-repository = "https://github.com/authlib/authlib"
dependencies = ["cryptography"]
2 changes: 1 addition & 1 deletion stubs/Deprecated/METADATA.toml
Original file line number Diff line number Diff line change
@@ -1,3 +1,3 @@
version = "~=1.3.1"
upstream_repository = "https://github.com/laurent-laporte-pro/deprecated"
upstream-repository = "https://github.com/laurent-laporte-pro/deprecated"
dependencies = []
2 changes: 1 addition & 1 deletion stubs/Flask-Cors/METADATA.toml
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
version = "6.0.*"
upstream_repository = "https://github.com/corydolphin/flask-cors"
upstream-repository = "https://github.com/corydolphin/flask-cors"
# Requires a version of flask with a `py.typed` file
dependencies = ["Flask>=2.0.0"]
2 changes: 1 addition & 1 deletion stubs/Flask-Migrate/METADATA.toml
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
version = "4.1.*"
upstream_repository = "https://github.com/miguelgrinberg/Flask-Migrate"
upstream-repository = "https://github.com/miguelgrinberg/Flask-Migrate"
# Requires versions of flask and Flask-SQLAlchemy with `py.typed` files
dependencies = ["Flask-SQLAlchemy>=3.0.1", "Flask>=2.0.0"]
2 changes: 1 addition & 1 deletion stubs/Flask-SocketIO/METADATA.toml
Original file line number Diff line number Diff line change
@@ -1,3 +1,3 @@
version = "5.6.*"
dependencies = ["Flask>=0.9"]
upstream_repository = "https://github.com/miguelgrinberg/flask-socketio"
upstream-repository = "https://github.com/miguelgrinberg/flask-socketio"
Loading
Loading