Skip to content

Commit a7cb2af

Browse files
authored
refactor: migrate from github3.py to PyGithub (#587)
* refactor: migrate from github3.py to PyGithub ## What/Why Replace github3.py (pinned at 4.0.1) with PyGithub (>=2.6.0) for GitHub API interactions. PyGithub is more actively maintained and provides first-class support for base_url, app auth, and installation tokens. ## Proof it works 176 tests passed, 32 subtests passed, 100% code coverage via `make test`. ## Risk + AI role Medium -- touches all GitHub API interaction code (auth, repo operations, exception handling). All changes AI-generated using Claude Opus 4.6 with human review. ## Review focus 1. Auth flow in auth.py -- AppAuth/installation auth two-step pattern replaces login_as_app_installation 2. dependabot_file.py -- directory_contents (tuples) merged into get_contents (objects with .name), verify the or [] pattern in test mocks handles the combined code path correctly Signed-off-by: jmeridth <jmeridth@gmail.com> * test: add coverage for missing app id/installation id guard Signed-off-by: jmeridth <jmeridth@gmail.com> * docs: fix return type in get_github_app_installation_token docstring Signed-off-by: jmeridth <jmeridth@gmail.com> * fix: narrow exception handling in get_github_app_installation_token ## What/Why Narrow except clause from bare Exception to GithubException so that misconfiguration errors (bad int() cast, JWT construction failures) propagate instead of silently returning None and causing confusing 401s downstream. Addresses PR review feedback from @zkoppert. ## Proof it works All 14 tests in test_auth.py pass. Updated the request_failure test to raise GithubException instead of bare Exception to match the narrowed catch. Pylint clean (no new warnings). ## Risk + AI role Low. AI-generated (Claude Opus 4.6) with human review. ## Review focus Whether GithubException covers all the API/network error cases we want to catch, or if additional specific exceptions should be included. Signed-off-by: jmeridth <jmeridth@gmail.com> * fix: use repo.owner.login for security-updates URLs and handle datetime in created_after filter ## What/Why Fix two bugs exposed by the PyGithub migration: (1) repo.owner is now a NamedUser object, not a string, so security-updates URLs rendered as "repos/NamedUser(login=...)/..." and 404'd silently; (2) repo.created_at is a datetime object but is_repo_created_date_before called fromisoformat() on it, which raises TypeError. ## Proof it works All 180 tests pass including 2 new tests for datetime input to is_repo_created_date_before (both before and after filter date). ## Risk + AI role Medium -- the repo.owner.login fix touches the security-updates code path which is pragma: no cover. AI-generated (Claude Opus 4.6) with human review. ## Review focus Whether repo.owner.login is the correct attribute for all PyGithub Repository objects (org-owned vs user-owned repos). Signed-off-by: jmeridth <jmeridth@gmail.com> --------- Signed-off-by: jmeridth <jmeridth@gmail.com>
1 parent 469649a commit a7cb2af

12 files changed

Lines changed: 506 additions & 416 deletions

.github/linters/.mypy.ini

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,5 @@
11
[mypy]
22
disable_error_code = attr-defined, import-not-found
33

4-
[mypy-github3.*]
4+
[mypy-github.*]
55
ignore_missing_imports = True

auth.py

Lines changed: 24 additions & 30 deletions
Original file line numberDiff line numberDiff line change
@@ -1,8 +1,7 @@
11
"""This is the module that contains functions related to authenticating to GitHub with a personal access token."""
22

33
import env
4-
import github3
5-
import requests
4+
from github import Auth, Github, GithubException, GithubIntegration
65

76

87
def auth_to_github(
@@ -13,7 +12,7 @@ def auth_to_github(
1312
ghe: str,
1413
gh_app_enterprise_only: bool,
1514
ghe_api_url: str = "",
16-
) -> github3.GitHub:
15+
) -> Github:
1716
"""
1817
Connect to GitHub.com or GitHub Enterprise, depending on env variables.
1918
@@ -27,33 +26,27 @@ def auth_to_github(
2726
ghe_api_url (str): the full GitHub Enterprise API endpoint URL override
2827
2928
Returns:
30-
github3.GitHub: the GitHub connection object
29+
Github: the GitHub connection object
3130
"""
3231
if gh_app_id and gh_app_private_key_bytes and gh_app_installation_id:
32+
app_auth = Auth.AppAuth(int(gh_app_id), gh_app_private_key_bytes.decode())
33+
installation_auth = app_auth.get_installation_auth(int(gh_app_installation_id))
3334
if ghe and gh_app_enterprise_only:
34-
gh = github3.github.GitHubEnterprise(url=ghe)
35-
if ghe_api_url:
36-
gh.session.base_url = ghe_api_url
35+
base_url = env.get_api_endpoint(ghe, ghe_api_url)
36+
github_connection = Github(base_url=base_url, auth=installation_auth)
3737
else:
38-
gh = github3.github.GitHub()
39-
gh.login_as_app_installation(
40-
gh_app_private_key_bytes, str(gh_app_id), gh_app_installation_id
41-
)
42-
github_connection = gh
38+
github_connection = Github(auth=installation_auth)
4339
elif ghe and token:
44-
github_connection = github3.github.GitHubEnterprise(url=ghe, token=token)
45-
if ghe_api_url:
46-
github_connection.session.base_url = ghe_api_url
40+
base_url = env.get_api_endpoint(ghe, ghe_api_url)
41+
github_connection = Github(base_url=base_url, auth=Auth.Token(token))
4742
elif token:
48-
github_connection = github3.login(token=token)
43+
github_connection = Github(auth=Auth.Token(token))
4944
else:
5045
raise ValueError(
5146
"GH_TOKEN or the set of [GH_APP_ID, GH_APP_INSTALLATION_ID, GH_APP_PRIVATE_KEY] environment variables are not set"
5247
)
5348

54-
if not github_connection:
55-
raise ValueError("Unable to authenticate to GitHub")
56-
return github_connection # type: ignore
49+
return github_connection
5750

5851

5952
def get_github_app_installation_token(
@@ -75,18 +68,19 @@ def get_github_app_installation_token(
7568
ghe_api_url (str): the full GitHub Enterprise API endpoint URL override
7669
7770
Returns:
78-
str: the GitHub App token
71+
str | None: the GitHub App token, or None if IDs are missing or the request fails
7972
"""
80-
jwt_headers = github3.apps.create_jwt_headers(
81-
gh_app_private_key_bytes, str(gh_app_id)
82-
)
83-
api_endpoint = env.get_api_endpoint(ghe, ghe_api_url)
84-
url = f"{api_endpoint}/app/installations/{gh_app_installation_id}/access_tokens"
85-
73+
if not gh_app_id or not gh_app_installation_id:
74+
return None
8675
try:
87-
response = requests.post(url, headers=jwt_headers, json=None, timeout=5)
88-
response.raise_for_status()
89-
except requests.exceptions.RequestException as e:
76+
app_auth = Auth.AppAuth(int(gh_app_id), gh_app_private_key_bytes.decode())
77+
if ghe:
78+
base_url = env.get_api_endpoint(ghe, ghe_api_url)
79+
gi = GithubIntegration(auth=app_auth, base_url=base_url)
80+
else:
81+
gi = GithubIntegration(auth=app_auth)
82+
installation_token = gi.get_access_token(int(gh_app_installation_id))
83+
return installation_token.token
84+
except GithubException as e:
9085
print(f"Request failed: {e}")
9186
return None
92-
return response.json().get("token")

dependabot_file.py

Lines changed: 10 additions & 16 deletions
Original file line numberDiff line numberDiff line change
@@ -5,9 +5,9 @@
55
import io
66
from collections.abc import Mapping
77

8-
import github3
98
import ruamel.yaml
109
from exceptions import OptionalFileNotFoundError, check_optional_file
10+
from github import UnknownObjectException
1111
from ruamel.yaml.scalarstring import SingleQuotedScalarString
1212

1313
# Define data structure for dependabot.yaml
@@ -349,8 +349,8 @@ def build_dependabot_file(
349349
# detect package managers with variable file names
350350
if "terraform" not in exempt_ecosystems_list:
351351
try:
352-
for file in repo.directory_contents("/"):
353-
if file[0].endswith(".tf"):
352+
for file in repo.get_contents("/"):
353+
if file.name.endswith(".tf"):
354354
package_managers_found["terraform"] = True
355355
make_dependabot_config(
356356
"terraform",
@@ -363,14 +363,12 @@ def build_dependabot_file(
363363
cooldown,
364364
)
365365
break
366-
except github3.exceptions.NotFoundError:
367-
# The file does not exist and is not required,
368-
# so we should continue to the next one rather than raising error or logging
366+
except UnknownObjectException:
369367
pass
370368
if "github-actions" not in exempt_ecosystems_list:
371369
try:
372-
for file in repo.directory_contents(".github/workflows"):
373-
if file[0].endswith(".yml") or file[0].endswith(".yaml"):
370+
for file in repo.get_contents(".github/workflows"):
371+
if file.name.endswith(".yml") or file.name.endswith(".yaml"):
374372
package_managers_found["github-actions"] = True
375373
make_dependabot_config(
376374
"github-actions",
@@ -383,14 +381,12 @@ def build_dependabot_file(
383381
cooldown,
384382
)
385383
break
386-
except github3.exceptions.NotFoundError:
387-
# The file does not exist and is not required,
388-
# so we should continue to the next one rather than raising error or logging
384+
except UnknownObjectException:
389385
pass
390386
if "devcontainers" not in exempt_ecosystems_list:
391387
try:
392-
for file in repo.directory_contents(".devcontainer"):
393-
if file[0] == "devcontainer.json":
388+
for file in repo.get_contents(".devcontainer"):
389+
if file.name == "devcontainer.json":
394390
package_managers_found["devcontainers"] = True
395391
make_dependabot_config(
396392
"devcontainers",
@@ -403,9 +399,7 @@ def build_dependabot_file(
403399
cooldown,
404400
)
405401
break
406-
except github3.exceptions.NotFoundError:
407-
# The file does not exist and is not required,
408-
# so we should continue to the next one rather than raising error or logging
402+
except UnknownObjectException:
409403
pass
410404

411405
if any(package_managers_found.values()):

evergreen.py

Lines changed: 31 additions & 26 deletions
Original file line numberDiff line numberDiff line change
@@ -7,11 +7,11 @@
77

88
import auth
99
import env
10-
import github3
1110
import requests
1211
import ruamel.yaml
1312
from dependabot_file import build_dependabot_file, validate_cooldown_config
1413
from exceptions import OptionalFileNotFoundError, check_optional_file
14+
from github import UnknownObjectException
1515

1616

1717
def main(): # pragma: no cover
@@ -213,10 +213,10 @@ def main(): # pragma: no cover
213213
# Get dependabot security updates enabled if possible
214214
if config.enable_security_updates:
215215
if not is_dependabot_security_updates_enabled(
216-
config.ghe, config.ghe_api_url, repo.owner, repo.name, token
216+
config.ghe, config.ghe_api_url, repo.owner.login, repo.name, token
217217
):
218218
enable_dependabot_security_updates(
219-
config.ghe, config.ghe_api_url, repo.owner, repo.name, token
219+
config.ghe, config.ghe_api_url, repo.owner.login, repo.name, token
220220
)
221221

222222
if config.follow_up_type == "issue":
@@ -290,7 +290,7 @@ def main(): # pragma: no cover
290290
print(
291291
f"\tLinked pull request to project {project_global_id}"
292292
)
293-
except github3.exceptions.NotFoundError:
293+
except UnknownObjectException:
294294
print("\tFailed to create pull request. Check write permissions.")
295295
continue
296296

@@ -300,9 +300,16 @@ def main(): # pragma: no cover
300300
append_to_github_summary(summary_content)
301301

302302

303-
def is_repo_created_date_before(repo_created_at: str, created_after_date: str):
303+
def is_repo_created_date_before(
304+
repo_created_at: str | datetime, created_after_date: str
305+
):
304306
"""Check if the repository was created before the created_after_date"""
305-
repo_created_at_date = datetime.fromisoformat(repo_created_at).replace(tzinfo=None)
307+
if isinstance(repo_created_at, datetime):
308+
repo_created_at_date = repo_created_at.replace(tzinfo=None)
309+
else:
310+
repo_created_at_date = datetime.fromisoformat(repo_created_at).replace(
311+
tzinfo=None
312+
)
306313
return created_after_date and repo_created_at_date < datetime.strptime(
307314
created_after_date, "%Y-%m-%d"
308315
)
@@ -332,11 +339,11 @@ def check_existing_config(repo, filename):
332339
repository and return the existing config if it does
333340
334341
Args:
335-
repo (github3.repos.repo.Repository): The repository to check
342+
repo: The repository to check
336343
filename (str): The configuration filename to check
337344
338345
Returns:
339-
github3.repos.contents.Contents | None: The existing config if it exists, otherwise None
346+
The existing config if it exists, otherwise None
340347
"""
341348
existing_config = None
342349
try:
@@ -376,36 +383,33 @@ def get_repos_iterator(
376383
# Use GitHub search API if REPOSITORY_SEARCH_QUERY is set
377384
if search_query:
378385
# Return repositories matching the search query
379-
repos = []
380-
# Search results need to be converted to a list of repositories since they are returned as a search iterator
381-
for repo in github_connection.search_repositories(search_query):
382-
repos.append(repo.repository)
386+
repos = list(github_connection.search_repositories(search_query))
383387
return repos
384388

385389
repos = []
386390
# Default behavior: list all organization/team repositories or specific repository list
387391
if organization and not repository_list and not team_name:
388-
repos = github_connection.organization(organization).repositories()
392+
repos = github_connection.get_organization(organization).get_repos()
389393
elif team_name and organization:
390394
# Get the repositories from the team
391-
team = github_connection.organization(organization).team_by_name(team_name)
395+
team = github_connection.get_organization(organization).get_team_by_slug(
396+
team_name
397+
)
392398
if team.repos_count == 0:
393399
print(f"Team {team_name} has no repositories")
394400
sys.exit(1)
395-
repos = team.repositories()
401+
repos = team.get_repos()
396402
else:
397403
# Get the repositories from the repository_list
398404
for repo in repository_list:
399-
repos.append(
400-
github_connection.repository(repo.split("/")[0], repo.split("/")[1])
401-
)
405+
repos.append(github_connection.get_repo(repo))
402406

403407
return repos
404408

405409

406410
def check_pending_pulls_for_duplicates(title, repo) -> bool:
407411
"""Check if there are any open pull requests for dependabot and return the bool skip"""
408-
pull_requests = repo.pull_requests(state="open")
412+
pull_requests = repo.get_pulls(state="open")
409413
skip = False
410414
for pull_request in pull_requests:
411415
if pull_request.title.startswith(title):
@@ -417,7 +421,7 @@ def check_pending_pulls_for_duplicates(title, repo) -> bool:
417421

418422
def check_pending_issues_for_duplicates(title, repo) -> bool:
419423
"""Check if there are any open issues for dependabot and return the bool skip"""
420-
issues = repo.issues(state="open")
424+
issues = repo.get_issues(state="open")
421425
skip = False
422426
for issue in issues:
423427
if issue.title.startswith(title):
@@ -439,21 +443,22 @@ def commit_changes(
439443
"""Commit the changes to the repo and open a pull request and return the pull request object"""
440444
default_branch = repo.default_branch
441445
# Get latest commit sha from default branch
442-
default_branch_commit = repo.ref("heads/" + default_branch).object.sha
443-
front_matter = "refs/heads/"
446+
default_branch_commit = repo.get_git_ref("heads/" + default_branch).object.sha
444447
branch_name = "dependabot-" + str(uuid.uuid4())
445-
repo.create_ref(front_matter + branch_name, default_branch_commit)
448+
repo.create_git_ref(ref="refs/heads/" + branch_name, sha=default_branch_commit)
446449
if existing_config:
447-
repo.file_contents(dependabot_filename).update(
450+
repo.update_file(
451+
path=dependabot_filename,
448452
message=message,
449-
content=dependabot_file.encode(), # Convert to bytes object
453+
content=dependabot_file,
454+
sha=existing_config.sha,
450455
branch=branch_name,
451456
)
452457
else:
453458
repo.create_file(
454459
path=dependabot_filename,
455460
message=message,
456-
content=dependabot_file.encode(), # Convert to bytes object
461+
content=dependabot_file,
457462
branch=branch_name,
458463
)
459464

exceptions.py

Lines changed: 11 additions & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -1,18 +1,20 @@
11
"""Custom exceptions for the evergreen application."""
22

3-
import github3.exceptions
3+
from github import UnknownObjectException
44

55

6-
class OptionalFileNotFoundError(github3.exceptions.NotFoundError):
6+
class OptionalFileNotFoundError(UnknownObjectException):
77
"""Exception raised when an optional file is not found.
88
9-
This exception inherits from github3.exceptions.NotFoundError but provides
9+
This exception inherits from github.UnknownObjectException but provides
1010
a more explicit name for cases where missing files are expected and should
1111
not be treated as errors. This is typically used for optional configuration
1212
files or dependency files that may not exist in all repositories.
1313
1414
Args:
15-
resp: The HTTP response object from the failed request
15+
status: The HTTP status code
16+
data: The response data
17+
headers: The response headers
1618
"""
1719

1820

@@ -35,15 +37,13 @@ def check_optional_file(repo, filename):
3537
Other exceptions: For unexpected errors (permissions, network issues, etc.)
3638
"""
3739
try:
38-
file_contents = repo.file_contents(filename)
39-
# Handle both real file contents objects and test mocks that return boolean
40+
file_contents = repo.get_contents(filename)
4041
if hasattr(file_contents, "size"):
41-
# Real file contents object
4242
if file_contents.size > 0:
4343
return file_contents
4444
return None
45-
# Test mock or other truthy value
4645
return file_contents if file_contents else None
47-
except github3.exceptions.NotFoundError as e:
48-
# Re-raise as our more specific exception type for better semantic clarity
49-
raise OptionalFileNotFoundError(resp=e.response) from e
46+
except UnknownObjectException as e:
47+
raise OptionalFileNotFoundError(
48+
status=e.status, data=e.data, headers=e.headers
49+
) from e

pyproject.toml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -4,7 +4,7 @@ version = "1.0.0"
44
description = "GitHub Action that enables Dependabot for all repositories in a GitHub organization."
55
requires-python = ">=3.11"
66
dependencies = [
7-
"github3-py==4.0.1",
7+
"PyGithub>=2.6.0",
88
"python-dotenv==1.2.2",
99
"requests==2.34.2",
1010
"ruamel-yaml==0.19.1",

0 commit comments

Comments
 (0)