Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
29 changes: 18 additions & 11 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -85,7 +85,8 @@ jobctl start-controller -r <release> --nightly --arch <architecture>
- `GithubUtil` - Handles file operations in the GitHub repository

**Environment Variables Required:**
- `GITHUB_TOKEN` - GitHub authentication
- `GITHUB_APP_WRITER_ID` - ERT Writer GitHub App ID (`openshift/release-tests`)
- `GITHUB_APP_WRITER_PRIVATE_KEY` - Path to Writer App private key `.pem` file
- `APITOKEN` - Prow/Gangway API authentication

---
Expand Down Expand Up @@ -122,7 +123,7 @@ jobctl start-aggregator --arch <architecture>
- `Artifacts` - Fetches test reports from GCS

**Environment Variables Required:**
- `GITHUB_TOKEN` - GitHub authentication
- `GITHUB_APP_WRITER_ID` / `GITHUB_APP_WRITER_PRIVATE_KEY` - Writer GitHub App for `openshift/release-tests`
- `APITOKEN` - Prow/Gangway API authentication
- `GCS_CRED_FILE` - Google Cloud Storage credentials for artifact access

Expand Down Expand Up @@ -163,7 +164,9 @@ oarctl jira-notificator [--dry-run] [--from-date YYYY-MM-DD]
- `LdapHelper` - LDAP integration for manager lookup

**Environment Variables Required:**
- `JIRA_TOKEN` - Jira authentication token
- `JIRA_TOKEN` - Jira personal access token for API access
- `GITHUB_APP_READER_ID` / `GITHUB_APP_READER_PRIVATE_KEY` - Reader GitHub App for PR label checks
- Kerberos ticket - For LDAP manager lookup (`kinit $kid@$domain`)

**Options:**
- `--dry-run` - Test mode that doesn't send actual Jira comments
Expand Down Expand Up @@ -248,7 +251,7 @@ python tools/auto_release_test_result_checker.py \
- File tracking system to avoid duplicate notifications

**Environment Variables Required:**
- `GITHUB_TOKEN` - GitHub authentication
- `GITHUB_APP_READER_ID` / `GITHUB_APP_READER_PRIVATE_KEY` - Reader GitHub App for `openshift/release-tests` (read-only)
- `SLACK_BOT_TOKEN` - Slack bot token

**Options:**
Expand Down Expand Up @@ -296,19 +299,23 @@ pip3 install -e .

**Release Detector:**
- All OAR CLI environment variables (calls `create-test-report` command)
- **GITHUB_TOKEN** - GitHub personal access token for monitoring repository file changes
- No GitHub token required (uses public raw.githubusercontent.com URL)

**OAR / StateBox / Release Discovery:**
- **GITHUB_APP_WRITER_ID** / **GITHUB_APP_WRITER_PRIVATE_KEY** - Writer GitHub App for `openshift/release-tests`

**Job Controller:**
- **GITHUB_TOKEN** - GitHub personal access token for repository operations
- **GITHUB_APP_WRITER_ID** / **GITHUB_APP_WRITER_PRIVATE_KEY** - Writer GitHub App for `openshift/release-tests`
- **APITOKEN** - Prow/Gangway API token for triggering test jobs

**Test Result Aggregator:**
- **GITHUB_TOKEN** - GitHub personal access token for repository operations
- **GITHUB_APP_WRITER_ID** / **GITHUB_APP_WRITER_PRIVATE_KEY** - Writer GitHub App for `openshift/release-tests`
- **APITOKEN** - Prow/Gangway API token for triggering test jobs
- **GCS_CRED_FILE** - Google Cloud Storage credentials file path for test artifact access

**Jira Notificator:**
- **JIRA_TOKEN** - Jira personal access token for API access
- **GITHUB_APP_READER_ID** / **GITHUB_APP_READER_PRIVATE_KEY** - Reader GitHub App for PR label checks
- **Kerberos ticket** - For LDAP manager lookup
Comment thread
coderabbitai[bot] marked this conversation as resolved.

**Slack Message Receiver (Release Bot):**
Expand All @@ -317,7 +324,7 @@ pip3 install -e .
- All OAR CLI environment variables (executes OAR commands)

**Test Result Checker:**
- **GITHUB_TOKEN** - GitHub personal access token for repository operations
- **GITHUB_APP_READER_ID** / **GITHUB_APP_READER_PRIVATE_KEY** - Reader GitHub App for `openshift/release-tests` (read-only)
- **SLACK_BOT_TOKEN** - Slack bot token for sending notifications

**Note:** `OAR_SLACK_CHANNEL` and `OAR_SLACK_THREAD` are set internally by the Slack bot when executing commands and should not be configured manually by users.
Expand Down Expand Up @@ -1018,9 +1025,9 @@ See individual agent sections above for specific environment variables required.
- **Solution:** Renew your Kerberos ticket: `kinit $kid@$domain`
- **Verify:** Check ticket status with `klist`

**Problem:** GitHub token permissions insufficient
- **Solution:** Ensure token has `repo` scope for private repositories
- **Verify:** Test with `gh auth status`
**Problem:** GitHub App credentials missing or invalid
- **Solution:** Set `GITHUB_APP_WRITER_ID` / `GITHUB_APP_WRITER_PRIVATE_KEY` (and Reader vars if needed); ensure `.pem` path is valid and the app is installed on the target repo
- **Verify:** Check env vars with `env | grep GITHUB_APP`; confirm app installation on `openshift/release-tests` or `openshift/release`

#### Agent-Specific Issues

Expand Down
7 changes: 4 additions & 3 deletions CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -348,7 +348,8 @@ For remote servers:
- Kerberos ticket required: `kinit $kid@$domain`

**For Controllers/Agents:**
- `GITHUB_TOKEN` - GitHub API access
- `GITHUB_APP_WRITER_ID` / `GITHUB_APP_WRITER_PRIVATE_KEY` - Writer App (`openshift/release-tests`)
- `GITHUB_APP_READER_ID` / `GITHUB_APP_READER_PRIVATE_KEY` - Reader App (`openshift/*`, e.g. release, PR checks)
- `APITOKEN` - Prow/Gangway API token
- `GCS_CRED_FILE` - Google Cloud Storage credentials (for test artifacts)

Expand Down Expand Up @@ -518,8 +519,8 @@ When adding new version support, update:

- **Kerberos required** for Errata Tool and LDAP access: `kinit $kid@$domain`
- **Bugzilla credentials** cached in `~/.config/python-bugzilla/bugzillarc`
- **GitHub token** needs `repo` scope for private repositories
- All tokens should be kept in secure storage (Bitwarden, environment variables)
- **GitHub Apps** — Writer (`GITHUB_APP_WRITER_*`) for `openshift/release-tests`; Reader (`GITHUB_APP_READER_*`) for `openshift/*` read access
- All tokens and keys should be kept in secure storage (Bitwarden, environment variables)

## Common Pitfalls

Expand Down
2 changes: 1 addition & 1 deletion deployment/systemd/release-progress-dashboard.service
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,7 @@ Environment="STREAMLIT_SERVER_ADDRESS=0.0.0.0"
Environment="STREAMLIT_SERVER_HEADLESS=true"

# Use bash login shell to load .bash_profile
# This inherits environment variables (GITHUB_TOKEN, etc.)
# This inherits environment variables (GITHUB_APP_WRITER_*, etc.)
ExecStart=/bin/bash -l -c 'cd /home/your-username/release-tests && streamlit run tools/release_progress_dashboard/release_progress_dashboard.py --server.port=${STREAMLIT_SERVER_PORT} --server.address=${STREAMLIT_SERVER_ADDRESS} --server.headless=${STREAMLIT_SERVER_HEADLESS}'

# Restart on failure
Expand Down
5 changes: 4 additions & 1 deletion mcp_server/check_env.sh
100644 → 100755
Original file line number Diff line number Diff line change
Expand Up @@ -59,7 +59,10 @@ echo "Checking controller/agent environment variables..."
echo "(These are not required for OAR CLI, but needed for separate components)"
echo

check_optional "GITHUB_TOKEN" "Release detector, job controller"
check_optional "GITHUB_APP_WRITER_ID" "StateBox, job controller, release discovery"
check_optional "GITHUB_APP_WRITER_PRIVATE_KEY" "StateBox, job controller, release discovery"
check_optional "GITHUB_APP_READER_ID" "Jira notificator, job CLI (openshift/release)"
check_optional "GITHUB_APP_READER_PRIVATE_KEY" "Jira notificator, job CLI (openshift/release)"
check_optional "APITOKEN" "Prow/Gangway job triggering"
check_optional "GCS_CRED_FILE" "GCS test result storage"

Expand Down
2 changes: 1 addition & 1 deletion oar/core/configstore.py
Original file line number Diff line number Diff line change
Expand Up @@ -38,7 +38,7 @@ def validate_environment():

- Prow controllers/aggregators: Validated separately
See: prow/job/controller.py (validate_environment function)
Vars: GITHUB_TOKEN, APITOKEN, GCS_CRED_FILE
Vars: GITHUB_APP_WRITER_*, GITHUB_APP_READER_*, APITOKEN, GCS_CRED_FILE

- MCP server: Uses this validation (wraps OAR CLI commands)
See: mcp_server/server.py
Expand Down
4 changes: 4 additions & 0 deletions oar/core/const.py
Original file line number Diff line number Diff line change
Expand Up @@ -135,6 +135,10 @@
ENV_APP_PASSWD = "GOOGLE_APP_PASSWD"
ENV_JENKINS_USER = "JENKINS_USER"
ENV_JENKINS_TOKEN = "JENKINS_TOKEN"
ENV_VAR_GITHUB_APP_WRITER_ID = "GITHUB_APP_WRITER_ID"
ENV_VAR_GITHUB_APP_WRITER_PRIVATE_KEY = "GITHUB_APP_WRITER_PRIVATE_KEY"
ENV_VAR_GITHUB_APP_READER_ID = "GITHUB_APP_READER_ID"
ENV_VAR_GITHUB_APP_READER_PRIVATE_KEY = "GITHUB_APP_READER_PRIVATE_KEY"
# jira status
JIRA_STATUS_CLOSED = "Closed"
JIRA_STATUS_IN_PROGRESS = "In Progress"
Expand Down
64 changes: 64 additions & 0 deletions oar/core/github_app.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,64 @@
"""GitHub App auth for ERT (Writer: release-tests, Reader: openshift/*)."""

from pathlib import Path

from github import Auth, Github, GithubIntegration


class GitHubApp:
"""PyGithub client via GitHub App installation token."""

def __init__(self, app_id: str, private_key_path: str):
"""
Initialize GitHub App authentication.

Args:
app_id: Application ID (not Client ID).
private_key_path: Path to the App private key ``.pem`` file.
"""
if "\n" in private_key_path or "-----BEGIN" in private_key_path:
raise ValueError(
"private_key_path must be a path to a .pem file, not inline key content"
)
key_file = Path(private_key_path).expanduser()
if not key_file.is_file():
raise FileNotFoundError("GitHub App private key file not found")
key = key_file.read_text()
auth = Auth.AppAuth(app_id, key)
self._integration = GithubIntegration(auth=auth)
Comment thread
coderabbitai[bot] marked this conversation as resolved.

def installation_token(self, owner: str, repo: str) -> str:

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is never called.

"""
Return a short-lived installation access token for ``owner/repo``.

Uses ``GithubIntegration.get_access_token`` (standard GitHub App API).

Args:
owner: GitHub org or user (e.g. ``openshift``).
repo: Repository name (e.g. ``release-tests``).

Returns:
Installation access token string for REST/GraphQL ``Authorization: Bearer``.

Raises:
GithubException: App not installed on the repo or invalid credentials.
"""
installation = self._integration.get_repo_installation(owner, repo)
return self._integration.get_access_token(installation.id).token

def client_for_repo(self, owner: str, repo: str) -> Github:
"""
Return a Github client for ``owner/repo``.

Args:
owner: GitHub org or user (e.g. ``openshift``).
repo: Repository name (e.g. ``release-tests``).

Returns:
Installation-scoped ``Github`` client.

Raises:
GithubException: App not installed on the repo or invalid credentials.
"""
installation = self._integration.get_repo_installation(owner, repo)
return self._integration.get_github_for_installation(installation.id)
42 changes: 29 additions & 13 deletions oar/core/release_discovery.py
Original file line number Diff line number Diff line change
Expand Up @@ -9,8 +9,8 @@
Usage:
from oar.core.release_discovery import ReleaseDiscovery

# Initialize with GitHub token
discovery = ReleaseDiscovery() # Uses GITHUB_TOKEN env var
# Initialize with GitHub App Writer credentials
discovery = ReleaseDiscovery()

# Get all supported y-streams
y_streams = discovery.get_supported_ystreams()
Expand All @@ -32,10 +32,11 @@
from typing import List, Optional

import yaml
from github import Auth, Github
from semver import VersionInfo

from oar.core.const import ENV_VAR_GITHUB_APP_WRITER_ID, ENV_VAR_GITHUB_APP_WRITER_PRIVATE_KEY
from oar.core.exceptions import ReleaseDiscoveryException
from oar.core.github_app import GitHubApp

logger = logging.getLogger(__name__)

Expand All @@ -53,37 +54,52 @@ class ReleaseDiscovery:

def __init__(
self,
github_token: Optional[str] = None,
repo_name: Optional[str] = None,
branch: Optional[str] = None
branch: Optional[str] = None,
):
"""
Initialize ReleaseDiscovery with authenticated GitHub API.

Args:
github_token: GitHub personal access token (default: from GITHUB_TOKEN env)
repo_name: GitHub repository name (default: "openshift/release-tests")
branch: Branch name (default: "z-stream")

Raises:
ReleaseDiscoveryException: If GitHub token is missing
ReleaseDiscoveryException: If GitHub App Writer credentials are missing
"""
token = github_token or os.environ.get("GITHUB_TOKEN")
if not token:
raise ReleaseDiscoveryException("GitHub token not found. Set GITHUB_TOKEN environment variable.")

self.repo_name = repo_name or self.DEFAULT_REPO
self.branch = branch or self.DEFAULT_BRANCH

if self.repo_name != self.DEFAULT_REPO:
raise ReleaseDiscoveryException(
f"ReleaseDiscovery only supports {self.DEFAULT_REPO}, got {self.repo_name}"
)
Comment on lines +73 to +76

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

With raising this exception, it doesn't make sense to keep optional repo_name parameter in constructor any more.


# Split repo_name into owner and repository for GraphQL queries
self.git_repo_owner, self.git_repo_name = self.repo_name.split('/', 1)

auth = Auth.Token(token)
self._github = Github(auth=auth)
self._github = self._init_github_client()

# Tracking files data (fetched via GraphQL)
self._tracking_data: Optional[dict] = None

def _init_github_client(self):

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Consider adding return type here for better readability:

Suggested change
def _init_github_client(self):
def _init_github_client(self) -> Github:

Requires Github import, so not sure if it's worth it.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Consider adding also docstring similar to _github_client_for_repo in statebox: https://github.com/tomasdavidorg/release-tests/blob/d079ad38627ae4949197023fe889a8f4f3e32825/oar/core/statebox.py#L343

app_id = os.environ.get(ENV_VAR_GITHUB_APP_WRITER_ID)
private_key_path = os.environ.get(ENV_VAR_GITHUB_APP_WRITER_PRIVATE_KEY)
if not app_id or not private_key_path:
raise ReleaseDiscoveryException(
f"{ENV_VAR_GITHUB_APP_WRITER_ID} and "
f"{ENV_VAR_GITHUB_APP_WRITER_PRIVATE_KEY} must be set."
)
try:
return GitHubApp(app_id, private_key_path).client_for_repo(
self.git_repo_owner, self.git_repo_name
)
except Exception as e:
raise ReleaseDiscoveryException(
f"Failed to initialize GitHub App Writer ({type(e).__name__})"
) from e

def get_supported_ystreams(self) -> List[str]:
"""
Get all supported y-streams by discovering directories in _releases/.
Expand Down
Loading