Skip to content

Add weekly LinkML toolkit compatibility check #248

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
matentzn wants to merge 1 commit into
base: main
Choose a base branch
from
Open

Add weekly LinkML toolkit compatibility check #248

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
120 changes: 120 additions & 0 deletions .github/workflows/linkml-compat.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,120 @@
# Weekly compatibility test against the linkml toolkit
#
# This workflow checks out the linkml toolkit and runs gen-project against
# the metamodel to ensure compatibility.

name: LinkML Toolkit Compatibility
env:
UV_VERSION: "0.7.13"
on:
schedule:
# Run every Monday at 10:00 UTC (1 hour after linkml's metamodel check)
- cron: "0 10 * * 1"
workflow_dispatch:

jobs:
linkml-compat:
runs-on: ubuntu-latest

Copilot AI Apr 17, 2026

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This job uses gh issue ... / gh label ... to create and update issues/labels, but the workflow doesn't declare any permissions:. In repositories with restricted default GITHUB_TOKEN permissions, these steps will fail (especially label creation and issue commenting). Consider explicitly setting job permissions (e.g., contents: read and issues: write) so scheduled/manual runs can reliably file/update the tracking issue.

Suggested change
runs-on: ubuntu-latest
runs-on: ubuntu-latest
permissions:
contents: read
issues: write

Copilot uses AI. Check for mistakes.
defaults:
run:
shell: bash
steps:
- name: Check out linkml-model repository
uses: actions/checkout@v4.2.2

- name: Check out linkml toolkit
uses: actions/checkout@v4.2.2
with:
repository: linkml/linkml
path: linkml-toolkit

- name: Install uv
uses: astral-sh/setup-uv@v7.2.0
with:
version: ${{ env.UV_VERSION }}
enable-cache: true

- name: Set up Python
uses: actions/setup-python@v5.6.0
Comment on lines +23 to +38

Copilot AI Apr 17, 2026

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This workflow pins older action versions (actions/checkout@v4.2.2, actions/setup-python@v5.6.0, astral-sh/setup-uv@v7.2.0) while the rest of the repo’s workflows use newer pins (e.g., .github/workflows/main.yaml uses actions/checkout@v6.0.2, actions/setup-python@v6.2.0, setup-uv@v7.3.0). To keep maintenance and security updates consistent across CI, align these action versions with the versions already used elsewhere in this repository (or explain why this workflow must differ).

Suggested change
uses: actions/checkout@v4.2.2
- name: Check out linkml toolkit
uses: actions/checkout@v4.2.2
with:
repository: linkml/linkml
path: linkml-toolkit
- name: Install uv
uses: astral-sh/setup-uv@v7.2.0
with:
version: ${{ env.UV_VERSION }}
enable-cache: true
- name: Set up Python
uses: actions/setup-python@v5.6.0
uses: actions/checkout@v6.0.2
- name: Check out linkml toolkit
uses: actions/checkout@v6.0.2
with:
repository: linkml/linkml
path: linkml-toolkit
- name: Install uv
uses: astral-sh/setup-uv@v7.3.0
with:
version: ${{ env.UV_VERSION }}
enable-cache: true
- name: Set up Python
uses: actions/setup-python@v6.2.0

Copilot uses AI. Check for mistakes.
with:
python-version: "3.12"

- name: Install linkml toolkit dependencies
working-directory: linkml-toolkit
run: uv sync --all-groups

- name: Setup Graphviz
uses: ts-graphviz/setup-graphviz@v2.0.2

- name: Run gen-project against metamodel
id: run_genproject
working-directory: linkml-toolkit
run: |
mkdir -p ../output
uv run gen-project ../linkml_model/model/schema/meta.yaml -d ../output 2>&1 | tee ../genproject_output.txt

- name: Verify key outputs were generated
id: verify_outputs
run: |
MISSING=""
# Python files go in root, others in subdirectories (per GEN_MAP in projectgen.py)
Comment on lines +54 to +60

Copilot AI Apr 17, 2026

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

gen-project is invoked without the repo’s gen_project_config.yaml, but this repository’s Makefile consistently runs gen-project with that config (which also excludes some generators like python and markdown). Running without the config may introduce failures unrelated to compatibility (extra generators, different output layout) and makes the output verification brittle. Consider using the same config file as the repo’s normal generation path, and if you want to validate Python output too, run the repo’s gen-python --genmeta step separately rather than assuming gen-project will emit meta.py.

Suggested change
uv run gen-project ../linkml_model/model/schema/meta.yaml -d ../output 2>&1 | tee ../genproject_output.txt
- name: Verify key outputs were generated
id: verify_outputs
run: |
MISSING=""
# Python files go in root, others in subdirectories (per GEN_MAP in projectgen.py)
uv run gen-project --config-file ../gen_project_config.yaml ../linkml_model/model/schema/meta.yaml -d ../output 2>&1 | tee ../genproject_output.txt
- name: Run gen-python --genmeta against metamodel
id: run_genpython
working-directory: linkml-toolkit
run: |
uv run gen-python --genmeta ../linkml_model/model/schema/meta.yaml > ../output/meta.py
- name: Verify key outputs were generated
id: verify_outputs
run: |
MISSING=""
# meta.py is generated separately via gen-python --genmeta; the remaining files come from gen-project.

Copilot uses AI. Check for mistakes.
for file in output/meta.py output/jsonschema/meta.schema.json output/owl/meta.owl.ttl output/shex/meta.shex output/graphql/meta.graphql; do
if [ ! -f "$file" ]; then
MISSING="${MISSING}${file}\n"
fi
done
Comment on lines +56 to +65

Copilot AI Apr 17, 2026

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The output checks hard-code output/meta.py, but in this repo the documented generation flow produces Python files via separate gen-python calls (see Makefile), and gen_project_config.yaml explicitly excludes the python generator. As written, this check can either (a) fail even when gen-project succeeded (if using the repo config), or (b) pass while diverging from the repo’s actual build artifacts. Adjust the verification list to match the outputs that are actually expected from the chosen generation command(s).

Copilot uses AI. Check for mistakes.

if [ -n "$MISSING" ]; then
echo "Missing outputs:"
echo -e "$MISSING"
exit 1
fi

- name: Check for existing issue
if: failure()
id: check_issue
run: |
EXISTING_ISSUE=$(gh issue list --label "linkml-compat" --state open --json number --jq '.[0].number // empty')
echo "existing_issue=${EXISTING_ISSUE}" >> $GITHUB_OUTPUT
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}

- name: Ensure linkml-compat label exists
if: failure()
run: |
gh label create "linkml-compat" --description "LinkML toolkit compatibility test" --color "d93f0b" 2>/dev/null || true
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}

- name: Create or update issue on failure
if: failure()
run: |
ISSUE_TITLE="LinkML toolkit compatibility test failure"
LOG_OUTPUT=$(tail -n 100 genproject_output.txt)

Copilot AI Apr 17, 2026

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The failure handler always does tail -n 100 genproject_output.txt, but that file is only created by the gen-project step. If the workflow fails earlier (e.g., checkout/uv sync/Graphviz), this tail will fail and prevent the issue from being created/updated. Consider guarding for a missing log file (fallback message), or narrowing the if: condition so this step runs only when the gen-project step actually executed.

Suggested change
LOG_OUTPUT=$(tail -n 100 genproject_output.txt)
if [ -f genproject_output.txt ]; then
LOG_OUTPUT=$(tail -n 100 genproject_output.txt)
else
LOG_OUTPUT="genproject_output.txt was not created. The workflow likely failed before the gen-project step ran."
fi

Copilot uses AI. Check for mistakes.
ISSUE_BODY="## LinkML Toolkit Compatibility Test Failure

The weekly LinkML toolkit compatibility test has failed. This indicates that the
current linkml-model metamodel may not be compatible with the latest linkml toolkit.

**Workflow run:** ${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }}

### gen-project Output (last 100 lines)
\`\`\`
${LOG_OUTPUT}
\`\`\`

### Next Steps
1. Review the output above
2. Check for recent changes in [linkml](https://github.com/linkml/linkml)
3. Update the metamodel if needed to maintain compatibility"

if [ -n "${{ steps.check_issue.outputs.existing_issue }}" ]; then
gh issue comment "${{ steps.check_issue.outputs.existing_issue }}" --body "${ISSUE_BODY}"
else
gh issue create \
--title "${ISSUE_TITLE}" \
--body "${ISSUE_BODY}" \
--label "linkml-compat"
fi
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
Loading