Skip to content

Optimize wolfprovider build time #167

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
ColtonWilley merged 1 commit into from
Jun 17, 2025
Merged

Optimize wolfprovider build time #167

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
111 changes: 99 additions & 12 deletions .github/workflows/build-wolfprovider.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -12,21 +12,21 @@ on:
outputs:
cache_key:
description: "Cache key for the build artifacts"
value: ${{ jobs.build_wolfprovider.outputs.cache_key }}
value: ${{ jobs.build_wolfprovider_common.outputs.cache_key }}

jobs:
build_wolfprovider:
build_wolfprovider_common:
name: Build wolfProvider
runs-on: ubuntu-22.04
timeout-minutes: 20
outputs:
cache_key: wolfprov-${{ inputs.wolfssl_ref }}-${{ github.sha }}
cache_key: wolfprov-${{ inputs.wolfssl_ref }}-${{ inputs.openssl_ref }}-${{ github.sha }}
steps:
- name: Checkout wolfProvider
uses: actions/checkout@v4

# Check if this version of wolfssl/wolfprovider has already been built,
# mark to cache these items on post if we do end up building
# On a push, the first workflow(s) will complete the build and cache it
# and any delayed workflow will be able to utilize it.
# This is not designed to cache builds across commits.
- name: Checking wolfSSL/wolfProvider in cache
uses: actions/cache@v4
id: wolfprov-cache
Expand All @@ -35,30 +35,117 @@ jobs:
wolfssl-source
wolfssl-install
wolfprov-install
openssl-install
provider.conf

key: wolfprov-${{ inputs.wolfssl_ref }}-${{ github.sha }}
key: wolfprov-${{ inputs.wolfssl_ref }}-${{ inputs.openssl_ref }}-${{ github.sha }}
lookup-only: true

# If wolfssl/wolfprovider have not yet been built, pull ossl from cache
# if cache hit, exit the workflow
- name: Exit workflow if cache hit
if: steps.wolfprov-cache.outputs.cache-hit == 'true'
run: |
echo "Found cached build, exiting workflow"
exit 0

- name: Checkout wolfProvider
if: steps.wolfprov-cache.outputs.cache-hit != 'true'
uses: actions/checkout@v4
with:
fetch-depth: 1

# Convert OpenSSL branch name to commit hash if needed
- name: Get OpenSSL commit hash
if: steps.wolfprov-cache.outputs.cache-hit != 'true'
id: openssl-ref
run: |
if [[ "${{ inputs.openssl_ref }}" =~ ^[0-9a-f]{40}$ ]]; then
echo "ref=${{ inputs.openssl_ref }}" >> $GITHUB_OUTPUT
else
sha=$(curl -s "https://api.github.com/repos/openssl/openssl/commits/${{ inputs.openssl_ref }}" | jq -r '.sha')
echo "ref=$sha" >> $GITHUB_OUTPUT
echo "OpenSSL ref ${{ inputs.openssl_ref }} is commit $sha"
fi

# Look for a cached version of OpenSSL
- name: Checking OpenSSL in cache
if: steps.wolfprov-cache.outputs.cache-hit != 'true'
uses: actions/cache@v4
uses: actions/cache/restore@v4
id: openssl-cache
with:
path: |
openssl-source
openssl-install

key: ossl-depends-${{ inputs.openssl_ref }}-${{ github.sha }}
lookup-only: true
key: ossl-depends-${{ steps.openssl-ref.outputs.ref }}
lookup-only: false

# Convert WolfSSL branch name to commit hash if needed
- name: Get WolfSSL commit hash
if: steps.wolfprov-cache.outputs.cache-hit != 'true'
id: wolfssl-ref
run: |
if [[ "${{ inputs.wolfssl_ref }}" =~ ^[0-9a-f]{40}$ ]]; then
echo "ref=${{ inputs.wolfssl_ref }}" >> $GITHUB_OUTPUT
else
sha=$(curl -s "https://api.github.com/repos/wolfssl/wolfssl/commits/${{ inputs.wolfssl_ref }}" | jq -r '.sha')
echo "ref=$sha" >> $GITHUB_OUTPUT
echo "WolfSSL ref ${{ inputs.wolfssl_ref }} is commit $sha"
fi

# Look for a cached version of WolfSSL
- name: Checking WolfSSL in cache
if: steps.wolfprov-cache.outputs.cache-hit != 'true'
uses: actions/cache/restore@v4
id: wolfssl-cache
with:
path: |
wolfssl-source
wolfssl-install

key: wolfssl-depends-${{ steps.wolfssl-ref.outputs.ref }}
lookup-only: false

# If not yet built this version, build it now
- name: Build wolfProvider
if: steps.wolfprov-cache.outputs.cache-hit != 'true'
run: |
OPENSSL_TAG=${{ inputs.openssl_ref }} WOLFSSL_TAG=${{ inputs.wolfssl_ref }} ./scripts/build-wolfprovider.sh

# This is potentially chewing up a lot of cache space.
# Could investigate reducing the size of this cache.
# Note that we use actions/cache/save and actions/cache/restore instead of
# actions/cache to reduce the chance of a cache lock issue.
- name: Save wolfProvider into cache
if: steps.wolfprov-cache.outputs.cache-hit != 'true'
uses: actions/cache/save@v4
with:
path: |
wolfssl-source
wolfssl-install
wolfprov-install
openssl-install
provider.conf
key: wolfprov-${{ inputs.wolfssl_ref }}-${{ inputs.openssl_ref }}-${{ github.sha }}

# If openssl cache miss, save it to the cache
- name: Save OpenSSL into cache
if: steps.openssl-cache.outputs.cache-hit != 'true'
uses: actions/cache/save@v4
with:
path: |
openssl-source
openssl-install
key: ossl-depends-${{ steps.openssl-ref.outputs.ref }}

- name: Save WolfSSL into cache
if: steps.wolfssl-cache.outputs.cache-hit != 'true'
uses: actions/cache/save@v4
with:
path: |
wolfssl-source
wolfssl-install
key: wolfssl-depends-${{ steps.wolfssl-ref.outputs.ref }}

- name: Print errors
if: ${{ failure() }}
run: |
Expand Down
25 changes: 7 additions & 18 deletions .github/workflows/cjose.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -35,35 +35,23 @@ jobs:
openssl_ref: [ 'openssl-3.5.0' ]
force_fail: [ 'WOLFPROV_FORCE_FAIL=1', '' ]
steps:
# Checkout the source so we can run the check-workflow-result script
- name: Checkout wolfProvider
uses: actions/checkout@v4
with:
sparse-checkout: |
.github
fetch-depth: 1

- name: Retrieving OpenSSL from cache
- name: Retrieving wolfProvider from cache
uses: actions/cache/restore@v4
id: openssl-cache
with:
path: |
openssl-source
openssl-install

key: ossl-depends-${{ matrix.openssl_ref }}-${{ github.sha }}
fail-on-cache-miss: true

- name: Retrieving wolfSSL/wolfProvider from cache
uses: actions/cache/restore@v4
id: wolfprov-cache
id: wolfprov-cache-restore
with:
path: |
wolfssl-source
wolfssl-install
wolfprov-install
openssl-install
provider.conf

key: wolfprov-${{ matrix.wolfssl_ref }}-${{ github.sha }}
key: wolfprov-${{ matrix.wolfssl_ref }}-${{ matrix.openssl_ref }}-${{ github.sha }}
fail-on-cache-miss: true

- name: Install cjose dependencies
Expand All @@ -78,12 +66,13 @@ jobs:
repository: OpenIDC/cjose
ref: ${{ matrix.cjose_ref }}
path: cjose
fetch-depth: 1

- name: Build cjose
working-directory: cjose
run: |
# Configure with OpenSSL
./configure CFLAGS="-Wno-error=deprecated-declarations" --with-openssl=/git/wolfProvider/openssl-install
./configure CFLAGS="-Wno-error=deprecated-declarations" --with-openssl=/git/wolfProvider/openssl-install

# Build cjose
make
Expand Down
16 changes: 4 additions & 12 deletions .github/workflows/curl.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -40,18 +40,9 @@ jobs:
steps:
- name: Checkout wolfProvider
uses: actions/checkout@v4

- name: Retrieving OpenSSL from cache
uses: actions/cache/restore@v4
id: openssl-cache
with:
path: |
openssl-source
openssl-install

key: ossl-depends-${{ matrix.openssl_ref }}-${{ github.sha }}
fail-on-cache-miss: true

fetch-depth: 1

- name: Retrieving wolfSSL/wolfProvider from cache
uses: actions/cache/restore@v4
id: wolfprov-cache
Expand All @@ -60,9 +51,10 @@ jobs:
wolfssl-source
wolfssl-install
wolfprov-install
openssl-install
provider.conf

key: wolfprov-${{ matrix.wolfssl_ref }}-${{ github.sha }}
key: wolfprov-${{ matrix.wolfssl_ref }}-${{ matrix.openssl_ref }}-${{ github.sha }}
fail-on-cache-miss: true

- name: Install dependencies
Expand Down
16 changes: 4 additions & 12 deletions .github/workflows/grpc.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -44,18 +44,9 @@ jobs:
steps:
- name: Checkout wolfProvider
uses: actions/checkout@v4

- name: Retrieving OpenSSL from cache
uses: actions/cache/restore@v4
id: openssl-cache
with:
path: |
openssl-source
openssl-install

key: ossl-depends-${{ matrix.openssl_ref }}-${{ github.sha }}
fail-on-cache-miss: true

fetch-depth: 1

- name: Retrieving wolfSSL/wolfProvider from cache
uses: actions/cache/restore@v4
id: wolfprov-cache
Expand All @@ -64,9 +55,10 @@ jobs:
wolfssl-source
wolfssl-install
wolfprov-install
openssl-install
provider.conf

key: wolfprov-${{ matrix.wolfssl_ref }}-${{ github.sha }}
key: wolfprov-${{ matrix.wolfssl_ref }}-${{ matrix.openssl_ref }}-${{ github.sha }}
fail-on-cache-miss: true

- name: Confirm IPv4 and IPv6 support
Expand Down
14 changes: 3 additions & 11 deletions .github/workflows/iperf.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -37,17 +37,8 @@ jobs:
steps:
- name: Checkout wolfProvider
uses: actions/checkout@v4

- name: Retrieving OpenSSL from cache
uses: actions/cache/restore@v4
id: openssl-cache
with:
path: |
openssl-source
openssl-install

key: ossl-depends-${{ matrix.openssl_ref }}-${{ github.sha }}
fail-on-cache-miss: true
fetch-depth: 1

- name: Retrieving wolfSSL/wolfProvider from cache
uses: actions/cache/restore@v4
Expand All @@ -57,9 +48,10 @@ jobs:
wolfssl-source
wolfssl-install
wolfprov-install
openssl-install
provider.conf

key: wolfprov-${{ matrix.wolfssl_ref }}-${{ github.sha }}
key: wolfprov-${{ matrix.wolfssl_ref }}-${{ matrix.openssl_ref }}-${{ github.sha }}
fail-on-cache-miss: true

- name: Install dependencies
Expand Down
14 changes: 3 additions & 11 deletions .github/workflows/ipmitool.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -37,17 +37,8 @@ jobs:
steps:
- name: Checkout wolfProvider
uses: actions/checkout@v4

- name: Retrieving OpenSSL from cache
uses: actions/cache/restore@v4
id: openssl-cache
with:
path: |
openssl-source
openssl-install

key: ossl-depends-${{ matrix.openssl_ref }}-${{ github.sha }}
fail-on-cache-miss: true
fetch-depth: 1

- name: Retrieving wolfSSL/wolfProvider from cache
uses: actions/cache/restore@v4
Expand All @@ -57,9 +48,10 @@ jobs:
wolfssl-source
wolfssl-install
wolfprov-install
openssl-install
provider.conf

key: wolfprov-${{ matrix.wolfssl_ref }}-${{ github.sha }}
key: wolfprov-${{ matrix.wolfssl_ref }}-${{ matrix.openssl_ref }}-${{ github.sha }}
fail-on-cache-miss: true

- name: Install dependencies
Expand Down
16 changes: 4 additions & 12 deletions .github/workflows/liboauth2.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -36,18 +36,9 @@ jobs:
steps:
- name: Checkout wolfProvider
uses: actions/checkout@v4

- name: Retrieving OpenSSL from cache
uses: actions/cache/restore@v4
id: openssl-cache
with:
path: |
openssl-source
openssl-install

key: ossl-depends-${{ matrix.openssl_ref }}-${{ github.sha }}
fail-on-cache-miss: true

fetch-depth: 1

- name: Retrieving wolfSSL/wolfProvider from cache
uses: actions/cache/restore@v4
id: wolfprov-cache
Expand All @@ -56,9 +47,10 @@ jobs:
wolfssl-source
wolfssl-install
wolfprov-install
openssl-install
provider.conf

key: wolfprov-${{ matrix.wolfssl_ref }}-${{ github.sha }}
key: wolfprov-${{ matrix.wolfssl_ref }}-${{ matrix.openssl_ref }}-${{ github.sha }}
fail-on-cache-miss: true

- name: Install liboauth2 dependencies
Expand Down
14 changes: 3 additions & 11 deletions .github/workflows/net-snmp.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -38,17 +38,8 @@ jobs:
steps:
- name: Checkout wolfProvider
uses: actions/checkout@v4

- name: Retrieving OpenSSL from cache
uses: actions/cache/restore@v4
id: openssl-cache
with:
path: |
openssl-source
openssl-install

key: ossl-depends-${{ matrix.openssl_ref }}-${{ github.sha }}
fail-on-cache-miss: true
fetch-depth: 1

- name: Retrieving wolfSSL/wolfProvider from cache
uses: actions/cache/restore@v4
Expand All @@ -58,9 +49,10 @@ jobs:
wolfssl-source
wolfssl-install
wolfprov-install
openssl-install
provider.conf

key: wolfprov-${{ matrix.wolfssl_ref }}-${{ github.sha }}
key: wolfprov-${{ matrix.wolfssl_ref }}-${{ matrix.openssl_ref }}-${{ github.sha }}
fail-on-cache-miss: true

- name: Install dependencies
Expand Down
Loading
Loading