Skip to content

RTD custom build #1096

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
atteggiani wants to merge 3 commits into
base: development
Choose a base branch
from
Draft

RTD custom build #1096

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
d681faa
Added custom build script to dynamically set edit_uri at build time
atteggiani Dec 4, 2025
0913f4c
Changed branch for custom build script to main
atteggiani Dec 4, 2025
e99afd0
Pinned jq and yq versions
atteggiani Dec 15, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 6 additions & 3 deletions .readthedocs/.readthedocs.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -21,9 +21,12 @@ build:
os: ubuntu-24.04
tools:
python: "3.13"
# jobs:
# post_checkout: # MODIFY IF NEEDED: uncomment to cancel the RTD build
# - exit 183
jobs:
# post_checkout: ['exit 183'] # MODIFY IF NEEDED: uncomment to cancel the RTD build
build:
html:
- curl -sSL https://raw.githubusercontent.com/ACCESS-NRI/ACCESS-Hive-Docs/main/.readthedocs/scripts/custom_build.sh | bash # Custom build


# Build documentation with Mkdocs
mkdocs:
Expand Down
59 changes: 59 additions & 0 deletions .readthedocs/scripts/custom_build.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,59 @@
# This is a custom build script for mkdocs that checks for the branch name and exports
# a EDIT_URI env variable to be used in the mkdocs build to set edit_uri to the proper branch,
# especially for PR preview builds.
# If the build comes from a forked repo or from a tag, the EDIT_URI variable will not be set.
# This is needed to avoid the link-check workflow to fail for new pages if the edit_uri points to
# a non-existing file in the main branch.

set -e

if [ "$READTHEDOCS_VERSION_TYPE" == external ]; then # PR preview build
# Download jq
export JQ_EXE=$(pwd)/jq
JQ_VERSION='jq-1.8.1'
wget -q https://github.com/stedolan/jq/releases/download/$JQ_VERSION/jq-linux64 -O "$JQ_EXE"
chmod +x "$JQ_EXE"
"$JQ_EXE" --version
# Get full repo
full_repo=$(sed -E 's|.*github\.com[:/](.+)$|\1|' <<< "$READTHEDOCS_GIT_CLONE_URL")
# Remove .git suffix if present
full_repo=${full_repo%.git}
# Get repo and owner
IFS='/' read owner repo <<< "$full_repo"
# Check if the PR comes from a forked repo
gh_api_url="https://api.github.com/repos/$owner/$repo"
pr_info=$(curl -s "$gh_api_url/pulls/$READTHEDOCS_GIT_IDENTIFIER")
pr_head_owner=$("$JQ_EXE" -r '.head.user.login' <<< "$pr_info")
if [ "$owner" == "$pr_head_owner" ]; then # Not a fork
# Get the branch name
branch_name=$("$JQ_EXE" -r '.head.ref' <<< "$pr_info")
fi
Comment on lines +24 to +30

@CodeGat CodeGat Dec 15, 2025

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Similarly, if this is running on GitHub Runners, gh is installed, and may be a bit easier to work with than calling the api via curl

@atteggiani atteggiani Dec 15, 2025 •
edited
Loading

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same as above, unfortunately I don't think gh is available out of the box within AWS instances.

@atteggiani atteggiani Dec 15, 2025 •
edited
Loading

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Here the choice might be whether to download gh (and using jq within it), or download jq and use curl ... for API queries (as it is done currently).
Not sure if there's any preferred approach as we would always need to download something.
What do you think?

@atteggiani atteggiani Dec 15, 2025

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also, as a general approach, do you think it might make more sense to pin the versions of the external software we download instead of using latest?

@CodeGat CodeGat Dec 15, 2025

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm, I think if you can get away with using gh with it's inbuilt jq that would be cleaner. However note that you would need to find a way to pass a suitable GitHub PAT to gh to enable use of it.

And yes, pin versions of external software. It means one day when any of the tools has a breaking change, you won't need to spend time figuring out why the workflow broke!

@atteggiani atteggiani Dec 15, 2025 •
edited
Loading

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, I remember now I chose to use curl over gh so I wouldn't need a PAT. The problem is that secrets cannot be set in RTD to be used by PRs. Secrets can only be used by "internal" builds, otherwise they will need to be set as plain env variables (i.e., they will be visible to anyone that has access to the RTD project settings/build, and that's definitely not what we want).

Therefore, I think I'll keep the use of jq and yq as it is, but update the script to pin specific versions.

elif git show-ref --heads "$READTHEDOCS_GIT_IDENTIFIER" --quiet; then # Branch build
branch_name="$READTHEDOCS_GIT_IDENTIFIER"
fi

# Download yq
export YQ_EXE=$(pwd)/yq
YQ_VERSION='v4.50.1'
wget -q https://github.com/mikefarah/yq/releases/download/$YQ_VERSION/yq_linux_amd64 -O "$YQ_EXE"
chmod +x "$YQ_EXE"
"$YQ_EXE" --version

# Get RTD config path
rtd_config_path=$(find "$READTHEDOCS_REPOSITORY_PATH" -type f \( -name ".readthedocs.yaml" -o -name ".readthedocs.yml" \))
# Get mkdocs config path
mkdocs_path=$("$YQ_EXE" '.mkdocs.configuration' "$rtd_config_path")
if [ -n "$branch_name" ]; then
# Get docs_dir from mkdocs config or set to default 'docs'
docs_dir=$("$YQ_EXE" '.docs_dir // "docs"' "$mkdocs_path")
EDIT_URI="edit/$branch_name/$docs_dir"
echo "This is a branch build. EDIT_URI set to to '$EDIT_URI'"
export EDIT_URI
fi

# Default RTD mkdocs build commands
echo ======= START OF mkdocs.yml =======
cat "$mkdocs_path"
echo ======= ENF OF mkdocs.yml =======
cmd="python -m mkdocs build --clean --site-dir $READTHEDOCS_OUTPUT/html --config-file $mkdocs_path"
eval "$cmd"
4 changes: 3 additions & 1 deletion mkdocs.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,9 @@ site_url: !ENV READTHEDOCS_CANONICAL_URL
repo_url: https://github.com/ACCESS-NRI/ACCESS-Hive-Docs
repo_name: ACCESS-Hive-Docs

edit_uri: edit/development/docs/
edit_uri: !ENV [EDIT_URI, ""]
# EDIT_URI is set within the website build (ReadTheDocs) through the ACCESS-Hive-Docs repo script
# https://github.com/ACCESS-NRI/ACCESS-Hive-Docs/blob/main/.readthedocs/scripts/custom_build.sh

theme:
name: material
Expand Down
Loading