Skip to content

Commit 35832eb

Browse files
authored
Separate S3 buckets for raw data and HTML exports (#41)
## Summary Splits Cloudflare R2 bucket configuration into two distinct buckets for better organization and access control: - **CLOUDFLARE_R2_RAWDATA_BUCKET_NAME** (`sfs-json`): Stores source JSON files from the API - **CLOUDFLARE_R2_HTMLEXPORT_BUCKET_NAME** (`web`): Stores generated HTML exports for public access ## Changes - ✅ Updated `fetch-sfs-workflow.yml` to upload JSON to rawdata bucket - ✅ Updated `html-export-workflow.yml` to download from rawdata and upload HTML to export bucket - ✅ Updated `github-pages-workflow.yml` to download JSON from rawdata bucket - ✅ Changed bucket names from secrets to repository variables for better transparency ## Benefits - Better organization with separate buckets for different data types - Allows different retention policies and access controls per bucket - More transparent configuration using variables instead of secrets for non-sensitive data ## Required Setup Before merging, add these two repository variables: ```bash gh variable set CLOUDFLARE_R2_RAWDATA_BUCKET_NAME --body "sfs-json" gh variable set CLOUDFLARE_R2_HTMLEXPORT_BUCKET_NAME --body "web" ``` Or via GitHub UI: Settings → Secrets and variables → Actions → Variables tab 🤖 Generated with [Claude Code](https://claude.com/claude-code)
1 parent 2124ad6 commit 35832eb

3 files changed

Lines changed: 9 additions & 13 deletions

File tree

.github/workflows/fetch-sfs-workflow.yml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -67,7 +67,7 @@ jobs:
6767
aws configure set output json
6868
6969
# Upload all JSON files to R2 for backup
70-
aws s3 sync data/sfs_json/ s3://${{ secrets.CLOUDFLARE_R2_BUCKET_NAME }}/sfs_json/ \
70+
aws s3 sync data/sfs_json/ s3://${{ vars.CLOUDFLARE_R2_RAWDATA_BUCKET_NAME }}/ \
7171
--endpoint-url https://${{ secrets.CLOUDFLARE_R2_ACCOUNT_ID }}.r2.cloudflarestorage.com \
7272
--content-type "application/json" \
7373
--exclude "*.md" \

.github/workflows/github-pages-workflow.yml

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,4 @@
1-
name: Exportera till HTML-format och publicera till GitHub Pages
1+
name: Exportera till HTML och publicera till GitHub Pages
22

33
on:
44
workflow_dispatch: # Tillåter manuell körning
@@ -70,7 +70,7 @@ jobs:
7070
7171
# Download all JSON files from R2
7272
mkdir -p data/sfs_json
73-
aws s3 sync s3://${{ secrets.CLOUDFLARE_R2_BUCKET_NAME }}/sfs_json/ data/sfs_json/ \
73+
aws s3 sync s3://${{ vars.CLOUDFLARE_R2_RAWDATA_BUCKET_NAME }}/ data/sfs_json/ \
7474
--endpoint-url https://${{ secrets.CLOUDFLARE_R2_ACCOUNT_ID }}.r2.cloudflarestorage.com \
7575
--exclude "*" \
7676
--include "*.json"

.github/workflows/html-export-workflow.yml

Lines changed: 6 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -62,7 +62,7 @@ jobs:
6262
6363
# Download all JSON files from R2
6464
mkdir -p data/sfs_json
65-
aws s3 sync s3://${{ secrets.CLOUDFLARE_R2_BUCKET_NAME }}/sfs_json/ data/sfs_json/ \
65+
aws s3 sync s3://${{ vars.CLOUDFLARE_R2_RAWDATA_BUCKET_NAME }}/ data/sfs_json/ \
6666
--endpoint-url https://${{ secrets.CLOUDFLARE_R2_ACCOUNT_ID }}.r2.cloudflarestorage.com \
6767
--exclude "*" \
6868
--include "*.json"
@@ -104,10 +104,6 @@ jobs:
104104
echo "Error: CLOUDFLARE_R2_SECRET_ACCESS_KEY secret is not set"
105105
exit 1
106106
fi
107-
if [ -z "${{ secrets.CLOUDFLARE_R2_BUCKET_NAME }}" ]; then
108-
echo "Error: CLOUDFLARE_R2_BUCKET_NAME secret is not set"
109-
exit 1
110-
fi
111107
if [ -z "${{ secrets.CLOUDFLARE_R2_ACCOUNT_ID }}" ]; then
112108
echo "Error: CLOUDFLARE_R2_ACCOUNT_ID secret is not set"
113109
exit 1
@@ -123,7 +119,7 @@ jobs:
123119
124120
- name: Upload HTML folder to Cloudflare R2
125121
run: |
126-
aws s3 sync output/html/ s3://${{ secrets.CLOUDFLARE_R2_BUCKET_NAME }}/sfs/ \
122+
aws s3 sync output/html/ s3://${{ vars.CLOUDFLARE_R2_HTMLEXPORT_BUCKET_NAME }}/sfs/ \
127123
--endpoint-url https://${{ secrets.CLOUDFLARE_R2_ACCOUNT_ID }}.r2.cloudflarestorage.com \
128124
--delete \
129125
--cache-control "public, max-age=3600" \
@@ -135,11 +131,11 @@ jobs:
135131

136132
- name: Upload index pages to Cloudflare R2
137133
run: |
138-
aws s3 cp index.html s3://${{ secrets.CLOUDFLARE_R2_BUCKET_NAME }}/index.html \
134+
aws s3 cp index.html s3://${{ vars.CLOUDFLARE_R2_HTMLEXPORT_BUCKET_NAME }}/index.html \
139135
--endpoint-url https://${{ secrets.CLOUDFLARE_R2_ACCOUNT_ID }}.r2.cloudflarestorage.com \
140136
--cache-control "public, max-age=1800" \
141137
--content-type "text/html"
142-
aws s3 cp latest.html s3://${{ secrets.CLOUDFLARE_R2_BUCKET_NAME }}/latest.html \
138+
aws s3 cp latest.html s3://${{ vars.CLOUDFLARE_R2_HTMLEXPORT_BUCKET_NAME }}/latest.html \
143139
--endpoint-url https://${{ secrets.CLOUDFLARE_R2_ACCOUNT_ID }}.r2.cloudflarestorage.com \
144140
--cache-control "public, max-age=1800" \
145141
--content-type "text/html"
@@ -149,7 +145,7 @@ jobs:
149145
- name: Upload summary
150146
run: |
151147
echo "HTML export completed at $(date)" > upload-summary.txt
152-
echo "Files uploaded to Cloudflare R2 bucket: ${{ secrets.CLOUDFLARE_R2_BUCKET_NAME }}/sfs/" >> upload-summary.txt
148+
echo "Files uploaded to Cloudflare R2 bucket: ${{ vars.CLOUDFLARE_R2_HTMLEXPORT_BUCKET_NAME }}/sfs/" >> upload-summary.txt
153149
echo "Index pages uploaded: index.html (30 senaste), latest.html (10 senaste)" >> upload-summary.txt
154-
aws s3 cp upload-summary.txt s3://${{ secrets.CLOUDFLARE_R2_BUCKET_NAME }}/sfs/last-update.txt \
150+
aws s3 cp upload-summary.txt s3://${{ vars.CLOUDFLARE_R2_HTMLEXPORT_BUCKET_NAME }}/sfs/last-update.txt \
155151
--endpoint-url https://${{ secrets.CLOUDFLARE_R2_ACCOUNT_ID }}.r2.cloudflarestorage.com

0 commit comments

Comments
 (0)