Commit 6f39c1b
authored
fix: pass --s3-no-head-object to rclone for dataset uploads (#146)
rclone v1.72 changed how its S3 backend handles a 403 from the FS-init
HeadObject probe (rclone/rclone#8975, commit rclone/rclone@6440052f,
2025-11-18): non-NotFound errors now propagate as fatal NewFs failures
instead of being silently treated as "this is a directory". For BlobTree
dataset uploads the platform's session credentials are scoped to
<prefix>/<version>/* and HEAD on the bare <version> key is denied — so
once Rclone_jll resolved to >= v1.72 (currently v1.73.5), every BlobTree
upload aborted with `JuliaHubError: Data upload failed`. The destination
is always a directory in our usage, so the probe serves no purpose;
suppress it with --s3-no-head-object.
Verified locally against internal.juliahub.com with Rclone_jll v1.73.5:
the live datasets testset now passes end-to-end, where it previously
failed at datasets-live.jl:70 (BlobTree upload). The flag has existed in
rclone since v1.55 (2021), so no compat impact.1 parent 5a4e3de commit 6f39c1b
1 file changed
Lines changed: 10 additions & 4 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
814 | 814 | | |
815 | 815 | | |
816 | 816 | | |
| 817 | + | |
| 818 | + | |
| 819 | + | |
| 820 | + | |
| 821 | + | |
| 822 | + | |
| 823 | + | |
| 824 | + | |
| 825 | + | |
817 | 826 | | |
818 | 827 | | |
819 | 828 | | |
820 | | - | |
821 | | - | |
822 | 829 | | |
823 | 830 | | |
824 | | - | |
825 | | - | |
826 | 831 | | |
827 | 832 | | |
828 | 833 | | |
| 834 | + | |
829 | 835 | | |
830 | 836 | | |
831 | 837 | | |
| |||
0 commit comments