Handle testing assets via Artifacts system

[j-fu]

We could have more tests reading and writing grids from different sources.
As it is handeled now, the test grids would increase the package size significantly.

This can be handeled through artifacts, though.

We could create a WIAS-PDELib/TestingArtifacts repo for storing them and access them during package tests.

[j-fu]

From ChatGPT: Below is a complete, end-to-end example workflow for hosting multiple Julia artifacts using GitHub Releases, including how to structure things when one repo hosts several artifacts (which is totally fine and common).

---

✅ Key answers up front

Can one repo host multiple artifacts?

Yes — absolutely.
A single GitHub repository (and even a single release) can host many artifacts, each as its own tarball.
Each artifact:

• Has its own directory

• Its own tarball

• Its own entry in Artifacts.toml

This is the recommended pattern.

You generally only need separate repos if:

• Artifacts have very different lifecycles

• Ownership differs

• Access control differs

---

Example setup

We’ll assume:

• Package repo: MyPackage.jl

• Artifact-hosting repo: MyPackageArtifacts

• Two artifacts:

  • testdata_small

  • testdata_large

---

1️⃣ Artifact-hosting repository layout

Repo: github.com/USER/MyPackageArtifacts

MyPackageArtifacts/
├─ testdata_small/
│  ├─ input1.bin
│  └─ input2.bin
├─ testdata_large/
│  ├─ dataset.h5
│  └─ metadata.json
└─ README.md

This repo is not a Julia package — it’s just storage.

---

2️⃣ Create tarballs (one per artifact)

From inside MyPackageArtifacts/:

tar -czf testdata_small.tar.gz testdata_small
tar -czf testdata_large.tar.gz testdata_large

---

3️⃣ Create a GitHub Release

1. Go to Releases → New release

2. Tag it, e.g. v1.0.0

3. Upload:

   • testdata_small.tar.gz

   • testdata_large.tar.gz

You now have immutable URLs like:

https://github.com/USER/MyPackageArtifacts/releases/download/v1.0.0/testdata_small.tar.gz
https://github.com/USER/MyPackageArtifacts/releases/download/v1.0.0/testdata_large.tar.gz

---

4️⃣ Compute hashes in Julia

In your package repo (MyPackage.jl):

using Pkg.Artifacts
using SHA

For testdata_small

artifact_hash_small = create_artifact() do dir
    run(`tar -xzf testdata_small.tar.gz -C $dir`)
end
sha256_small = bytes2hex(open(sha256, "testdata_small.tar.gz"))

For testdata_large

artifact_hash_large = create_artifact() do dir

run(tar -xzf testdata_large.tar.gz -C $dir)

end

sha256_large = bytes2hex(open(sha256, "testdata_large.tar.gz"))

---

5️⃣ Artifacts.toml with multiple artifacts

This lives in MyPackage.jl/Artifacts.toml

[testdata_small]

git-tree-sha1 = "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa"

[[testdata_small.download]]

url = "https://github.com/USER/MyPackageArtifacts/releases/download/v1.0.0/testdata_small.tar.gz"

sha256 = "bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb"
[testdata_large]

git-tree-sha1 = "cccccccccccccccccccccccccccccccccccccccc"
[[testdata_large.download]]

url = "https://github.com/USER/MyPackageArtifacts/releases/download/v1.0.0/testdata_large.tar.gz"

sha256 = "dddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddd"

✔ One Artifacts.toml
✔ Multiple artifacts
✔ Same repo
✔ Same release

---

6️⃣ Using the artifacts in tests

using Pkg.Artifacts

using Test

@testset "Small test data" begin

small_path = artifact"testdata_small"

file = joinpath(small_path, "testdata_small", "input1.bin")

@test isfile(file)

end
@testset "Large test data" begin

large_path = artifact"testdata_large"

file = joinpath(large_path, "testdata_large", "dataset.h5")

@test isfile(file)

end

⚠️ Note: the top-level directory (testdata_small/) is preserved inside the artifact.

---

7️⃣ Updating artifacts later

Option A — new release, same artifact names (recommended)

• Add new data

• Create v1.1.0

• Update URLs + hashes in Artifacts.toml

Option B — new artifact name

[testdata_large_v2]

...

Option A is usually cleaner.

---

8️⃣ CI behavior (important!)

• Artifacts are cached automatically

• Download happens once per machine

• Offline tests work after first download

• No special CI configuration needed

---

🏆 Recommended pattern summary

Question	Answer
One repo for many artifacts?	✅ Yes
One release for many artifacts?	✅ Yes
One tarball per artifact?	✅ Yes
Artifacts.toml entries	One per artifact
Best host	GitHub Releases

---

If you want, I can:

• Generate a script to automate tarball + release creation

• Show a monorepo pattern (artifacts + package in one repo)

• Explain private artifact access in CI

Just tell me which direction you want to go.

[jpthiele]

That is a very good idea and sounds like a reasonable approach.
I had just thought about these kinds of tests when reading through the PR for .dom files since most of what we are checking is that the format we prescribe in code can be read again without checking external ressources.

[j-fu]

Right, the good test files I have are around 200k, and I would not like to add these to the package.

... though Makie has 52M of assets ... and DelaunayTriangulation (used by Makie) has 70M in tests...

[jpthiele]

But I don't see a specific 'testing only' artifact type, so it will still be downloaded when adding the package, right?

[j-fu]

The oracle says artifact download is lazy, see the end of the conversation.

https://chatgpt.com/share/6970ee24-4d48-8011-aa4e-b786eb83ac15

Confirmation:

• https://docs.julialang.org/en/v1/stdlib/Artifacts/#Artifacts.@artifact_str
• https://pkgdocs.julialang.org/v1/artifacts/#Artifact-types-and-properties

[jpthiele]

Ah very good, but we should not forget it then