Skip to content

cmd/rofl/build: Validate image platform and size#488

Merged
abukosek merged 1 commit into
masterfrom
andrej/feature/rofl-build-check-img-sizes
Jun 6, 2025
Merged

cmd/rofl/build: Validate image platform and size#488
abukosek merged 1 commit into
masterfrom
andrej/feature/rofl-build-check-img-sizes

Conversation

@abukosek
Copy link
Copy Markdown
Contributor

@abukosek abukosek commented Jun 5, 2025

Closes #416.

@netlify
Copy link
Copy Markdown

netlify Bot commented Jun 5, 2025

Deploy Preview for oasisprotocol-cli canceled.

Name Link
🔨 Latest commit 8870d9a
🔍 Latest deploy log https://app.netlify.com/projects/oasisprotocol-cli/deploys/6842e6161024fa00085bee54

@abukosek abukosek force-pushed the andrej/feature/rofl-build-check-img-sizes branch 2 times, most recently from 153ab78 to a7ffd24 Compare June 5, 2025 13:09
@abukosek abukosek marked this pull request as ready for review June 5, 2025 13:10
@abukosek abukosek requested a review from matevz June 5, 2025 13:10
@abukosek abukosek force-pushed the andrej/feature/rofl-build-check-img-sizes branch from a7ffd24 to 8954ce5 Compare June 5, 2025 13:21
@abukosek
Copy link
Copy Markdown
Contributor Author

abukosek commented Jun 5, 2025

The OCI manifest only specifies the compressed layer size, so this is kind of useless at the moment...

For ghcr.io/oasisprotocol/demo-rofl-tgbot, the total size of the compressed layers is 24MB, but the actual uncompressed size is 86MB, which is a significant difference.

The only way to get the uncompressed size is to download and uncompress all the layers, which seems dumb. What to do? 🤔

@abukosek abukosek added blocked and removed blocked labels Jun 5, 2025
@abukosek
Copy link
Copy Markdown
Contributor Author

abukosek commented Jun 5, 2025

OK, so after chatting with Matevž, the summary is that the most popular images, even compressed, exceed the default 512MB storage limit, so having this check is still useful.

@abukosek abukosek force-pushed the andrej/feature/rofl-build-check-img-sizes branch from 8954ce5 to f8bf3f9 Compare June 5, 2025 14:35
Comment thread cmd/rofl/build/container.go Outdated
@kostko
Copy link
Copy Markdown
Member

kostko commented Jun 5, 2025

You could estimate that data is compressed and multiply by some factor. Obviously this will still be incorrect as it depends on entropy and other factors, but it could still be a better estimate.

@abukosek abukosek force-pushed the andrej/feature/rofl-build-check-img-sizes branch from f8bf3f9 to ea0587a Compare June 6, 2025 08:15
@abukosek
Copy link
Copy Markdown
Contributor Author

abukosek commented Jun 6, 2025

I've multiplied the final size by 2, which seems like a good estimate for the commonly-used images 🤷‍♂️

@matevz
Copy link
Copy Markdown
Member

matevz commented Jun 6, 2025

I've multiplied the final size by 2, which seems like a good estimate for the commonly-used images 🤷‍♂️

I can confirm that docker.io/ollama/ollama:latest is ~1.6GB compressed and ~ 3.4GB uncompressed so it's a reasonable factor.

Comment on lines +125 to +132
// Validate platform if given.
platform := mainDescriptor.Platform
if platform != nil {
if platform.Architecture != "amd64" || platform.OS != "linux" {
return fmt.Errorf("compose file validation failed: image '%s' has incorrect platform (expected linux/amd64, got %s/%s)", imageFull, platform.OS, platform.Architecture)
}
}
platform = mf.Config.Platform
if platform != nil {
if platform.Architecture != "amd64" || platform.OS != "linux" {
return fmt.Errorf("compose file validation failed: image '%s' has incorrect platform (expected linux/amd64, got %s/%s)", imageFull, platform.OS, platform.Architecture)
}
}
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can't you deduplicate this code with

for platform := range map[string]string{mainDescriptor, mf.Config} {
...
}

?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No, because this code won't work, but I've done it in a similar way :)

@abukosek abukosek force-pushed the andrej/feature/rofl-build-check-img-sizes branch from ea0587a to 8870d9a Compare June 6, 2025 12:59
@abukosek abukosek merged commit 750214c into master Jun 6, 2025
4 checks passed
@abukosek abukosek deleted the andrej/feature/rofl-build-check-img-sizes branch June 6, 2025 13:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

rofl build: Warn, if .orc size and images to be fetched is greater than resources.storage.size

3 participants