Add zstd compression support (#18)

* Initial plan

* Add zstd compression support

Co-authored-by: SeriousBug <1008124+SeriousBug@users.noreply.github.com>

* Add test coverage for DynamicFile compression methods and fix clippy warning

Co-authored-by: SeriousBug <1008124+SeriousBug@users.noreply.github.com>

* Fix clippy warnings and add explicit test coverage for DynamicFile compression methods

Co-authored-by: SeriousBug <1008124+SeriousBug@users.noreply.github.com>

* Fix dynamic tests to handle both always-embed and debug modes correctly

Co-authored-by: SeriousBug <1008124+SeriousBug@users.noreply.github.com>

* Format code with cargo fmt

Co-authored-by: SeriousBug <1008124+SeriousBug@users.noreply.github.com>

* Make zstd compression an optional feature

Co-authored-by: SeriousBug <1008124+SeriousBug@users.noreply.github.com>

* Add comprehensive tests for optional zstd feature

Co-authored-by: SeriousBug <1008124+SeriousBug@users.noreply.github.com>

* Make compression-zstd opt-in and add documentation

Changes:
- Remove compression-zstd from default features (requires C bindings)
- Add comprehensive documentation for zstd compression feature
- Document compression functions and config methods
- Reorganize README features section for clarity

The zstd feature is now opt-in due to its C library dependency,
which may not be compatible with all build environments.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: SeriousBug <1008124+SeriousBug@users.noreply.github.com>
Co-authored-by: Kaan Barmore-Genc <kaan@bgenc.dev>
Co-authored-by: Claude <noreply@anthropic.com>

Author: 3 people

Cargo.toml

@@ -19,6 +19,7 @@ rust-embed-for-web-utils = { version = "11.2.1", path = "utils" }
 chrono = { version = "0.4", default-features = false }
 flate2 = "1.0"
 brotli = "6.0"
+zstd = "0.13"
 actix-web = "4.4"
 
 [features]
@@ -32,20 +33,31 @@ include-exclude = [
   "rust-embed-for-web-impl/include-exclude",
   "rust-embed-for-web-utils/include-exclude",
 ]
+compression-zstd = ["rust-embed-for-web-impl/compression-zstd", "rust-embed-for-web-utils/compression-zstd"]
 
 [workspace]
 members = ["impl", "utils"]
 
 [[test]]
 name = "compression"
 path = "tests/compression.rs"
+required-features = ["always-embed", "compression-zstd"]
+
+[[test]]
+name = "compression_without_zstd"
+path = "tests/compression_without_zstd.rs"
 required-features = ["always-embed"]
 
 [[test]]
 name = "gzip"
 path = "tests/gzip.rs"
 required-features = ["always-embed"]
 
+[[test]]
+name = "zstd"
+path = "tests/zstd.rs"
+required-features = ["always-embed", "compression-zstd"]
+
 [[test]]
 name = "include-exclude"
 path = "tests/include-exclude.rs"

README.md

@@ -24,7 +24,7 @@ executable in exchange for better performance at runtime. In particular:
   or decompress anything at runtime.
   - If the compression makes little difference, for example a jpeg file won't
     compress much further if at all, then the compressed version is not included.
-  - You can also disable this behavior by adding an attribute `#[gzip = false]` and `#[br = false]`
+  - You can also disable this behavior by adding an attribute `#[gzip = false]`, `#[br = false]`, or `#[zstd = false]`
     When disabled, the compressed files won't be included for that embed.
 - Some metadata that is useful for web headers like `ETag` and `Last-Modified`
   are computed ahead of time and embedded into the executable. This makes it
@@ -68,17 +68,19 @@ The path for the `folder` is resolved relative to where `Cargo.toml` is.
 
 ### Disabling compression
 
-You can add `#[gzip = false]` and/or `#[br = false]` attributes to your embed to
-disable gzip and brotli compression for the files in that embed.
+You can add `#[gzip = false]`, `#[br = false]`, and/or `#[zstd = false]` attributes to your embed to
+disable gzip, brotli, and/or zstd compression for the files in that embed.
 `rust-embed-for-web` will only include compressed files where the compression
 actually makes files smaller so files that won't compress well like images or
 archives already don't include their compressed versions. However you can
 
 ## Features
 
-Both of the following features are enabled by default.
+### Default Features
 
-### `interpolate-folder-path`
+The following features are enabled by default.
+
+#### `interpolate-folder-path`
 
 Allow environment variables and `~`s to be used in the `folder` path. Example:
 
@@ -91,7 +93,7 @@ struct Asset;
 `~` will expand into your home folder, and `${PROJECT_NAME}` will expand into
 the value of the `PROJECT_NAME` environment variable.
 
-### `include-exclude`
+#### `include-exclude`
 
 You can filter which files are embedded by adding one or more `#[include = "*.txt"]` and `#[exclude = "*.jpg"]` attributes.
 Matching is done on relative file paths --the paths you use for the `.get` call-- via [`globset`](https://docs.rs/globset/latest/globset/).
@@ -111,7 +113,26 @@ For example, if you wanted to exclude all `.svg` files except for one named
 struct Assets;
 ```
 
-### `prefix`
+### Optional Features
+
+#### `compression-zstd`
+
+Enables zstd compression support for embedded files. When enabled, files will be compressed with zstd (in addition to gzip and brotli), allowing you to serve zstd-compressed content to clients that support it.
+
+**Note:** This feature is **not enabled by default** because the `zstd` crate uses C bindings, which may not be compatible with all build environments.
+
+To enable zstd compression, add this to your `Cargo.toml`:
+
+```toml
+[dependencies]
+rust-embed-for-web = { version = "11.2.1", features = ["compression-zstd"] }
+```
+
+You can also disable zstd compression for specific embeds using the `#[zstd = false]` attribute as described in the "Disabling compression" section above.
+
+### Other Configuration
+
+#### `prefix`
 
 You can specify a prefix, which will be added to the path of the files. For example:
 

impl/Cargo.toml

@@ -27,6 +27,7 @@ walkdir = "2.4.0"
 # Compression
 flate2 = "1.0"
 brotli = "6.0"
+zstd = { version = "0.13", optional = true }
 
 globset = { version = "0.4", optional = true }
 
@@ -39,3 +40,4 @@ default = []
 interpolate-folder-path = ["shellexpand"]
 include-exclude = ["rust-embed-for-web-utils/include-exclude", "globset"]
 always-embed = []
+compression-zstd = ["zstd", "rust-embed-for-web-utils/compression-zstd"]

impl/src/attributes.rs

@@ -42,6 +42,7 @@ pub(crate) fn read_attribute_config(ast: &syn::DeriveInput) -> Config {
                 "exclude" => parse_str(attribute).map(|v| config.add_exclude(v)),
                 "gzip" => parse_bool(attribute).map(|v| config.set_gzip(v)),
                 "br" => parse_bool(attribute).map(|v| config.set_br(v)),
+                "zstd" => parse_bool(attribute).map(|v| config.set_zstd(v)),
                 _ => None,
             };
         }

impl/src/compress.rs

@@ -2,6 +2,8 @@ use std::io::{BufReader, Write};
 
 use brotli::enc::BrotliEncoderParams;
 use flate2::{write::GzEncoder, Compression};
+#[cfg(feature = "compression-zstd")]
+use zstd::stream::write::Encoder as ZstdEncoder;
 
 /// Only include the compressed version if it is at least this much smaller than
 /// the uncompressed version.
@@ -39,3 +41,31 @@ pub(crate) fn compress_br(data: &[u8]) -> Option<Vec<u8>> {
         None
     }
 }
+
+/// Compresses data using zstd compression.
+///
+/// Returns the compressed data if it's smaller than the threshold, `None` otherwise.
+/// Uses compression level 3 as a balance between compression ratio and speed.
+#[cfg(feature = "compression-zstd")]
+pub(crate) fn compress_zstd(data: &[u8]) -> Option<Vec<u8>> {
+    let mut data_zstd: Vec<u8> = Vec::new();
+    // Level 3 provides good compression with reasonable speed for build-time compression
+    let mut encoder = ZstdEncoder::new(&mut data_zstd, 3).expect("Failed to create zstd encoder");
+    encoder
+        .write_all(data)
+        .expect("Failed to compress zstd data");
+    encoder
+        .finish()
+        .expect("Failed to finish compression of zstd data");
+
+    if data_zstd.len() < ((data.len() as f64) * COMPRESSION_INCLUDE_THRESHOLD) as usize {
+        Some(data_zstd)
+    } else {
+        None
+    }
+}
+
+#[cfg(not(feature = "compression-zstd"))]
+pub(crate) fn compress_zstd(_data: &[u8]) -> Option<Vec<u8>> {
+    None
+}

impl/src/embed.rs

@@ -1,7 +1,7 @@
 use proc_macro2::TokenStream as TokenStream2;
 use rust_embed_for_web_utils::{get_files, Config, DynamicFile, EmbedableFile, FileEntry};
 
-use crate::compress::{compress_br, compress_gzip};
+use crate::compress::{compress_br, compress_gzip, compress_zstd};
 
 /// Anything that can be embedded into the program.
 ///
@@ -70,6 +70,11 @@ impl<'t> MakeEmbed for EmbedDynamicFile<'t> {
         } else {
             None::<Vec<u8>>.make_embed()
         };
+        let data_zstd = if self.config.should_zstd() {
+            compress_zstd(&data).make_embed()
+        } else {
+            None::<Vec<u8>>.make_embed()
+        };
         let data = data.make_embed();
         let hash = file.hash().make_embed();
         let etag = file.etag().make_embed();
@@ -83,6 +88,7 @@ impl<'t> MakeEmbed for EmbedDynamicFile<'t> {
                 #data,
                 #data_gzip,
                 #data_br,
+                #data_zstd,
                 #hash,
                 #etag,
                 #last_modified,

impl/src/lib.rs

@@ -87,7 +87,10 @@ fn impl_rust_embed_for_web(ast: &syn::DeriveInput) -> TokenStream2 {
     }
 }
 
-#[proc_macro_derive(RustEmbed, attributes(folder, prefix, include, exclude, gzip, br))]
+#[proc_macro_derive(
+    RustEmbed,
+    attributes(folder, prefix, include, exclude, gzip, br, zstd)
+)]
 /// A folder that is embedded into your program.
 ///
 /// For example:

tests/basic.rs

@@ -41,6 +41,7 @@ fn file_name_exists() {
 #[test]
 fn readme_example() {
     let index = Embed::get("index.html").unwrap().data();
+    #[allow(clippy::useless_asref)]
     let contents = std::str::from_utf8(index.as_ref()).unwrap();
     assert!(!contents.is_empty());
 }

tests/compression.rs

@@ -11,6 +11,7 @@ struct Embed;
 fn html_files_are_compressed() {
     assert!(Embed::get("index.html").unwrap().data_gzip().is_some());
     assert!(Embed::get("index.html").unwrap().data_br().is_some());
+    assert!(Embed::get("index.html").unwrap().data_zstd().is_some());
 }
 
 #[test]
@@ -20,14 +21,18 @@ fn image_files_are_not_compressed() {
         .data_gzip()
         .is_none());
     assert!(Embed::get("images/flower.jpg").unwrap().data_br().is_none());
+    assert!(Embed::get("images/flower.jpg")
+        .unwrap()
+        .data_zstd()
+        .is_none());
 }
 
 #[test]
 fn compression_gzip_roundtrip() {
     let compressed = Embed::get("index.html").unwrap().data_gzip().unwrap();
     let mut decompressed: Vec<u8> = Vec::new();
     let mut decoder = GzDecoder::new(&mut decompressed);
-    decoder.write_all(&compressed[..]).unwrap();
+    decoder.write_all(compressed).unwrap();
     decoder.finish().unwrap();
     let decompressed_body = String::from_utf8_lossy(&decompressed[..]);
     assert!(decompressed_body.starts_with("<!DOCTYPE html>"));
@@ -37,8 +42,16 @@ fn compression_gzip_roundtrip() {
 fn compression_br_roundtrip() {
     let compressed = Embed::get("index.html").unwrap().data_br().unwrap();
     let mut decompressed: Vec<u8> = Vec::new();
-    let mut data_read = BufReader::new(&compressed[..]);
+    let mut data_read = BufReader::new(compressed);
     brotli::BrotliDecompress(&mut data_read, &mut decompressed).unwrap();
     let decompressed_body = String::from_utf8_lossy(&decompressed[..]);
     assert!(decompressed_body.starts_with("<!DOCTYPE html>"));
 }
+
+#[test]
+fn compression_zstd_roundtrip() {
+    let compressed = Embed::get("index.html").unwrap().data_zstd().unwrap();
+    let decompressed = zstd::bulk::decompress(compressed, 1024 * 1024).unwrap();
+    let decompressed_body = String::from_utf8_lossy(&decompressed[..]);
+    assert!(decompressed_body.starts_with("<!DOCTYPE html>"));
+}

tests/compression_without_zstd.rs

@@ -0,0 +1,64 @@
+use std::io::{BufReader, Write};
+
+use flate2::write::GzDecoder;
+use rust_embed_for_web::{EmbedableFile, RustEmbed};
+
+#[derive(RustEmbed)]
+#[folder = "examples/public"]
+struct Embed;
+
+#[test]
+fn html_files_gzip_and_br_compression() {
+    assert!(Embed::get("index.html").unwrap().data_gzip().is_some());
+    assert!(Embed::get("index.html").unwrap().data_br().is_some());
+}
+
+#[test]
+fn zstd_behavior_without_feature() {
+    // When compression-zstd feature is not enabled, data_zstd should return None
+    #[cfg(not(feature = "compression-zstd"))]
+    {
+        assert!(Embed::get("index.html").unwrap().data_zstd().is_none());
+    }
+
+    // When compression-zstd feature is enabled, it may return Some or None based on effectiveness
+    #[cfg(feature = "compression-zstd")]
+    {
+        // Just test that the method doesn't panic
+        let _ = Embed::get("index.html").unwrap().data_zstd();
+    }
+}
+
+#[test]
+fn image_files_are_not_compressed() {
+    assert!(Embed::get("images/flower.jpg")
+        .unwrap()
+        .data_gzip()
+        .is_none());
+    assert!(Embed::get("images/flower.jpg").unwrap().data_br().is_none());
+    assert!(Embed::get("images/flower.jpg")
+        .unwrap()
+        .data_zstd()
+        .is_none());
+}
+
+#[test]
+fn compression_gzip_roundtrip() {
+    let compressed = Embed::get("index.html").unwrap().data_gzip().unwrap();
+    let mut decompressed: Vec<u8> = Vec::new();
+    let mut decoder = GzDecoder::new(&mut decompressed);
+    decoder.write_all(compressed).unwrap();
+    decoder.finish().unwrap();
+    let decompressed_body = String::from_utf8_lossy(&decompressed[..]);
+    assert!(decompressed_body.starts_with("<!DOCTYPE html>"));
+}
+
+#[test]
+fn compression_br_roundtrip() {
+    let compressed = Embed::get("index.html").unwrap().data_br().unwrap();
+    let mut decompressed: Vec<u8> = Vec::new();
+    let mut data_read = BufReader::new(compressed);
+    brotli::BrotliDecompress(&mut data_read, &mut decompressed).unwrap();
+    let decompressed_body = String::from_utf8_lossy(&decompressed[..]);
+    assert!(decompressed_body.starts_with("<!DOCTYPE html>"));
+}