Remove image detection and related deps to image_detection lib

Author: kipcole9

CHANGELOG.md

@@ -1,5 +1,74 @@
 # Changelog
 
+## Image 0.67.0
+
+### Breaking Changes
+
+* **`Image.Classification` and `Image.Generation` have moved to a new sibling package, [`:image_detection`](https://hex.pm/packages/image_detection).** The two modules keep their fully-qualified names (`Image.Classification` and `Image.Generation`) so that existing call sites continue to work — they just live in a different OTP application now. To restore the previous functionality, add `:image_detection` to your `mix.exs`:
+
+      def deps do
+        [
+          {:image, "~> 0.67"},
+          {:image_detection, "~> 0.1"},
+          # plus your preferred Nx backend, e.g.
+          {:exla, "~> 0.10"}
+        ]
+      end
+
+* **`:bumblebee` is no longer a dependency of `:image`.** It is brought in transitively only when you add `:image_detection`. The `:nx`, `:nx_image`, `:scholar`, `:exla`, and `:rustler` optional deps stay in `:image` because `Image.to_nx/2`, `Image.from_nx/1`, `Image.k_means/2`, and a few internal pipelines still use them.
+
+* **`Image.bumblebee_configured?/0` has been removed.** It is replaced by `ImageDetection.bumblebee_configured?/0` which lives in the new package.
+
+* **`Image.Application` no longer autostarts the classification or generation services.** That responsibility moved to `ImageDetection.Application`, which manages its own `ImageDetection.Supervisor`. Both services default to `autostart: false` in the new package — you opt in via `config :image_detection, :classifier, autostart: true` (or `:generator`).
+
+* **`config :image, :classifier` and `config :image, :generator` no longer have any effect.** Move them to `config :image_detection, :classifier` / `:generator` in your `runtime.exs` (or equivalent).
+
+* The `doctest Image.Classification` line has been removed from `test/image_test.exs` since the module is no longer part of `:image`.
+
+### Enhancements
+
+* `:image` no longer pulls `:bumblebee` (or its transitive deps) into the dependency graph. For users who don't need ML-backed classification/generation, the install footprint shrinks substantially. The Hex tarball is unchanged in size, but the dependency tree is much lighter.
+
+* The `image_detection` package's `bumblebee_configured?/0` predicate only checks for `Nx` and `Bumblebee` at compile time; the Nx backend (e.g. `:exla`) is now a runtime-only requirement. Users who want to compile against `image_detection` no longer have to install EXLA on their build machine.
+
+## Image 0.66.0
+
+This release replaces the eVision-backed video frame extraction with an FFmpeg-backed implementation via `:xav`.
+
+### Breaking Changes
+
+* **`Image.Video` is now backed by [Xav](https://hex.pm/packages/xav)** (a thin Elixir wrapper around FFmpeg) instead of `:evision` / OpenCV. The public API surface is largely unchanged but the underlying type, options, and a few semantic details have moved:
+
+  * The video struct is now `%Image.Video{}` (with fields `:reader`, `:source`, `:fps`, `:duration_seconds`, `:frame_count`, `:width`, `:height`) rather than `%Evision.VideoCapture{}`. Pattern-match on the new struct module if your code does so.
+
+  * `Image.Video.open/2`'s `:backend` option has been removed. FFmpeg picks the demuxer automatically and there is no concept of pluggable backends in Xav.
+
+  * `Image.Video.known_backends/0`, `available_backends/0`, `known_backend?/1`, `known_backend_values/0`, and `available_backend?/1` have been removed for the same reason. `Image.Options.Video` (the module that owned the backend table) has been deleted.
+
+  * Camera input is now opened via a platform-specific device path. `:default_camera` resolves to `/dev/video0` on Linux, `"0"` (AVFoundation device 0) on macOS, and `"video=0"` on Windows. An integer camera index is mapped to the corresponding `/dev/videoN` (or platform equivalent). For non-default cameras you can also pass an explicit FFmpeg device string.
+
+  * Frame-based seeking (`Image.Video.seek/2` with `frame: n`, and `Image.Video.image_from_video/2` with `frame: n`) is now implemented as a time-based seek to `n / fps` followed by zero or more `next_frame` calls. For most files this lands on the requested frame; for very inter-frame-compressed files FFmpeg may snap to the nearest preceding keyframe.
+
+  * `Image.Video.close/1` is now a no-op that returns `{:ok, %Image.Video{reader: nil}}`. Xav garbage-collects the underlying FFmpeg context, so explicit close is no longer necessary. The function is retained for source compatibility; subsequent operations on the closed struct return `{:error, %Image.Error{reason: :video_closed}}`.
+
+  * Image and audio frames are decoded by FFmpeg + libswscale rather than by OpenCV's videoio backend. Pixel-exact comparisons against fixtures generated by the previous version will not match; the test fixture `test/support/validate/video/video_sample_frame_0.png` has been regenerated.
+
+* **`:xav` is now an optional dependency.** Add it to your `mix.exs` if you use `Image.Video`:
+
+      {:xav, "~> 0.10", optional: true}
+
+  Xav requires FFmpeg ≥ 6.0 on the system.
+
+* **`:evision` is no longer needed for `Image.Video`.** It is still required for `Image.QRcode` and for the `Image.to_evision/2` / `Image.from_evision/1` interop helpers, which are unchanged. The README's optional-dependency table reflects the new split.
+
+### Enhancements
+
+* `Image.Video` now supports HTTP/HTTPS/RTSP/RTMP URLs as video sources for free, since FFmpeg supports them natively.
+
+* `Image.xav_configured?/0` is the new compile-time predicate that gates the `Image.Video` module (analogous to `Image.evision_configured?/0` and `Image.bumblebee_configured?/0`).
+
+* The `Image.Video.frame_to_image/1` helper exposes the raw `Xav.Frame` → `Vix.Vips.Image.t()` conversion (used internally by `image_from_video/2` and `stream!/2`). Useful if you have a frame from elsewhere in the Xav ecosystem and want to bring it into `Image`.
+
 ## Image 0.65.0
 
 This release prepares the library for a 1.0 tag. It is dominated by API hygiene rather than new features.

README.md

@@ -6,7 +6,7 @@ Elixir. It is a high-level wrapper around
 [libvips](https://www.libvips.org) C library, and provides an
 idiomatic functional API for image manipulation, drawing, text
 rendering, EXIF/XMP metadata, classification (via Bumblebee),
-object detection (via eVision), QR code encoding and decoding,
+video frame extraction (via Xav/FFmpeg), QR code encoding and decoding (via eVision),
 video frame extraction, blurhash, perceptual hashing, and many
 other image-related operations.
 
@@ -66,13 +66,20 @@ Documentation can be found at <https://hexdocs.pm/image>.
   Plug `Conn`s, in-memory binaries, and S3 sources.
 
 * **Optional ML integrations** —
-  * `Image.Classification` and `Image.Generation` via
-    [Bumblebee](https://hex.pm/packages/bumblebee).
-  * `Image.QRcode` (encode + decode) and `Image.Video` (frame
-    extraction, seek, webcam) via
+  * `Image.Video` (frame extraction, seek, webcam) via
+    [Xav](https://hex.pm/packages/xav), an Elixir wrapper around
+    FFmpeg. Requires FFmpeg ≥ 6.0 on the system.
+  * `Image.QRcode` (encode + decode) via
     [eVision](https://hex.pm/packages/evision).
   * `Image.k_means` via [Scholar](https://hex.pm/packages/scholar).
   * `Image.to_nx/2` / `Image.from_nx/1` via [Nx](https://hex.pm/packages/nx).
+  * **Object detection, image classification, and image
+    generation** live in the separate
+    [`:image_detection`](https://hex.pm/packages/image_detection)
+    package. Add it alongside `:image` in your `mix.exs` to get
+    `Image.Detection`, `Image.Classification`, and
+    `Image.Generation` (which depend on `:axon_onnx` and
+    [Bumblebee](https://hex.pm/packages/bumblebee) respectively).
 
 * **Hashing** — perceptual difference hash (`Image.dhash/2`),
   blurhash encode/decode (`Image.Blurhash`), Hamming distance.
@@ -256,9 +263,10 @@ optional dependencies enable specific features:
 | Dependency | Enables |
 |---|---|
 | `:nx` | `Image.to_nx/2`, `Image.from_nx/1`, tensor interop |
-| `:bumblebee` | `Image.Classification`, `Image.Generation` |
 | `:scholar` | `Image.k_means/2` |
-| `:evision` | `Image.QRcode`, `Image.Video` |
+| `:xav` | `Image.Video` (FFmpeg-backed frame extraction) |
+| `:evision` | `Image.QRcode`, `Image.to_evision/2`, `Image.from_evision/1` |
+| `:image_detection` | `Image.Detection`, `Image.Classification`, `Image.Generation` (object detection, classification, image generation — pulls Bumblebee, Nx, Axon as transitive deps) |
 | `:plug` | streaming via `Plug.Conn` |
 | `:req` | streaming over HTTP |
 | `:kino` | `Image.Kino` (Livebook integration) |
@@ -286,6 +294,37 @@ You can also set the concurrency programmatically with
 `Image.put_concurrency/1` and read it back with
 `Image.get_concurrency/0`.
 
+## FFmpeg / Xav log noise
+
+If you use `Image.Video` (which is backed by
+[Xav](https://hex.pm/packages/xav) / FFmpeg) you may see lines like
+
+    [swscaler @ 0x1490a0000] No accelerated colorspace conversion found from yuv420p to rgb24.
+
+written to `stderr` during frame decoding. These are
+**informational notices** from FFmpeg's `libswscale`, **not
+errors**. They mean that `libswscale` does not have a
+hand-optimised SIMD path for that particular pixel-format
+conversion on your CPU, so it is using its generic C fallback.
+Decoded frames are bit-for-bit correct either way.
+
+The messages come from FFmpeg writing directly to `stderr` at its
+default log level (`AV_LOG_INFO`). Xav does not currently expose
+`av_log_set_level/1`, so the only way to silence them from
+application code is to install an FFmpeg build that has the
+SIMD path for your architecture (typically an FFmpeg compiled
+with `--enable-runtime-cpudetect` and any of `--enable-asm`,
+`--enable-x86asm`, or platform ASM flags — most distribution
+packages already do this). On Apple Silicon the arm64 optimised
+path for `yuv420p → rgb24` is not in FFmpeg's `swscale` as of
+FFmpeg 7.x, which is why macOS users on M-series machines see
+the notice most often.
+
+If the noise is disruptive during tests or automation, you can
+redirect stderr for the command in question, e.g.
+`mix test 2> /dev/null`. Do not do this for production —
+suppressing stderr will also hide real FFmpeg errors.
+
 ## Security considerations
 
 * `libvips` and the underlying loaders are written in C; a malicious

config/dev.exs

@@ -24,11 +24,12 @@ config :nx, :default_defn_options, compiler: EXLA
 config :logger,
   level: :warning
 
-config :image, :classifier,
-  model: {:hf, "Falconsai/nsfw_image_detection"},
-  featurizer: {:hf, "Falconsai/nsfw_image_detection"},
-  featurizer_options: [module: Bumblebee.Vision.VitFeaturizer],
-  name: Image.Classification.Server,
-  autostart: true
-
-config :image, :generator, autostart: false
+# Image classification and generation moved to the
+# `:image_detection` package. Configure them there:
+#
+#   config :image_detection, :classifier,
+#     model: {:hf, "Falconsai/nsfw_image_detection"},
+#     featurizer: {:hf, "Falconsai/nsfw_image_detection"},
+#     autostart: true
+#
+#   config :image_detection, :generator, autostart: false

lib/application.ex

@@ -1,62 +1,24 @@
 defmodule Image.Application do
   @moduledoc false
 
-  # This env var, if set, will prevent `libvips` from loading
-  # untrusted loaders.  We set this to true if it is not otherwise set.
-  # See https://github.com/kipcole9/image/issues/9
+  # This env var, if set, will prevent `libvips` from loading untrusted
+  # loaders. We set it to "TRUE" unless the user has explicitly set
+  # something else. See https://github.com/kipcole9/image/issues/9
   @untrusted_env_var "VIPS_BLOCK_UNTRUSTED"
 
   use Application
-  require Logger
 
   @doc false
   def start(_type, _args) do
     set_safe_loader()
 
     Supervisor.start_link(
-      children(Code.ensure_loaded?(Bumblebee)),
+      [],
       strategy: :one_for_one,
       name: Image.Supervisor
     )
   end
 
-  # When Bumblebee is available
-  if Image.bumblebee_configured?() do
-    @services [
-      {{Image.Classification, :classifier, []}, true},
-      {{Image.Generation, :generator, []}, false}
-    ]
-
-    defp children(true) do
-      Enum.reduce(@services, [], fn {{module, function, args}, start?}, acc ->
-        if autostart?(function, start?) do
-          case apply(module, function, args) do
-            {:error, reason} ->
-              Logger.warning("Cannot autostart #{inspect(function)}. Error: #{inspect(reason)}")
-              acc
-
-            server ->
-              [server | acc]
-          end
-        else
-          acc
-        end
-      end)
-    end
-  end
-
-  # When bumblebee is not available
-  defp children(_) do
-    []
-  end
-
-  @doc false
-  def autostart?(service, start?) do
-    :image
-    |> Application.get_env(service, autostart: start?)
-    |> Keyword.get(:autostart)
-  end
-
   # Sets `VIPS_BLOCK_UNTRUSTED=TRUE` in the environment unless the user
   # has already set it themselves. Prevents libvips from loading
   # untrusted format loaders, which is the secure default.

lib/image.ex

@@ -11893,17 +11893,12 @@ defmodule Image do
   end
 
   @doc false
-  def bumblebee_configured? do
-    Enum.reduce_while([Nx, EXLA, Bumblebee], true, fn mod, flag ->
-      case Code.ensure_compiled(mod) do
-        {:module, _module} -> {:cont, flag}
-        _other -> {:halt, false}
-      end
-    end)
+  def evision_configured? do
+    match?({:module, _module}, Code.ensure_compiled(Evision))
   end
 
   @doc false
-  def evision_configured? do
-    match?({:module, _module}, Code.ensure_compiled(Evision))
+  def xav_configured? do
+    match?({:module, _module}, Code.ensure_compiled(Xav.Reader))
   end
 end