Rennovate to newer models

Author: kipcole9

.dialyzer_ignore_warnings

@@ -1,10 +1,11 @@
 # Changelog
 
-## Image Detection version  0.1.0
+## ImageVision v0.2.0
 
-This is the changelog for Image Detection version 0.1.0 released on ____, 2023.  For older changelogs please consult the release tag on [GitHub](https://github.com/kipcole9/image/tags)
+This is the initial release of `image_vision` (formerly `image_detection`). It covers three vision tasks — classification, segmentation, and object detection — with permissively-licensed default models, automatic weight downloads, and a simple API designed for developers who are not ML experts.
 
-### Enhancements
+See the [README](https://github.com/elixir-image/image_vision/blob/v0.2.0/README.md) for the full feature overview and quick-start examples.
 
-* Initial version.
+## image_detection v0.1.0
 
+Initial release of the experimental `image_detection` library. Provided YOLO v8-based object detection, ResNet-50 classification, and Stable Diffusion image generation.

CHANGELOG.md

@@ -0,0 +1,83 @@
+# CLAUDE.md — image_vision
+
+## Project identity
+
+`image_vision` (formerly `image_detection`) is a thin, opinionated wrapper around the Elixir ML ecosystem (Bumblebee, Ortex, Nx) that sits next to the [`image`](https://hex.pm/packages/image) library. It exposes three vision tasks — **classification**, **segmentation**, and **object detection** — through a small, friendly API.
+
+## Audience
+
+The target user is a working Elixir developer who:
+
+* Wants to know "is this a cat?" or "how many people are in this photo?" or "give me the alpha mask for the foreground object".
+
+* Is **not** an ML researcher. They should not need to know what a featurizer is, what a serving is, what shape a tensor must be, or which model variant to pick.
+
+* Already uses [`image`](https://hex.pm/packages/image) (`Vix.Vips.Image`) for their image handling and expects this library to feel like a natural extension of it.
+
+## Design priorities (in order)
+
+1. **Developer ease of use.** The default call path is one function with one argument: `Image.Classification.labels(image)`, `Image.Segmentation.segment(image)`, `Image.Detection.detect(image)`. The user should never *have to* pick a model, configure a backend, manage a server, or download weights manually. Sensible defaults handle every step.
+
+2. **Strong, opinionated defaults.** Each task has exactly one default model that is good enough for ~90% of use cases. Defaults are chosen for: permissive licensing (Apache 2.0 / MIT only), reasonable size (<500 MB), broad applicability, and proven quality. Power users can override every default through options, but they should not have to.
+
+3. **Great documentation.** Every public function has the standard `### Arguments / ### Options / ### Returns / ### Examples` template (see `~/.claude/CLAUDE.md`). Module docs explain the *task* in plain language before the API. Doctest examples use real images from `test/support/images/`. Guides exist for the three common workflows (classify, segment, detect) and walk through what's happening at each step.
+
+4. **Tight interop with `image`.** All public functions accept `t:Vix.Vips.Image.t/0` directly and return masks/overlays as `Vix.Vips.Image.t` where appropriate. Tensor conversion happens internally via `Image.to_nx/2` and `Image.from_nx/1`. The user should never need to touch Nx directly to use this library.
+
+5. **Small install footprint.** Avoid eVision (huge — pulls in OpenCV). Use Ortex (~10–20 MB ORT shared library) for ONNX inference. ML deps are `optional: true` so `mix.exs` users opt into the runtime they want.
+
+## What this library is *not*
+
+* Not a research toolkit. If a user wants to swap encoder backbones or fine-tune a head, they should drop down to Bumblebee / Ortex directly.
+
+* Not a generation library. Image generation (Stable Diffusion, etc.) lives elsewhere.
+
+* Not a model zoo. We pick one default per task and document one or two upgrade paths. We do not enumerate every checkpoint on HuggingFace.
+
+## Default models (April 2026)
+
+| Task | Default | License | Runtime |
+|---|---|---|---|
+| Classification | `facebook/convnext-tiny-224` | Apache 2.0 | Bumblebee |
+| Embedding | `facebook/dinov2-base` | Apache 2.0 | Bumblebee |
+| Promptable segmentation | `facebook/sam2.1-hiera-tiny` (ONNX) | Apache 2.0 | Ortex |
+| Semantic / instance segmentation | `facebook/mask2former-swin-tiny-coco-instance` (ONNX) | MIT | Ortex |
+| Object detection | `PekingU/rtdetr_v2_r18vd` (ONNX) | Apache 2.0 | Ortex |
+
+All defaults are permissive-licensed. We never default to AGPL/GPL (e.g. YOLOv8/11) or non-commercial (e.g. SegFormer's NVIDIA SSL, ConvNeXt-V2 Meta weights, DINOv3) models, even when they're more accurate.
+
+## Model weight management
+
+* ONNX weights for Ortex-backed tasks are auto-downloaded on first use from the configured HuggingFace repo, and cached on disk.
+
+* Cache directory is configurable: `config :image_vision, :cache_dir, "/path/to/cache"`. Default falls back to `:filename.basedir(:user_cache, "image_vision")` (XDG-compliant per-user cache).
+
+* Bumblebee-backed tasks defer to Bumblebee's own HF cache (controlled by `BUMBLEBEE_CACHE_DIR` and friends).
+
+* Users who want fully offline operation can pre-download into the cache directory.
+
+## Dependency posture
+
+* `:image` is a hard dependency.
+
+* `:bumblebee`, `:nx`, `:nx_image` are `optional: true` — required only for classification/embedding.
+
+* `:ortex` is `optional: true` — required only for segmentation/detection.
+
+* `:req` is required — used for ONNX weight downloads. Small, ~no transitive cost.
+
+* No `:axon_onnx` (dormant, doesn't handle modern transformer vision graphs).
+
+* No `:evision` (huge OpenCV footprint).
+
+## Code style notes specific to this project
+
+* Module structure follows the existing pattern: each task is its own top-level `Image.*` module (`Image.Classification`, `Image.Segmentation`, `Image.Detection`) so the user sees them as natural neighbours of `Image.Text`, `Image.Shape`, etc. from the `image` library.
+
+* Compile-time gating (`if ImageVision.bumblebee_configured?(), do: defmodule ...`) lets the package compile cleanly when optional deps are absent.
+
+* Use `Nx.select/3` (not `Nx.map/2`) for elementwise tensor operations — `Nx.map` does per-element backend transfers and is unusably slow.
+
+* Always normalise via `NxImage.normalize/3` for ImageNet preprocessing. Don't hand-roll.
+
+* When a serving is required (Bumblebee path), prefer the supervised-process pattern already in `lib/classification.ex`: child spec via `classifier/1`, optional autostart from app config.

CLAUDE.md

@@ -1,43 +1,123 @@
-# Image Detection
+# ImageVision
 
-Image Detection is an [Image](https://hex.pm/packages/image)-based object detection library based upon the [YOLO V8](https://docs.ultralytics.com) ML model.
+`ImageVision` is a simple, opinionated image vision library for Elixir. It sits alongside the [`image`](https://hex.pm/packages/image) library and answers three questions about any image — **what's in it**, **where are the objects**, and **which pixels belong to which object** — with strong defaults and no ML expertise required.
 
-The documentation can be found at [https://hexdocs.pm/image_detection](https://hexdocs.pm/image_detection).
+## Tasks
 
-## Installation
+| Task | Module | What it does |
+|---|---|---|
+| Classification | [`Image.Classification`](https://hexdocs.pm/image_vision/Image.Classification.html) | Labels like `"sports car"` or `"Blenheim spaniel"` |
+| Embedding | [`Image.Classification`](https://hexdocs.pm/image_vision/Image.Classification.html) | 768-dim feature vector for similarity search |
+| Segmentation | [`Image.Segmentation`](https://hexdocs.pm/image_vision/Image.Segmentation.html) | Pixel masks — promptable ("cut out this object") or class-labeled ("every region in the image") |
+| Detection | [`Image.Detection`](https://hexdocs.pm/image_vision/Image.Detection.html) | Bounding boxes with class labels |
 
-`Image Detection` can be installed by adding `image_detection` to your list of dependencies in `mix.exs`:
+## Installation
 
 ```elixir
 def deps do
   [
-    {:image_detection_, "~> 0.1.0"}
+    {:image_vision, "~> 0.2"},
+    # Nx backend for classification (pick one)
+    {:exla, "~> 0.9"},
+    # Required for classification
+    {:bumblebee, "~> 0.6"},
+    # Required for segmentation and detection
+    {:ortex, "~> 0.1"}
   ]
 end
 ```
 
+## Quick examples
+
+```elixir
+# Classification — what's in this image?
+iex> puppy = Image.open!("puppy.jpg")
+iex> Image.Classification.labels(puppy)
+["Blenheim spaniel"]
+
+# Embedding — feature vector for similarity search
+iex> Image.Classification.embed(puppy)
+#Nx.Tensor<f32[768]>
+
+# Promptable segmentation — mask the object at the centre
+iex> %{mask: mask} = Image.Segmentation.segment(puppy)
+iex> cutout = Image.Segmentation.apply_mask!(puppy, mask)
+
+# Promptable segmentation — mask the object at a specific point
+iex> %{mask: mask} = Image.Segmentation.segment(puppy, prompt: {:point, 320, 240})
+
+# Class-labeled segmentation — every region
+iex> street = Image.open!("street.jpg")
+iex> segments = Image.Segmentation.segment_panoptic(street)
+iex> Enum.map(segments, & &1.label)
+["person", "car", "road", "sky"]
+
+# Overlay coloured segments on the original image
+iex> overlay = Image.Segmentation.compose_overlay(street, segments)
+
+# Object detection — bounding boxes with labels
+iex> detections = Image.Detection.detect(street)
+iex> hd(detections)
+%{label: "person", score: 0.94, box: {120, 45, 60, 180}}
+
+# Draw bounding boxes on the image
+iex> annotated = Image.Detection.draw_bbox_with_labels(detections, street)
+```
+
+## Default models
+
+All defaults are permissively licensed (Apache 2.0 / MIT). Models are downloaded automatically on first call and cached to disk — no manual setup needed.
+
+| Task | Model | License | Size |
+|---|---|---|---|
+| Classification | `facebook/convnext-tiny-224` | Apache 2.0 | ~110 MB |
+| Embedding | `facebook/dinov2-base` | Apache 2.0 | ~340 MB |
+| Promptable segmentation | `SharpAI/sam2-hiera-tiny-onnx` (SAM 2) | Apache 2.0 | ~150 MB |
+| Class-labeled segmentation | `Xenova/detr-resnet-50-panoptic` | Apache 2.0 | ~175 MB |
+| Object detection | `onnx-community/rtdetr_r50vd` (RT-DETR) | Apache 2.0 | ~175 MB |
+
 ## Configuration
 
-THIS IS AN EXPERIMENTAL LIBRARY. Please do not use in production. Testing is not yet complete.
+### Model cache
+
+ONNX models are cached in a per-user directory by default. Override in `config/runtime.exs`:
+
+```elixir
+config :image_vision, :cache_dir, "/var/lib/my_app/models"
+```
 
-The code is an adaptation of the livecoding demo by Hans Elias (@hansihe) at his [talk](https://www.youtube.com/watch?v=OsxGB6MbA8o&t=1s) at the Elixir Warsaw meetup in March 2023.
+The default is `~/Library/Caches/image_vision` on macOS and `~/.cache/image_vision` on Linux.
 
-## Accessing the Yolo V8 model
+### Classification serving
 
-The YOLO v8 model is GPL 3.0 licensed and must be built separately. Python 3.10 is required (apparently it won't work with 3.11).
+`Image.Classification` runs a supervised Bumblebee serving. It autostarts by default:
 
-```bash
-% pip3.10 install ultralytics
-% pip3.10 install onnx
-% yolo export model=yolov8n.pt format=onnx imgsz=640
+```elixir
+# config/runtime.exs
+config :image_vision, :classifier,
+  model: {:hf, "facebook/convnext-large-224-22k-1k"},  # larger model
+  featurizer: {:hf, "facebook/convnext-large-224-22k-1k"},
+  autostart: true
+```
+
+Set `autostart: false` to manage the serving in your own supervision tree:
+
+```elixir
+# application.ex
+children = [
+  Image.Classification.classifier(),
+  Image.Classification.embedder()
+]
 ```
 
-The "n" model is the smallest - we can maybe tolerate a larger one. And we need to find way to host the model or download the `.onnx` from somewhere.
+## Guides
+
+- [Classification](https://github.com/elixir-image/image_vision/blob/v0.2.0/guides/classification.md) — classifying images and computing embeddings
+- [Segmentation](https://github.com/elixir-image/image_vision/blob/v0.2.0/guides/segmentation.md) — promptable and class-labeled segmentation
+- [Detection](https://github.com/elixir-image/image_vision/blob/v0.2.0/guides/detection.md) — bounding-box object detection
 
-## References
+## Design
 
-* The original talk by @hansihe is at https://www.youtube.com/watch?v=OsxGB6MbA8o&t=1s
-* The original models are stored at https://github.com/ultralytics/assets/releases
-* Learning: https://learnopencv.com/ultralytics-yolov8/
-* Exploration: https://medium.com/mlearning-ai/yolo-v8-the-real-state-of-the-art-eda6c86a1b90
+`ImageVision` is for developers who want answers, not ML configuration. The defaults are chosen for permissive licensing, broad applicability, and reasonable size — not raw benchmark ranking. Power users can override every default via options; most users never need to.
 
+See [CLAUDE.md](https://github.com/elixir-image/image_vision/blob/v0.2.0/CLAUDE.md) for the full design rationale.

README.md

@@ -0,0 +1,79 @@
+# Image Classification
+
+`Image.Classification` answers "what's in this image?" and "how similar are these two images?".
+
+## Getting labels
+
+The simplest entry point is `labels/2`. It returns a list of human-readable labels for whatever is most prominent in the image:
+
+```elixir
+iex> puppy = Image.open!("puppy.jpg")
+iex> Image.Classification.labels(puppy)
+["Blenheim spaniel"]
+
+iex> car = Image.open!("lamborghini.jpg")
+iex> Image.Classification.labels(car)
+["sports car", "sport car"]
+```
+
+Labels come from the model's training dataset (ImageNet-1k for the default ConvNeXt model — 1000 everyday categories). The default minimum confidence threshold is 0.5; adjust with `:min_score`:
+
+```elixir
+iex> Image.Classification.labels(puppy, min_score: 0.8)
+["Blenheim spaniel"]
+
+iex> Image.Classification.labels(puppy, min_score: 0.1)
+["Blenheim spaniel", "cocker spaniel", "papillon"]
+```
+
+## Getting raw predictions with scores
+
+`classify/2` returns the full prediction map including scores:
+
+```elixir
+iex> %{predictions: preds} = Image.Classification.classify(puppy)
+iex> hd(preds)
+%{label: "Blenheim spaniel", score: 0.9327}
+```
+
+## Computing embeddings
+
+`embed/2` returns a 768-dimensional feature vector. Vectors for visually similar images will be close together in this space — useful for "find images like this one" or feeding into a custom classifier.
+
+```elixir
+iex> v1 = Image.Classification.embed(puppy)
+iex> v2 = Image.Classification.embed(other_puppy)
+
+# Cosine similarity: closer to 1.0 = more similar
+iex> cos_sim = Nx.dot(v1, v2) / (Nx.norm(v1) * Nx.norm(v2)) |> Nx.to_number()
+0.97
+```
+
+## Configuration
+
+The classification serving is autostarted when the `:image_vision` application starts. To use a larger, more accurate model, set it in `config/runtime.exs`:
+
+```elixir
+config :image_vision, :classifier,
+  model: {:hf, "facebook/convnext-large-224-22k-1k"},
+  featurizer: {:hf, "facebook/convnext-large-224-22k-1k"}
+```
+
+To manage the serving yourself (e.g. in an umbrella app):
+
+```elixir
+# config/runtime.exs
+config :image_vision, :classifier, autostart: false
+
+# application.ex
+children = [Image.Classification.classifier()]
+```
+
+## Dependencies
+
+Classification requires `:bumblebee`, `:nx`, and an Nx backend such as `:exla`. Add to `mix.exs`:
+
+```elixir
+{:bumblebee, "~> 0.6"},
+{:exla, "~> 0.9"},
+```