Extract RGB and HEX value from the image

[ridzria]

Let's say I have an image and I would like to extract the RGB and HEX value from that. Is there a function already available in Image?

    image_url = "some_image_url"
    {:ok, colors} = get_color_details(Image.open!(image_url))
    
    # output colors_list = [r: 90, g: 90, b: 90, a: 0.4, hex: 5a5a5a]
   
    def get_color_details(image) do
       {:ok, colors} = Image.ExtractColor(image) # some function to get all the colors in RGB and HEX
    end

Wouldn't be nicer if those colors are embedded in the metadata (exif) itself?

   iex> Image.Color.rgb_color("gray")
   {:ok, [hex: "#808080", rgb: [128, 128, 128]]}

   {:ok, metadata} = Image.exif(Image.open!(image_url))
    metadata.colors #returns [hex: "#808080", rgb: [128, 128, 128]]

ImageMagick returns average color:
Right now I'm doing like this to get an average color of any given image.

def identify_hex(path) do
    result =
      System.cmd(
        "convert",
        [
          "#{path}[0]",
          "-resize",
          "1x1\!",
          "-format",
          "%[fx:int(255*r+.5)],%[fx:int(255*g+.5)],%[fx:int(255*b+.5)]",
          "info:-"
        ],
        stderr_to_stdout: false
      )

    case result do
      {rgb, 0} ->
        [{r}, {g}, {b}] =
          String.split(rgb, ",")
          |> Enum.map(fn s ->
            {v, ""} = String.trim(s) |> Integer.parse()
            {v}
          end)

        %ColorUtils.RGB{red: r, blue: b, green: g} |> ColorUtils.rgb_to_hex()

      :error ->
        %ColorUtils.RGB{red: 255, blue: 255, green: 255} |> ColorUtils.rgb_to_hex()
    end
  end

[kipcole9]

Sorry to be a bit slow to get back to you on this.

Average color

Computing the average is straight forward since that is a function already in Image:

iex> image = Image.open! "./test/support/images/Kamchatka-2019-8754.jpg"
%Vix.Vips.Image{ref: #Reference<0.2751592336.3974758430.163424>}
iex> Image.average(image)
[128, 116, 95]

Getting all colors

Although possible, this is a very expensive operation and I strongly recommend not doing it. Examining all pixels results in one NIF call per pixel which will be extremely expensive. See Image.get_pixel/3.

Deriving a color palette

I think deriving a color pallet for an image would be very helpful. I opened an issue on this topic but that has not born fruit at this stage.

Using a histogram

It's possible to create a histogram of colors that fall into different bins. You could use this to at least establish broad color usage. The function to use Vix.Vips.Operation.hist_find_ndim/2 and its been on my list for a while to work out how to apply this in order to so a simple paletisation of image colors. Any help on this part would be very welcome. See some discussion at libvips/libvips#2520

Conversion to/from hex/rgb

What I really need to do is make the time to finish up a proper color library. The current approach to colors in Image isn't very sophisticated at all. I can improve the current color functions in Image.Color for pragmatic requirements but really I just need to get on with finishing up color as a proper library.

I will add conversion from RGB to Hex at least (give me a week or so).

[kipcole9]

I've pushed a commit that adds Image.Color.rgb_to_hex/1 and Image.Color.hex_to_rgb/1. They aren't perfect but they should work for basic use cases. I'm deferring as much as I can until I can work on color as a library.

[ridzria]

How to get the new codes pulled to my current project since it is still 0.33.0?

[kipcole9]

You can configure dependencies to come from GitHub. So in your mix.exs:

def deps do
  [
    {:image, github: "elixir-image/image"},
    ...
  ]
end

[ridzria]

Awesome. That works as expected. Thanks

[kipcole9]

Also, on getting all the colors of an image, Nx can get most of the way there:

iex> i = Image.open! "./test/support/images/Kamchatka-2019-8754.jpg"
%Vix.Vips.Image{ref: #Reference<0.1315682744.4087480349.157106>}
# Convert to a Nx tensor. This comes at the expense of having to stream
# the entire image and allocate it in memory - libvips operations typically
# stream content and never have to allocated memory for the full image.
iex> {:ok, tensor} = Image.to_nx(i)
{:ok,
 #Nx.Tensor<
   u8[width: 1000][height: 542][bands: 3]
   EXLA.Backend<host:0, 0.1315682744.4087480340.156769>
   [
     [
       [220, 223, 232],
       [220, 223, 232],
       [220, 223, 232],
       ...
     ],
     ...
   ]
 >}
# Now we need turn the tensor into a simple 2 dimensional
# list of colors
iex> {width, height, bands} = Nx.shape(tensor)
{1000, 542, 3}
iex> colors = Nx.reshape(tensor, {width * height, bands})
#Nx.Tensor<
  u8[542000][3]
  EXLA.Backend<host:0, 0.1315682744.4087480340.156771>
  [
    [220, 223, 232],
    [220, 223, 232],
    [220, 223, 232],
    [220, 223, 232],
    ...
  ]
>

What is currently missing (and I've asked the question in the Elixir slack) is the equivalent of bumpy's np.unique(colors, axis=0) function to return a unique list of colors. Maybe its there and I just couldn't find it.

Using scholar it's also possible then to use Scholar.Cluster.KMeans.fit/2 to produce a k-means clustering of colors which would be useful.