classification.mdx

title	Classification
description	Learn about how to classify objects in your images with imgproxy

Classification ((pro))

imgproxy can classify images by assigning them to predefined categories based on the overall content of the image. Unlike object detection, which answers “Where is a cat in this image?”, image classification answers “Is there a cat in this image?” by labeling the entire image as a whole.

Specialized image classification models are typically faster and simpler than object detection because they do not need to locate objects within the image. They can also classify images based on broader concepts such as scenes, activities, or safety categories (e.g., NSFW vs. SFW), and require only labeled images for training instead of annotated bounding boxes.

Configuration

:::tip If you're using an imgproxy Pro Docker image with a tag suffixed with -ml, a basic classification model is included. For advanced classification, you may want to configure your own model.

The list of classes available in the bundled model

``` Accordion Adhesive tape Aircraft Alarm clock Alpaca Ambulance Ant Antelope Apple Armadillo Artichoke Axe Backpack Bagel Baked goods Balance beam Ball (Object) Balloon Banana Band-aid Banjo Barge Barrel Baseball bat Baseball glove Bat (Animal) Bathroom accessory Bathroom cabinet Bathtub Beaker Bear Bed Bee Beehive Beer Beetle Bell pepper Belt Bench Bicycle Bicycle helmet Bidet Billboard Billiard table Binoculars Bird Blender Blue jay Boat Bomb Book Bookcase Boot Bottle Bottle opener Bow and arrow Bowl Bowling equipment Box Boy Brassiere Bread Briefcase Broccoli Bronze sculpture Brown bear Bull Burrito Bus Bust Butterfly Cabbage Cabinetry Cake Cake stand Calculator Camel Camera Can opener Canary Candle Candy Cannon Canoe Cantaloupe Car Carrot Cart Cassette deck Castle Cat Cat furniture Caterpillar Cattle Ceiling fan Cello Centipede Chainsaw Chair Cheese Cheetah Chest of drawers Chicken Chime Chisel Chopsticks Christmas tree Clock Closet Coat Cocktail Cocktail shaker Coconut Coffee (drink) Coffee cup Coffee table Coffeemaker Coin Common fig Common sunflower Computer keyboard Computer monitor Computer mouse Container Convenience store Cookie Cooking spray Corded phone Cosmetics Couch Countertop Cowboy hat Crab Cream Cricket ball Crocodile Croissant Crown Crutch Cucumber Cupboard Curtain Cutting board Dagger Dairy Product Deer Desk Dessert Diaper Dice Digital clock Dinosaur Dishwasher Dog Dog bed Doll Dolphin Door Door handle Doughnut Dragonfly Drawer Dress Drill (Tool) Drink Drinking straw Drum Duck Dumbbell Eagle Earring Egg Elephant Envelope Eraser Face powder Facial tissue holder Falcon Fast food Fax Fedora Filing cabinet Fire hydrant Fireplace Fish Fixed-wing aircraft Flag Flashlight Flower Flowerpot Flute Flying disc Food processor Football Football helmet Footwear Fork Fountain Fox French fries French horn Frog Fruit Frying pan Garden Asparagus Gas stove Giraffe Girl Glasses Glove Goat Goggles Goldfish Golf ball Golf cart Gondola Goose Grape Grapefruit Grinder Guacamole Guitar Hair dryer Hair spray Hamburger Hammer Hamster Hand dryer Handbag Handgun Harbor seal Harmonica Harp Harpsichord Hat Headphones Heater Hedgehog Helicopter Helmet High heels Hiking equipment Hippopotamus Honeycomb Horizontal bar Horse Hot dog House Houseplant Humidifier Ice cream Indoor rower Infant bed Insect Ipod Isopod Jacket Jacuzzi Jaguar (Animal) Jeans Jellyfish Jet ski Jug Juice Kangaroo Kettle Kitchen & dining room table Kitchen appliance Kitchen knife Kitchen utensil Kite Knife Koala Ladder Ladle Ladybug Lamp Lantern Laptop Lavender (Plant) Lemon (plant) Leopard Light bulb Light switch Lighthouse Lily Limousine Lion Lipstick Lizard Lobster Loveseat Luggage and bags Lynx Magpie Man Mango Maple Maraca Measuring cup Mechanical fan Medical equipment Microphone Microwave oven Milk Miniskirt Mirror Missile Mixer Mixing bowl Mobile phone Monkey Moths and butterflies Motorcycle Mouse Muffin Mug Mule Mushroom Musical instrument Musical keyboard Nail (Construction) Necklace Nightstand Oboe Office building Orange (fruit) Organ (Musical Instrument) Ostrich Otter Oven Owl Oyster Paddle Palm tree Pancake Panda Paper cutter Paper towel Parachute Parking meter Parrot Pasta Pastry Peach Pear Pen Pencil case Pencil sharpener Penguin Perfume Person Personal flotation device Piano Picnic basket Picture frame Pig Pillow Pineapple Pitcher (Container) Pizza Pizza cutter Plastic bag Plate Platter Polar bear Pomegranate Popcorn Porch Porcupine Poster Potato Power plugs and sockets Pressure cooker Pretzel Printer Pumpkin Punching bag Rabbit Raccoon Racket Radish Ratchet (Device) Raven Rays and skates Red panda Refrigerator Remote control Reptile Rhinoceros Rifle Ring binder Rocket Roller skates Rose Rugby ball Ruler Salad Salt and pepper shakers Sandal Sandwich Saucer Saxophone Scale Scarf Scissors Scoreboard Scorpion Screwdriver Sculpture Sea lion Sea turtle Seafood Seahorse Segway Serving tray Sewing machine Shark Sheep Shelf Shellfish Shirt Shorts Shotgun Shower Shrimp Sink Skateboard Ski Skirt Skull Skunk Skyscraper Slow cooker Snack Snail Snake Snowboard Snowman Snowmobile Snowplow Soap dispenser Sock Sofa bed Sombrero Sparrow Spatula Spice rack Spider Spoon Sports uniform Squash (Plant) Squid Squirrel Stairs Stapler Starfish Stationary bicycle Stethoscope Stool Stop sign Strawberry Street light Stretcher Studio couch Submarine Submarine sandwich Suit Suitcase Sun hat Sunglasses Surfboard Sushi Swan Swim cap Swimming pool Swimwear Sword Syringe Table Table tennis racket Tablet computer Tableware Taco Tank Tap Tart Taxi Tea Teapot Teddy bear Telephone Television Tennis ball Tennis racket Tent Tiara Tick Tie Tiger Tin can Toaster Toilet Toilet paper Tomato Toothbrush Torch Tortoise Towel Tower Toy Traffic light Traffic sign Train Training bench Treadmill Tree Tree house Tripod Trombone Trousers Truck Trumpet Turkey Turtle Umbrella Unicycle Van Vase Vegetable Violin Volleyball (Ball) Waffle Waffle iron Wall clock Wardrobe Washing machine Waste container Watch Watermelon Weapon Whale Wheelchair Whisk Whiteboard Willow Window Window blind Wine Wine glass Wine rack Winter melon Wok Woman Wood-burning stove Woodpecker Worm Wrench Zebra Zucchini ```

:::

You need to define the following config variables to enable object classification:

• [IMGPROXY_CLASSIFICATION_NET]: a path to the classification neural network model in ONNX format

• [IMGPROXY_CLASSIFICATION_CLASSES]: a path to the class names file

• [IMGPROXY_CLASSIFICATION_NET_SIZE]: the size of the neural network input. The width and the heights of the inputs should be the same, so this config value should be a single number. Default: 224

• [IMGPROXY_CLASSIFICATION_THRESHOLD]: classifications with confidence below this value will be discarded. Default: 0.5

• [IMGPROXY_CLASSIFICATION_NORMALIZATION]: the normalization type to apply to the input image. Possible values:

  • none: no normalization
  • half: normalize to [0, 1] range
  • full: normalize to [-1, 1] range
  • imagenet: normalize using ImageNet mean and standard deviation

  Default: none

• [IMGPROXY_CLASSIFICATION_LAYOUT]: the data layout of the neural network input. Possible values:

  • nhwc: channels last (TensorFlow default)
  • nchw: channels first (PyTorch default)

  Default: nhwc

Class names file

The class names file maps the neural network's class indexes to human-readable class names. The path to the class names file should be defined in the IMGPROXY_CLASSIFICATION_CLASSES config variable.

The class names file should contain one class name per line. The class names should match the order of the classes in the neural network output. Example:

person
bicycle
car

Getting classification info

Object classification is available via the info handler using the cl (classify) endpoint. Fetch classification results by specifying the number of top classes to return:

/info/.../cl:5/...

Where 5 is the number of top classes to return.

The response is an array of objects, each containing:

• class_id: The numeric ID of the class
• name: The class name from the class names file
• confidence: The confidence score for this classification

See the getting info documentation for more details.