|
| 1 | +--- |
| 2 | +name: image-annotations |
| 3 | +description: 'Annotate screenshots, diagrams, and images with callout rectangles, arrows, labels, and color-coded highlights using PIL. Includes rules for animated GIF annotations with timing and pacing.' |
| 4 | +--- |
| 5 | + |
| 6 | +# Image Annotations |
| 7 | + |
| 8 | +Add visual callouts to any image — screenshots, diagrams, architecture docs, demo frames — using PIL/Pillow. Highlights what changed or what to look at, so reviewers don't have to guess. |
| 9 | + |
| 10 | +## When to Use This Skill |
| 11 | + |
| 12 | +Use this skill when you need to: |
| 13 | + |
| 14 | +- Highlight a specific area in a screenshot for a PR description |
| 15 | +- Annotate before/after images to show what changed |
| 16 | +- Add labels and callouts to diagrams or architecture images |
| 17 | +- Create annotated frames for animated GIF demos |
| 18 | + |
| 19 | +## Prerequisites |
| 20 | + |
| 21 | +```bash |
| 22 | +pip install Pillow -q |
| 23 | +``` |
| 24 | + |
| 25 | +## Color Rules |
| 26 | + |
| 27 | +- **Red (`#E63946`)** — only for "bad" / "removed" things (e.g., circling a bug being fixed) |
| 28 | +- **Yellowish-orange (`#FF9F1C`)** — for neutral highlights ("look here", "new feature", etc.) |
| 29 | +- Never use red just because it's eye-catching — red = bad/removed |
| 30 | + |
| 31 | +## Font |
| 32 | + |
| 33 | +- Use **Ink Free** (`C:/Windows/Fonts/Inkfree.ttf`) for a handwritten look on Windows |
| 34 | +- On Linux/macOS, fall back to `ImageFont.load_default()` |
| 35 | +- Size **36** for annotations on ~1400px-wide images |
| 36 | +- `stroke_width=1` with `stroke_fill=<same color as fill>` — gives body without being too thick |
| 37 | +- Do NOT use white stroke — looks like a bad glow effect |
| 38 | + |
| 39 | +## Shapes |
| 40 | + |
| 41 | +- Prefer **rounded rectangles** over circles/ellipses — less pixelation at edges |
| 42 | +- `draw.rounded_rectangle([x1, y1, x2, y2], radius=14, outline=color, width=5)` |
| 43 | +- **Padding 18px** around the target content |
| 44 | + |
| 45 | +## Reference Snippet |
| 46 | + |
| 47 | +```python |
| 48 | +from PIL import Image, ImageDraw, ImageFont |
| 49 | + |
| 50 | +# Setup |
| 51 | +font = ImageFont.truetype('C:/Windows/Fonts/Inkfree.ttf', 36) # or load_default() |
| 52 | +color = '#FF9F1C' # orange for highlights |
| 53 | +stroke = 5 |
| 54 | +pad = 18 |
| 55 | + |
| 56 | +img = Image.open('screenshot.png') |
| 57 | +draw = ImageDraw.Draw(img) |
| 58 | + |
| 59 | +# Rounded rect with padding |
| 60 | +draw.rounded_rectangle( |
| 61 | + [x1 - pad, y1 - pad, x2 + pad, y2 + pad], |
| 62 | + radius=14, outline=color, width=stroke |
| 63 | +) |
| 64 | + |
| 65 | +# Leader line (same thickness as rect) |
| 66 | +draw.line([x2 + pad, cy, x2 + pad + 40, cy - 30], fill=color, width=stroke) |
| 67 | + |
| 68 | +# Label — same-color stroke for body, NO white stroke |
| 69 | +draw.text( |
| 70 | + (x2 + pad + 45, cy - 60), 'label text', |
| 71 | + fill=color, font=font, stroke_width=1, stroke_fill=color |
| 72 | +) |
| 73 | + |
| 74 | +img.save('annotated.png') |
| 75 | +``` |
| 76 | + |
| 77 | +## Algorithmic Annotation |
| 78 | + |
| 79 | +For images with multiple elements to annotate, use an algorithmic approach that automatically places labels without overlapping. |
| 80 | + |
| 81 | +### Quick start |
| 82 | + |
| 83 | +```python |
| 84 | +from annotate import annotate_image |
| 85 | + |
| 86 | +result = annotate_image( |
| 87 | + 'screenshot.png', |
| 88 | + [ |
| 89 | + {'elem': (560, 275, 635, 390), 'label': 'arrow on card', 'draw_box': True}, |
| 90 | + {'elem': (105, 453, 236, 470), 'label': 'direction text'}, |
| 91 | + ], |
| 92 | + debug=True, |
| 93 | +) |
| 94 | +result.save('annotated.png') |
| 95 | +``` |
| 96 | + |
| 97 | +- `elem`: `(x1, y1, x2, y2)` tight bounding box — must be exact pixel coordinates |
| 98 | +- `label`: text label (supports `\n` for multi-line) |
| 99 | +- `draw_box`: if `True`, draws a rounded rectangle around the element. If `False` (default), draws a V-arrowhead pointing at the element |
| 100 | +- `debug`: shows targeting rectangles and candidate heatmap for placement validation |
| 101 | + |
| 102 | +### Coordinate grid helper |
| 103 | + |
| 104 | +**Always use `grid_image()` before annotating an unfamiliar image.** Scaled-down previews display images smaller than actual pixel dimensions — the error compounds as you move away from (0,0). |
| 105 | + |
| 106 | +```python |
| 107 | +from annotate import grid_image |
| 108 | + |
| 109 | +grid = grid_image('screenshot.png', step=100) |
| 110 | +grid.save('grid.png') |
| 111 | +``` |
| 112 | + |
| 113 | +Then verify with small crops: |
| 114 | + |
| 115 | +```python |
| 116 | +from PIL import Image |
| 117 | +img = Image.open('screenshot.png') |
| 118 | +crop = img.crop((x1 - 20, y1 - 20, x2 + 20, y2 + 20)) |
| 119 | +crop.save('verify.png') |
| 120 | +``` |
| 121 | + |
| 122 | +### Algorithm overview |
| 123 | + |
| 124 | +1. **Ring search**: candidates between MIN_ARROW (25px) and MAX_ARROW (120px) from element edge |
| 125 | +2. **Contrast scoring**: prefers placements where label text is readable — `abs(avg_brightness - 147) - std * 0.3 - dist * 0.02` |
| 126 | +3. **Joint resolution**: candidates computed independently, placed greedily (best score first) |
| 127 | +4. **Hard blocks**: labels cannot overlap any other annotation's element or breathing box |
| 128 | +5. **Proximity penalty**: labels within 40px of other placed boxes get a score penalty |
| 129 | +6. **Arrow crossing penalty**: -50 for arrows crossing already-placed arrows |
| 130 | + |
| 131 | +### Debug mode colors |
| 132 | + |
| 133 | +| Color | Meaning | |
| 134 | +|-------|---------| |
| 135 | +| Cyan | Target element box (elem + padding) | |
| 136 | +| Gray | Exclusion zone (MIN_ARROW buffer) | |
| 137 | +| Red→Green | Candidate heatmap (red=bad, green=good) | |
| 138 | +| Magenta | Chosen label position | |
| 139 | +| Orange | Final rendered annotation | |
| 140 | + |
| 141 | +### Arrow styles |
| 142 | + |
| 143 | +- **`draw_box=True`**: rounded rectangle + straight line to label, no arrowhead |
| 144 | +- **`draw_box=False`**: V-shaped arrowhead with rounded line caps |
| 145 | + |
| 146 | +## Image Diffing |
| 147 | + |
| 148 | +Find what changed between two screenshots programmatically. Use as a safety net for subtle changes — when the difference is obvious, annotate directly instead. |
| 149 | + |
| 150 | +```python |
| 151 | +from annotate import diff_images |
| 152 | + |
| 153 | +clusters, debug_img = diff_images( |
| 154 | + 'before.png', 'after.png', |
| 155 | + threshold=30, # pixel difference floor (0-255) |
| 156 | + min_pixels=300, # ignore tiny noise clusters |
| 157 | + dilate=5, # merge nearby changed pixels |
| 158 | + debug=True, # render heatmap overlay |
| 159 | +) |
| 160 | + |
| 161 | +# clusters = [(x1, y1, x2, y2, pixel_count), ...] sorted largest-first |
| 162 | +if debug_img: |
| 163 | + debug_img.save('diff-debug.png') |
| 164 | + |
| 165 | +# Feed clusters into annotate_image: |
| 166 | +annotations = [ |
| 167 | + {'elem': (x1, y1, x2, y2), 'label': f'Change #{i+1}', 'draw_box': True} |
| 168 | + for i, (x1, y1, x2, y2, _) in enumerate(clusters[:3]) |
| 169 | +] |
| 170 | +``` |
| 171 | + |
| 172 | +**Debug heatmap colors:** Blue = small difference, Yellow = medium, Red = large, Cyan boxes = cluster bounding boxes. |
| 173 | + |
| 174 | +**When to use:** subtle opacity changes, dashed lines, minor color shifts, anti-aliasing differences. |
| 175 | +**When NOT to use:** any change you can see by eye — annotate directly for better labels. |
| 176 | + |
| 177 | +## Animated GIF Annotations |
| 178 | + |
| 179 | +Different from static images — animations have timing, transitions, and competing visual motion. |
| 180 | + |
| 181 | +### Element highlighting |
| 182 | + |
| 183 | +1. **Rects for big areas, arrows for small elements** — 500x300px area = rect, 200x25px element = arrow |
| 184 | +2. **Labels go RIGHT NEXT to what they describe** — short arrow (30-80px), label adjacent. Viewer's eye shouldn't travel more than ~100px |
| 185 | +3. **Arrow must not cross its own label** — pick the edge closest to the target |
| 186 | +4. **No bottom bar / subtitle approach** — eyes jump between content and bar. Contextual placement only |
| 187 | +5. **Hero message gets a bigger font** — main takeaway 64pt+, detail annotations 38pt |
| 188 | + |
| 189 | +### Timing and pacing |
| 190 | + |
| 191 | +6. **Fade: 2-frame pop-in at 10fps** — 50% → 100% opacity (0.2s total). Easing curves look bad at low FPS |
| 192 | +7. **Type → pause → annotate** — during fast action, show NO annotation. Pause, then add it |
| 193 | +8. **Variable frame duration** — fast during action (100ms), slow during pauses (600-800ms), long hold for hero (500ms) |
| 194 | +9. **Higher FPS for smooth motion** — 10fps minimum for typing/interaction |
| 195 | + |
| 196 | +### Pop-in fade implementation |
| 197 | + |
| 198 | +```python |
| 199 | +# 2-frame pop-in at 10fps |
| 200 | +FADE_ALPHAS = [0.50, 1.00] |
| 201 | + |
| 202 | +for frame_idx in range(total_frames): |
| 203 | + if annotation_just_changed and local_idx < len(FADE_ALPHAS): |
| 204 | + alpha = FADE_ALPHAS[local_idx] |
| 205 | + else: |
| 206 | + alpha = 1.0 |
| 207 | + # Apply alpha to annotation elements: |
| 208 | + # - pill background: fill=(r, g, b, int(base_alpha * alpha)) |
| 209 | + # - text: fill=(*color, int(255 * alpha)) |
| 210 | + # - rect outline: outline=(*color, int(255 * alpha)) |
| 211 | +``` |
| 212 | + |
| 213 | +## Guidelines |
| 214 | + |
| 215 | +1. **All elements same thickness** — rect `width`, line `width`, and visual text weight should feel consistent (~5px) |
| 216 | +2. Place labels **close to the rect** — short leader line (25-35px) |
| 217 | +3. Labels can overlap content — the stroke gives enough contrast |
| 218 | +4. **Show locally first** — verify before uploading to a PR |
| 219 | +5. **Take screenshots at native 1x, control display size in HTML** — use `<img width="300">` in markdown, never resize with PIL (creates artifacts) |
| 220 | +6. **Always check `Image.open(path).size` first** — HiDPI screenshots are larger than they appear (150% scaling = 1.5x CSS pixel dimensions) |
| 221 | +7. **Short labels work better** — wide labels have fewer valid placements. Use 1-3 words when possible |
| 222 | +8. **Verify with debug=True** — always check the first annotation of a new image with debug mode |
| 223 | + |
| 224 | +## Limitations |
| 225 | + |
| 226 | +- Ink Free font is Windows-only; other platforms need a fallback font |
| 227 | +- PIL text rendering is basic — no rich text, no markdown |
| 228 | +- Animated GIF annotations require frame-by-frame processing which can be slow for long recordings |
| 229 | +- Algorithmic placement works best with 2-6 annotations; more than that may produce crowded results |
0 commit comments