Skip to content

GESTURE_CONTROLS.md

Latest commit

 

History

History
261 lines (200 loc) · 5.94 KB

GESTURE_CONTROLS.md

File metadata and controls

261 lines (200 loc) · 5.94 KB

🖐️ Gesture Controls Guide

Works with EITHER Left or Right Hand! ✨

🖱️ MOUSE MODE (Default)

1. Move Cursor 👆

Gesture: Index finger pointing up
         (other fingers down)

     ☝️
    👊

Action: Moves cursor smoothly and quickly
Speed: 1.8x faster than before
Response: Instant (no lag)

2. Left Click 🤏

Gesture: Pinch thumb and index finger together
         (distance < 35px)

    👌
   (pinch)

Action: Left mouse click
Protection: 0.3s cooldown (no double-clicks)
Visual: Red circle appears

3. Right Click 🤏

Gesture: Pinch thumb and middle finger together
         (index finger down)

    🤌
   (pinch)

Action: Right mouse click
Protection: 0.3s cooldown
Visual: Purple circle appears

4. Scroll ✌️

Gesture: Index and middle fingers up
         (thumb down)

    ✌️
    👊

Action: Smooth scrolling
Movement: Up/down with fingers
Speed: 25x multiplier
Smoothing: Averaged between both fingers
Visual: Yellow circle + "SCROLL MODE" text

⌨️ KEYBOARD MODE (Press 'K')

Hover to Type 👆

Gesture: Index finger pointing at key
         Hold for 0.8 seconds

     ☝️
    👊
     ↓
   [KEY]

Action: Types the key
Visual: Green progress bar fills up
Feedback: Key highlights in green when pressed

Keyboard Layout

┌────┬────┬────┬────┬────┬────┬────┬────┬────┬────┐
│ Q  │ W  │ E  │ R  │ T  │ Y  │ U  │ I  │ O  │ P  │
├────┼────┼────┼────┼────┼────┼────┼────┼────┼────┤
│ A  │ S  │ D  │ F  │ G  │ H  │ J  │ K  │ L  │    │
├────┼────┼────┼────┼────┼────┼────┼────┼──────────┤
│ Z  │ X  │ C  │ V  │ B  │ N  │ M  │ BACKSPACE   │
├──────────────────────────────────┼──────────────┤
│         SPACE (300px)            │    ENTER     │
└──────────────────────────────────┴──────────────┘

Key Size: 80x80px (33% larger!)
Spacing: 20px gaps (easier targeting)
Activation: 0.8 seconds (faster)

🎮 Mode Controls

Switch Modes

Keyboard Shortcuts:
┌─────┬──────────────────────────┐
│  M  │ Switch to Mouse mode     │
│  K  │ Switch to Keyboard mode  │
│  F  │ Toggle fullscreen        │
│  Q  │ Quit application         │
└─────┴──────────────────────────┘

🎯 Hand Position Guide

Optimal Setup

        [Camera]
           |
           | 1-2 feet
           |
           ↓
        [Your Hand]
        
✅ Good lighting
✅ Plain background
✅ Hand fully visible
✅ Smooth movements
✅ Either left or right hand!

Hand Detection

The system automatically detects:
- Left hand: Thumb on left side
- Right hand: Thumb on right side

No configuration needed!
Just use whichever hand is comfortable.

📊 Visual Feedback

Mouse Mode Indicators

🟢 Green Circle    = Cursor position (normal)
🔴 Red Circle      = Left click detected
🟣 Purple Circle   = Right click detected
🟡 Yellow Circle   = Scroll mode active
📝 "SCROLL MODE"   = Scrolling enabled

Keyboard Mode Indicators

🟢 Green Circle    = Finger position
📊 Green Bar       = Hover progress (0-100%)
🟩 Green Key       = Key being pressed
🔵 Blue Key        = Key being hovered
⬛ Gray Key        = Normal key state

⚡ Performance Features

Speed Optimizations

  • ✅ 40% faster hand detection
  • ✅ Instant cursor movement (no delay)
  • ✅ 3-frame smoothing buffer (reduced from 5)
  • ✅ 1.8x cursor speed multiplier

Smoothness Optimizations

  • ✅ Scroll smoothing buffer (2 frames)
  • ✅ Averaged finger positions for stability
  • ✅ Click debouncing (0.3s cooldown)
  • ✅ Screen bounds clamping

Usability Optimizations

  • ✅ 33% larger keyboard keys
  • ✅ 100% more spacing between keys
  • ✅ 20% faster key activation
  • ✅ Better key layout organization

🎨 Gesture Tips

For Best Results:

  1. Cursor Movement

    • Keep other fingers closed
    • Move smoothly, not jerky
    • Stay within camera frame

  2. Clicking

    • Pinch firmly but briefly
    • Wait for visual feedback
    • Don't hold the pinch

  3. Scrolling

    • Keep both fingers parallel
    • Move up/down steadily
    • Larger movements = faster scroll

  4. Typing

    • Point directly at key center
    • Hold steady for 0.8 seconds
    • Watch the progress bar

🔄 Hand Orientation

How It Works

Right Hand Detection:
  Wrist ← Pinky
  (Pinky is left of wrist)

Left Hand Detection:
  Pinky → Wrist
  (Pinky is right of wrist)

The system automatically adjusts
thumb detection based on orientation!

🚀 Quick Reference

Want to... Do this...
Move cursor ☝️ Index up
Click 🤏 Thumb + Index pinch
Right-click 🤌 Thumb + Middle pinch
Scroll ✌️ Index + Middle up
Type 👆 Hover 0.8s
Switch mode Press M or K

💡 Pro Tips

  1. Use either hand - System works with both!
  2. Smooth gestures - Avoid sudden movements
  3. Good lighting - Helps detection speed
  4. Practice scrolling - Two fingers give best control
  5. Keyboard spacing - Larger keys are easier to hit
  6. Fullscreen mode - Press F for better visibility

Enjoy your optimized hand tracking system! 🎉

For technical details, see OPTIMIZATION_SUMMARY.md For quick start, see QUICK_START_OPTIMIZED.md