Human Follower ReID Pose is a modular person tracking system that combines:
- Detection: YOLO11m for person and pose detection.
- Tracking: BoTSORT with Kalman filtering.
- ReIdentification: FastReID for person feature extraction.
- Visualization: Color matched pose skeletons and trajectories.
- Analytics: Zone based tracking and data export.
┌─────────────────────────────────────────────────────────────────┐
│ INPUT (Video/Webcam) │
└────────────────────────────┬────────────────────────────────────┘
│
▼
┌─────────────────────────────────────────────────────────────────┐
│ DETECTION MODULE │
│ ┌──────────────────┐ ┌──────────────────┐ │
│ │ PersonDetector │ │ PoseDetector │ │
│ │ (YOLO11m) │ │ (YOLO11m-Pose) │ │
│ └────────┬─────────┘ └────────┬─────────┘ │
│ │ │ │
│ ├─────────────────────────────┤ │
│ │ Bounding Boxes, and Keypoints │ │
└───────────┼─────────────────────────────┼───────────────────────┘
│ │
▼ ▼
┌────────────────────────────────────────────────────────────────┐
│ REID MODULE │
│ ┌──────────────────────────────────────────────────────────┐ │
│ │ FastReIDFeatureExtractor (Market-1501 SBS) │ │
│ │ Extracts 2048-dim feature vectors for each person │ │
│ └────────────────────────┬─────────────────────────────────┘ │
└───────────────────────────┼────────────────────────────────────┘
│
▼
┌────────────────────────────────────────────────────────────────┐
│ TRACKING MODULE │
│ ┌──────────────────────────────────────────────────────────┐ │
│ │ BoTSORT Tracker │ │
│ │ - Kalman Filter for motion prediction │ │
│ │ - IoU, and ReID cost matrix for association │ │
│ │ - Track management (creation, update, deletion) │ │
│ └────────────────────────┬─────────────────────────────────┘ │
└───────────────────────────┼────────────────────────────────────┘
│
▼
┌────────────────────────────────────────────────────────────────┐
│ VISUALIZATION MODULE │
│ ┌──────────────┐ ┌──────────────┐ ┌──────────────┐ │
│ │ BBoxDrawer │ │ PoseDrawer │ │ Trajectory │ │
│ │ (Color-coded)│ │ (Color-match)│ │ Drawer │ │
│ └──────────────┘ └──────────────┘ └──────────────┘ │
└───────────────────────────┬────────────────────────────────────┘
│
▼
┌────────────────────────────────────────────────────────────────┐
│ ANALYTICS MODULE │
│ ┌──────────────┐ ┌──────────────┐ ┌──────────────┐ │
│ │ Trajectory │ │ Zone │ │ Analytics │ │
│ │ Tracker │ │ Manager │ │ Exporter │ │
│ └──────────────┘ └──────────────┘ └──────────────┘ │
└───────────────────────────┬────────────────────────────────────┘
│
▼
┌─────────────────────────────────────────────────────────────────┐
│ OUTPUT (Display/Video/Analytics) │
└─────────────────────────────────────────────────────────────────┘
HumanFollowerReIDPose/
│
├── main.py #Entry point and main processing loop.
├── requirements.txt #Python dependencies.
│
├── README.md #User documentation (1202 lines).
├── CodeBaseIndex.md #This file: Architecture and code documentation.
│
├── BUGFIX_INDEXERROR_DISAPPEARED_TRACKS.md #Bug fix documentation (300 lines).
├── ENHANCED_REID_CONFIGURATION_SUMMARY.md #ReID config summary (300 lines).
├── COMPREHENSIVE_TEST_RESULTS.md #Test results documentation (300 lines).
├── REID_PERSISTENCE_AND_WEBCAM_UPDATE.md #ReID persistence update summary.
│
├── core/ #Core modules.
│ │
│ ├── __init__.py #Core module exports.
│ │
│ ├── detection/ #Detection modules.
│ │ ├── __init__.py #Detection exports.
│ │ ├── person_detector.py #YOLO11m person detection.
│ │ └── pose_detector.py #YOLO11m-pose estimation.
│ │
│ ├── tracking/ #Tracking modules.
│ │ ├── __init__.py #Tracking exports.
│ │ ├── botsort.py #BoTSORT tracker implementation.
│ │ ├── kalman_tracker.py #Kalman filter for motion prediction.
│ │ └── crowd_tracking.py #Crowd analysis and continuity.
│ │
│ ├── reid/ #ReIdentification.
│ │ ├── __init__.py #ReID exports.
│ │ └── feature_extractor.py #FastReID feature extraction.
│ │
│ ├── visualization/ #Visualization modules.
│ │ ├── __init__.py #Visualization exports.
│ │ ├── drawer.py #Pose/trajectory/bbox drawing.
│ │ └── bbox_drawer.py #Bounding box utilities.
│ │
│ ├── analytics/ #Analytics modules.
│ │ ├── __init__.py #Analytics exports.
│ │ ├── trajectory_tracker.py #Trajectory recording.
│ │ ├── zone_manager.py #Zone based event detection.
│ │ └── analytics_exporter.py #CSV/JSON export.
│ │
│ └── ui_controller.py #UI controls and keyboard handling.
│
├── models/ #Model files (not in git).
│ ├── market_sbs_R101-ibn.pth #FastReID model (506 MB).
│ ├── yolo11m.pt #Person detection (39 MB).
│ ├── yolo11m-pose.pt #Pose estimation (41 MB).
│ ├── yolo11n.pt #Legacy (5.4 MB).
│ └── yolo11n-pose.pt #Legacy (6.0 MB).
│
├── configs/ #Configuration files.
│ ├── default_tracking.yml #Default tracking configuration.
│ └── market_sbs_R101-ibn.yml #FastReID model configuration.
│
├── output/ # Output directory.
│ └── videos/ # Saved output videos.
│
├── utils/ # Utility functions.
│ ├── __init__.py # Utils exports.
│ └── config.py # Configuration loading utilities.
│
└── Analytics_Data/ # Analytics output (created at runtime).
├── trajectories/ # Trajectory CSV files.
├── statistics/ # Statistics JSON files.
└── zone_events/ # Zone event logs.
Purpose: YOLO11m-based person detection with FP16 optimization.
Key Methods:
class PersonDetector:
def __init__(self, model_path='yolo11m.pt', conf_threshold=0.4, device='cuda'):
"""Initialize person detector with FP16 support"""
def detect(self, frame, conf_threshold=None):
"""
Detect persons in frame
Args:
frame (np.ndarray): Input frame (BGR)
conf_threshold (float): Override confidence threshold
Returns:
list: Detections [[x1, y1, x2, y2, confidence], ...]
"""Features:
- FP16 mixed precision inference on CUDA.
- Configurable confidence threshold (default: 0.4).
- Automatic model download if not present.
- GPU/CPU device selection.
Purpose: YOLO11m-pose for 17-keypoint COCO pose estimation.
Key Methods:
class PoseDetector:
def __init__(self, model_path='yolo11m-pose.pt',
conf_threshold=0.4, keypoint_threshold=0.4, device='cuda'):
"""Initialize pose detector with FP16 support"""
def detect(self, frame, conf_threshold=None):
"""
Detect persons and estimate poses
Returns:
tuple: (detections, keypoints_list)
- detections: [[x1, y1, x2, y2, conf], ...]
- keypoints: [np.ndarray(17, 3), ...] # [x, y, conf]
"""
def infer_pose(self, frame):
"""Infer poses without bounding boxes"""
@staticmethod
def match_pose_to_detections(detections, pose_preds, iou_threshold=0.3):
"""Match pose predictions to detections using IoU"""COCO Keypoints (17 points):
0. Nose, 1-2. Eyes, 3-4. Ears, 5-6. Shoulders, 7-8. Elbows,
9-10. Wrists, 11-12. Hips, 13-14. Knees, 15-16. Ankles
Purpose: Multi-object tracking with ReID integration and disappeared track reappearance
File: core/tracking/botsort.py (467 lines)
Key Methods:
class BoTSORT:
def __init__(self, feature_extractor, max_disappeared=60, min_hits=3,
iou_threshold=0.3, reid_threshold=0.7, reid_weight=0.3,
reid_history_frames=120, reid_reappear_threshold=0.75,
adaptive_iou_enabled=False, adaptive_iou_min=0.2,
adaptive_iou_max=0.5):
"""
Initialize BoTSORT tracker with enhanced ReID persistence
Args:
feature_extractor: FastReID feature extractor
max_disappeared (int): Max frames before removing lost track (default: 60)
reid_history_frames (int): Frames to keep disappeared track features (default: 120)
reid_reappear_threshold (float): ReID similarity for reappearance (default: 0.75)
adaptive_iou_enabled (bool): Enable motion-based adaptive IoU (default: False)
"""
def update(self, detections, frame=None):
"""
Update tracker with new detections
Args:
detections (np.ndarray): [[x1, y1, x2, y2, conf], ...]
frame (np.ndarray): Frame for ReID feature extraction
Returns:
np.ndarray: Tracked objects [[x1, y1, x2, y2, track_id], ...]
"""
def _match_disappeared_tracks(self, detections, reid_features):
"""
Match unmatched detections with disappeared tracks using ReID.
Uses greedy matching to ensure each disappeared track is matched
to at most one detection (Complete correspondence).
Args:
detections (np.ndarray): Unmatched detections
reid_features (np.ndarray): ReID features for unmatched detections
Returns:
list: List of (detection_idx, disappeared_idx, similarity) tuples
"""
def _compute_adaptive_iou_threshold(self, tracker):
"""
Compute adaptive IoU threshold based on tracker motion speed.
Fast-moving objects get lower IoU threshold (more tolerant matching).
Slow-moving objects get higher IoU threshold (stricter matching).
Args:
tracker (KalmanBoxTracker): Tracker object
Returns:
float: Adaptive IoU threshold
"""
def _cleanup_disappeared_tracks(self):
"""Remove disappeared tracks that exceed reid_history_frames"""Algorithm:
- Predict: Kalman filter predicts track positions.
- Extract Features: ReID features for new detections.
- Associate: IoU, and ReID cost matrix matching (Hungarian algorithm).
- Update: Update matched tracks with new detections.
- Reappearance Matching: Match unmatched detections with disappeared tracks using ReID.
- Complete greedy matching (highest similarity first).
- Prevents duplicate reactivations (bug fix).
- Bounds checking before list operations.
- Create New Tracks: Initialize new tracks for remaining unmatched detections.
- Cleanup: Remove old disappeared tracks (> reid_history_frames).
Key Features:
- Disappeared Track History: Maintains features for 120 frames (4 seconds at 30 FPS).
- ReID-Based Reappearance: Matches persons who reappear after occlusion.
- Adaptive IoU (Optional): Adjusts IoU threshold based on motion speed.
- Robust Error Handling: Bounds checking and duplicate prevention.
Purpose: Kalman filter for motion prediction.
State Vector: [x, y, w, h, vx, vy, vw, vh]
- Position: (x, y) - center coordinates
- Size: (w, h) - width and height
- Velocity: (vx, vy, vw, vh) - velocities
Purpose: Extract 2048-dim feature vectors for person reidentification.
Key Methods:
class FastReIDFeatureExtractor:
def __init__(self, model_path, config_path, device='cuda'):
"""Initialize FastReID with Market-1501 SBS model"""
def extract_features(self, frame, bboxes):
"""
Extract ReID features for multiple persons
Args:
frame (np.ndarray): Input frame
bboxes (list): [[x1, y1, x2, y2], ...]
Returns:
np.ndarray: Features (N, 2048)
"""
def compute_similarity(self, feat1, feat2):
"""Compute cosine similarity between features"""Model: Market-1501 SBS ResNet101-IBN
- Input: 256x128 RGB images
- Output: 2048-dim feature vectors
- Similarity: Cosine similarity (0-1)
Purpose: Draw poses, bounding boxes, and trajectories with color matching.
Key Methods:
class PoseDrawer:
def draw(self, frame, keypoints_list, track_ids=None):
"""
Draw poses with color matched to track IDs
Args:
frame (np.ndarray): Frame to draw on
keypoints_list (list): [np.ndarray(17, 3), ...]
track_ids (list): Track IDs for color matching
Returns:
np.ndarray: Frame with poses drawn
"""
def draw_single_pose(self, frame, keypoints, color=(0, 255, 0)):
"""Draw a single pose skeleton"""Color Palette System:
COLORS = [
(255, 0, 0), #Red
(0, 255, 0), #Green
(0, 0, 255), #Blue
(255, 255, 0), #Yellow
#... 16 more colors
]
def get_color(track_id):
"""Get consistent color for track ID"""
return COLORS[track_id % len(COLORS)]Purpose: Record and analyze person trajectories.
Key Methods:
class TrajectoryTracker:
def update(self, track_id, bbox):
"""Update trajectory for track"""
def get_trajectory(self, track_id):
"""Get full trajectory for track"""
def export_trajectories(self, output_path):
"""Export trajectories to CSV"""Purpose: Define zones and track entry/exit events.
Key Methods:
class ZoneManager:
def add_zone(self, name, polygon):
"""Add a zone defined by polygon"""
def update(self, detections_dict):
"""
Update zone occupancy
Returns:
list: Zone events (entry/exit)
"""Purpose: Handle keyboard controls and UI overlays.
Key Methods:
class UIController:
def handle_keyboard(self, key, permanent_storage, analytics_exporter):
"""Handle keyboard input"""
def create_mouse_callback(self):
"""Create mouse callback for person selection"""
def draw_selection_mode_instruction(self, frame):
"""Draw instruction overlay"""Keyboard Mapping:
- P: Toggle pose, Z: Toggle zones, I: Toggle info
- A: Toggle analytics, T: Toggle trajectories
- C: Toggle selection mode, S: Save screenshot
- E: Export analytics, Y: Delete data, Q: Quit
1. READ FRAME:
↓
2. DETECTION:
- PersonDetector.detect() OR PoseDetector.detect().
- Output: detections, and keypoints (if pose mode).
↓
3. REID FEATURE EXTRACTION:
- FastReIDFeatureExtractor.extract_features().
- Output: feature vectors (N, 2048).
↓
4. TRACKING:
- BoTSORT.update(detections, frame).
- Kalman prediction, and IoU/ReID association.
- Output: tracked_objects [[x1, y1, x2, y2, track_id], ...].
↓
5. KEYPOINT REORDERING (if pose mode):
- Match keypoints to tracked objects using IoU.
- Output: reordered_keypoints matching track order.
↓
6. VISUALIZATION:
- BBoxDrawer.draw(): bounding boxes.
- PoseDrawer.draw(): color matched poses.
- TrajectoryDrawer.draw(): trajectories.
- UIController overlays.
↓
7. OUTPUT:
- Display: cv2.imshow()
- Video: VideoWriter.write()
- Analytics: Export to CSV/JSON
Problem: Pose skeletons must match bounding box colors.
Solution: IoU based keypoint reordering.
#In main.py processing loop:
for obj in tracked_objects:
bbox = obj[:4]
track_id = int(obj[4])
#Find best matching detection using IoU.
best_iou = 0
best_idx = -1
for det_idx, det in enumerate(detections):
iou = calculate_iou(bbox, det[:4])
if iou > best_iou:
best_iou = iou
best_idx = det_idx
#Add matched keypoints.
if best_idx >= 0:
reordered_keypoints.append(keypoints_list[best_idx])
#Draw with color matching.
pose_drawer.draw(frame, reordered_keypoints, track_ids)Result: Each person's pose uses get_color(track_id) matching their bbox.
Updated: 2025-10-07 with enhanced ReID persistence parameters.
SYSTEM:
DEVICE: cuda
VERBOSE: true
DETECTION:
MODEL_PATH: yolo11m.pt #Options: yolo11n.pt, yolo11s.pt, yolo11m.pt, yolo11l.pt, yolo11x.pt.
CONF_THRESHOLD: 0.4
IOU_THRESHOLD: 0.45
MAX_DETECTIONS: 100
POSE:
MODEL_PATH: yolo11m-pose.pt #Options: yolo11n-pose.pt, yolo11s-pose.pt, yolo11m-pose.pt, yolo11l-pose.pt, yolo11x-pose.pt.
CONF_THRESHOLD: 0.4
IOU_THRESHOLD: 0.45
KEYPOINT_THRESHOLD: 0.4
REID:
MODEL_PATH: models/market_sbs_R101-ibn.pth
CONFIG_PATH: configs/market_sbs_R101-ibn.yml
THRESHOLD: 0.7
WEIGHT: 0.3
TRACKING:
MAX_DISAPPEARED: 60 #UPDATED from 30 to 60 (2 seconds at 30 FPS).
MIN_HITS: 3
IOU_THRESHOLD: 0.3
#ReID Persistence Parameters.
REID_HISTORY_FRAMES: 120 #Frames to keep disappeared track features (4 seconds at 30 FPS).
REID_REAPPEAR_THRESHOLD: 0.75 #ReID similarity threshold for reappearance matching.
REID_FEATURE_BUFFER_SIZE: 10 #Number of ReID features to store per track.
REID_SIMILARITY_METHOD: 'cosine' #Similarity computation method.
#Adaptive IoU Parameters.
ADAPTIVE_IOU_ENABLED: false # Enable motion-based adaptive IoU threshold.
ADAPTIVE_IOU_MIN: 0.2 # Minimum IoU for fast moving objects.
ADAPTIVE_IOU_MAX: 0.5 # Maximum IoU for slow moving objects.
VIDEO:
SOURCE: 0 #0 for webcam, or path to video file.
DISPLAY_WIDTH: 1280
DISPLAY_HEIGHT: 720| Parameter | Default | Description | Range |
|---|---|---|---|
MAX_DISAPPEARED |
60 | Max frames before removing lost track | 30-90 |
MIN_HITS |
3 | Min detections to confirm track | 1-5 |
IOU_THRESHOLD |
0.3 | IoU threshold for bbox matching | 0.2-0.5 |
REID_HISTORY_FRAMES |
120 | Frames to keep disappeared track features | 60-180 |
REID_REAPPEAR_THRESHOLD |
0.75 | ReID similarity for reappearance | 0.6-0.9 |
REID_FEATURE_BUFFER_SIZE |
10 | Features to store per track | 5-20 |
REID_SIMILARITY_METHOD |
'cosine' | Similarity computation method | 'cosine', 'euclidean' |
ADAPTIVE_IOU_ENABLED |
false | Enable adaptive IoU | true/false |
ADAPTIVE_IOU_MIN |
0.2 | Min IoU for fast-moving objects | 0.1-0.3 |
ADAPTIVE_IOU_MAX |
0.5 | Max IoU for slow-moving objects | 0.4-0.7 |
See README.md "Model Configuration Guide" section for detailed customization examples.
def main():
#Parse arguments.
args = parse_arguments()
#Load configuration.
config = load_config(args.config) if args.config else get_default_config()
#Initialize components.
detector = PoseDetector(...) if pose_mode else PersonDetector(...)
tracker = BoTSORT(...)
feature_extractor = FastReIDFeatureExtractor(...)
#Open video source.
cap = cv2.VideoCapture(source)
#Initialize video writer (if --save-video).
if args.save_video:
video_writer = cv2.VideoWriter(...)
#Main processing loop.
while True:
ret, frame = cap.read()
#Detect.
detections, keypoints = detector.detect(frame)
#Track.
tracked_objects = tracker.update(detections, frame)
#Reorder keypoints (if pose mode).
reordered_keypoints = match_keypoints_to_tracks(...)
#Visualize.
vis_frame = draw_all(frame, tracked_objects, reordered_keypoints, ...)
#Output.
cv2.imshow('Reid Pose Tracker', vis_frame)
if video_writer:
video_writer.write(vis_frame)
#Handle keyboard.
key = cv2.waitKey(1)
if key == ord('q'):
break
#Cleanup.
cap.release()
if video_writer:
video_writer.release()
cv2.destroyAllWindows()- Classes: PascalCase (
PersonDetector,BoTSORT). - Functions: snake_case (
detect(),extract_features()). - Constants: UPPER_SNAKE_CASE (
COLORS,SKELETON_PAIRS). - Private methods:
_method_name().
def method_name(self, arg1, arg2):
"""
Brief description.
Detailed description if needed.
Args:
arg1 (type): Description
arg2 (type): Description
Returns:
type: Description
"""try:
#Operation:
except SpecificException as e:
print(f"[Error] {e}")
#Fallback or cleanup.