Skip to content

README_camera_inference.md

Latest commit

 

History

History
191 lines (143 loc) · 5.18 KB

README_camera_inference.md

File metadata and controls

191 lines (143 loc) · 5.18 KB
Raw

Real-time Facial Expression Recognition Camera Inference

This project provides a real-time facial expression recognition system using a webcam and the FER2013 CNN model trained in the accompanying Jupyter notebook.

Features

  • Real-time emotion detection from webcam feed
  • 7 emotion classes: Angry 😠, Disgust 🤢, Fear 😨, Happy 😊, Sad 😢, Surprise 😲, Neutral 😐
  • Visual feedback with bounding boxes, emotion labels, and confidence scores
  • Performance monitoring with FPS display
  • Interactive controls for saving frames and toggling fullscreen
  • GPU acceleration support (CUDA/CPU automatic detection)

Requirements

Hardware

  • Webcam or built-in camera
  • Optional: CUDA-capable GPU for faster inference

Software

  • Python 3.7+
  • Required packages (install via requirements file)

Installation

  1. Clone or download the repository

    cd fer_and_ter_model

  2. Install required packages

    pip install -r requirements_camera_inference.txt

  3. Ensure model file is present

    • The trained model file fer2013_final_model.pth should be in the same directory
    • This file is generated by running the training notebook fer2013_model_training.ipynb

Usage

Basic Usage

python camera_fer_inference.py

Advanced Usage with Options

# Use specific model file
python camera_fer_inference.py --model_path path/to/your/model.pth

# Use specific camera (if multiple cameras available)
python camera_fer_inference.py --camera_id 1

# Force CPU usage (disable GPU)
python camera_fer_inference.py --device cpu

# Force CUDA usage
python camera_fer_inference.py --device cuda

Interactive Controls

While the application is running:

  • Q: Quit the application
  • S: Save the current frame with predictions
  • F: Toggle fullscreen mode

Testing

Before running the camera inference, you can test the model loading and basic functionality:

python test_fer_model.py

This will verify:

  • Model loading from the .pth file
  • Basic inference functionality
  • Image preprocessing pipeline

Model Architecture

The emotion recognition model is a Convolutional Neural Network (CNN) with the following architecture:

  • Input: 48x48 grayscale face images
  • Convolutional layers: 4 conv layers with batch normalization
  • Pooling: Max pooling and adaptive average pooling
  • Fully connected layers: 3 FC layers with dropout
  • Output: 7 emotion classes

Model Details

  • Parameters: ~1.2M trainable parameters
  • Input preprocessing: Resize to 48x48, normalize to [-1, 1]
  • Output: Softmax probabilities for 7 emotion classes

Performance Notes

  • FPS: Typically 15-30 FPS depending on hardware
  • Latency: ~30-50ms per frame on modern hardware
  • Accuracy: Model accuracy depends on lighting conditions and face visibility

Optimization Tips

  1. Good lighting: Ensure faces are well-lit
  2. Face positioning: Keep faces relatively centered and upright
  3. Distance: Maintain reasonable distance from camera (1-3 feet)
  4. GPU usage: Use CUDA-capable GPU for better performance

Troubleshooting

Common Issues

  1. "Model file not found"

    • Ensure fer2013_final_model.pth is in the current directory
    • Run the training notebook to generate the model file

  2. "Failed to open camera"

    • Check if camera is being used by another application
    • Try different camera IDs (0, 1, 2, etc.)
    • Check camera permissions

  3. Low FPS or high latency

    • Close other applications using the camera
    • Use GPU acceleration with --device cuda
    • Reduce camera resolution in the script if needed

  4. Poor emotion recognition

    • Ensure good lighting conditions
    • Keep face clearly visible and unobstructed
    • Model works best with frontal face views

Dependencies Issues

If you encounter package conflicts:

# Create a virtual environment
python -m venv fer_env
source fer_env/bin/activate  # On Windows: fer_env\Scripts\activate

# Install requirements
pip install -r requirements_camera_inference.txt

Model Training

To retrain or fine-tune the model:

  1. Open fer2013_model_training.ipynb in Jupyter
  2. Follow the training process in the notebook
  3. The trained model will be saved as fer2013_final_model.pth

Technical Details

Face Detection

  • Uses OpenCV's Haar Cascade classifier for face detection
  • Processes detected faces individually for emotion recognition

Emotion Classes

emotion_labels = {
    0: 'Angry',     # 😠
    1: 'Disgust',   # 🤢
    2: 'Fear',      # 😨
    3: 'Happy',     # 😊
    4: 'Sad',       # 😢
    5: 'Surprise',  # 😲
    6: 'Neutral'    # 😐
}

Color Coding

Each emotion has a unique color for the bounding box:

  • Angry: Red
  • Disgust: Green
  • Fear: Purple
  • Happy: Yellow
  • Sad: Blue
  • Surprise: Orange
  • Neutral: Gray

Contributing

Feel free to contribute improvements:

  1. Better face detection algorithms
  2. Model architecture improvements
  3. Additional emotion classes
  4. Performance optimizations
  5. UI/UX enhancements

License

This project is for educational and research purposes. Please ensure proper attribution when using the code or model.