Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
163 changes: 163 additions & 0 deletions LENOVO_IR_DEBUG.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,163 @@
# Lenovo IR Camera — Debug Guide

## Symptom

```
Response: ENROLL_FAIL Found 0 faces in ir. Expecting exactly 1.
```

## Root Cause

The YuNet face detector assigns **significantly lower confidence scores** to faces
captured by IR cameras (grayscale image, low contrast, often overexposed).

The default threshold of `0.9` was tuned for standard RGB webcams. On IR cameras
it is nearly impossible to reach. Lowering it to `0.5–0.6` resolves the issue
in most cases.

---

## Quick Fix (no recompilation required)

Edit `/etc/linuxcampam/config.ini`:

```ini
[Auth]
detection_threshold = 0.5

[Capture]
enroll_averaging = on
enroll_average_frames = 7
```

Then restart the daemon:
```bash
sudo systemctl restart linuxcampam
```

---

## Diagnosing with logs

During enrollment, watch the logs in real time:

```bash
journalctl -u linuxcampam -f
```

What to look for:

| Log message | Meaning |
|-------------|---------|
| `0 faces found above threshold (0.9)` | Threshold too high → lower it |
| `Best score: 0.65 \| Threshold: 0.9` | Face found but below threshold → lower to 0.55 |
| `Brightness: 12` | Frame nearly black → IR emitter not activating |
| `Brightness: 240` | Frame overexposed → camera exposure issue |

---

## Inspecting the failed frame

On every failed enrollment the captured frame is automatically saved to:

```
/var/log/linuxcampam/failed_enroll_ir_<username>.jpg
```

Open it to check what the camera actually captured:
```bash
xdg-open /var/log/linuxcampam/failed_enroll_ir_<username>.jpg
```

- **Nearly black frame** → IR emitter did not activate (see section below)
- **Grainy / blurry frame** → increase `enroll_average_frames`
- **Face visible but not detected** → lower `detection_threshold` further
- **No face in frame** → positioning issue during enrollment

---

## IR emitter not activating

If the saved frame is nearly black:

1. Verify `linux-enable-ir-emitter` is installed:
```bash
ls /usr/local/bin/linux-enable-ir-emitter
```

2. Test it manually:
```bash
sudo linux-enable-ir-emitter run
```

3. If not installed:
```bash
sudo apt install linux-enable-ir-emitter
# or from source: https://github.com/EmixamPP/linux-enable-ir-emitter
```

4. Configure for your specific hardware model:
```bash
sudo linux-enable-ir-emitter configure
```
Follow the interactive procedure — move your head in front of the camera
while it tries different configurations. When the emitter blinks and you see
`The infrared emitter has been successfully enabled!` you are done.

---

## Finding the correct IR camera device

```bash
# List all webcams
v4l2-ctl --list-devices

# Identify the IR camera by checking shape and brightness
python3 -c "
import cv2
for i in range(4):
cap = cv2.VideoCapture(i)
if not cap.isOpened():
print(f'video{i}: could not open')
continue
ret, frame = cap.read()
cap.release()
if ret:
print(f'video{i}: shape={frame.shape} brightness={frame.mean():.0f}')
"
```

The IR camera has **1 channel** (shape like `(360, 640)`) instead of 3 (RGB).
On Lenovo laptops it is typically `/dev/video2`.

---

## Recommended settings for Lenovo ThinkPad / IdeaPad

```ini
[Auth]
detection_threshold = 0.5
timeout_ms = 5000

[Capture]
enroll_hdr = off ; IR cameras do not support HDR
enroll_averaging = on
enroll_average_frames = 7

[Hardware]
camera_path_ir = /dev/video2 ; verify with v4l2-ctl --list-devices
```

---

## Code changes (this fix)

| File | Change |
|------|--------|
| `include/constants.hpp` | `IR_TRIGGER_DELAY_MS`: 200 → 1500 ms |
| `include/constants.hpp` | `CAMERA_WARMUP_FRAMES`: 10 → 15 |
| `include/constants.hpp` | `CAMERA_WARMUP_DELAY_MS`: 100 → 200 ms |
| `src/service/config.hpp` | `DEFAULT_DETECTION_THRESHOLD`: 0.9 → 0.6 |
| `src/service/auth_engine.cpp` | Log YuNet score + brightness in `generateEmbedding` |
| `src/service/auth_engine.cpp` | Always save failed enrollment frame with diagnostic info |
| `config/config.ini` | Updated default `detection_threshold` + averaging enabled |
10 changes: 7 additions & 3 deletions config/config.ini
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,10 @@ ir_emitter_path = /usr/local/bin/linux-enable-ir-emitter
; How confident the AI must be that it found a face.
; High values (0.9) are safe but fail in poor light/IR.
; Lower values (0.5-0.6) improve IR success but risk false detections.
detection_threshold = 0.9
; Lowered from 0.9 to 0.6: IR cameras (e.g. Lenovo, Dell) produce grayscale,
; low-contrast images where YuNet rarely scores above 0.7.
; Use 0.5-0.6 for IR cameras, 0.8-0.9 for high-quality RGB cameras only.
detection_threshold = 0.6

; Timeout in milliseconds to wait for a successful match.
; timeout_ms = 3000
Expand Down Expand Up @@ -67,8 +70,9 @@ detection_threshold = 0.9
; Enhanced capture settings for enrollment quality.
; HDR uses multiple exposures if camera supports manual exposure control.
; enroll_hdr = auto ; auto | on | off
; enroll_averaging = on ; on | off
; enroll_average_frames = 5
; For IR cameras (e.g. Lenovo, Dell), enable averaging to reduce noise:
enroll_averaging = on
enroll_average_frames = 7

; Verification capture (speed-focused, defaults to fast single-frame).
; verify_averaging = off
Expand Down
163 changes: 163 additions & 0 deletions docs/LENOVO_IR_DEBUG.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,163 @@
# Lenovo IR Camera — Debug Guide

## Symptom

```
Response: ENROLL_FAIL Found 0 faces in ir. Expecting exactly 1.
```

## Root Cause

The YuNet face detector assigns **significantly lower confidence scores** to faces
captured by IR cameras (grayscale image, low contrast, often overexposed).

The default threshold of `0.9` was tuned for standard RGB webcams. On IR cameras
it is nearly impossible to reach. Lowering it to `0.5–0.6` resolves the issue
in most cases.

---

## Quick Fix (no recompilation required)

Edit `/etc/linuxcampam/config.ini`:

```ini
[Auth]
detection_threshold = 0.5

[Capture]
enroll_averaging = on
enroll_average_frames = 7
```

Then restart the daemon:
```bash
sudo systemctl restart linuxcampam
```

---

## Diagnosing with logs

During enrollment, watch the logs in real time:

```bash
journalctl -u linuxcampam -f
```

What to look for:

| Log message | Meaning |
|-------------|---------|
| `0 faces found above threshold (0.9)` | Threshold too high → lower it |
| `Best score: 0.65 \| Threshold: 0.9` | Face found but below threshold → lower to 0.55 |
| `Brightness: 12` | Frame nearly black → IR emitter not activating |
| `Brightness: 240` | Frame overexposed → camera exposure issue |

---

## Inspecting the failed frame

On every failed enrollment the captured frame is automatically saved to:

```
/var/log/linuxcampam/failed_enroll_ir_<username>.jpg
```

Open it to check what the camera actually captured:
```bash
xdg-open /var/log/linuxcampam/failed_enroll_ir_<username>.jpg
```

- **Nearly black frame** → IR emitter did not activate (see section below)
- **Grainy / blurry frame** → increase `enroll_average_frames`
- **Face visible but not detected** → lower `detection_threshold` further
- **No face in frame** → positioning issue during enrollment

---

## IR emitter not activating

If the saved frame is nearly black:

1. Verify `linux-enable-ir-emitter` is installed:
```bash
ls /usr/local/bin/linux-enable-ir-emitter
```

2. Test it manually:
```bash
sudo linux-enable-ir-emitter run
```

3. If not installed:
```bash
sudo apt install linux-enable-ir-emitter
# or from source: https://github.com/EmixamPP/linux-enable-ir-emitter
```

4. Configure for your specific hardware model:
```bash
sudo linux-enable-ir-emitter configure
```
Follow the interactive procedure — move your head in front of the camera
while it tries different configurations. When the emitter blinks and you see
`The infrared emitter has been successfully enabled!` you are done.

---

## Finding the correct IR camera device

```bash
# List all webcams
v4l2-ctl --list-devices

# Identify the IR camera by checking shape and brightness
python3 -c "
import cv2
for i in range(4):
cap = cv2.VideoCapture(i)
if not cap.isOpened():
print(f'video{i}: could not open')
continue
ret, frame = cap.read()
cap.release()
if ret:
print(f'video{i}: shape={frame.shape} brightness={frame.mean():.0f}')
"
```

The IR camera has **1 channel** (shape like `(360, 640)`) instead of 3 (RGB).
On Lenovo laptops it is typically `/dev/video2`.

---

## Recommended settings for Lenovo ThinkPad / IdeaPad

```ini
[Auth]
detection_threshold = 0.5
timeout_ms = 5000

[Capture]
enroll_hdr = off ; IR cameras do not support HDR
enroll_averaging = on
enroll_average_frames = 7

[Hardware]
camera_path_ir = /dev/video2 ; verify with v4l2-ctl --list-devices
```

---

## Code changes (this fix)

| File | Change |
|------|--------|
| `include/constants.hpp` | `IR_TRIGGER_DELAY_MS`: 200 → 1500 ms |
| `include/constants.hpp` | `CAMERA_WARMUP_FRAMES`: 10 → 15 |
| `include/constants.hpp` | `CAMERA_WARMUP_DELAY_MS`: 100 → 200 ms |
| `src/service/config.hpp` | `DEFAULT_DETECTION_THRESHOLD`: 0.9 → 0.6 |
| `src/service/auth_engine.cpp` | Log YuNet score + brightness in `generateEmbedding` |
| `src/service/auth_engine.cpp` | Always save failed enrollment frame with diagnostic info |
| `config/config.ini` | Updated default `detection_threshold` + averaging enabled |
18 changes: 13 additions & 5 deletions include/constants.hpp
Original file line number Diff line number Diff line change
Expand Up @@ -18,10 +18,18 @@ inline constexpr double RGB_CHANNELS = 3.0;
inline constexpr uid_t DEFAULT_MIN_UID = 1000;

// Camera & Auth constants
inline constexpr int CAMERA_WARMUP_FRAMES = 10;
inline constexpr int CAMERA_WARMUP_DELAY_MS = 100;
// WARMUP_FRAMES and WARMUP_DELAY_MS increased from 10/100 to 15/200:
// IR cameras (e.g. Lenovo) need more frames discarded and longer settle time
// before the auto-exposure stabilizes and produces a usable image.
inline constexpr int CAMERA_WARMUP_FRAMES = 15;
inline constexpr int CAMERA_WARMUP_DELAY_MS = 200;
inline constexpr int CAMERA_AVERAGE_FRAMES = 5;
inline constexpr int IR_TRIGGER_DELAY_MS = 200;

// IR_TRIGGER_DELAY_MS increased from 200ms to 1500ms:
// On Lenovo laptops the IR emitter hardware needs significantly more time
// to activate after being triggered before the sensor receives usable IR light.
// 200ms was too short, resulting in near-black frames (brightness ~14/255).
inline constexpr int IR_TRIGGER_DELAY_MS = 1500;
inline constexpr int CAPTURE_RETRY_DELAY_S = 1;

// HDR Constants
Expand All @@ -36,5 +44,5 @@ inline constexpr int CAPTURE_RETRY_ATTEMPTS = 3;

inline constexpr float MIRROR_THRESHOLD_DEFAULT = 0.6f; // detection confidence
inline constexpr int MIRROR_SIZE = 640;
inline constexpr int MIRROR_NMS = 5000; // keep top K bboxes before NMS .
} // namespace linuxcampam
inline constexpr int MIRROR_NMS = 5000; // keep top K bboxes before NMS
} // namespace linuxcampam
Loading
Loading