feat(go2rtc): codec-aware registration + live re-register on codec change (#374)

Reolink 810A (and anything else that streams H.265) couldn't negotiate
WebRTC: go2rtc offered H.265 video, browsers only accept VP8/VP9/H.264/AV1,
SDP failed with "codecs not matched". go2rtc's answer is a multi-source
stream with an ffmpeg H.264 fallback, but lightNVR was registering a
single raw-RTSP source and never consulting the codec field it already
had in stream_config_t.

A/B hybrid: let the user declare the codec up-front, but still auto-
correct and re-register live if they (or we) got it wrong.

Backend
- go2rtc_stream.h / .c: thread `const char *codec` through
  go2rtc_stream_register(). When codec is anything other than
  "h264" (case-insensitive) — including empty/unknown — append
  `ffmpeg:<id>#video=h264#hardware` to the source list. Unknown is
  treated as "possibly H.265" since detection only populates the codec
  field after the first packet; the cost is an idle ffmpeg entry that
  go2rtc never activates unless a consumer actually asks for H.264.
  The composition now stacks cleanly: primary RTSP + optional AAC
  (for MP4 audio muxing) + optional H.264 (for WebRTC).
- All nine call sites threaded: ensure_stream_registered_with_go2rtc,
  register_all_configured_streams main+sub, sync_db_streams main+sub,
  reload_stream_config, the helper in go2rtc_integration.c that runs
  on stream start, and the sub-stream path in api_handlers_streams_modify.c.
- go2rtc_disabled.c stub updated to match the new signature.
- api_handlers_streams_modify.c: accept `codec` on both POST (create,
  takes as initial hint) and PUT (update, triggers a stream restart so
  re-registration fires immediately instead of waiting for detection).
- unified_detection_thread.c: snapshot the stored codec before calling
  update_stream_video_params, and if the detected codec differs (e.g.
  user declared H.264 but camera is actually HEVC), call
  go2rtc_integration_reload_stream_config() with all-defaults to
  re-register with the freshly-written value. This is the "users are
  going to want it corrected without a program restart" piece.

Frontend
- StreamConfigModal.jsx: codec field is now an editable select
  (Auto-detect / H.264 / H.265) instead of a read-only span.
- StreamsView.jsx: include lower-cased `codec` in the outbound stream
  payload so the user's choice reaches the backend.
- en.json: two new keys (streamsConfig.codecAutoDetect,
  streamsConfig.codecHelp) describing what the field actually does.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

Author: matteius

include/video/go2rtc/go2rtc_stream.h

@@ -30,12 +30,21 @@ bool go2rtc_stream_init(const char *binary_path, const char *config_dir, int api
  * @param backchannel_enabled Whether to enable two-way audio (backchannel) support
  * @param protocol Stream protocol (TCP or UDP) - used to set transport parameter for go2rtc
  * @param record_audio Whether audio recording is enabled - if true, adds FFmpeg AAC transcoding source
+ * @param codec Detected or user-declared source video codec (e.g. "h264", "hevc").
+ *              May be NULL or empty when unknown. Used to decide whether to add
+ *              an `ffmpeg:<id>#video=h264#hardware` fallback source so that
+ *              WebRTC clients (which only accept H.264/VP8/VP9/AV1) can consume
+ *              H.265 streams via on-demand transcoding. The fallback is added
+ *              for anything that isn't explicitly "h264" — unknown codecs are
+ *              treated as "might be H.265" to be safe; once the detection
+ *              thread learns the real codec it re-registers via
+ *              go2rtc_integration_reregister_stream().
  * @return true if stream was registered successfully, false otherwise
  */
 bool go2rtc_stream_register(const char *stream_id, const char *stream_url,
                            const char *username, const char *password,
                            bool backchannel_enabled, stream_protocol_t protocol,
-                           bool record_audio);
+                           bool record_audio, const char *codec);
 
 /**
  * @brief Unregister a stream from go2rtc

src/video/go2rtc/go2rtc_disabled.c

@@ -78,9 +78,10 @@ bool go2rtc_stream_init(const char *binary_path, const char *config_dir, int api
 bool go2rtc_stream_register(const char *stream_id, const char *stream_url,
                             const char *username, const char *password,
                             bool backchannel_enabled, stream_protocol_t protocol,
-                            bool record_audio) {
+                            bool record_audio, const char *codec) {
     UNUSED(stream_id); UNUSED(stream_url); UNUSED(username); UNUSED(password);
-    UNUSED(backchannel_enabled); UNUSED(protocol); UNUSED(record_audio); return true;
+    UNUSED(backchannel_enabled); UNUSED(protocol); UNUSED(record_audio);
+    UNUSED(codec); return true;
 }
 bool go2rtc_stream_unregister(const char *stream_id) { UNUSED(stream_id); return true; }
 bool go2rtc_stream_get_webrtc_url(const char *stream_id, char *buffer, size_t buffer_size) {

src/video/go2rtc/go2rtc_integration.c

@@ -528,7 +528,7 @@ static bool ensure_stream_registered_with_go2rtc(const char *stream_name) {
                                config.onvif_username[0] != '\0' ? config.onvif_username : NULL,
                                config.onvif_password[0] != '\0' ? config.onvif_password : NULL,
                                config.backchannel_enabled, config.protocol,
-                               config.record_audio)) {
+                               config.record_audio, config.codec)) {
         log_error("Failed to register stream %s with go2rtc", stream_name);
         return false;
     }
@@ -1398,7 +1398,7 @@ bool go2rtc_integration_register_all_streams(void) {
                                            streams[i].onvif_username[0] != '\0' ? streams[i].onvif_username : NULL,
                                            streams[i].onvif_password[0] != '\0' ? streams[i].onvif_password : NULL,
                                            streams[i].backchannel_enabled, streams[i].protocol,
-                                           streams[i].record_audio)) {
+                                           streams[i].record_audio, streams[i].codec)) {
                     log_error("Failed to register stream %s with go2rtc", streams[i].name);
                     all_success = false;
                 } else {
@@ -1415,7 +1415,8 @@ bool go2rtc_integration_register_all_streams(void) {
                 if (!go2rtc_stream_register(sub_name, streams[i].sub_stream_url,
                                            streams[i].onvif_username[0] != '\0' ? streams[i].onvif_username : NULL,
                                            streams[i].onvif_password[0] != '\0' ? streams[i].onvif_password : NULL,
-                                           false, streams[i].protocol, false)) {
+                                           false, streams[i].protocol, false,
+                                           streams[i].codec)) {
                     log_warn("Failed to register sub-stream %s with go2rtc", sub_name);
                 }
             }
@@ -1510,7 +1511,7 @@ bool go2rtc_sync_streams_from_database(void) {
             if (!go2rtc_stream_register(db_streams[i].name, db_streams[i].url,
                                         username, password,
                                         db_streams[i].backchannel_enabled, db_streams[i].protocol,
-                                        db_streams[i].record_audio)) {
+                                        db_streams[i].record_audio, db_streams[i].codec)) {
                 log_error("Failed to register stream %s with go2rtc", db_streams[i].name);
                 all_success = false;
                 failed++;
@@ -1529,7 +1530,8 @@ bool go2rtc_sync_streams_from_database(void) {
                 log_info("Registering missing sub-stream %s with go2rtc", sub_name);
                 go2rtc_stream_register(sub_name, db_streams[i].sub_stream_url,
                                        username, password,
-                                       false, db_streams[i].protocol, false);
+                                       false, db_streams[i].protocol, false,
+                                       db_streams[i].codec);
             }
         }
     }
@@ -1762,8 +1764,10 @@ bool go2rtc_integration_reload_stream_config(const char *stream_name,
     // Wait a moment for go2rtc to clean up
     usleep(500000); // 500ms
 
-    // Re-register with new configuration
-    if (!go2rtc_stream_register(stream_name, url, username, password, backchannel, protocol, record_audio)) {
+    // Re-register with new configuration (passing the known codec so the
+    // H.264 transcoding fallback is added/omitted appropriately — #374/WebRTC)
+    const char *codec = have_config ? config.codec : NULL;
+    if (!go2rtc_stream_register(stream_name, url, username, password, backchannel, protocol, record_audio, codec)) {
         log_error("Failed to re-register stream %s with go2rtc", stream_name);
         return false;
     }
@@ -1895,7 +1899,7 @@ bool go2rtc_integration_register_stream(const char *stream_name) {
                                    username[0] != '\0' ? username : NULL,
                                    password[0] != '\0' ? password : NULL,
                                    config.backchannel_enabled, config.protocol,
-                                   config.record_audio)) {
+                                   config.record_audio, config.codec)) {
             log_info("Successfully registered stream %s with go2rtc", stream_name);
         } else {
             log_warn("Failed to register stream %s with go2rtc", stream_name);
@@ -1912,7 +1916,8 @@ bool go2rtc_integration_register_stream(const char *stream_name) {
         go2rtc_stream_register(sub_name, config.sub_stream_url,
                                username[0] != '\0' ? username : NULL,
                                password[0] != '\0' ? password : NULL,
-                               false, config.protocol, false);
+                               false, config.protocol, false,
+                               config.codec);
     }
 
     return main_ok || skip_main;

src/video/go2rtc/go2rtc_stream.c

@@ -93,7 +93,7 @@ bool go2rtc_stream_init(const char *binary_path, const char *config_dir, int api
 bool go2rtc_stream_register(const char *stream_id, const char *stream_url,
                            const char *username, const char *password,
                            bool backchannel_enabled, stream_protocol_t protocol,
-                           bool record_audio) {
+                           bool record_audio, const char *codec) {
     if (!g_initialized) {
         log_error("go2rtc stream integration not initialized");
         return false;
@@ -185,58 +185,74 @@ bool go2rtc_stream_register(const char *stream_id, const char *stream_url,
 
     log_info("Prepared go2rtc source URL for stream registration of %s: %s", stream_id, modified_url);
 
-    bool result;
-
-    // Register the stream with go2rtc.
-    //
-    // Audio strategy: rely on go2rtc's built-in on-demand transcoding rather
-    // than spawning persistent ffmpeg producer processes.  When a WebRTC viewer
-    // connects, go2rtc transcodes audio to OPUS in-process on the fly; no
-    // long-lived ffmpeg process is needed just to keep OPUS available.
-    //
-    // When audio recording is enabled we still need a persistent AAC producer
-    // so that the MP4 recording muxer gets an AAC audio track.  In that case
-    // we register two sources: primary RTSP + ffmpeg AAC.  go2rtc will
-    // transcode to OPUS for WebRTC viewers on demand from the AAC feed.
-    //
-    // NOTE: The FFmpeg AAC source is audio-only.  Do NOT add #video=copy —
-    //       that would open a second connection to the camera, creating a
-    //       duplicate video producer and RTSP 404 errors.  Video is served
-    //       directly from the primary RTSP source.
+    /*
+     * Compose the multi-source list for go2rtc.
+     *
+     * Source 0 is always the primary RTSP URL. We optionally append:
+     *
+     *   - ffmpeg:<id>#audio=aac   when record_audio is true, so the MP4
+     *     muxer has a persistent AAC producer. go2rtc still transcodes to
+     *     OPUS on demand for WebRTC viewers without a second ffmpeg.
+     *
+     *   - ffmpeg:<id>#video=h264#hardware   when the source codec is
+     *     anything other than "h264" (including unknown/empty). Browsers'
+     *     WebRTC stacks don't accept H.265, so go2rtc needs a transcoded
+     *     fallback to negotiate with them — this one only spawns its
+     *     ffmpeg process when a consumer actually asks for H.264 and the
+     *     primary doesn't supply it (H.264 sources leave it idle).
+     *     #hardware lets go2rtc pick VAAPI/NVENC/v4l2m2m if the host has
+     *     it and fall back to libx264.
+     *     (Neither #video=copy on the AAC source nor transcoding from
+     *     the AAC feed — each ffmpeg: entry opens its own producer.)
+     *
+     * codec may be NULL/empty on first registration since the detection
+     * thread only populates stream_config_t.codec once it has read a
+     * packet. Treat unknown as "might be H.265" — safer to pay the idle
+     * cost of an unused ffmpeg entry than to ship a stream that breaks
+     * WebRTC silently. When the detection thread later learns the real
+     * codec, go2rtc_integration_reregister_stream() re-issues this call
+     * with the corrected value.
+     */
+    const char *sources[3];
+    int num_sources = 0;
+    sources[num_sources++] = modified_url;
+
+    char ffmpeg_aac_source[URL_BUFFER_SIZE];
     if (record_audio) {
-        log_info("Audio recording enabled for stream %s, adding FFmpeg AAC source for MP4 recording", stream_id);
-
-        // Build the FFmpeg AAC transcoding source URL for recording.
-        // Audio-only: FFmpeg reads from go2rtc's internal RTSP, transcodes to
-        // AAC, and publishes back.  go2rtc will transcode to OPUS for WebRTC
-        // on demand without a separate persistent ffmpeg process.
-        char ffmpeg_aac_source[URL_BUFFER_SIZE];
-        snprintf(ffmpeg_aac_source, URL_BUFFER_SIZE, "ffmpeg:%s#audio=aac", encoded_stream_id);
-
-        const char *sources[2] = { modified_url, ffmpeg_aac_source };
-        result = go2rtc_api_add_stream_multi(encoded_stream_id, sources, 2);
+        snprintf(ffmpeg_aac_source, sizeof(ffmpeg_aac_source),
+                 "ffmpeg:%s#audio=aac", encoded_stream_id);
+        sources[num_sources++] = ffmpeg_aac_source;
+    }
+
+    bool is_h264 = (codec && codec[0] != '\0' && strcasecmp(codec, "h264") == 0);
+    char ffmpeg_h264_source[URL_BUFFER_SIZE];
+    if (!is_h264) {
+        snprintf(ffmpeg_h264_source, sizeof(ffmpeg_h264_source),
+                 "ffmpeg:%s#video=h264#hardware", encoded_stream_id);
+        sources[num_sources++] = ffmpeg_h264_source;
+        log_info("Stream %s codec=%s; adding ffmpeg H.264 fallback source for WebRTC",
+                 stream_id, (codec && codec[0]) ? codec : "unknown");
+    } else {
+        log_info("Stream %s codec=h264; no transcoding fallback needed", stream_id);
+    }
 
-        if (result) {
-            log_info("Successfully registered stream with go2rtc (with AAC audio for recording): %s", encoded_stream_id);
-        } else {
-            log_error("Failed to register stream with go2rtc (with AAC audio): %s", encoded_stream_id);
-            // Fall back to primary RTSP source only
-            log_warn("Falling back to single source registration without audio transcoding");
+    bool result;
+    if (num_sources > 1) {
+        result = go2rtc_api_add_stream_multi(encoded_stream_id, sources, num_sources);
+        if (!result) {
+            log_error("Failed to register stream %s with go2rtc (%d sources); falling back to primary-only",
+                      encoded_stream_id, num_sources);
             result = go2rtc_api_add_stream(encoded_stream_id, modified_url);
         }
     } else {
-        // No audio recording: register only the primary RTSP source.
-        // go2rtc transcodes audio to OPUS for WebRTC viewers on demand
-        // without requiring a persistent ffmpeg process.
-        log_info("Registering stream %s with primary RTSP source only (on-demand OPUS transcoding by go2rtc)", stream_id);
-
         result = go2rtc_api_add_stream(encoded_stream_id, modified_url);
+    }
 
-        if (result) {
-            log_info("Successfully registered stream with go2rtc: %s", encoded_stream_id);
-        } else {
-            log_error("Failed to register stream with go2rtc: %s", encoded_stream_id);
-        }
+    if (result) {
+        log_info("Successfully registered stream %s with go2rtc (%d source%s)",
+                 encoded_stream_id, num_sources, num_sources == 1 ? "" : "s");
+    } else {
+        log_error("Failed to register stream %s with go2rtc", encoded_stream_id);
     }
 
     // Intentionally do NOT preload here.

src/video/unified_detection_thread.c

@@ -54,6 +54,7 @@
 #include "video/streams.h"
 #include "video/go2rtc/go2rtc_stream.h"
 #include "video/go2rtc/go2rtc_snapshot.h"
+#include "video/go2rtc/go2rtc_integration.h"
 #include "database/db_recordings.h"
 #include "database/db_detections.h"
 #include "database/db_streams.h"
@@ -1184,9 +1185,34 @@ static void udt_update_stream_video_params(unified_detection_ctx_t *ctx,
     }
     pthread_mutex_unlock(&ctx->mutex);
 
+    /* Snapshot the previously-stored codec before writing the new row so we
+     * can detect a codec transition and re-register the stream with go2rtc.
+     * This covers the case where the user declared (or we guessed) H.264 at
+     * stream-add time but the camera is actually delivering H.265 — without
+     * the re-register, go2rtc keeps its original H.264-only source list and
+     * WebRTC clients can't negotiate (#374). */
+    char prev_codec[16] = {0};
+    {
+        stream_config_t prev = {0};
+        if (get_stream_config_by_name(ctx->stream_name, &prev) == 0) {
+            safe_strcpy(prev_codec, prev.codec, sizeof(prev_codec), 0);
+        }
+    }
+
     // Persist the (possibly provisional) values to the database
     update_stream_video_params(ctx->stream_name, det_width, det_height,
                                det_fps, det_codec);
+
+    if (det_codec && det_codec[0] != '\0' &&
+        strcasecmp(prev_codec, det_codec) != 0) {
+        log_info("[%s] Source codec transitioned %s -> %s; re-registering with go2rtc",
+                 ctx->stream_name,
+                 prev_codec[0] ? prev_codec : "(unknown)", det_codec);
+        /* Pass all defaults so reload_stream_config reads the freshly-written
+         * codec out of the DB itself. */
+        go2rtc_integration_reload_stream_config(ctx->stream_name,
+                                                NULL, NULL, NULL, -1, -1, -1);
+    }
 }
 
 /**

src/web/api_handlers_streams_modify.c

@@ -384,7 +384,8 @@ static void put_stream_worker(put_stream_task_t *task) {
                 go2rtc_stream_register(sub_name, task->config.sub_stream_url,
                                        task->config.onvif_username[0] != '\0' ? task->config.onvif_username : NULL,
                                        task->config.onvif_password[0] != '\0' ? task->config.onvif_password : NULL,
-                                       false, task->config.protocol, false);
+                                       false, task->config.protocol, false,
+                                       task->config.codec);
             } else {
                 log_info("Sub-stream %s removed from go2rtc", sub_name);
             }
@@ -578,6 +579,16 @@ void handle_post_stream(const http_request_t *req, http_response_t *res) {
         config.protocol = (stream_protocol_t)protocol->valueint;
     }
 
+    /* codec is user-settable as a hint (select Auto / H.264 / H.265) so
+     * lightNVR can make the right call on the initial go2rtc registration —
+     * specifically whether to pre-add the ffmpeg H.264 fallback for WebRTC
+     * (#374). The detection thread overwrites this with the real codec once
+     * it reads a packet, and re-registers with go2rtc if the user was wrong. */
+    cJSON *codec_hint = cJSON_GetObjectItem(stream_json, "codec");
+    if (codec_hint && cJSON_IsString(codec_hint)) {
+        safe_strcpy(config.codec, codec_hint->valuestring, sizeof(config.codec), 0);
+    }
+
     cJSON *record_audio = cJSON_GetObjectItem(stream_json, "record_audio");
     if (record_audio && cJSON_IsBool(record_audio)) {
         config.record_audio = cJSON_IsTrue(record_audio);
@@ -1330,6 +1341,24 @@ void handle_put_stream(const http_request_t *req, http_response_t *res) {
         }
     }
 
+    /* codec hint (#374): lets the user declare H.264 vs H.265 so the initial
+     * go2rtc registration picks the right source list. If the value changes,
+     * require a stream restart so the go2rtc re-registration picks it up
+     * immediately rather than waiting for the detection thread's next run. */
+    cJSON *codec_put = cJSON_GetObjectItem(stream_json, "codec");
+    if (codec_put && cJSON_IsString(codec_put)) {
+        char new_codec[sizeof(config.codec)];
+        safe_strcpy(new_codec, codec_put->valuestring, sizeof(new_codec), 0);
+        if (strcasecmp(config.codec, new_codec) != 0) {
+            safe_strcpy(config.codec, new_codec, sizeof(config.codec), 0);
+            config_changed = true;
+            non_dynamic_config_changed = true;
+            requires_restart = true;
+            log_info("Codec hint changed to '%s' for stream %s - restart required",
+                     config.codec, config.name);
+        }
+    }
+
     // Parse tags setting (metadata only, no restart needed)
     cJSON *tags_put = cJSON_GetObjectItem(stream_json, "tags");
     if (tags_put && cJSON_IsString(tags_put)) {