MacOS flaky tests

* enable recording, streaming, dual-output tests on MacOS
* set CI env var for MacOS github runner for obs_handler.ts
* fix warning we cant assign null: `let secondContext: osn.IVideo = null;`
* do not invoke `AddVideoContext` on CI builds in enhanced broadcasting tests since these are disabled.
* main.yml: upgrade to macos-15 for arm64 ARCH (same OS Intel ARCH is using).
* main.yml: upload OBS logs to troubleshoot unit tests.

main.yml: set fail-fast to false

* turn off fail-fast so the tests can continue running which will fix issue where if macos-15-intel fails, macos-arm64 is cancelled before it can finish running.

osn_handler: log time delta for long duration

* add warning when signal received took longer then expectedDeadline

obs_handler: assign explicit log filename for tests

Fix orphaned worker thread:

The WorkerSignals::worker keeps polling Query. When a test fails before
calling destroy(), the server-side recording gets freed when OBS shuts
down, but the orphaned worker thread continues. When the next test file
creates a fresh OBS connection, Controller::GetInstance().GetConnection()
now returns the new session's connection. The orphaned worker sends
  Query(oldUID) to the new server, which has no knowledge of that
uid → "Recording reference is not valid."

Remove source message listener when test ends

* stops worker thread properly
* test_osn_simple_recording: remove lamda to enable timeout

Tests: simple_recording,test_osn_advanced_replayBuffer,test_osn_advanced_recording,dual_output - wrap with try/finally

* update any test that starts a worker thread via create() to properly invoke
destroy() within a finally block to guanrantee the worker thread is stopped even if the test fails. Prevents orphaned worker threads from invoking Query() every 33ms on the new IPC connection created by other tests.

guarantees cleanup even when tests throw so they tests will clean up proactively, and any orphaned workers that slip through will stop themselves rather than polluting subsequent test runs.

validate totalSleepMS does not have a negative value

When the IPC call takes longer than sleepIntervalMS on an overloaded CI
runner), sleepIntervalMS - dur.count() produces a negative result that
wraps to a huge size_t value, causing the worker to sleep for an
effectively infinite duration. After that, no more signals are ever
polled — any test waiting for a signal hits the 30-second timeout.

do not share stdout/stderr with parent process

On macOS, the server is spawned with posix_spawnp with NULL for
file_actions, which means it inherits the parent's stdout/stderr.
On Windows this doesn't happen because CreateProcessW uses
DETACHED_PROCESS | CREATE_NO_WINDOW, which cuts the server off
from the parent's console entirely.

obs_handler: add more retryable timeouts for flaky tests

advanced-recording fix

* Removed extraneous set_video_mix since audio encoders don't have a video mix which produced the "encoder 'track1' is not a video encoder" warning
* Reduce defaultVideoContext from 1280x720@60fps to 1280x720@30fps to reduce CPU load on the slow x86_64 CI machine.

(MacOS) controller.cpp: kill prev obs64 before starting

* Should make running multiple tests in succession on this slow CI more
reliable. Add a waitpid(pids[i], nullptr, 0) call that blocks until the
SIGKILLed process is fully reaped by the kernel before the new
posix_spawnp runs, eliminating the race.

osn-replay-buffer: invoke output handler to save

* makes test more reliable
* follows similar pattern used by OBS::ReplayBufferSave

Author: sandboxcoder

.github/workflows/main.yml

@@ -42,7 +42,7 @@ jobs:
             ReleaseName: release
             Architecture: x86_64
           - BuildReleases: Release-arm64
-            image: macos-14
+            image: macos-15
             BuildConfig: RelWithDebInfo
             ReleaseName: release
             Architecture: arm64
@@ -99,6 +99,7 @@ jobs:
     needs: build-macos
     runs-on: ${{ matrix.image }}
     strategy:
+      fail-fast: false # Don't cancel the other architecture's tests if one fails, so we can get test results for both.
       matrix:
         BuildReleases: [Release-x86_64, Release-arm64]
         include:
@@ -108,7 +109,7 @@ jobs:
             ReleaseName: release
             Architecture: x86_64
           - BuildReleases: Release-arm64
-            image: macos-14
+            image: macos-15
             BuildConfig: RelWithDebInfo
             ReleaseName: release
             Architecture: arm64
@@ -151,6 +152,15 @@ jobs:
           OSN_ACCESS_KEY_ID: ${{secrets.AWS_RELEASE_ACCESS_KEY_ID}}
           OSN_SECRET_ACCESS_KEY: ${{secrets.AWS_RELEASE_SECRET_ACCESS_KEY}}
           RELEASE_NAME: ${{matrix.ReleaseName}}
+          CI: true
+          SUPPRESS_STREAMLABS_OBS_LOGS: true # Prevents logs from being printed in the test output, but still generates log files that can be uploaded as artifacts.
+      - name: Upload OBS logs
+        if: ${{ always() }}
+        uses: actions/upload-artifact@v4
+        with:
+          name: obs-logs-mac-${{ matrix.Architecture }}
+          path: tests/osn-tests/osnData/slobs-client/node-obs/logs/
+          if-no-files-found: ignore
       # Run even after test failures so the PR still gets the flaky summary.
       - name: Publish flaky test check
         if: ${{ always() }}
@@ -181,7 +191,7 @@ jobs:
             ReleaseName: release
             Architecture: x86_64
           - BuildReleases: Release-arm64
-            image: macos-14
+            image: macos-15
             BuildConfig: RelWithDebInfo
             ReleaseName: release
             Architecture: arm64
@@ -234,7 +244,7 @@ jobs:
             ReleaseName: release
             Architecture: x86_64
           - BuildReleases: Release-arm64
-            image: macos-14
+            image: macos-15
             BuildConfig: RelWithDebInfo
             ReleaseName: release
             Architecture: arm64
@@ -348,6 +358,13 @@ jobs:
           OSN_ACCESS_KEY_ID: ${{secrets.AWS_RELEASE_ACCESS_KEY_ID}}
           OSN_SECRET_ACCESS_KEY: ${{secrets.AWS_RELEASE_SECRET_ACCESS_KEY}}
           RELEASE_NAME: release
+      - name: Upload OBS logs
+        if: ${{ always() }}
+        uses: actions/upload-artifact@v4
+        with:
+          name: obs-logs-mac-${{ matrix.Architecture }}
+          path: tests/osn-tests/osnData/slobs-client/node-obs/logs/
+          if-no-files-found: ignore
       # Run even after test failures so the PR still gets the flaky summary.
       - name: Publish flaky test check
         if: ${{ always() }}

obs-studio-client/source/controller.cpp

@@ -53,6 +53,9 @@ std::wstring utfWorkingDir = L"";
 #include <libproc.h>
 #include <iostream>
 #include <spawn.h>
+#include <fcntl.h>
+#include <unistd.h>
+#include <sys/wait.h>
 extern char **environ;
 #endif
 
@@ -301,16 +304,32 @@ std::shared_ptr<ipc::client> Controller::host(const std::string &uri)
 		int st = proc_pidinfo(pids[i], PROC_PIDTBSDINFO, 0, &proc, PROC_PIDTBSDINFO_SIZE);
 		if (st == PROC_PIDTBSDINFO_SIZE) {
 			if (strcmp("obs64", proc.pbi_name) == 0) {
-				if (pids[i] != 0)
+				if (pids[i] != 0) {
 					kill(pids[i], SIGKILL);
+					waitpid(pids[i], nullptr, 0);
+				}
 			}
 		}
 	}
 
 	pid_t pid;
 	std::vector<const char *> argv = {"obs64", uri.c_str(), version.c_str(), serverBinaryPath.c_str(), nullptr};
 
-	int ret = posix_spawnp(&pid, serverBinaryPath.c_str(), NULL, NULL, const_cast<char *const *>(argv.data()), environ);
+	const char *suppressLogsEnv = std::getenv("SUPPRESS_STREAMLABS_OBS_LOGS");
+	int ret = 0;
+	if (suppressLogsEnv == nullptr || strcasecmp(suppressLogsEnv, "false") == 0) {
+		// For development, it can be helpful for process to share stdout/stderr.
+		ret = posix_spawnp(&pid, serverBinaryPath.c_str(), NULL, NULL, const_cast<char *const *>(argv.data()), environ);
+	} else {
+		// Do not send the blog(s) to stdout/stderr.
+		posix_spawn_file_actions_t file_actions;
+		posix_spawn_file_actions_init(&file_actions);
+		posix_spawn_file_actions_addopen(&file_actions, STDOUT_FILENO, "/dev/null", O_WRONLY, 0);
+		posix_spawn_file_actions_addopen(&file_actions, STDERR_FILENO, "/dev/null", O_WRONLY, 0);
+		ret = posix_spawnp(&pid, serverBinaryPath.c_str(), &file_actions, NULL, const_cast<char *const *>(argv.data()), environ);
+		posix_spawn_file_actions_destroy(&file_actions);
+	}
+
 	if (ret != 0) {
 		std::cerr << "Could not spawn the server at " << serverBinaryPath.c_str() << " with error code: " << ret << std::endl;
 		return nullptr;

obs-studio-client/source/nodeobs_api.cpp

@@ -36,12 +36,15 @@ Napi::Value api::OBS_API_initAPI(const Napi::CallbackInfo &info)
 	std::string language;
 	std::string version;
 	std::string crashserverurl;
+	std::string logFilename;
 
 	ASSERT_GET_VALUE(info, info[0], language);
 	ASSERT_GET_VALUE(info, info[1], path);
 	ASSERT_GET_VALUE(info, info[2], version);
 	if (info.Length() > 3)
 		ASSERT_GET_VALUE(info, info[3], crashserverurl);
+	if (info.Length() > 4)
+		ASSERT_GET_VALUE(info, info[4], logFilename);
 
 	auto conn = GetConnection(info);
 	if (!conn)
@@ -50,7 +53,7 @@ Napi::Value api::OBS_API_initAPI(const Napi::CallbackInfo &info)
 	conn->set_freeze_callback(ipc_freeze_callback, path);
 
 	std::vector<ipc::value> response = conn->call_synchronous_helper(
-		"API", "OBS_API_initAPI", {ipc::value(path), ipc::value(language), ipc::value(version), ipc::value(crashserverurl)});
+		"API", "OBS_API_initAPI", {ipc::value(path), ipc::value(language), ipc::value(version), ipc::value(crashserverurl), ipc::value(logFilename)});
 
 	// The API init method will return a response error + graphical error
 	// If there is a problem with the IPC the number of responses here will be zero so we must validate the

obs-studio-client/source/nodeobs_autoconfig.cpp

@@ -66,7 +66,8 @@ void autoConfig::worker()
 	do_sleep:
 		auto tp_end = std::chrono::high_resolution_clock::now();
 		auto dur = std::chrono::duration_cast<std::chrono::milliseconds>(tp_end - tp_start);
-		totalSleepMS = sleepIntervalMS - dur.count();
+		auto durCount = dur.count();
+		totalSleepMS = durCount < sleepIntervalMS ? sleepIntervalMS - durCount : 0;
 		std::this_thread::sleep_for(std::chrono::milliseconds(totalSleepMS));
 	}
 	return;

obs-studio-client/source/nodeobs_service.cpp

@@ -348,7 +348,8 @@ void service::worker()
 
 		auto tp_end = std::chrono::high_resolution_clock::now();
 		auto dur = std::chrono::duration_cast<std::chrono::milliseconds>(tp_end - tp_start);
-		totalSleepMS = sleepIntervalMS - dur.count();
+		auto durCount = dur.count();
+		totalSleepMS = durCount < sleepIntervalMS ? sleepIntervalMS - durCount : 0;
 		std::this_thread::sleep_for(std::chrono::milliseconds(totalSleepMS));
 	}
 

obs-studio-client/source/worker-signals.hpp

@@ -88,6 +88,16 @@ class WorkerSignals {
 			auto conn = Controller::GetInstance().GetConnection();
 			if (conn) {
 				std::vector<ipc::value> response = conn->call_synchronous_helper(name, "Query", {ipc::value(refID)});
+				if (!response.empty()) {
+					ErrorCode firstError = (ErrorCode)response[0].value_union.ui64;
+					if (firstError == ErrorCode::InvalidReference) {
+						// This typically happens if the worker thread is orphaned.
+						std::string errorMessage = response.size() > 1 ? response[1].value_str : "";
+						std::cout << "Worker thread exiting due to Invalid reference error encountered: " << errorMessage << std::endl;
+						isWorkerRunning = false;
+						break;
+					}
+				}
 				if ((response.size() == 5) && signalsList.size() < maximum_signals_in_queue) {
 					ErrorCode error = (ErrorCode)response[0].value_union.ui64;
 					if (error == ErrorCode::Ok) {
@@ -122,7 +132,8 @@ class WorkerSignals {
 
 			auto tp_end = std::chrono::high_resolution_clock::now();
 			auto dur = std::chrono::duration_cast<std::chrono::milliseconds>(tp_end - tp_start);
-			totalSleepMS = sleepIntervalMS - dur.count();
+			auto durCount = dur.count();
+			totalSleepMS = durCount < sleepIntervalMS ? sleepIntervalMS - durCount : 0;
 			std::this_thread::sleep_for(std::chrono::milliseconds(totalSleepMS));
 		}
 
@@ -140,4 +151,4 @@ class WorkerSignals {
 			workerThread->join();
 		}
 	}
-};
+};

obs-studio-server/source/nodeobs_api.cpp

@@ -874,10 +874,18 @@ void OBS_API::OBS_API_initAPI(void *data, const int64_t id, const std::vector<ip
 	std::string appdata = args[0].value_str;
 	std::string locale = args[1].value_str;
 	currentVersion = args[2].value_str;
+	std::string logFilename;
+	// Skip index 3 which is reserved for crash-handler server
+	if (args.size() > 4) {
+		// Parse the log filename
+		std::ostringstream ss;
+		ss << args[4].value_str << '-' << GenerateTimeDateFilename("txt");
+		logFilename = ss.str();
+	}
 	utility::osn_current_version(currentVersion);
 
 	/* Logging */
-	std::string filename = GenerateTimeDateFilename("txt");
+	std::string filename = logFilename.size() > 0 ? logFilename : GenerateTimeDateFilename("txt");
 	std::string log_path = appdata;
 	log_path.append("/node-obs/logs/");
 

obs-studio-server/source/osn-replay-buffer.cpp

@@ -152,19 +152,23 @@ void osn::IReplayBuffer::Query(void *data, const int64_t id, const std::vector<i
 
 void osn::IReplayBuffer::Save(void *data, const int64_t id, const std::vector<ipc::value> &args, std::vector<ipc::value> &rval)
 {
-	obs_enum_hotkeys(
-		[](void *data, obs_hotkey_id id, obs_hotkey_t *key) {
-			if (obs_hotkey_get_registerer_type(key) == OBS_HOTKEY_REGISTERER_OUTPUT) {
-				std::string key_name = obs_hotkey_get_name(key);
-				if (key_name.compare("ReplayBuffer.Save") == 0) {
-					obs_hotkey_enable_callback_rerouting(true);
-					obs_hotkey_trigger_routed_callback(id, true);
-				}
-			}
-			return true;
-		},
-		nullptr);
+	ReplayBuffer *replayBuffer = static_cast<ReplayBuffer *>(osn::IFileOutput::Manager::GetInstance().find(args[0].value_union.ui64));
+	if (!replayBuffer) {
+		PRETTY_ERROR_RETURN(ErrorCode::InvalidReference, "ReplayBuffer reference is not valid.");
+	}
 
+	obs_output_t *output = replayBuffer->GetOutput();
+	if (!output) {
+		PRETTY_ERROR_RETURN(ErrorCode::InvalidReference, "Invalid replay buffer output.");
+	}
+
+	calldata_t cd = {0};
+	proc_handler_t *ph = obs_output_get_proc_handler(output);
+	bool hasInvoked = proc_handler_call(ph, "save", &cd);
+	calldata_free(&cd);
+
+	if (!hasInvoked)
+		PRETTY_ERROR_RETURN(ErrorCode::NotFound, "Could not find ReplayBuffer::Save");
 	rval.push_back(ipc::value((uint64_t)ErrorCode::Ok));
 	AUTO_DEBUG;
-}
+}

package.json

@@ -22,7 +22,7 @@
     "local:build": "cmake --build build --target install --config Debug",
     "local:clean": "rm -rf build/*",
     "test": "electron-mocha -t 80000 --js-flags=\"--expose-gc\" --color -r ts-node/register tests/osn-tests/src/**/*.ts --reporter tests/osn-tests/util/list-reporter.js",
-    "test:ci": "yarn run test --retries 2"
+    "test:ci": "yarn run test --retries 3"
   },
   "devDependencies": {
     "@aws-sdk/client-s3": "^3.0.0",

tests/osn-tests/src/test_nodeobs_autoconfig.ts

@@ -50,9 +50,6 @@ describe(testName, function() {
     });
 
     it('Run autoconfig', async function() {
-        if (obs.isDarwin()) {
-            this.skip();
-        }
         const start = performance.now();
         let progressInfo: IConfigProgress;
 	    let settingValue: any;