record-mp4-h264

Recording a Video File

General Concept

A video file is created in 4 steps:

1. Create a VideoWriter object.
2. Call VideoWriter::beginFile to start a new video file.
3. Feed frames into the video file by repeatedly calling VideoWriter::addFrame.
4. Finally, call VideoWriter::finishFile to finalize the recording.

Practical Implementation

Step 0: Preparation

A video capture device is opened with the helper function from previous examples:

ic4-examples/cpp/image-acquisition/record-mp4-h264/src/record-mp4-h264.cpp

Lines 73 to 88 in 6abd8de

	// Let the user select a device
	auto device_list = ic4::DeviceEnum::getDevices();
	auto it = ic4_examples::console::select_from_list(device_list);
	if (it == device_list.end())
	{
	return -1;
	}
	// Open the selected device
	ic4::Error err;
	ic4::Grabber grabber;
	if (!grabber.deviceOpen(*it, err))
	{
	std::cerr << "Failed to open device: " << err.message() << std::endl;
	return -2;
	}

To get access to the image data received from the device, we have to create a sink object. Since we want to record a video file of a portion of the data stream from the video capture device, we need to access all image buffers that are received from the device during the time of recording. The QueueSink is the perfect solution in this situation.

To use a QueueSink, a class has to be derived from QueueSinkListener that handles sink events, especially QueueSinkListener::framesQueued:

ic4-examples/cpp/image-acquisition/record-mp4-h264/src/record-mp4-h264.cpp

Lines 10 to 66 in 6abd8de

	// Define QueueSinkListener-derived class that can add the received frames into a video writer
	class AddFrameListener : public ic4::QueueSinkListener
	{
	private:
	ic4::VideoWriter& writer_;
	std::atomic<bool> do_write_frames_;
	std::atomic<int> num_frames_written_;
	public:
	AddFrameListener(ic4::VideoWriter& writer)
	: writer_(writer)
	, do_write_frames_(false)
	, num_frames_written_(0)
	{
	}
	// Inherited via QueueSinkListener, called when there are frames available in the sink's output queue
	void framesQueued(ic4::QueueSink& sink) override
	{
	ic4::Error err;
	// Remove a buffer from the sink's output queue
	// We have to remove buffers from the queue even if not recording; otherwise the device will not have
	// buffers to write new video data into
	auto buffer = sink.popOutputBuffer(err);
	if (buffer == nullptr)
	{
	std::cerr << "Failed to get frame from sink: " << err.message() << std::endl;
	return;
	}
	if (do_write_frames_)
	{
	// Pass the image buffer to the video writer
	if (!writer_.addFrame(buffer, err))
	{
	std::cerr << "Failed to add frame to video file: " << err.message() << std::endl;
	}
	num_frames_written_ += 1;
	}
	}
	void enable_recording(bool enable)
	{
	if (enable)
	{
	num_frames_written_ = 0;
	}
	do_write_frames_ = enable;
	}
	int num_frames_written()
	{
	return num_frames_written_;
	}
	};

The sink is created by passing the listener instance to QueueSink::create:

ic4-examples/cpp/image-acquisition/record-mp4-h264/src/record-mp4-h264.cpp

Lines 95 to 104 in 6abd8de

	// Create an instance of the listener type defined above
	AddFrameListener listener(writer);
	// Create a QueueSink to capture all images arriving from the video capture device
	auto sink = ic4::QueueSink::create(listener, err);
	if (!sink)
	{
	std::cerr << "Failed to create sink: " << err.message() << std::endl;
	return -3;
	}

The data stream is started by calling Grabber::streamSetup, passing int he newly-created sink:

ic4-examples/cpp/image-acquisition/record-mp4-h264/src/record-mp4-h264.cpp

Lines 106 to 111 in 6abd8de

	// Start the video stream into the sink
	if (!grabber.streamSetup(sink, ic4::StreamSetupOption::AcquisitionStart, err))
	{
	std::cerr << "Failed to setup stream: " << err.message() << std::endl;
	return -4;
	}

Step 1: Create a VideoWriter Object

A video writer is always created specifying the file type and encoder. In our example, we want to create MP4 files with H264 encoding:

ic4-examples/cpp/image-acquisition/record-mp4-h264/src/record-mp4-h264.cpp

Lines 92 to 93 in 6abd8de

	// Create a video writer for H264-compressed MP4 files
	ic4::VideoWriter writer(ic4::VideoWriterType::MP4_H264);

Step 2: Start a New Video File

A new video file is started by a call to VideoWriter::beginFile. The function expects a file name, the format of the buffers that are going to be fed into VideoWriter::addFrame, and the video file's playback rate:

ic4-examples/cpp/image-acquisition/record-mp4-h264/src/record-mp4-h264.cpp

Lines 143 to 148 in 6abd8de

	// Begin writing a video file with a name, image type and playback rate
	if (!writer.beginFile(file_name.c_str(), image_type, frame_rate, err))
	{
	std::cerr << "Failed to begin recording: " << err.message() << std::endl;
	continue;
	}

The required image type and frame rate information is gathered from the sink and device beforehand:

ic4-examples/cpp/image-acquisition/record-mp4-h264/src/record-mp4-h264.cpp

Lines 113 to 129 in 6abd8de

	// Query the sink's output image type
	// The image type is required when starting the video recording
	auto image_type = sink->getOutputImageType(err);
	if (err.isError())
	{
	std::cerr << "Failed to query sink output frame type: " << err.message() << std::endl;
	return -5;
	}
	// Query the device's configured frame rate.
	// The frame rate is later set as the video file's playback rate.
	auto frame_rate = grabber.devicePropertyMap().find(ic4::PropId::AcquisitionFrameRate).getValue(err);
	if (err.isError())
	{
	std::cerr << "Failed to query acquisition frame rate: " << err.message() << std::endl;
	return -6;
	}

Step 3: Add Frames to the Video File

In our queue sink's framesQueued callback function, the oldest available image buffer is extracted from the sink's output queue. Is important to always pop and ultimately requeue the buffers to not starve the drivers of image buffers, even if the program does not always use the buffers:

ic4-examples/cpp/image-acquisition/record-mp4-h264/src/record-mp4-h264.cpp

Lines 27 to 51 in 6abd8de

	void framesQueued(ic4::QueueSink& sink) override
	{
	ic4::Error err;
	// Remove a buffer from the sink's output queue
	// We have to remove buffers from the queue even if not recording; otherwise the device will not have
	// buffers to write new video data into
	auto buffer = sink.popOutputBuffer(err);
	if (buffer == nullptr)
	{
	std::cerr << "Failed to get frame from sink: " << err.message() << std::endl;
	return;
	}
	if (do_write_frames_)
	{
	// Pass the image buffer to the video writer
	if (!writer_.addFrame(buffer, err))
	{
	std::cerr << "Failed to add frame to video file: " << err.message() << std::endl;
	}
	num_frames_written_ += 1;
	}
	}

The function adds the image buffer to the current video file if the do_write_frames_ flag indicates that a recording is active.

Step 4: Finalize the Video File

ic4-examples/cpp/image-acquisition/record-mp4-h264/src/record-mp4-h264.cpp

Lines 159 to 164 in 6abd8de

	// Finalize the currently opened video file
	if (!writer.finishFile(err))
	{
	std::cerr << "Failed to finish recording: " << err.message() << std::endl;
	continue;
	}