Is it possible to use grab() and enableRecording() functions at the same time?

onakany · November 20, 2023, 3:00am

Hi there.

I’m making a program to get ZED images.
That program first calls sl::Camera::open, then sl::Camera::grab and sl::Camera::retrieveImage repeatedly. It is working as expected.

I have added to this program the ability to do SVO recording at the same time.
Now the program crashes on std::out_of_range. The following message is output at the moment of crash.

terminate called after throwing an instance of 'std::out_of_range
  what(): deque::_M_range_check: __n (which is 11)>= this->size() (which is 1)

This crash occurs suddenly and irregularly.

My program was using std::queue and std::queue is implemented using std::deque in the libstdc++, so I tried using try-catch on all the places where I used queue, but could not catch it. My program uses only the ZED SDK as an external library. It may be that I am just not using the ZED SDK properly.

Is it possible to use the sl::Camera::grab function and sl::Camera::enableRecording function at the same time?

My environment:
- Jetson AGX Orin 32GB L4T 35.1
- Stereolabs Capture Card (x1)
- ZED X Mini (x1)
- ZED SDK 4.0.3

Myzhar · November 20, 2023, 8:31am

Hi @onakany
Welcome to the Stereolabs community.

Can you please provide the code you’re using to analyze and discover possible causes of the problem you described?

onakany · November 21, 2023, 1:04am

Hi @Myzhar
Thank you for your reply.
I will prepare the code soon. Please wait…

onakany · November 24, 2023, 10:39am

The pseudo source code is here:

// Grab and Recording are used simultaneously.

// In the code shown here, error handling is omitted
using namespace std;
using namespace sl;
int main() {
	Camera zed;
	InitParameters init_parameters;

	init_parameters.camera_resolution = RESOLUTION::HD1080;
	init_parameters.depth_mode = DEPTH_MODE::NONE;
	init_parameters.camera_fps = 60;
	init_parameters.camera_image_flip = FLIP_MODE::OFF;

	ERROR_CODE zedStat = zed.open(init_parameters);
	if (zedStat != ERROR_CODE::SUCCESS) {
		// NO ERRORS at this point - I confirmed the zed.open returned with no errors.
		return -1;
	}

	// Start recording to the SVO file.
	RecordingParameters recParam;
	recParam.compression_mode = SVO_COMPRESSION_MODE::H264;
	recParam.video_filename = String("/test/svo_filepath"); // I confirmed that the path is valid and writable.
	ERROR_CODE recStat = zed.enableRecording(recParam);
	if (recStat != ERROR_CODE::SUCCESS) {
		// NO ERRORS at this point - I confirmed the enableRecording function returned with no errors.
		return -1;
	}

	Mat leftImage;
	Mat rightImage;
	while (true) {
		try {
			if (every_4_minutes) { // We try to record to a new SVO file every 4 minutes.
				zed.disableRecording();

				recParam.compression_mode = SVO_COMPRESSION_MODE::H264;
				recParam.video_filename = String("/test/svo_filepath"); // I confirmed that the path is valid and writable.
				cout << "Enabling SVO Recording..." << endl;
				recStat = zed.enableRecording(recParam);
				if (recStat != ERROR_CODE::SUCCESS) {
					// NO ERRORS at this point - I confirmed the enableRecording function returned with no errors.
					return -1;
				}
				cout << "Enabled SVO Recording" << endl;
			}

			zedStat = zed.grab();
			if (zedStat != ERROR_CODE::SUCCESS) {
				// NO ERRORS at this point - I confirmed the grab function returned with no errors.
				return -1;
			}

			zed.retrieveImage(leftImage, VIEW::LEFT_UNRECTIFIED, MEM::CPU);
			zed.retrieveImage(rightImage, VIEW::RIGHT_UNRECTIFIED, MEM::CPU);

			// There are some stuff to do with the retrieved images.

		} catch (std::out_of_range& e) {
			// I tried to catch the std::out_of_range, but I cannot catch it.
			// still this program has crashed with "std::out_of_range".
		}
	}
	return 0;
}

This problem occurs irregularly.
Sometimes after a few minutes, sometimes after an hour.

The program output: (The number after “MyApplication” is a process id)

Oct  6 23:24:34 MyApplication[456]: Enabling SVO Recording...
Oct  6 23:24:35 MyApplication[456]: NvMMLiteOpen : Block : BlockType = 4
Oct  6 23:24:35 MyApplication[456]: ===== NVMEDIA: NVENC =====
Oct  6 23:24:35 MyApplication[456]: NvMMLiteBlockCreate : Block : BlockType = 4
Oct  6 23:24:35 MyApplication[456]: H264: Profile = 77, Level = 50
Oct  6 23:24:35 MyApplication[456]: Opening in BLOCKING MODE
Oct  6 23:24:35 MyApplication[456]: Enabled SVO Recording
Oct  6 23:24:35 MyApplication[456]: NVMEDIA: Need to set EMC bandwidth : 5744000
Oct  6 23:24:35 MyApplication[456]: NVMEDIA_ENC: bBlitMode is set to TRUE
Oct  6 23:25:37 MyApplication[456]: terminate called after throwing an instance of 'std::out_of_range'
Oct  6 23:25:37 MyApplication[456]:   what():  deque::_M_range_check: __n (which is 11)>= this->size() (which is 1)
Oct  6 23:25:38 nvargus-daemon[1425]: SCF: Error InvalidState: Failed to set the metadata for the frame (propagating from src/services/gl/EGLStreamProducer.cpp, function setFrameMetadata(), line 468)
Oct  6 23:25:38 nvargus-daemon[1425]: SCF: Error InvalidState:  (propagating from src/services/gl/EGLStreamProducer.cpp, function presentBufferInternal(), line 535)
Oct  6 23:25:38 nvargus-daemon[1425]: SCF: Error Disconnected:  (propagating from src/services/gl/EGLStreamProducer.cpp, function presentBufferInternal(), line 537)
Oct  6 23:25:38 nvargus-daemon[1425]: SCF: Error InvalidState: Failed to set the metadata for the frame (propagating from src/services/gl/EGLStreamProducer.cpp, function setFrameMetadata(), line 468)
Oct  6 23:25:38 nvargus-daemon[1425]: SCF: Error InvalidState:  (propagating from src/services/gl/EGLStreamProducer.cpp, function presentBufferInternal(), line 535)
Oct  6 23:25:38 nvargus-daemon[1425]: SCF: Error InsufficientMemory:  (propagating from src/services/gl/EGLStreamProducer.cpp, function allocateAndRegisterBuffers(), line 262)
Oct  6 23:25:38 nvargus-daemon[1425]: SCF: Error InvalidState:  (propagating from src/services/gl/EGLStreamProducer.cpp, function allocateAndRegisterBuffers(), line 262)
Oct  6 23:25:38 nvargus-daemon[1425]: SCF: Error InvalidState: Failed to set the metadata for the frame (propagating from src/services/gl/EGLStreamProducer.cpp, function setFrameMetadata(), line 468)
Oct  6 23:25:38 nvargus-daemon[1425]: SCF: Error InvalidState:  (propagating from src/services/gl/EGLStreamProducer.cpp, function presentBufferInternal(), line 535)
Oct  6 23:25:38 nvargus-daemon[1425]: SCF: Error InvalidState:  (propagating from src/services/gl/EGLStreamProducer.cpp, function presentBufferInternal(), line 537)
Oct  6 23:25:38 nvargus-daemon[1425]: SCF: Error InsufficientMemory:  (propagating from src/services/gl/EGLStreamProducer.cpp, function getBuffer(), line 413)
Oct  6 23:25:38 nvargus-daemon[1425]: SCF: Error InvalidState:  (propagating from src/services/gl/EGLStreamProducer.cpp, function presentBufferInternal(), line 537)
Oct  6 23:25:38 nvargus-daemon[1425]: SCF: Error InvalidState:  (propagating from src/services/gl/EGLStreamProducer.cpp, function returnFrame(), line 370)
Oct  6 23:25:38 nvargus-daemon[1425]: SCF: Error InvalidState:  (propagating from src/services/gl/EGLStreamProducer.cpp, function getBuffer(), line 432)
Oct  6 23:25:38 nvargus-daemon[1425]: SCF: Error InvalidState:  (propagating from src/components/CaptureContainerImpl.cpp, function assignAllBuffersFromStream(), line 241)
Oct  6 23:25:38 nvargus-daemon[1425]: SCF: Error InvalidState:  (propagating from src/components/stages/CCDataSetupStage.cpp, function doHandleRequest(), line 68)
Oct  6 23:25:38 nvargus-daemon[1425]: SCF: Error InvalidState:  (propagating from src/components/stages/OrderedStage.cpp, function doExecute(), line 158)
The log output of nvargus-daemon will continue for a bit longer, but it is long and will be omitted.

Another log output:

Sep 27 15:38:03 MyApplication[457]: Enabling SVO Recording...
Sep 27 15:38:03 MyApplication[457]: NvMMLiteOpen : Block : BlockType = 4
Sep 27 15:38:03 MyApplication[457]: ===== NVMEDIA: NVENC =====
Sep 27 15:38:03 MyApplication[457]: NvMMLiteBlockCreate : Block : BlockType = 4
Sep 27 15:38:03 MyApplication[457]: H264: Profile = 77, Level = 50
Sep 27 15:38:03 MyApplication[457]: Opening in BLOCKING MODE
Sep 27 15:38:03 MyApplication[457]: Enabled SVO Recording
Sep 27 15:38:03 MyApplication[457]: NVMEDIA: Need to set EMC bandwidth : 5744000
Sep 27 15:38:03 MyApplication[457]: NVMEDIA_ENC: bBlitMode is set to TRUE
Sep 27 15:38:42 MyApplication[457]: terminate called after throwing an instance of 'std::out_of_range'
Sep 27 15:38:42 MyApplication[457]:   what():  deque::_M_range_check: __n (which is 11)>= this->size() (which is 1)
Sep 27 15:38:53 nvargus-daemon[1405]: SCF: Error InvalidState: Failed to set the metadata for the frame (propagating from src/services/gl/EGLStreamProducer.cpp, function setFrameMetadata(), line 468)
Sep 27 15:38:53 nvargus-daemon[1405]: SCF: Error InvalidState:  (propagating from src/services/gl/EGLStreamProducer.cpp, function presentBufferInternal(), line 535)
Sep 27 15:38:53 nvargus-daemon[1405]: SCF: Error InvalidState: Failed to set the metadata for the frame (propagating from src/services/gl/EGLStreamProducer.cpp, function setFrameMetadata(), line 468)
Sep 27 15:38:53 nvargus-daemon[1405]: SCF: Error InvalidState:  (propagating from src/services/gl/EGLStreamProducer.cpp, function presentBufferInternal(), line 535)
Sep 27 15:38:53 nvargus-daemon[1405]: SCF: Error InsufficientMemory:  (propagating from src/services/gl/EGLStreamProducer.cpp, function allocateAndRegisterBuffers(), line 262)
Sep 27 15:38:53 nvargus-daemon[1405]: SCF: Error InvalidState:  (propagating from src/services/gl/EGLStreamProducer.cpp, function presentBufferInternal(), line 537)
Sep 27 15:38:53 nvargus-daemon[1405]: SCF: Error InvalidState:  (propagating from src/services/gl/EGLStreamProducer.cpp, function presentBufferInternal(), line 537)
Sep 27 15:38:53 nvargus-daemon[1405]: SCF: Error InsufficientMemory:  (propagating from src/services/gl/EGLStreamProducer.cpp, function getBuffer(), line 413)
Sep 27 15:38:53 nvargus-daemon[1405]: SCF: Error InvalidState:  (propagating from src/services/gl/EGLStreamProducer.cpp, function returnFrame(), line 370)
Sep 27 15:38:53 nvargus-daemon[1405]: SCF: Error InvalidState:  (propagating from src/services/gl/EGLStreamProducer.cpp, function getBuffer(), line 432)
Sep 27 15:38:53 nvargus-daemon[1405]: SCF: Error InvalidState:  (propagating from src/components/CaptureContainerImpl.cpp, function assignAllBuffersFromStream(), line 241)
Sep 27 15:38:53 nvargus-daemon[1405]: SCF: Error InvalidState:  (propagating from src/components/stages/CCDataSetupStage.cpp, function doHandleRequest(), line 68)
Sep 27 15:38:53 nvargus-daemon[1405]: SCF: Error InvalidState:  (propagating from src/components/stages/OrderedStage.cpp, function doExecute(), line 158)
The log output of nvargus-daemon will continue for a bit longer, but it is long and will be omitted.

Is there anywhere in the ZED SDK that uses the std::deque (or std::queue’s) at() function?

As a workaround, currently when my program crashes, I can do the following to restart both nvargus-daemon and zed_x_daemon and my application will start up again, so this is what I am doing now.

systemctl stop nvargus-daemon
systemctl stop zed_x_daemon
systemctl start nvargus-daemon
systemctl start zed_x_daemon

alassagne · November 24, 2023, 2:06pm

Hi,

The enableRecording method should only be called once, in the beginning. You don’t need to run it for every grab. Check out our example here => https://github.com/stereolabs/zed-sdk/tree/master/recording/recording/mono

onakany · November 27, 2023, 11:09am

Thanks for the response.
I am calling enableRecording at every 4 minute grab instead of every grab, is that a problem too?
I am intending to do like as the operation in ZED Explorer that stops recording every 4 minutes and immediately resumes.

alassagne · November 27, 2023, 11:45am

You can enable recordings every 4 minutes, as long as you also disable it.