SVO _Export - videois shorter than the timestamp

Hi I have recorded a video on a Jetson nano 2gb for almost 2 hours with
Set params:

  • depth mode = ultra
  • resolution = 720 HD
  • .h265 enconding
  • adusted the bitrate = 1024 kbps
    When I convert the recorded .svo to a avi video format i can see that the time duration is only 32 minutes on my video player but the timestamp runs as usual for 2 hours?

Would be great to get some help.Thanks

I used the following script to convert .svo file to a .avi format (mode 1 (left +Depth))

########################################################################
#
# Copyright (c) 2022, STEREOLABS.
#
# All rights reserved.
#
# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
# "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
# LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
# A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
# OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
# SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
# LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
# DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
# THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
# (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
# OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
#
########################################################################

import sys
import pyzed.sl as sl
import numpy as np
import cv2
from pathlib import Path
import enum
import argparse
import os 
import datetime
class AppType(enum.Enum):
    LEFT_AND_RIGHT = 1
    LEFT_AND_DEPTH = 2
    LEFT_AND_DEPTH_16 = 3


def progress_bar(percent_done, bar_length=50):
    #Display a progress bar
    done_length = int(bar_length * percent_done / 100)
    bar = '=' * done_length + '-' * (bar_length - done_length)
    sys.stdout.write('[%s] %i%s\r' % (bar, percent_done, '%'))
    sys.stdout.flush()


def main():
    # Get input parameters
    svo_input_path = opt.input_svo_file
    output_dir = opt.output_path_dir
    avi_output_path = opt.output_avi_file 
    output_as_video = True    
    app_type = AppType.LEFT_AND_RIGHT
    if opt.mode == 1 or opt.mode == 3:
        app_type = AppType.LEFT_AND_DEPTH
    if opt.mode == 4:
        app_type = AppType.LEFT_AND_DEPTH_16
    
    # Check if exporting to AVI or SEQUENCE
    if opt.mode !=0 and opt.mode !=1:
        output_as_video = False

    if not output_as_video and not os.path.isdir(output_dir):
        sys.stdout.write("Input directory doesn't exist. Check permissions or create it.\n",
                         output_dir, "\n")
        exit()

    # Specify SVO path parameter
    init_params = sl.InitParameters()
    init_params.set_from_svo_file(svo_input_path)
    init_params.svo_real_time_mode = False  # Don't convert in realtime
    init_params.coordinate_units = sl.UNIT.MILLIMETER  # Use milliliter units (for depth measurements)

    # Create ZED objects
    zed = sl.Camera()

    # Open the SVO file specified as a parameter
    err = zed.open(init_params)
    if err != sl.ERROR_CODE.SUCCESS:
        sys.stdout.write(repr(err))
        zed.close()
        exit()
    
    # Get image size
    image_size = zed.get_camera_information().camera_configuration.resolution
    width = image_size.width
    height = image_size.height
    width_sbs = width * 2
    
    # Prepare side by side image container equivalent to CV_8UC4
    svo_image_sbs_rgba = np.zeros((height, width_sbs, 4), dtype=np.uint8)

    # Prepare single image containers
    left_image = sl.Mat()
    right_image = sl.Mat()
    depth_image = sl.Mat()

    video_writer = None
    if output_as_video:
        # Create video writer with MPEG-4 part 2 codec
        video_writer = cv2.VideoWriter(avi_output_path,
                                       cv2.VideoWriter_fourcc('M', '4', 'S', '2'),
                                       max(zed.get_camera_information().camera_configuration.fps, 30),
                                       (width_sbs, height))
        if not video_writer.isOpened():
            sys.stdout.write("OpenCV video writer cannot be opened. Please check the .avi file path and write "
                             "permissions.\n")
            zed.close()
            exit()
    
    rt_param = sl.RuntimeParameters()

    # Start SVO conversion to AVI/SEQUENCE
    sys.stdout.write("Converting SVO... Use Ctrl-C to interrupt conversion.\n")

    nb_frames = zed.get_svo_number_of_frames()
    
    font = cv2.FONT_HERSHEY_SIMPLEX
    #bottom_left_corner = (10, image_cv2.shape[0] - 10)
    font_scale = 0.5
    font_color = (255, 255, 255)  # White
    line_type = 1



    while True:
        timestamp_ns = zed.get_timestamp(sl.TIME_REFERENCE.IMAGE)
        timestamp_ms = int(timestamp_ns.get_milliseconds())
        tm = datetime.datetime.fromtimestamp(timestamp_ms / 1000.0).strftime('%H:%M:%S:%f')[:-3]
        #tm = zed.get_timestamp(sl.TIME_REFERENCE.IMAGE) #zed.get_timestamp(sl.TIME_REFERENCE.CURRENT).get_seconds()
        print(f'time:{tm},type:{type(tm)}')
        err = zed.grab(rt_param)
        if err == sl.ERROR_CODE.SUCCESS:
            svo_position = zed.get_svo_position()

            # Retrieve SVO images
            zed.retrieve_image(left_image, sl.VIEW.LEFT)

            if app_type == AppType.LEFT_AND_RIGHT:
                zed.retrieve_image(right_image, sl.VIEW.RIGHT)
            elif app_type == AppType.LEFT_AND_DEPTH:
                zed.retrieve_image(right_image, sl.VIEW.DEPTH)
            elif app_type == AppType.LEFT_AND_DEPTH_16:
                zed.retrieve_measure(depth_image, sl.MEASURE.DEPTH)

            if output_as_video:
                # Copy the left image to the left side of SBS image
                svo_image_sbs_rgba[0:height, 0:width, :] = left_image.get_data()

                # Copy the right image to the right side of SBS image
                svo_image_sbs_rgba[0:, width:, :] = right_image.get_data()

                # Convert SVO image from RGBA to RGB
                ocv_image_sbs_rgb = cv2.cvtColor(svo_image_sbs_rgba, cv2.COLOR_RGBA2RGB)
                

                bottom_left_corner = (10, height - 10)

                cv2.putText(
                        ocv_image_sbs_rgb, 
                        tm, 
                        bottom_left_corner, 
                        font, font_scale, font_color, line_type)


                # Write the RGB image in the video
                video_writer.write(ocv_image_sbs_rgb)
            else:
                # Generate file names
                filename1 = output_dir +"/"+ ("left%s.png" % str(svo_position).zfill(6))
                filename2 = output_dir +"/"+ (("right%s.png" if app_type == AppType.LEFT_AND_RIGHT
                                           else "depth%s.png") % str(svo_position).zfill(6))
                # Save Left images
                cv2.imwrite(str(filename1), left_image.get_data())

                if app_type != AppType.LEFT_AND_DEPTH_16:
                    # Save right images
                    cv2.imwrite(str(filename2), right_image.get_data())
                    
                else:
                    # Save depth images (convert to uint16)
                    cv2.imwrite(str(filename2), depth_image.get_data().astype(np.uint16))
        
            # Display progress
            progress_bar((svo_position + 1) / nb_frames * 100, 30)
        if err == sl.ERROR_CODE.END_OF_SVOFILE_REACHED:
            progress_bar(100 , 30)
            sys.stdout.write("\nSVO end has been reached. Exiting now.\n")
            break
    if output_as_video:
        # Close the video writer
        video_writer.release()

    zed.close()
    return 0


if __name__ == "__main__":
    parser = argparse.ArgumentParser(formatter_class=argparse.RawTextHelpFormatter)
    parser.add_argument('--mode', type = int, required=True, help= " Mode 0 is to export LEFT+RIGHT AVI. \n Mode 1 is to export LEFT+DEPTH_VIEW Avi. \n Mode 2 is to export LEFT+RIGHT image sequence. \n Mode 3 is to export LEFT+DEPTH_View image sequence. \n Mode 4 is to export LEFT+DEPTH_16BIT image sequence.")
    parser.add_argument('--input_svo_file', type=str, required=True, help='Path to the .svo file')
    parser.add_argument('--output_avi_file', type=str, help='Path to the output .avi file, if mode includes a .avi export', default = '')
    parser.add_argument('--output_path_dir', type = str, help = 'Path to a directory, where .png will be written, if mode includes image sequence export', default = '')
    opt = parser.parse_args()
    if opt.mode > 4 or opt.mode < 0 :
        print("Mode shoud be between 0 and 4 included. \n Mode 0 is to export LEFT+RIGHT AVI. \n Mode 1 is to export LEFT+DEPTH_VIEW Avi. \n Mode 2 is to export LEFT+RIGHT image sequence. \n Mode 3 is to export LEFT+DEPTH_View image sequence. \n Mode 4 is to export LEFT+DEPTH_16BIT image sequence.")
        exit()
    if not opt.input_svo_file.endswith(".svo"): 
        print("--input_svo_file parameter should be a .svo file but is not : ",opt.input_svo_file,"Exit program.")
        exit()
    if not os.path.isfile(opt.input_svo_file):
        print("--input_svo_file parameter should be an existing file but is not : ",opt.input_svo_file,"Exit program.")
        exit()
    if opt.mode < 2 and len(opt.output_avi_file)==0:
        print("In mode ",opt.mode,", output_avi_file parameter needs to be specified.")
        exit()
    if opt.mode < 2 and not opt.output_avi_file.endswith(".avi"):
        print("--output_avi_file parameter should be a .avi file but is not : ",opt.output_avi_file,"Exit program.")
        exit()
    if opt.mode >=2  and len(opt.output_path_dir)==0 :
        print("In mode ",opt.mode,", output_path_dir parameter needs to be specified.")
        exit()
    if opt.mode >=2 and not os.path.isdir(opt.output_path_dir):
        print("--output_path_dir parameter should be an existing folder but is not : ",opt.output_path_dir,"Exit program.")
        exit()
    main()

Hi @mn172

Is the generated video also accelerated?
If that’s the case, when the video_writer is created, please replace the fps parameters with the framerate of the SVO: max(zed.get_camera_information().camera_configuration.fps, <actual framerate of the svo>).

If your issue is reproducible with a shorter SVO, maybe try it out before the big one :sweat_smile:

Hi,
The video doesnt seem that accelerated on 15fps but on 30 fps it does feel like that.
I have : max(zed.get_camera_information().camera_configuration.fps, <actual framerate of the svo>)
set it to 15 or 30 sepending on how much i have set during recording.

I notice that when I view the svo file on explorer the i can see the frames dropped increase over time which in a way adds up to why the video is shorter. I also have the smae issue with a shorter video

If there are more and more frames dropped and the framerate is not stable enough, I think you have to add a check on the timestamps to the script, so that you add the image several times to the video when needed to stabilize the rate.

I would store ocv_image_sbs_rgb, and add it before the next image in video_writer.write(ocv_image_sbs_rgb), as many times as necessary to stabilize the timestamp.

To clarify, the SVO manages the frame drops well, but the AVI converter expects a stable framerate, it will provide a video with the settings provided when creating cv2.VideoWriter(...). It just adds the images grabbed sequentially, assuming they are at the given framerate.

Sure. Adding frames manually seem very unreliable to compensate though.
However when i set the depth mode in the python script to NONE it seems to grab more frames and reduces frame dropping than setting in ultra mode on the jetson 2gb module. Will this make a difference in choosing the modes later

Since I keep the files as .svo format will i be okay to record them in none depth and then convert them back to via sdk or pyzed to the preferred depth mode.

Getting as many frames as possible, as stable as possible is indeed the goal here.
I overlooked the depth mode, but yeah set it to NONE if you only need to record.
There will be no issue afterward, you will be able to choose the depth mode you want on the SVO.

If you keep the .svo file, you will be able to use it in the SDK. However, you won’t be able to convert the AVI back to SVO.

Thanks for getting back.
I have a question regarding using a ssd as a bootable device on the jetson. Will this help with faster write speed and reduce the farme drop rate. Currently my samsung memory card has 150 mb/s write speed

Will change compression mode from .h265 to .h264 help? How can i test if the issue with the performance is coming from the Jetson limitation or its software related.

It’s probably a bandwidth issue, so using something with better speed will most likely improve the recording quality a lot.

For the compression mode, you should try, h265 should write less data on disk and so improve the bandwidth usage efficiency.

1 Like

Do you know if I changed the bit rate to very low ex:2048 kbps will that impact the quality of the depth information to a great extent whil transforming back the data from .svo to .required_file format

Reducing the bit rate to this extent may cause blocky images that will give a bad depth.
The resolution or framerate should be lowered in priority.

1 Like