Hi I am getting lower FPS than expected using the ZED X camera with Orin. The camera should be able to do 120 FPS(using SVGA resolution), but writing and timing a simple program that only grab() reveals that I am not able to get more than around 60FPS. 30 and 60 FPS settings work, but 120 is the same as 60 FPS.
Adding some functionality like retrieveImage() only slows down the program slightly and the time spent on grab() is reduced, showing that is could be done quicker(and getting a higher FPS).
Adding more functionality like object detection from the sample code quickly reduces the FPS way more (to around 10). Anything I am doing wrong, or could do to speed up the process? I have looked at threading, but did not fully understand how to implement it correctly. Thanks for any help:)
What is the model of Orin that you are using? Orin NX or AGX? 8 GB, 16 GB, 32 GB, or 64 GB?
Did you enable the MAXN power mode to leverage the full power of the Jetson device?
Did you run the jetson_clocks script to maximize the clock frequency of each module of the Jetson device?
Please provide more information concerning the configuration of your system and the source code of the test programs you are executing.
Hi!
I am using the Orin NX 16GB. Using the normal power mode(15W). I will try different power mode and the mentioned script, thanks! The programs was just a loop with zed.grab() , then with zed.retrieveImage, and the last program was the zed sdk sample for birds eye viewer object detection (https://github.com/stereolabs/zed-sdk/tree/master/object%20detection/birds%20eye%20viewer/python)
Hi, changing to MAXN helped a lot and I was able to get around 105 FPS when just grabbing images and nothing else. I did not understand how to run the jetson_clocks script if that is relevant for my Orin?
The performance on the object detection script I am working on also improved to about 30FPS and that is acceptable, but would love more room for expansion without going below 30FPS. The script in its ugly form with lots of timings and prints is below. In the future I want to run dual cameras with a different object detection for the other camera. Will this halve the framerate or is it possible with some threading or other ways to get similar performance? Thank you soo much for the help, much appreciated!
import pyzed.sl as sl
import ogl_viewer.viewer as gl
import numpy as np
debug = True
show_image = False
timing = True
init = sl.InitParameters()
camera = sl.Camera()
if not camera.is_opened():
print("Opening ZED Camera...")
status = camera.open(init)
if status != sl.ERROR_CODE.SUCCESS:
print(repr(status))
exit(0)
mat = sl.Mat()
runtime = sl.RuntimeParameters()
init = sl.InitParameters()
fps=0
resolution=sl.RESOLUTION.SVGA
init.camera_resolution = resolution
#init.camera_resolution = sl.RESOLUTION.RESOLUTION_HD1080
init.camera_fps = fps # The framerate is lowered to avoid any USB3 bandwidth issues
init.coordinate_units = sl.UNIT.METER
init.coordinate_system = sl.COORDINATE_SYSTEM.RIGHT_HANDED_Y_UP
camera = sl.Camera()
cam_status = camera.open(init)
print(cam_status)
detection_confidence = 40
# def emergency_braking(image, camera, detection_confidence = 40):
# Set initialization parameters
detection_parameters = sl.ObjectDetectionParameters()
detection_parameters.enable_tracking = True # Objects will keep the same ID between frames
detection_parameters.enable_segmentation = True # Outputs 2D masks over detected objects
# Set runtime parameters
obj_runtime_param = sl.ObjectDetectionRuntimeParameters()
obj_runtime_param.object_class_filter = [sl.OBJECT_CLASS.VEHICLE, sl.OBJECT_CLASS.PERSON]
obj_runtime_param.object_class_detection_confidence_threshold[sl.OBJECT_CLASS.PERSON] = detection_confidence
obj_runtime_param.object_class_detection_confidence_threshold[sl.OBJECT_CLASS.VEHICLE] = detection_confidence
# choose a detection model
detection_parameters.detection_model = sl.OBJECT_DETECTION_MODEL.MULTI_CLASS_BOX_FAST
if detection_parameters.enable_tracking :
# Set positional tracking parameters
positional_tracking_parameters = sl.PositionalTrackingParameters()
# Enable positional tracking
camera.enable_positional_tracking(positional_tracking_parameters)
# Enable object detection with initialization parameters
zed_error = camera.enable_object_detection(detection_parameters)
if zed_error != sl.ERROR_CODE.SUCCESS:
print("enable_object_detection", zed_error, "\nExit program.")
camera.close()
exit(-1)
# Viewer:
if show_image:
camera_info = camera.get_camera_information()
viewer = gl.GLViewer()
viewer.init(camera_info.camera_configuration.calibration_parameters.left_cam, detection_parameters.enable_tracking)
objects = sl.Objects() # Structure containing all the detected objects
prev = 0
import time
# while viewer.is_available():
start=time.time()
for i in range(100):
s=time.time()
if camera.grab(runtime) == sl.ERROR_CODE.SUCCESS:
print(time.time()-s)
camera.retrieve_image(mat, sl.VIEW.LEFT)
print(time.time()-s)
camera.retrieve_objects(objects, obj_runtime_param) # Retrieve the detected objects
print(time.time()-s)
if timing:
timestamp = camera.get_timestamp(sl.TIME_REFERENCE.CURRENT) # Get the timestamp at the time the image was captured
print(timestamp.get_milliseconds()-prev)
prev=timestamp.get_milliseconds()
if show_image:
viewer.update_view(mat, objects)
if debug:
for object in objects.object_list:
print(object.id,object.position,object.velocity)
print(time.time()-s)
print()
print(time.time()-start)
print("FPS: ", 100/(time.time()-start))
jetson_clocks is just a command to run in a terminal. It will increase your performance a little.
If your final app will not need to run faster than 30 fps, you should not ask the zed x to run at 120 because it will consume more CPU in the background. You’ll free some resources by lowering the target FPS to 60 or even 30.
The Orin NX 16 is supposed to be able to run 2 ZED X, so your assumptions are right and if there is a performance issue, we will solve it
About your code:
Can you try to remove the viewer? It’s probably only there for debug
Can you try to enable set_as_static in the PositionalTrackingParameters?
