I’m encountering an issue while trying to receive streams from multiple cameras and serve them on a web interface using Flask.
My objectives are:
Ensure that the streaming runs smoothly and continuously without any interruptions.
Guarantee that if one camera fails, it does not affect the streaming of the other cameras.
My current approach involves receiving the streams and serving them via Flask on the same laptop, which can be accessed by anyone on the same network.
Here’s what I’ve tried so far:
Multiprocessing: This method handles failures gracefully, but I experience a brief glitch (like a blink) in all camera streams every 10 seconds.
Sequential code or threading: This method provides perfect streaming, but if one camera fails, all streams stop for approximately 15 seconds before resuming, This occurs regardless of whether there are 2 or more cameras.
Could you please advise on how to resolve these issues? I’m looking for a solution within the existing approach, as I do not have the time to explore entirely different methods.
Hi @ashfaq
we do not provide official plugins for Flask, so we cannot provide you with the correct reply to solve your problem.
In my opinion, when using multiple cameras, a multi-threading approach is the correct solution to make the system stable and fail-proof, but you must be sure that everything is correctly synchronized.
A wrong synchronization mechanism probably causes the glitch that you are facing every 10 seconds.
I tried the multi camera code which is on ur website and once disconnect one cam. Both is stopped and second one is resumed again only frame in each 5 s3conds mainly ???
This means the single failure isnt handled 100% on ur samples, can assist only on ur code u provided on the website ?
########################################################################
#
# Copyright (c) 2020, STEREOLABS.
#
# All rights reserved.
#
# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
# "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
# LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
# A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
# OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
# SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
# LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
# DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
# THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
# (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
# OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
#
########################################################################
"""
Multi cameras sample showing how to open multiple ZED in one program
"""
import pyzed.sl as sl
import cv2
import numpy as np
import threading
import time
import signal
zed_list = []
left_list = []
depth_list = []
timestamp_list = []
thread_list = []
stop_signal = False
def signal_handler(signal, frame):
global stop_signal
stop_signal=True
time.sleep(0.5)
exit()
def grab_run(index):
global stop_signal
global zed_list
global timestamp_list
global left_list
global depth_list
runtime = sl.RuntimeParameters()
while not stop_signal:
err = zed_list[index].grab(runtime)
if err == sl.ERROR_CODE.SUCCESS:
zed_list[index].retrieve_image(left_list[index], sl.VIEW.LEFT)
zed_list[index].retrieve_measure(depth_list[index], sl.MEASURE.DEPTH)
timestamp_list[index] = zed_list[index].get_timestamp(sl.TIME_REFERENCE.CURRENT).data_ns
time.sleep(0.001) #1ms
zed_list[index].close()
def main():
global stop_signal
global zed_list
global left_list
global depth_list
global timestamp_list
global thread_list
signal.signal(signal.SIGINT, signal_handler)
print("Running...")
init = sl.InitParameters()
init.camera_resolution = sl.RESOLUTION.HD720
init.camera_fps = 30 # The framerate is lowered to avoid any USB3 bandwidth issues
#List and open cameras
name_list = []
last_ts_list = []
cameras = sl.Camera.get_device_list()
index = 0
for cam in cameras:
init.set_from_serial_number(cam.serial_number)
name_list.append("ZED {}".format(cam.serial_number))
print("Opening {}".format(name_list[index]))
zed_list.append(sl.Camera())
left_list.append(sl.Mat())
depth_list.append(sl.Mat())
timestamp_list.append(0)
last_ts_list.append(0)
status = zed_list[index].open(init)
if status != sl.ERROR_CODE.SUCCESS:
print(repr(status))
zed_list[index].close()
index = index +1
#Start camera threads
for index in range(0, len(zed_list)):
if zed_list[index].is_opened():
thread_list.append(threading.Thread(target=grab_run, args=(index,)))
thread_list[index].start()
#Display camera images
key = ''
while key != 113: # for 'q' key
for index in range(0, len(zed_list)):
if zed_list[index].is_opened():
if (timestamp_list[index] > last_ts_list[index]):
cv2.imshow(name_list[index], left_list[index].get_data())
x = round(depth_list[index].get_width() / 2)
y = round(depth_list[index].get_height() / 2)
err, depth_value = depth_list[index].get_value(x, y)
if np.isfinite(depth_value):
print("{} depth at center: {}MM".format(name_list[index], round(depth_value)))
last_ts_list[index] = timestamp_list[index]
key = cv2.waitKey(10)
cv2.destroyAllWindows()
#Stop the threads
stop_signal = True
for index in range(0, len(thread_list)):
thread_list[index].join()
print("\nFINISH")
if __name__ == "__main__":
main()
Our code examples shared on our Github are code samples. They are meant to be used to get started with using the ZED SDK, and are by no means designed or meant for production applications.
There are many comments I can make about this sample regarding it not being a production-ready application, but generally speaking:
error codes must be checked. Here only the successful error code of the camera is handled, the application should behave different if the camera returns an error
What I would recommend is to either write the sample in C++, which provides really multi-threading, or make the sample sequential as you’ve stated before, and handle the errors from the ZED SDK on a failure.
I believe the cameras hang for 15s after a camera is disconnected, because the grab() call is blocking after a failure, as it runs its automatic recovery process. You can change this by setting this parameter to True: async_grab_camera_recovery
ashfaq: