Distorted output from manual made sl.Mat()

Software Python
1

Hi there,

I am creating a manual sl.Mat() from a numpy array that corresponds to a color image. Everything looks fine however when I try to visualize the content of the sl.Mat() the output is distorted. I guess it has with the order of the pixel values or something but I am not quite sure how to resolve it. Any idea why this could be happening?

The code snippet I use is the following:

color_image = np.asarray(my_array_data) # this gives me a 480x640x3 matrix
tmp = sl.Mat(color_image.shape[1], color_image.shape[0],  sl.MAT_TYPE.U8_C4, sl.MEM.CPU)
tmp_np = tmp.get_data() # tmp_np and tmp are sharing the same pointer so I can change the data in tmp through the get_data()
tmp_np = color_image

cv2.imshow("original image", color_image)
cv2.imshow("sl image", tmp.get_data())

and the output is the following:

image
image1279×483 84.6 KB

2

Hi @ttsesm,

Your matrix type seems wrong here, instead of sl.MAT_TYPE.U8_C4, it should be sl.MAT_TYPE.U8_C3 according to the comment above.

3

Hi @mattrouss,

Well what you are suggesting it makes sense. However, it seems to be related with something else.

Actually to give you a better understanding I am having a for loop from where I am retrieving multiple RGB images from 3 intel realsense cameras. So I am looping through each device to retrieve the color (and later the depth images) and then for the created color image I want to create a corresponding sl.Mat(). However, for some reason I cannot get the correct output in the sl.Mat() matrix. I think that it might be related to the memory sharing or something :thinking:

Here is the loop:

if any(available_devices):
    frames_devices = device_manager.poll_frames()
    for (device_info, frame) in frames_devices.items():
        device = device_info[0]  # serial number
        color_image = np.asarray(frame[rs.stream.color].get_data())
        
        tmp = sl.Mat(color_image.shape[1], color_image.shape[0],  sl.MAT_TYPE.U8_C3, sl.MEM.CPU)
        tmp_np = tmp.get_data()
        tmp_np = np.copy(test_image)
                  
        # Visualise the results
        cv2.imshow("Intel_{}_{}".format(device_info[1], device_info[0]), color_image)
        cv2.imshow("Intel_{}_{}_sl".format(device_info[1], device_info[0]), tmp.get_data())

    key = cv2.waitKey(0)

Even with only one camera is not working and when I compare the two matrices color_image == tmp.get_data() I am getting a False which means that my two images are not the same.