Accessing GstZedSrcMeta from Buffer (CPP)

Software C++
1

How do you access GstZedSrcMeta from the buffer?

The documentation for metadata at stereolabs.com says to reference the Zeddatacsvsink for an example. I copied how the gst_zeddatacsvsink_render() function maps the buffer to the ZedSrcMeta. However, when I do so, I only get uninitialized data (sometimes 0, sometimes -1255031952, etc). I am working in a .cpp script that rebuilds the example pipeline from GStreamer - ZED Sensors CSV Sink - Stereolabs with a pad probe onto the zedsrc element’s src pad. The following is a condensed version of the probe:

include <gst/zed/gstzedmeta.h>

static GstPadProbeReturn zed_metadata_probe(GstPad *pad, GstPadProbeInfo *info, gpointer u_data) {
    GstBuffer *buf = (GstBuffer *) info->buffer; 
    GstMapInfo map_in;

    if (gst_buffer_map(buf, &map_in, GST_MAP_READ)) {
        GstZedSrcMeta *meta = (GstZedSrcMeta *) info->data;
        g_print("%d %d %d\n", meta->info.cam_model, meta->info.stream_type, meta->pose.pose_avail);
        gst_buffer_unmap(buf, &map_in);
    }

    return GST_PAD_PROBE_OK;
}

I know the GstZedSrcMeta is available and correct because I have it linked into a zeddatacsvsink element which shows valid data. I referenced the zed-gstreamer source code https://github.com/stereolabs/zed-gstreamer/blob/master/gst-zed-meta/gstzedmeta.h to try to utilize the functions defined to get the metadata but was also unable to. For example, gst_buffer_get_zed_src_meta(b). It may likely I am referencing this function wrong.

I have added these lines to me CMakeLists.txt:

find_package(ZED 3.7 REQUIRED)
include_directories(${ZED_INCLUDE_DIRS})
link_directories(${ZED_LIBRARY_DIR})
target_link_libraries(${PROJECT_NAME} ${ZED_LIBRARIES})

But compiling results in:

...undefined reference to 'gst_zed_src_meta_api_get_type'
collect2: error: ld returned 1 exit status

Any help or an example of how to access it through a pad probe would be great!

Specifications:

  • NVidia Jetson Xavier NX
  • Jetpack 4.6.0
  • ZED SDK 3.7.2
  • Zed-Gstreamer (master, latest)
  • Ubuntu 18.04 LTS
2

Hi @penguin
you can find useful code to access the metadata in the OD overlay element plugin:

Regarding the linking error, have you linked the GStreamer library?

3

The linking suggestion solved it. I found libgstzedmeta.so in /usr/lib/aarch64-linux-gnu/, so I added the line to my CMakelists.txt (inserting into the list of other libraries in the target_link_libraries call):

target_link_libraries( ... /usr/lib/aarch64-linux-gnu/libgstzedmeta.so ...)

In case it helps anyone else:
I also had added the dependencies to my CMakelists.txt earlier for the ZED Libraries that I later found out (by accident) are required as well to get the GstZedSrcMeta. Below are required:

find_package(ZED 3.7 REQUIRED)
include_directories(... ${ZED_INCLUDE_DIRS})
link_directories($(ZED_LIBRARY_DIR})
target_link_libraries(... ${ZED_LIBRARIES} ...)

Here are 2 example probes for accessing GstZedSrcMeta. 2 different ones are required in case you want to probe the buffer from zeddemux's src_data pad.

Probing a buffer that contains both image/depth data and zedmeta (ex: zedsrc's src_pad):

include <gst/zed/gstzedmeta.h>

static GstPadProbeReturn zedmeta_zedsrc_probe(GstPad *pad, GstPadProbeInfo *info, gpointer u_data) {
    GstBuffer *buf = (GstBuffer *) info->buffer; 
    GstMapInfo map_buf;

    if (gst_buffer_map(buf, &map_in, GstMapFlags(GST_MAP_READ) == FALSE) {
        g_print("Failed to map buffer\n");
        return GST_PAD_PROBE_OK;
    }

    GstZedSrcMeta *meta = (GstZedSrcMeta *) gst_buffer_get_meta(buf, GST_ZED_SRC_META_API_TYPE);

    //The following also works 
    //GstZedSrcMeta *meta = (GstZedSrcMeta *) gst_buffer_get_zed_src_meta(buf);
    if (meta == NULL) {
        g_print("No meta data\n"); 
        return GST_PAD_PROBE_OK;
    }

   gint model = meta->info.cam_model;

    return GST_PAD_PROBE_OK;
}

If you want to probe the src_data pad of the zeddemux element, you must use the following pad:

include <gst/zed/gstzedmeta.h>

static GstPadProbeReturn zedmeta_src_data_probe(GstPad *pad, GstPadProbeInfo *info, gpointer u_data) {
    GstBuffer *buf = (GstBuffer *) info->buffer; 
    GstMapInfo map_buf;

    if (gst_buffer_map(buf, &map_buf)) {
        GstZedSrcMeta *meta = (GstZedSrcMeta *) map_buf.data;
        gint model = meta->info.cam_model;

        gst_buffer_unmap(buf, &map_buf);
    }

    return GST_PAD_PROBE_OK;
}

When probing the src_data pad of zeddemux, the buffer is only the GstZedSrcMeta. Thus, a simple typecast can be used to access it (the gst_buffer_map call is used to verify its a good buffer and can frankly be removed). Using gst_buffer_get_zed_src_meta on this buffer does not work and will result in the meta returned containing all uninitialized data. Why? I figure it has something to do with how gst_buffer_get_meta works. Maybe it’s a bug, maybe not. Anyways, here’s the solution for either scenario