Minimal Configurations for lower CPU & GPU usage and better performance ZED X Mini, Jetson Orin AGX

karmenlu · August 3, 2023, 6:58pm

Running into performance issues when running ZED X Mini on the Jetson Orin. Using SDK version 4.0.1, what are configurations that I could set (perhaps in the common.yaml) that could lower CPU usage and GPU usage. For example, how can we turn off all depth computation that ZED X Mini may be running on the Orin?

Myzhar · August 4, 2023, 8:16am

Hi @karmenlu

can you please provide more information (number and benchmarking methods) regarding this?

karmenlu · August 4, 2023, 10:05am

Hi @Myzhar,

Thanks for getting back to us!

Our perception stack uses images published by the ZED X Mini to find targets. The performance of the ZED has been measured against the same perception stack running with a different camera. The key indicator that ZED-related performance issues are present include that the ZED/Orin/perception setup crashes/disconnected while the other camera/Orin/perception setup connects properly and runs without disconnecting. Crashing / freezing led us to look into ZED performance. We have been measuring with jtop, the Jetson Stats tool. By default, ZED X Mini has been running with the Depth Quality setting of Ultra. (Ideally, we would want to set Depth Quality to 0/None. Running into issues discussed here). In the logs before, key columns to compare include the temperature, power, and GPU. We believe that minimizing power and compute usage by the ZED would prevent the overheat/crashing of our ZED testbench setup.

ZED X Mini Running w/ Depth Quality Ultra:

time	uptime	CPU1	CPU2	CPU3	CPU4	CPU5	CPU6	CPU7	CPU8	CPU9	CPU10	CPU11	CPU12	RAM	GPU	APE	DLA0_CORE	DLA0_FALCON	DLA1_CORE	DLA1_FALCON	NVDEC	NVENC	NVJPG	NVJPG1	PVA0_CPU_AXI	PVA0_VPS	SE	VIC	Fan pwmfan0	Temp CPU	Temp CV0	Temp CV1	Temp CV2	Temp GPU	Temp SOC0	Temp SOC1	Temp SOC2	Temp Tboard	Temp Tdiode	Temp tj	Power tj	Power TOT	jetson_clocks	nvp model
2023-08-02 19:24:38	0:06:09	26	25	24	24	25	24	26	24	23	24	24	23	0.2774543256	98.9	OFF	OFF	OFF	OFF	OFF	OFF	OFF	OFF	OFF	OFF	OFF	473600	115200	61.17647059	76.187	-256	-256	-256	72.375	71.468	71.187	70.968	61	64.5	76.187	9359	41406	OFF	MAXN
2023-08-02 19:24:38	0:06:09	26	25	24	24	25	24	26	24	23	24	24	23	0.2774543256	98.9	OFF	OFF	OFF	OFF	OFF	OFF	OFF	OFF	OFF	OFF	OFF	473600	115200	61.17647059	76.187	-256	-256	-256	72.375	71.468	71.187	70.968	61	64.5	76.187	9359	41406	OFF	MAXN
2023-08-02 19:24:39	0:06:10	27	29	24	39	23	36	37	68	50	32	36	37	0.2782506435	98.7	OFF	OFF	OFF	OFF	OFF	OFF	OFF	OFF	OFF	OFF	OFF	473600	115200	61.17647059	75.937	-256	-256	-256	72.906	71.531	71.187	70.937	61	64.75	75.937	9359	41082	OFF	MAXN
2023-08-02 19:24:40	0:06:11	39	29	27	26	21	34	25	72	27	30	39	29	0.2782028005	95.3	OFF	OFF	OFF	OFF	OFF	OFF	OFF	OFF	OFF	OFF	OFF	473600	115200	61.17647059	75.718	-256	-256	-256	72.562	71.5	71.156	70.937	61	64.5	75.718	9359	41103	OFF	MAXN
2023-08-02 19:24:41	0:06:12	33	32	30	28	38	31	30	37	73	20	34	24	0.2781731225	99.1	OFF	OFF	OFF	OFF	OFF	OFF	OFF	OFF	OFF	OFF	OFF	473600	115200	61.17647059	76.218	-256	-256	-256	72.781	71.5	71.125	71.125	61	64.5	75.812	9359	41324	OFF	MAXN
2023-08-02 19:24:42	0:06:13	34	29	32	28	29	17	22	31	77	35	26	33	0.2782475734	96.3	OFF	OFF	OFF	OFF	OFF	OFF	OFF	OFF	OFF	OFF	OFF	473600	115200	61.17647059	76.218	-256	-256	-256	72.531	71.531	71.187	71.062	61	64.5	76.218	9258	40804	OFF	MAXN
2023-08-02 19:24:43	0:06:14	34	29	32	30	35	31	37	35	41	20	71	32	0.278165703	83.4	OFF	OFF	OFF	OFF	OFF	OFF	OFF	OFF	OFF	OFF	OFF	473600	115200	61.17647059	75.968	-256	-256	-256	72.656	71.406	71.156	70.968	62	64.25	75.968	9158	40195	OFF	MAXN
2023-08-02 19:24:44	0:06:15	33	29	30	31	31	31	32	34	29	16	48	59	0.278215209	99.1	OFF	OFF	OFF	OFF	OFF	OFF	OFF	OFF	OFF	OFF	OFF	473600	115200	61.17647059	75.687	-256	-256	-256	72.906	71.406	71.406	71.156	62	64.5	76	9258	40565	OFF	MAXN
2023-08-02 19:24:45	0:06:16	34	31	29	29	30	33	46	30	27	84	18	24	0.2783069295	98.8	OFF	OFF	OFF	OFF	OFF	OFF	OFF	OFF	OFF	OFF	OFF	473600	115200	61.17647059	75.906	-256	-256	-256	72.593	71.562	71.343	71.093	62	64.5	75.906	9258	41022	OFF	MAXN

Other Camera:

time	uptime	CPU1	CPU2	CPU3	CPU4	CPU5	CPU6	CPU7	CPU8	CPU9	CPU10	CPU11	CPU12	RAM	GPU	APE	DLA0_CORE	DLA0_FALCON	DLA1_CORE	DLA1_FALCON	NVDEC	NVENC	NVJPG	NVJPG1	PVA0_CPU_AXI	PVA0_VPS	SE	VIC	Fan pwmfan0	Temp CPU	Temp CV0	Temp CV1	Temp CV2	Temp GPU	Temp SOC0	Temp SOC1	Temp SOC2	Temp Tboard	Temp Tdiode	Temp tj	Power tj	Power TOT	jetson_clocks	nvp model
2023-08-02 21:24:50	0:12:49	16	14	13	14	13	13	13	14	13	14	13	14	0.3111295092	42.7	OFF	OFF	OFF	OFF	OFF	OFF	OFF	OFF	OFF	OFF	OFF	473600	OFF	30.19607843	56.156	-256	-256	-256	52.187	51.968	51.031	52.218	42	45.25	56.156	8291	34789	OFF	MAXN
2023-08-02 21:24:50	0:12:49	16	14	13	14	13	13	13	14	13	14	13	14	0.3111295092	42.7	OFF	OFF	OFF	OFF	OFF	OFF	OFF	OFF	OFF	OFF	OFF	473600	OFF	30.19607843	56.156	-256	-256	-256	52.187	51.968	51.031	52.218	42	45.25	56.156	8291	34789	OFF	MAXN
2023-08-02 21:24:51	0:12:50	41	52	45	53	28	51	36	71	33	40	40	32	0.3107495791	99	OFF	OFF	OFF	OFF	OFF	OFF	OFF	OFF	OFF	OFF	OFF	473600	OFF	30.19607843	55.968	-256	-256	-256	54.468	52.156	50.968	52.75	42	45	55.968	8696	37348	OFF	MAXN
2023-08-02 21:24:52	0:12:51	29	33	29	47	25	98	18	22	45	27	42	57	0.311450467	53.3	OFF	OFF	OFF	OFF	OFF	OFF	OFF	OFF	OFF	OFF	OFF	473600	OFF	30.19607843	56.031	-256	-256	-256	52.531	52.031	51.093	52.281	42	45.25	56.031	8291	35278	OFF	MAXN
2023-08-02 21:24:53	0:12:52	35	46	43	31	32	73	59	33	44	20	61	41	0.3117581209	99.4	OFF	OFF	OFF	OFF	OFF	OFF	OFF	OFF	OFF	OFF	OFF	473600	OFF	30.19607843	56.031	-256	-256	-256	55.125	52.25	51.062	52.531	42	45	56.031	8595	36853	OFF	MAXN
2023-08-02 21:24:54	0:12:53	31	37	44	37	32	32	56	76	48	26	21	52	0.3123647298	60.9	OFF	OFF	OFF	OFF	OFF	OFF	OFF	OFF	OFF	OFF	OFF	473600	OFF	30.19607843	56.656	-256	-256	-256	52.656	52.031	51.062	52.531	42	45	56.656	8291	35579	OFF	MAXN
2023-08-02 21:24:55	0:12:54	56	60	40	37	48	63	39	96	34	32	37	37	0.3138743444	99.4	OFF	OFF	OFF	OFF	OFF	OFF	OFF	OFF	OFF	OFF	OFF	473600	OFF	30.19607843	57.718	-256	-256	-256	54.031	52.562	51.187	52.75	42	45.25	57.625	8595	38140	OFF	MAXN
2023-08-02 21:24:56	0:12:55	50	51	56	43	97	44	28	39	39	43	38	38	0.3127868744	73.4	OFF	OFF	OFF	OFF	OFF	OFF	OFF	OFF	OFF	OFF	OFF	473600	OFF	30.19607843	56.437	-256	-256	-256	53.218	52.5	51.281	52.625	42	45	56.437	8392	35774	OFF	MAXN
2023-08-02 21:24:57	0:12:56	77	34	38	45	33	44	26	31	39	37	41	87	0.3128693845	99.2	OFF	OFF	OFF	OFF	OFF	OFF	OFF	OFF	OFF	OFF	OFF	473600	OFF	30.19607843	56.25	-256	-256	-256	54.343	52.843	51.5	53.062	42	45.75	56.25	8696	37844	OFF	MAXN

alassagne · August 7, 2023, 12:09pm

Hi, what is your other camera ? Does it run depth computing ?
Also, could you publish the code you run so I can check what can be using your CPU ?

karmenlu · August 7, 2023, 3:24pm

Other camera: Realsense. Runs depth computation too.

Our code runs deep learning algorithms on top of ZED data. Stepping aside from our code, the ZED X Mini is an edge device and shouldn’t be consuming too much of the Orin’s compute anyway.
Below are jtop logs with ZED running on the Orin without our code at different depth qualities. The difference between GPU usage on 0 and GPU usage on 3 looks great! Any configuration suggestions for lowering CPU, Temperature, and Power that would be compatible with either/both depth quality settings?

0/None:

time	uptime	CPU1	CPU2	CPU3	CPU4	CPU5	CPU6	CPU7	CPU8	CPU9	CPU10	CPU11	CPU12	RAM	GPU	APE	DLA0_CORE	DLA0_FALCON	DLA1_CORE	DLA1_FALCON	NVDEC	NVENC	NVJPG	NVJPG1	PVA0_CPU_AXI	PVA0_VPS	SE	VIC	Fan pwmfan0	Temp CPU	Temp CV0	Temp CV1	Temp CV2	Temp GPU	Temp SOC0	Temp SOC1	Temp SOC2	Temp Tboard	Temp Tdiode	Temp tj	Power tj	Power TOT	jetson_clocks	nvp model
2023-08-07 11:01:39	0:19:11	19	16	17	14	16	16	16	16	18	16	16	17	0.1063119817	13.6	OFF	OFF	OFF	OFF	OFF	OFF	OFF	OFF	OFF	OFF	OFF	473600	115200	27.45098039	53.562	-256	-256	-256	48.031	49.375	49.781	48.531	41	42	53.25	5846	13548	OFF	MAXN
2023-08-07 11:01:39	0:19:11	19	16	17	14	16	16	16	16	18	16	16	17	0.1063119817	13.6	OFF	OFF	OFF	OFF	OFF	OFF	OFF	OFF	OFF	OFF	OFF	473600	115200	27.45098039	53.562	-256	-256	-256	48.031	49.375	49.781	48.531	41	42	53.25	5846	13548	OFF	MAXN
2023-08-07 11:01:40	0:19:12	30	23	21	25	32	14	15	14	30	28	30	27	0.1067281139	0	OFF	OFF	OFF	OFF	OFF	OFF	OFF	OFF	OFF	OFF	OFF	473600	115200	27.45098039	52.843	-256	-256	-256	48.031	49.375	49.781	48.5	41	42	52.843	5846	13319	OFF	MAXN
2023-08-07 11:01:41	0:19:13	29	20	19	19	18	18	13	17	26	25	25	25	0.10676099	0	OFF	OFF	OFF	OFF	OFF	OFF	OFF	OFF	OFF	OFF	OFF	473600	115200	27.45098039	52.531	-256	-256	-256	48.125	49.406	49.812	48.468	41	42	52.531	5745	12755	OFF	MAXN
2023-08-07 11:01:42	0:19:14	25	25	24	24	23	23	23	25	21	23	20	24	0.1067448717	0	OFF	OFF	OFF	OFF	OFF	OFF	OFF	OFF	OFF	OFF	OFF	473600	115200	27.45098039	52.625	-256	-256	-256	48.062	49.593	50	48.5	41	42	52.718	5745	12754	OFF	MAXN
2023-08-07 11:01:43	0:19:15	34	20	20	26	23	22	25	24	29	26	23	23	0.1067584315	0	OFF	OFF	OFF	OFF	OFF	OFF	OFF	OFF	OFF	OFF	OFF	473600	115200	27.45098039	52.656	-256	-256	-256	47.906	49.343	50.093	48.468	41	42	52.656	5745	12989	OFF	MAXN
2023-08-07 11:01:44	0:19:16	29	21	20	16	29	25	26	26	25	21	24	23	0.1068635839	0	OFF	OFF	OFF	OFF	OFF	OFF	OFF	OFF	OFF	OFF	OFF	473600	115200	27.45098039	52.687	-256	-256	-256	48.125	49.593	49.906	48.562	41	42	52.812	5745	13679	OFF	MAXN
2023-08-07 11:01:45	0:19:17	29	27	26	22	17	23	23	22	20	25	21	26	0.1068840515	0	OFF	OFF	OFF	OFF	OFF	OFF	OFF	OFF	OFF	OFF	OFF	473600	115200	27.45098039	52.687	-256	-256	-256	48.031	49.406	49.906	48.531	41	42	52.687	5745	12757	OFF	MAXN
2023-08-07 11:01:46	0:19:18	28	26	24	23	23	19	24	23	20	23	28	22	0.1068493845	0	OFF	OFF	OFF	OFF	OFF	OFF	OFF	OFF	OFF	OFF	OFF	473600	115200	27.45098039	52.781	-256	-256	-256	48.125	49.312	50.125	48.5	41	42	52.781	5745	12755	OFF	MAXN
2023-08-07 11:01:47	0:19:19	31	23	24	25	24	22	22	22	19	19	20	22	0.1068990184	0	OFF	OFF	OFF	OFF	OFF	OFF	OFF	OFF	OFF	OFF	OFF	473600	115200	27.45098039	52.625	-256	-256	-256	47.968	49.343	50.031	48.468	41	42	53.031	5745	12755	OFF	MAXN
2023-08-07 11:01:48	0:19:20	31	27	19	20	21	24	23	26	21	20	21	23	0.1069284406	0	OFF	OFF	OFF	OFF	OFF	OFF	OFF	OFF	OFF	OFF	OFF	473600	115200	27.45098039	52.562	-256	-256	-256	47.906	49.468	49.968	48.593	41	42	52.562	5745	12755	OFF	MAXN
2023-08-07 11:01:49	0:19:21	24	23	22	21	21	20	25	25	29	27	25	27	0.1068797021	0	OFF	OFF	OFF	OFF	OFF	OFF	OFF	OFF	OFF	OFF	OFF	473600	115200	27.45098039	52.656	-256	-256	-256	47.968	49.656	50.062	48.5	41	42	52.656	5745	12754	OFF	MAXN
2023-08-07 11:01:50	0:19:22	27	22	19	25	21	19	20	19	24	22	25	27	0.1068857145	0	OFF	OFF	OFF	OFF	OFF	OFF	OFF	OFF	OFF	OFF	OFF	473600	115200	27.45098039	52.593	-256	-256	-256	47.875	49.5	50.062	48.531	41	42.25	52.906	5745	12755	OFF	MAXN

3/Ultra:

time	uptime	CPU1	CPU2	CPU3	CPU4	CPU5	CPU6	CPU7	CPU8	CPU9	CPU10	CPU11	CPU12	RAM	GPU	APE	DLA0_CORE	DLA0_FALCON	DLA1_CORE	DLA1_FALCON	NVDEC	NVENC	NVJPG	NVJPG1	PVA0_CPU_AXI	PVA0_VPS	SE	VIC	Fan pwmfan0	Temp CPU	Temp CV0	Temp CV1	Temp CV2	Temp GPU	Temp SOC0	Temp SOC1	Temp SOC2	Temp Tboard	Temp Tdiode	Temp tj	Power tj	Power TOT	jetson_clocks	nvp model
2023-08-07 11:03:59	0:21:09	34	28	27	25	19	18	21	23	23	24	24	27	0.1289226821	43.2	OFF	OFF	OFF	OFF	OFF	OFF	OFF	OFF	OFF	OFF	OFF	473600	115200	27.45098039	54.312	-256	-256	-256	49.218	50.343	50.718	49.406	42	43	53.875	5745	13913	OFF	MAXN
2023-08-07 11:03:59	0:21:09	34	28	27	25	19	18	21	23	23	24	24	27	0.1289226821	43.2	OFF	OFF	OFF	OFF	OFF	OFF	OFF	OFF	OFF	OFF	OFF	473600	115200	27.45098039	54.312	-256	-256	-256	49.218	50.343	50.718	49.406	42	43	53.875	5745	13913	OFF	MAXN
2023-08-07 11:03:59	0:21:32	20	17	17	15	17	16	16	16	18	17	16	17	0.1319689025	43.9	OFF	OFF	OFF	OFF	OFF	OFF	OFF	OFF	OFF	OFF	OFF	473600	115200	27.45098039	54.406	-256	-256	-256	49.093	50.312	50.625	49.468	42	43	54.125	6652	16458	OFF	MAXN
2023-08-07 11:04:00	0:21:33	37	32	27	22	21	25	25	19	31	25	28	29	0.132158228	21.3	OFF	OFF	OFF	OFF	OFF	OFF	OFF	OFF	OFF	OFF	OFF	473600	115200	27.45098039	53.843	-256	-256	-256	49.093	50.468	50.843	49.468	42	43	54	6552	15346	OFF	MAXN
2023-08-07 11:04:01	0:21:34	33	26	25	24	29	24	31	27	27	26	28	24	0.1321716598	21.3	OFF	OFF	OFF	OFF	OFF	OFF	OFF	OFF	OFF	OFF	OFF	473600	115200	27.45098039	53.812	-256	-256	-256	49.281	50.281	50.718	49.406	42	43	53.812	6552	15349	OFF	MAXN
2023-08-07 11:04:02	0:21:35	29	25	24	19	26	30	27	23	27	26	25	23	0.132125096	21.8	OFF	OFF	OFF	OFF	OFF	OFF	OFF	OFF	OFF	OFF	OFF	473600	115200	27.45098039	53.687	-256	-256	-256	49.062	50.437	50.843	49.437	42	43	53.718	6552	15117	OFF	MAXN
2023-08-07 11:04:03	0:21:36	31	27	27	22	26	27	26	32	29	28	25	27	0.1322114437	32.9	OFF	OFF	OFF	OFF	OFF	OFF	OFF	OFF	OFF	OFF	OFF	473600	115200	27.45098039	53.781	-256	-256	-256	49.125	50.531	50.968	49.343	42	43	53.937	6552	15116	OFF	MAXN
2023-08-07 11:04:04	0:21:37	31	23	24	20	27	29	26	34	22	26	24	28	0.1321344344	42.8	OFF	OFF	OFF	OFF	OFF	OFF	OFF	OFF	OFF	OFF	OFF	473600	115200	27.45098039	54	-256	-256	-256	49.062	50.375	50.625	49.437	42	43	53.906	6451	15015	OFF	MAXN
2023-08-07 11:04:05	0:21:38	30	27	26	19	31	27	30	28	23	22	27	24	0.1322214217	21.3	OFF	OFF	OFF	OFF	OFF	OFF	OFF	OFF	OFF	OFF	OFF	473600	115200	27.45098039	53.781	-256	-256	-256	49.218	50.343	50.75	49.375	42	43	54.187	6552	15349	OFF	MAXN
2023-08-07 11:04:06	0:21:39	30	24	23	28	29	28	34	29	27	28	26	26	0.132277196	21.3	OFF	OFF	OFF	OFF	OFF	OFF	OFF	OFF	OFF	OFF	OFF	473600	115200	27.45098039	53.843	-256	-256	-256	49.25	50.5	50.968	49.437	42	43	54.125	6552	15447	OFF	MAXN
2023-08-07 11:04:07	0:21:40	27	21	22	19	29	30	30	30	28	25	26	28	0.132262229	29.9	OFF	OFF	OFF	OFF	OFF	OFF	OFF	OFF	OFF	OFF	OFF	473600	115200	27.45098039	53.843	-256	-256	-256	49.25	50.5	50.812	49.593	42	43	53.843	6552	15349	OFF	MAXN
2023-08-07 11:04:08	0:21:41	24	23	16	17	29	31	29	28	32	27	33	30	0.1322617173	32.7	OFF	OFF	OFF	OFF	OFF	OFF	OFF	OFF	OFF	OFF	OFF	473600	115200	27.45098039	54.312	-256	-256	-256	49.437	50.406	50.968	49.531	42	43	54.437	6552	15447	OFF	MAXN
2023-08-07 11:04:09	0:21:42	24	21	17	18	29	31	25	29	28	31	30	27	0.1322816732	24.1	OFF	OFF	OFF	OFF	OFF	OFF	OFF	OFF	OFF	OFF	OFF	473600	115200	27.45098039	54.187	-256	-256	-256	49.343	50.531	50.781	49.406	42	43	54.031	6552	15217	OFF	MAXN

alassagne · August 7, 2023, 4:49pm

Hi,
Intel Realsense works quite differently than a purely passive camera. One of the advantage are indeed a lower compute, it’s normal that the ZED takes more compute since it does not run “simple” disparity algorithm but neural networks too.

However, ULTRA should be quite light for an AGX. In our benchmarks, an AGX can run 4 cameras at 15 FPS or 2 cameras at 15. We work on improving that, we want the AGX to run 4 cameras with more power available.

Can you share the code you use for your benchmark ? What module do you use apart Depth ?
Also, be aware that you can cap the FPS of the SDK so that it does not take up all the power available. Otherwise, it will try to reach the target framerate.

karmenlu · August 7, 2023, 10:26pm

Hi,

Is it possible to turn the neural networks running on the ZED off? We are very interested in simplifying whatever is running on the ZED. The benchmark code is simply jtop exported to csv. We used jtop_logger.py here.

We have been using the common.yaml pulled from the Stereolabs GitHub. Only changing Depth Quality between 0, 1, and 3. Unable to get 0 to work without ASYNC/RGB warnings.

alassagne · August 8, 2023, 8:51am

If you set the depth mode NONE, you should run at full speed. I’ve seen you get a few issues, that are related to ROS – I suggest you don’t use ROS for a benchmark, it’s more accurate to use directly the C++.

karmenlu · August 8, 2023, 1:05pm

Okay, great. I can try running with a lower fps and the max speed. Is the lower speed that I have been running with the probable cause for the ASYNC/RGB warning?

alassagne · August 8, 2023, 2:47pm

I don’t know what errors you encountered. Can you share these warnings ?