Found a fix for this, in case anyone else hits upon this issue.
In my case, there are reported issues between UE5, certain RTX series cards and DX12.
I changed the following line in my DefaultEngine.ini
[/Script/WindowsTargetPlatform.WindowsTargetSettings]
DefaultGraphicsRHI=DefaultGraphicsRHI_DX11
;DefaultGraphicsRHI=DefaultGraphicsRHI_DX12
Switching from DX12 down to DX11 - and on the RTX3080, this “fixed” the issue.
It definitely looks to not be a problem with Stereolabs/ZED SDK nor CUDA, but something in Unreal and/or DX12. I’d also like to say a huge thank you to Stereolabs’ support team for their great back-and-forth discussion with me, to narrow down what is or is not the problem!