hi there. thanks Cpene1 for the super resolution suggestion. That is probably a better route indeed.
I tried the DeepGyro repo (just the DeepBlind version, I was not brave enough to dev the IMU capture from the zed and do the file prep work required to feed the DeepGyro network properly) : it does not work great on real world images and drinks memory like crazy. In the paper, they use “synthetic” blurred images, so not real stuff. That is likely why it does not work as well on real prod use cases.
Then I tried your super resolution idea.I have mitigated results so far.
First observation: super resolution algos/models are sensitive to noise, and the ZED generates quite some noise, at least in indoor conditions and at high refresh rates like the WGA 100fps I tried.
So I decided to denoise the WGA image first using this model. It works quite well based on my limited testing, especially on flat surfaces, but some noise remains on details like edges.
Then I used this super resolution model, that works ok on test images, but not so well on images in the wild, it generates a lot of edges artefacts on my test frames, even after deep denoising from the model above.
Maybe there are better models to try. I ay try on outdoor captured frames as well to limit the noise issue.
Each model takes between 3 to 5GB of mem to infer. Not sure on runtime per frame, but since the models have to be called sequentially, just the model load time is a fps killer.
Maybe you will have better results with opencv.
Just wanted to share, hope that helps.