I am working with ZED Open Capture (sl_oc::video) to build a custom calibration pipeline where a C++ service streams rectified stereo frames to a Java application.
My Workflow:
The system starts in Auto Mode (AECAGC + AWB enabled).
For calibration steps, I need to switch to Manual Mode, set specific Gain/Exposure values, run the process, and then switch back to Auto Mode.
Once back in Auto, the camera begins recalculating Exposure and White Balance based on the new scene lighting.
Here is the problem: There seems to be no public API flag or callback in VideoCapture to indicate when the auto-algorithms have “stabilized” or finished their initial calculation cycle after being re-enabled.
Currently, when I toggle setAECAGC(true) or setAutoWhiteBalance(true):
The registers for Gain/Exposure/WB are updated immediately by the firmware loop.
However, if I read them immediately (e.g., via getGain()), they often still hold the old manual values or unstable intermediate values.
After ~1-2 seconds of frame processing, they finally converge to stable values.
What I’m trying to achieve: I want my Java script to only request/display frames after the camera has finished stabilizing its exposure and white balance, ensuring consistent data during the calibration phase.
So my questions:
Does VideoCapture expose any method like isAutoStabilized() or an event listener for this?
If not, what is the recommended pattern to detect stability?
Should I simply wait a fixed duration (e.g., 1.5s) after enabling auto?
Is it better to force a few getLastFrame() calls and monitor the variance of getGain()/getExposure() until it drops below a threshold?
Is there a specific “frame ready” signal from the FPGA/DSP that indicates the exposure loop has converged?
Any insights or examples of how others handle switching between Auto/Manual modes without introducing artifacts would be greatly appreciated!
Good question, and the short answer is: no, there’s no hidden convergence flag or “frame ready” signal exposed for this.
To clarify what’s happening under the hood: ZED Open Capture talks to the camera over UVC and writes the AEC/AGC and AWB control registers directly. When you call setAECAGC(true) or setAutoWhiteBalance(true), you’re just re-enabling the on-sensor/ISP control loops. That loop runs asynchronously on the camera side, frame by frame, and the library has no back-channel telling it when the loop considers itself “settled.” There’s no isAutoStabilized(), no event listener, and no DSP/FPGA convergence interrupt surfaced through VideoCapture. That’s why reading getGain()/getExposure() immediately after the toggle gives you stale manual values or intermediate ones, the registers are only updated as the loop iterates.
So you’re left with detecting convergence yourself, and your instinct in option 2 is the right one. A fixed wait (1.5s) is fragile because convergence time depends on how far the scene is from the previous manual setpoint and on framerate, so under a big lighting delta you’ll occasionally ship a frame that’s still drifting.
The robust pattern is to poll the control values and watch for them to settle:
After re-enabling auto, grab frames in your normal capture loop and read getGain(), getExposure() (and white balance if you toggle AWB) once per frame.
Keep a short sliding window (e.g. the last N readings) and compute the spread (max−min, or stddev).
Declare “stabilized” when the spread stays below a small threshold for K consecutive frames. Requiring K consecutive hits avoids false positives when the loop momentarily plateaus mid-adjustment.
Add a hard timeout as a safety net so a flickering/changing scene can’t hang you forever.
A couple of practical notes:
Read the registers on a frame cadence, not in a tight CPU loop. The values only change when the firmware processes a new frame, so polling faster than the framerate just burns cycles. Gate each read on a fresh getLastFrame().
Don’t read back the register right after the write expecting the new manual value either, same asynchronous-update reason. When you set manual gain/exposure, give it a frame or two before trusting the read.
If you also want pixel-level confirmation, you can complement the register check by monitoring mean luma (and per-channel means for WB) of the rectified frame over the same window. In practice the register-spread method alone is usually enough and far cheaper.
This is essentially the variance-threshold approach you described in 2b, and it’s what I’d recommend over the fixed delay. It self-adjusts to the magnitude of the scene change instead of guessing a worst-case wait.