Grab() stalls with area memory, getting worse with speed

[bajaj10]

One more finding from the car, separate from the GIL issue.

Setup: ZED X Mini, SDK 5.3.1, SVGA@60, GEN_3, enable_area_memory = True, enable_localization_only = True, a saved .area map of the track (about 54 m loop), enable_imu_fusion = True, depth disabled in RuntimeParameters. Jetson Orin, L4T R36.4.7, jetson_clocks on.

Observation: grab() periodically takes 100–220 ms instead of about 16 ms, and the stalls get more frequent at higher speed:

	stalls (grab() > 100 ms)	frames dropped
parked, with map	about once per second, 110–120 ms	about 8%
driving at roughly 75% speed, with map	every 0.2–1 s; in the last 1.6 s before LOST: every 0.2–0.3 s, 150–224 ms	48 stalls and about 920 dropped frames in 26 s; about 77 of 95 frames dropped in the 1.6 s before LOST
same speed, enable_area_memory = False, no map	1 stall in about 26 s	62 dropped, startup included

During the stalls the GPU is at 0–7%, and one CPU core goes to 90–95% (tegrastats). Per-thread top with area memory off shows no thread above about 60%. So it looks like CPU-bound map matching running inside grab(). At speed it falls behind, grab() drops most frames, and tracking ends up LOST.

After a reset_positional_tracking() (the call itself takes about 3.1 s; INITIALIZING → KNOWN_MAP then took 0.19 s, so 3.35 s from calling the reset to KNOWN_MAP), grab() stalled about every 0.12 s (about 105 ms each) for the next several seconds.

Questions

1. With enable_area_memory on and a loaded map, does relocalization / map matching run synchronously inside grab(), or in a background thread that grab() waits on?
2. Is there a way to limit how often it runs, or how much time it may take per frame, so VIO keeps running at the full frame rate?
3. Can the VIO pose be read without waiting for map matching? For example, does get_position(…, REFERENCE_FRAME.CAMERA) come from a separate path?
4. For high-speed use (a small car, several m/s, fast turns), what setup do you recommend for continuous VIO plus map relocalization? Would two SDK instances on a local stream (one VIO-only, one with area memory) be a supported approach, and do IMU data come through the stream?
5. Is the faster stall pattern right after a reset expected?
6. The API notes that reset_positional_tracking() “will also flush the accumulated or loaded spatial memory”. With a loaded .area file, does the reset reload the map from area_file_path, and is that why it takes ~3 s? Is there a lighter way to re-seed the pose after LOST without flushing the map?

I can share bags, logs and the tegrastats / top captures if useful.

---

[Myzhar]

Hi @bajaj10,
thank you for the detailed report and the measurements; they are very useful.

@bajaj10

1. With enable_area_memory on and a loaded map, does relocalization / map matching run synchronously inside grab(), or in a background thread that grab() waits on?

The pose is updated with every new frame, and grab() returns when the pose for that frame is available. The CPU spikes you measured with area memory enabled are not expected at this level, so I forwarded your data to the StereoLabs Positional Tracking team for analysis.

@bajaj10

2. Is there a way to limit how often it runs, or how much time it may take per frame, so VIO keeps running at the full frame rate?

No, the current API does not expose parameters to limit the frequency or the time budget of the relocalization process.

@bajaj10

3. Can the VIO pose be read without waiting for map matching? For example, does get_position(…, REFERENCE_FRAME.CAMERA) come from a separate path?

No, REFERENCE_FRAME.CAMERA returns the pose change relative to the previous frame, computed by the same pipeline as the WORLD pose; see Getting Velocity.

@bajaj10

4. For high-speed use (a small car, several m/s, fast turns), what setup do you recommend for continuous VIO plus map relocalization? Would two SDK instances on a local stream (one VIO-only, one with area memory) be a supported approach, and do IMU data come through the stream?

Your map is a single 54 m loop, while we recommend that single-loop trajectories do not exceed roughly 20 m; please remap the track by splitting it into shorter sections following the mapping best practices, and disable pose smoothing for high-speed use. I do not recommend running two SDK instances on the same Jetson through a local stream, because they would compete for the same CPU and GPU resources and add encoding latency.

@bajaj10

5. Is the faster stall pattern right after a reset expected?

This is one of the points we will analyze with your data.

@bajaj10

6. The API notes that reset_positional_tracking() “will also flush the accumulated or loaded spatial memory”. With a loaded .area file, does the reset reload the map from area_file_path, and is that why it takes ~3 s? Is there a lighter way to re-seed the pose after LOST without flushing the map?

Yes, the reset reloads the map from area_file_path, which explains most of the 3 s. In localization mode, you do not need to reset after LOST; the module relocalizes automatically when the camera is back in a mapped region, as described in Relocalization within an Area Map.

To allow the team to reproduce the issue offline, please share an SVO2 recording of the same drive, the .area file, your script, and the tegrastats/top captures.

You can send a download link to support@stereolabs.com with a reference to this thread.