How would I go about modifying the tracked skeletons' transforms?


I am using the Fusion API to connect two cameras which I’m then streaming into an unreal engine scene. I’m using this documentation as a base : docs

When I walk from one camera to another, the mapped movement in my scene does not really match up. Essentially, my virtual character moves 1/3rd the distance of me in the real environment.

I want to get all the tracked Actors from the scene and take their predicted world position and multiply it by some vector (I’m trying to make the actors move more distance in scene compared to my actual movement).

Any tips on how to do this?

I want this as a blueprint from this pseudocode:
actorPositionInWorld = // computed by zed SDK somehow somewhere
newActorPositionInWorld = (scaled movement vector I computed) * actorPositionInWorld
actor.position = newActorPositionInWorld

What I’ve tried:

I tried inspecting the C++ file for the animation instance on each spawned actor to possibly modify the computed location of the bones from the live link data. I got a bit too confused and possibly misdirected.

I tried getting the actors computed from BPZed_Livelink_Manager and then updating their position. This seemed to have changed their spawn position, but did not update thereafter.

1 Like

Hi @ZanderCodes,

This is not an expected behaviour, the movement should not be scaled if you did not change the avatar’s scale.
I will take a look shortly and get back to you.
In the meanwhile, if you want to relocate the avatar, you should only have to move it’s root position, as only the rotations are used for the animation.

I tried to access the spawned BP_ZED_Manny root and root component. Printing the location resulted in (0,0,0) even though I move around the scene.

Maybe this is important, but I do this print statement in the BP_ZEDLiveLink_Manager blueprint after the actor is cast to a BP_ZED_Manny.

Also, I tried to print out the socket position of the hips of BP_ZED_Manny avatar and I got what looked like proper coordinates, but I couldn’t find a way to change where the avatar appeared in the world.

@ZanderCodes Hey,

Sorry for the delay.
You probably want to do how we manage the manual height offset in the animation blueprint, that is offset the “RootPosition” by using a vector input.
Look around this line in LiveLinkOrientationRemapAsset.

ManualHeightOffset is using HeightOffset from the Animation blueprint, itself set from the livelink manager.


1 Like

No worries! So, I tried to modify and build the LiveLinkOrientationRemapAsset.cpp so that I could apply my own transformation scaling, however building caused lots of errors.

I read here that if I wanted to make edits to the Live Link ZED plugin, that I would need to build from source - both the engine and Fusion API? I hope this isn’t necessary, but if I have to, then I’ll try.

…And sorry for all the confusion, I am actually quite new to unreal. Could you possibly walk me through:

  1. Build the provided unreal sample for windows on engine 5.3.2?**
  2. Add a slot for vector transformation scaling that will get applied to the root bone?

**Here’s the error when I try to package my project for windows on Unreal 5.3.2:

Hi @ZanderCodes,

You should not have to rebuild the sender, which is what requires the UE5 sources. To be clear, the sender is the only thing that requires the sources.

  1. Build the provided unreal sample for windows on engine 5.3.2?**

If you need to modify the code of the sender, you can follow the instructions here: Building ZED Live Link - Stereolabs

  1. Add a slot for vector transformation scaling that will get applied to the root bone?

In the code, look for “HeightOffset” and mimic exactly what uses this attribute. Add your variable (which would be an FVector) to the AnimNode_ZEDLiveLinkPose.h, use it in AnimNode_ZEDLiveLinkPose.cpp just like HeightOffset.
You’ll have to create a

void SetCustomOffset(FVector CustomOffset);

in LiveLinkOrientationsRemapAsset.h and implement it in LiveLinkOrientationsRemapAsset.cpp just like SetHeightOffset(float Offset). And then use it at the same place that ManualHeightOffset is used. (You also have to create a FVector CustomOffset in LiveLinkOrientationsRemapAsset.h, next to ManualOffset).

For your packaging issue, in which configuration are you packaging? Shipping, development… ?


Also, I see this in my output logs when packaging for windows development:

LogUObjectHash: Compacting FUObjectHashTables data took   0.82ms
LogTurnkeySupport: Selected target: ZEDUnrealLiveLink
LogLauncherProfile: Unable to use promoted target - D:/Zed Live Link/zed-livelink-main/zed-livelink-main/UnrealProject/Binaries/Win64/ does not exist.
LogMonitoredProcess: Running Serialized UAT: [ cmd.exe /c ""C:/Program Files/Epic Games/UE_5.3/Engine/Build/BatchFiles/RunUAT.bat"  -ScriptsForProject="D:/Zed Live Link/zed-livelink-main/zed-livelink-main/UnrealProject/ZEDUnrealLiveLink.uproject" Turnkey -command=VerifySdk -platform=Win64 -UpdateIfNeeded -EditorIO -EditorIOPort=54093  -project="D:/Zed Live Link/zed-livelink-main/zed-livelink-main/UnrealProject/ZEDUnrealLiveLink.uproject" BuildCookRun -nop4 -utf8output -nocompileeditor -skipbuildeditor -cook  -project="D:/Zed Live Link/zed-livelink-main/zed-livelink-main/UnrealProject/ZEDUnreal
LiveLink.uproject" -target=ZEDUnrealLiveLink  -unrealexe="C:\Program Files\Epic Games\UE_5.3\Engine\Binaries\Win64\UnrealEditor-Cmd.exe" -platform=Win64 -installed -stage -archive -package -build -pak -iostore -compressed -prereqs -archivedirectory="D:/Zed Live Link/zed-livelink-main/zed-livelink-main/UnrealProject/Builds/win1" -clientconfig=Development" -nocompile -nocompileuat ]
UATHelper: Packaging (Windows): Running AutomationTool...
UATHelper: Packaging (Windows): Using bundled DotNet SDK version: 6.0.302
UATHelper: Packaging (Windows): Starting AutomationTool...
UATHelper: Packaging (Windows): Parsing command line: -ScriptsForProject="D:/Zed Live Link/zed-livelink-main/zed-livelink-main/UnrealProject/ZEDUnrealLiveLink.uproject" Turnkey -command=VerifySdk -platform=Win64 -UpdateIfNeeded -EditorIO -EditorIOPort=54093 -project="D:/Zed Live Link/zed-livelink-main/zed-livelink-main/UnrealProject/ZEDUnrealLiveLink.uproject" BuildCookRun -nop4 -utf8output -nocompileeditor -skipbuildeditor -cook -project="D:/Zed Live Link/zed-livelink-main/zed-livelink-main/UnrealProject/ZEDUnrealLiveLink.uproject" -target=ZEDUnrealLiveLink -unrealexe="C:\Program Files\Epic G
ames\UE_5.3\Engine\Binaries\Win64\UnrealEditor-Cmd.exe" -platform=Win64 -installed -stage -archive -package -build -pak -iostore -compressed -prereqs -archivedirectory="D:/Zed Live Link/zed-livelink-main/zed-livelink-main/UnrealProject/Builds/win1" -clientconfig=Development -nocompile -nocompileuat
UATHelper: Packaging (Windows): Initializing script modules...
UATHelper: Packaging (Windows): Total script module initialization time: 0.13 s.
UATHelper: Packaging (Windows): Executing commands...
UATHelper: Packaging (Windows): Installed Sdk validity:
UATHelper: Packaging (Windows): Win64: (Status=Valid, MinAllowed_Sdk=10.0.00000.0, MaxAllowed_Sdk=10.9.99999.0, Current_Sdk=10.0.22621.0, Allowed_AutoSdk=10.0.18362.0, Current_AutoSdk=, Flags="InstalledSdk_ValidVersionExists")
UATHelper: Packaging (Windows): Scanning for envvar changes...
UATHelper: Packaging (Windows): ... done! 
UATHelper: Packaging (Windows): Cleaning Temp Paths...
UATHelper: Packaging (Windows): BUILD SUCCESSFUL
UATHelper: Packaging (Windows): Setting up ProjectParams for D:\Zed Live Link\zed-livelink-main\zed-livelink-main\UnrealProject\ZEDUnrealLiveLink.uproject
UATHelper: Packaging (Windows): ********** BUILD COMMAND STARTED **********
UATHelper: Packaging (Windows): Running: C:\Program Files\Epic Games\UE_5.3\Engine\Binaries\ThirdParty\DotNet\6.0.302\windows\dotnet.exe "C:\Program Files\Epic Games\UE_5.3\Engine\Binaries\DotNET\UnrealBuildTool\UnrealBuildTool.dll" ZEDUnrealLiveLink Win64 Development -Project="D:\Zed Live Link\zed-livelink-main\zed-livelink-main\UnrealProject\ZEDUnrealLiveLink.uproject" -Manifest="D:\Zed Live Link\zed-livelink-main\zed-livelink-main\UnrealProject\Intermediate\Build\Manifest.xml"  -remoteini="D:\Zed Live Link\zed-livelink-main\zed-livelink-main\UnrealProject"  -skipdeploy  -log="C:\Users\My Name\AppData\Roaming\Unreal Engine\AutomationTool\Logs\C+Program+Files+Epic+Games+UE_5.3\UBT-ZEDUnrealLiveLink-Win64-Development.txt"
UATHelper: Packaging (Windows): Log file: C:\Users\My Name\AppData\Roaming\Unreal Engine\AutomationTool\Logs\C+Program+Files+Epic+Games+UE_5.3\UBT-ZEDUnrealLiveLink-Win64-Development.txt
UATHelper: Packaging (Windows): Creating makefile for ZEDUnrealLiveLink (no existing makefile)
UATHelper: Packaging (Windows): Total execution time: 0.43 seconds
UATHelper: Packaging (Windows): ZEDUnrealLiveLink modifies the value of GlobalDefinitions. This is not allowed, as ZEDUnrealLiveLink has build products in common with UnrealGame.
UATHelper: Packaging (Windows): Remove the modified setting, change ZEDUnrealLiveLink to use a unique build environment by setting 'BuildEnvironment = TargetBuildEnvironment.Unique;' in the ZEDUnrealLiveLinkTarget constructor, or set bOverrideBuildEnvironment = true to force this setting on.
UATHelper: Packaging (Windows): Took 0.55s to run dotnet.exe, ExitCode=6
UATHelper: Packaging (Windows): UnrealBuildTool failed. See log for more details. (C:\Users\My Name\AppData\Roaming\Unreal Engine\AutomationTool\Logs\C+Program+Files+Epic+Games+UE_5.3\UBT-ZEDUnrealLiveLink-Win64-Development.txt)
UATHelper: Packaging (Windows): AutomationTool executed for 0h 0m 2s
UATHelper: Packaging (Windows): AutomationTool exiting with ExitCode=6 (6)
UATHelper: Packaging (Windows): BUILD FAILED
PackagingResults: Error: Unknown Error

I made a new post because this is different from the original question. I found a solution there.