Skip to main content

Embodied Space Studio [UE Plugin]

Embodied Space Studio is an Unreal Editor workflow panel for authoring embodied motion from text, spatial constraints, saved motion data, and batch prompt collections. It connects the editor to the Kimodo bridge/runtime service, lets you stage prompt timing and body constraints in the Timeline, previews generated motion on either the embedded Preview viewport or the main Level viewport, and turns the selected motion into UE Animation Sequence assets for SOMA, Manny, Quinn, MetaHuman, or a custom retarget target.

The motions were generated using Kimodo, a controllable human motion generation system described in: Davis Rempe et al., "Kimodo: Scaling Controllable Human Motion Generation," arXiv:2603.15546, 2026. https://arxiv.org/abs/2603.15546

Main workflow
  1. Use Generate to start or check the bridge service, refresh the model catalog, choose dataset/skeleton/version, load a model, choose the source preview mesh or default SOMA, and generate motion from Timeline prompts and constraints.
  2. Use Preview Target for fast isolated inspection, or Level Target to work directly in the main Level viewport with Spawn SOMA, Use Selected, Focus, and Unlink.
  3. Use the Timeline to create prompt sections, scrub frames, play generated samples, import bulk prompt text, and place Full-Body, 2D Root, hand, and foot constraints.
  4. Use Constraints when a motion needs exact body poses, hand/foot placement, root path control, saved skeleton edit states, or external constraints.json input.
  5. Use Load/Save to reopen saved ue_preview_manifest.json outputs, save reusable example metadata, and open generated output folders.
  6. Use Visualize to control playback speed, mesh/skeleton/foot-contact overlays, floor/environment display, camera FOV, Timeline visibility, and constraint label display.
  7. Use Export to bake the active motion as a SOMA Animation Sequence, build or refresh rooted SOMA retarget assets, select target character assets, tune collision/foot anchoring options, and export a target Animation Sequence.
  8. Use Batch to process many prompts or already-generated SOMA jobs, export multiple target families, preview generated naming, and resume or stop a batch run.
  9. Use Misc for production utilities: placing batch animations into a level, creating a MetaHuman player character, merging MetaHuman body/garment meshes, packaging a Fab product, offline collision/grounding cleanup, MetaHuman Level Sequences, and prompt subtitles.

When a button is disabled, check the status summaries above the active section. Most actions require a healthy bridge service, a loaded model, a valid preview clip, a matching source/target mesh, a selected Level Target, or a valid /Game output path. Runtime Output and Bridge Output logs are designed for troubleshooting and can be copied from the UI.

ess

Kimodo Docker Runtime Setup

AI-assisted setup

You can use AI coding agents such as Codex, GitHub Copilot, or Claude Code to help configure the Kimodo Docker runtime automatically by following the instructions in this section. Give the agent access to the bridge repository and ask it to complete and verify the documented setup steps. Before running the runtime, review any generated commands and confirm that local paths, credentials, model files, and GPU settings are correct for your machine.

Embodied Space Studio uses a local Docker Compose runtime to serve Kimodo motion generation to Unreal Engine. The Unreal Engine plugin is distributed through Fab, while the Docker bridge runtime is distributed separately through GitHub:

https://github.com/richieBao/embodied-space-studio-kimodo-bridge

In this bridge repository, kimodo-main is the Docker Compose project name declared in docker-compose.yaml. The always-on services are text-encoder and ue-bridge; demo is optional for the standalone Kimodo web demo.

The Docker bridge is self-contained. It can be placed in any local folder and does not need to live inside or beside an Unreal project. Any Unreal project with the Embodied Space Studio / KimodoMotionAuthoring plugin can connect to the same local bridge endpoint:

http://127.0.0.1:18027/bridge/v1

ess

Install order

  1. Buy/download and install the Embodied Space Studio plugin from Fab into any Unreal project that will use Kimodo generation.
  2. Install and start Docker Desktop.
  3. Download or clone the GitHub bridge repository to a user-chosen folder.
  4. Request Hugging Face access to meta-llama/Meta-Llama-3-8B-Instruct, then authenticate with a Hugging Face token.
  5. Download the required Kimodo checkpoint and text encoder model files into the bridge folder.
  6. Copy .env.example to .env, then edit paths for your machine.
  7. Build/start the Docker bridge and connect to http://127.0.0.1:18027/bridge/v1 from the plugin's Generate tab.

Prerequisite: install Docker Desktop

Docker is required before you can start the Kimodo runtime. Embodied Space Studio does not install Docker automatically.

  1. Install WSL for the Docker Desktop WSL 2 backend. Open PowerShell as Administrator and run:

    wsl --install --no-distribution
    wsl --update

    The --no-distribution option installs WSL without installing an additional Linux distribution such as Ubuntu. Docker Desktop manages the WSL distributions it requires. If the installation prompts you to restart, restart your computer.

  2. Verify the installed WSL version:

    wsl --version

    Docker Desktop requires WSL version 2.1.5 or later; using the latest available WSL version is recommended.

  3. Install Docker Desktop for Windows from the official Docker website.

  4. Start Docker Desktop and wait until the Docker Engine is running. In Settings > General, enable Use WSL 2 based engine. On supported systems, this option may already be enabled by default and may not be visible.

  5. For GPU generation, install a compatible NVIDIA driver and make sure Docker Desktop can access the GPU.

  6. Open PowerShell and confirm Docker is available:

docker --version
docker compose version
note

Docker is required for Generate, Refresh Models, Connect Docker Bridge, and other Kimodo runtime features. Loading already generated previews or using some offline Unreal-only utilities may not require the Docker runtime to be running.

Python environment note

The official Kimodo local virtual-environment setup uses Python 3.10:

conda create -n kimodo python=3.10
conda activate kimodo

Embodied Space Studio normally runs Kimodo through Docker, so users do not need to create this local Python environment unless they want to run Kimodo scripts outside the containers.

Download the bridge repository

Choose any local folder for the bridge. The commands below use <BridgeRoot> as the folder where you cloned or extracted the GitHub repository.

git clone https://github.com/richieBao/embodied-space-studio-kimodo-bridge.git
cd embodied-space-studio-kimodo-bridge

Downloading the repository ZIP from GitHub and extracting it to a folder such as D:/EmbodiedSpaceStudio/KimodoBridge is also fine.

The bridge folder should contain:

<BridgeRoot>/docker-compose.yaml
<BridgeRoot>/Dockerfile
<BridgeRoot>/ue_bridge_service
<BridgeRoot>/scripts
<BridgeRoot>/checkpoints
<BridgeRoot>/text-encoders

The ue-bridge container runs the bridge API service packaged in <BridgeRoot>/ue_bridge_service; it no longer mounts a specific Unreal project or runs a server.py from the Fab plugin folder.

Runtime layout

ServiceContainer namePurposeHost access
text-encodertext-encoderLoads the LLM2Vec text encoder and provides embeddings to Kimodo services.http://127.0.0.1:9550
ue-bridgeue-bridgeRuns the bridge API used by the plugin Generate / Models / Health actions.http://127.0.0.1:18027/bridge/v1
demodemoOptional Kimodo interactive demo service.http://127.0.0.1:7860 by default

The services are built from Dockerfile into the kimodo:1.0 image. The image is based on NVIDIA PyTorch, installs the Kimodo Python runtime, includes the local kimodo-viser fork, and runs through the container entrypoint that maps files back to the host user.

Download model files

Model weights are intentionally not included in the GitHub repository. Download them from the official model pages and review their licenses before use.

Install the Hugging Face CLI and authenticate on the host machine:

pip install --upgrade huggingface_hub
hf auth login

Download the Kimodo checkpoint:

https://huggingface.co/nvidia/Kimodo-SOMA-RP-v1.1

Place it under:

<BridgeRoot>/checkpoints/Kimodo-SOMA-RP-v1.1/model.safetensors
<BridgeRoot>/checkpoints/Kimodo-SOMA-RP-v1.1/config.yaml

Example command:

hf download nvidia/Kimodo-SOMA-RP-v1.1 --local-dir checkpoints/Kimodo-SOMA-RP-v1.1

Before downloading the text encoder, open the gated Meta Llama model page, request access, and wait until your Hugging Face account has been granted access:

https://huggingface.co/meta-llama/Meta-Llama-3-8B-Instruct

Kimodo's LLM2Vec text encoder uses three pieces:

Raw gated base model:
https://huggingface.co/meta-llama/Meta-Llama-3-8B-Instruct

MNTP LoRA adapter:
https://huggingface.co/McGill-NLP/LLM2Vec-Meta-Llama-3-8B-Instruct-mntp

Supervised LoRA adapter:
https://huggingface.co/McGill-NLP/LLM2Vec-Meta-Llama-3-8B-Instruct-mntp-supervised

Place them under:

<BridgeRoot>/text-encoders/McGill-NLP/LLM2Vec-Meta-Llama-3-8B-Instruct-mntp/
<BridgeRoot>/text-encoders/McGill-NLP/LLM2Vec-Meta-Llama-3-8B-Instruct-mntp-adapter/
<BridgeRoot>/text-encoders/McGill-NLP/LLM2Vec-Meta-Llama-3-8B-Instruct-mntp-supervised/

Example manual download commands:

hf download meta-llama/Meta-Llama-3-8B-Instruct --local-dir text-encoders/McGill-NLP/LLM2Vec-Meta-Llama-3-8B-Instruct-mntp
hf download McGill-NLP/LLM2Vec-Meta-Llama-3-8B-Instruct-mntp --local-dir text-encoders/McGill-NLP/LLM2Vec-Meta-Llama-3-8B-Instruct-mntp-adapter
hf download McGill-NLP/LLM2Vec-Meta-Llama-3-8B-Instruct-mntp-supervised --local-dir text-encoders/McGill-NLP/LLM2Vec-Meta-Llama-3-8B-Instruct-mntp-supervised

The LLM2Vec-Meta-Llama-3-8B-Instruct-mntp local folder should include the gated raw Meta Llama files such as model.safetensors.index.json and shard files. The ...-mntp-adapter and ...-mntp-supervised folders are LoRA adapters and should include adapter_model.safetensors and adapter_config.json.

The McGill-NLP/LLM2Vec-Meta-Llama-3-8B-Instruct-mntp page may look small in the Hugging Face file browser because it is the MNTP adapter, not the full 8B base model. The full 8B shard files come from the gated meta-llama/Meta-Llama-3-8B-Instruct repository.

Do not download a separate McGill-NLP/LLM2Vec-Meta-Llama-3-8B-Instruct-mntp-adapter repository. The recommended local folder is named LLM2Vec-Meta-Llama-3-8B-Instruct-mntp-adapter only to make its runtime role clear. Its source is McGill-NLP/LLM2Vec-Meta-Llama-3-8B-Instruct-mntp.

Runtime loading order:

  1. Load raw meta-llama/Meta-Llama-3-8B-Instruct files from the local LLM2Vec-Meta-Llama-3-8B-Instruct-mntp folder as the base model.
  2. Merge the MNTP adapter from TEXT_ENCODER_MNTP_ADAPTER_HOST; by default this is the local LLM2Vec-Meta-Llama-3-8B-Instruct-mntp-adapter folder downloaded from McGill-NLP/LLM2Vec-Meta-Llama-3-8B-Instruct-mntp.
  3. Apply McGill-NLP/LLM2Vec-Meta-Llama-3-8B-Instruct-mntp-supervised as the final supervised adapter exposed by the text-encoder service.

Configure .env

From <BridgeRoot>, copy the example file:

Copy-Item .env.example .env

Then edit .env for your machine. Use forward slashes in Windows paths for Docker volume mapping.

HF_HOME_HOST=./.cache/huggingface
HOST_USER=<your-windows-user-name>
SERVER_PORT=7860
TEXT_ENCODER_DEVICE=auto
TEXT_ENCODER_BASE_MODEL_HOST=./text-encoders/McGill-NLP/LLM2Vec-Meta-Llama-3-8B-Instruct-mntp
TEXT_ENCODER_MNTP_ADAPTER_HOST=./text-encoders/McGill-NLP/LLM2Vec-Meta-Llama-3-8B-Instruct-mntp-adapter
TEXT_ENCODER_ADAPTER_HOST=./text-encoders/McGill-NLP/LLM2Vec-Meta-Llama-3-8B-Instruct-mntp-supervised
UE_BRIDGE_HOST_PORT=18027
KIMODO_OUTPUT_HOST=D:/EmbodiedSpaceStudio/KimodoBridge/outputs

HF_HOME_HOST should point to the Hugging Face cache that contains the token created by hf auth login. If you prefer to keep the cache inside the bridge folder, place the token under <BridgeRoot>/.cache/huggingface/token and use HF_HOME_HOST=./.cache/huggingface.

KIMODO_OUTPUT_HOST is the host folder where generated motion.npz, ue_preview_manifest.json, logs, diagnostics, and prompt metadata are written. The packaged start script resolves KIMODO_OUTPUT_HOST=./outputs to an absolute bridge-local path automatically. If you run raw docker compose commands, set KIMODO_OUTPUT_HOST to an absolute Windows path so Unreal can open generated results.

TEXT_ENCODER_DEVICE is a Docker startup-level setting for the text-encoder service:

TEXT_ENCODER_DEVICE=auto
TEXT_ENCODER_DEVICE=cuda
TEXT_ENCODER_DEVICE=cpu

Use auto for CUDA when available with CPU fallback, cuda to force GPU, or cpu to force CPU text encoding. Use cuda, not gpu; Kimodo passes this value to PyTorch device selection.

Start the Unreal bridge runtime

From PowerShell:

cd "<BridgeRoot>"
.\scripts\Start-EmbodiedSpaceStudio-Bridge.ps1 -Build

This builds the image if needed, starts text-encoder and ue-bridge in detached mode, and prints the bridge health/model URLs plus the host output directory.

After the first build, start without rebuilding:

.\scripts\Start-EmbodiedSpaceStudio-Bridge.ps1

To also start the optional Kimodo demo service:

.\scripts\Start-EmbodiedSpaceStudio-Bridge.ps1 -WithDemo

Equivalent Docker Compose commands:

docker compose build
docker compose up -d text-encoder ue-bridge

To start the optional demo with raw Docker Compose:

docker compose up -d demo

Connect from Unreal Engine

Open any Unreal project that has the Embodied Space Studio / KimodoMotionAuthoring plugin installed. In the plugin's Docker bridge settings or Generate tab, connect to:

http://127.0.0.1:18027/bridge/v1

The Docker bridge is independent from the UE project folder. Multiple compatible UE projects can use the same local bridge by connecting to the same endpoint.

Apply .env changes and recreate containers

Docker reads environment variables when a container is created. If you edit .env, especially TEXT_ENCODER_DEVICE, model paths, ports, or KIMODO_OUTPUT_HOST, simply clicking Stop/Start or Restart in Docker Desktop can keep the old container configuration and the old value.

After changing .env, run:

cd "<BridgeRoot>"
docker compose up -d --force-recreate --no-build text-encoder ue-bridge

Use --build only when the Docker image itself needs to be rebuilt, such as after changing Dockerfile, docker_requirements.txt, ue_bridge_service, or dependency files:

cd "<BridgeRoot>"
docker compose up -d --build --force-recreate text-encoder ue-bridge

Verify the active device setting inside the recreated container:

docker exec text-encoder python -c "import os; print(os.environ.get('TEXT_ENCODER_DEVICE'))"

Verify the runtime

Run the packaged verification script:

cd "<BridgeRoot>"
.\scripts\Verify-EmbodiedSpaceStudio-Bridge.ps1

A healthy bridge should respond at:

http://127.0.0.1:18027/bridge/v1/health
http://127.0.0.1:18027/bridge/v1/models

The expected model key for this packaged setup is:

kimodo-soma-rp-v1.1

You can also inspect the containers directly:

docker ps --filter "name=text-encoder"
docker ps --filter "name=ue-bridge"
cd "<BridgeRoot>"
docker compose logs -f ue-bridge

Stop or reset the runtime

To stop the services while keeping containers available for a faster restart:

cd "<BridgeRoot>"
.\scripts\Stop-EmbodiedSpaceStudio-Bridge.ps1

To remove the compose containers completely:

.\scripts\Stop-EmbodiedSpaceStudio-Bridge.ps1 -RemoveContainers
tip

If Unreal Engine cannot connect from the Generate tab, first check http://127.0.0.1:18027/bridge/v1/health, then confirm that Docker Desktop is running and that text-encoder and ue-bridge are active. The Unreal plugin's Connect Docker Bridge button is intended to connect to the same local bridge endpoint expected by these scripts.

User Manual

Purpose

Connect Unreal Editor to the Kimodo runtime, choose the active generation model and source character, configure sampling/postprocess settings, and submit the current Timeline prompts plus native or external constraints for motion generation.

ess

Action Buttons

  1. Generate / Generating submits the current request. The button is enabled only when the bridge is healthy, a model is loaded, there is valid prompt content, and the service is not busy.
  2. Stop Compute asks the bridge/runtime to stop the active compute job without resetting all editor state.
  3. Connect Docker Bridge and Disconnect start or stop the local bridge service. Refresh Health rechecks service health, and Reset clears editor authoring state and preview state.

Runtime Output

  1. Output Name Prefix is used when naming generated job outputs and follow-up assets.
  2. Text Encoder Device is configured in Docker, not per Generate request. Edit TEXT_ENCODER_DEVICE in KimodoBridge/.env, then recreate text-encoder and ue-bridge with docker compose up -d --force-recreate --no-build text-encoder ue-bridge.
  3. Runtime Log records user-facing Generate/Export/Batch actions. Copy Log sends it to the clipboard; Clear removes the local display text.

Model Selection

  1. Refresh Models queries the bridge model catalog.
  2. Training dataset, Skeleton, and Version filter the model list. Load model makes that model active in the current bridge session.
  3. The selected model summary explains whether the model is loaded, unavailable, or waiting on service health.

Kimodo Mesh / Skeleton

  1. Kimodo Source Preview Mesh overrides the source mesh used for preview binding and request metadata.
  2. Source Skeleton Override can provide a skeleton when a mesh is not enough or when a specific binding is required.
  3. Reset To SOMA Default clears custom source selections and returns to the default Kimodo/SOMA source character.

Generate Settings

  1. Num Samples controls how many candidates the runtime returns for the same authored prompt/constraint request.
  2. Seed becomes active when edited and is used for reproducible generation.
  3. Denoising Steps controls diffusion refinement cost and quality.
  4. Reduce Foot Sliding enables runtime foot-contact stabilization. Level 1 is lighter; Level 2 adds stable curves and XY foot lock.
  5. Classifier-Free Guidance balances Text Weight and Constraint Weight when CFG is enabled.
  6. Transition Frames controls blending between adjacent prompt segments.
  7. Post Processing toggles runtime postprocess, and Root Margin adjusts the root-trajectory margin used by that postprocess.

First author prompts in the Timeline, then add constraints if needed, confirm the bridge health and loaded model, generate one sample set, and inspect it in Preview or Level Target before exporting or batching.

Example Prompts