fernandotonon/QtMeshEditor-rmib-inbetween

QtMeshEditor — RMIB Animation In-Betweening

A small transformer that fills the gap between two sparse keyframes with smooth, plausible intermediate motion — Robust Motion In-Betweening (RMIB) style, exported to ONNX for in-app inference via ONNX Runtime.

Built for QtMeshEditor (issue #409) — a free, open-source 3D mesh & animation editor — and its companion QtMesh Cloud asset service. The app downloads this model on first use and runs it locally (offline) to power the dope-sheet "AI in-between → Fill gap" action, the qtmesh anim --in-between CLI, and the motion_in_between MCP tool.

Model

• Input: pose_pair float32 [1, 2, 220] — the start and end pose, each 22 core-body joints × 10 DoF [tx,ty,tz, qx,qy,qz,qw, sx,sy,sz].
• Output: interior float32 [1, 8, 220] — 8 predicted in-between poses.
• Architecture: a 4-layer Transformer encoder (start + end pose tokens + learnable interior-frame queries → per-frame pose). ~13 MB.
• Skeleton: the 22 CMU core-body joints (hips / spine / neck / head + both arms + both legs). QtMeshEditor maps arbitrary rig bone names (Mixamo, generic, CMU) onto these roles at runtime; rigs that don't map fall back to a deterministic spline.

Training data & license

Trained from scratch on the CMU Graphics Lab Motion Capture Database — permissively licensed (free to use/modify/redistribute, including in commercial products; you may not resell the motion data itself; credit mocap.cs.cmu.edu). This is what makes the weights redistributable under QtMeshEditor's permissive (MIT) distribution — unlike the field-standard LAFAN1 (CC-BY-NC-ND), which we deliberately avoided.

On a held-out CMU split the model beats spherical-linear interpolation (the shipped spline fallback) by more than 2× on rotation error.

Weights released under CC-BY-4.0; please credit QtMeshEditor and the CMU Graphics Lab Motion Capture Database.

Reproducing

scripts/export-rmib-onnx.py in the QtMeshEditor repo (one-time, offline; not shipped with the app).

Versions

• v1.0.0 (current) — initial RMIB in-betweening model trained on CMU MoCap (22 core-body joints, ~421k windows; beats slerp by >2× on a held-out split).