SeMoCo: A Semantic-First Motion Codec for Motion Language Modeling

Introduction

SeMoCo is a semantic-first motion codec for motion language modeling. Instead of organizing codebooks purely by reconstruction residuals, each motion token contains one semantic token and a residual sequence of kinematic tokens, so that action-level meaning and fine-grained kinematic detail are encoded explicitly. A dual-axis generator built on these tokens models semantic progression across time and autoregressively refines the residual entries.

This repository contains the pretrained tokenizer, its ablation variants, the Lite / Base text-to-motion generators trained on SeMoCo tokens, and the unconditional motion priors used for motion prediction.

Model Files

File	Model	Params
tokenizer/split_branch_sem.pt	SeMoCo tokenizer	22.9M
tokenizer/ablation/	Tokenizer ablation variants (split-branch / single-chain, ± semantic)	~22M
generator/lite.pt	Ours-Lite T2M generator	188M
generator/base.pt	Ours-Base T2M generator	391M
generator/prior_lite.pt	Lite unconditional motion prior	199M
generator/prior_base.pt	Base unconditional motion prior	419M

Tokenizer checkpoints are self-contained (codec_config + net + norm); generator checkpoints contain model + model_config, plus data_meta describing the text encoder for the text-to-motion ones. Optimizer states are stripped. The motion priors take no text conditioning and are only needed for the prediction benchmark.

Evaluation

Model	Headline results
SeMoCo tokenizer	MPJPE 12.83 mm / PA-MPJPE 11.07 mm (SMPL-22, symmetric-FK protocol); codebook usage 100%, perplexity 998, 198.4 tokens/s
Ours-Lite (T2M)	FID .186, R@1 .484, MedR 1.70
Ours-Base (T2M)	FID .181, R@1 .494, MedR 1.50

Usage

# tokenizer only
hf download poisonousID/SeMoCo --include 'tokenizer/*' --local-dir checkpoints/

# generators and motion priors
hf download poisonousID/SeMoCo --include 'generator/*' --local-dir checkpoints/

Tokenizer reconstruction is evaluated from SeMoCo-Tokenizer, whose README covers building test.txt and the recordings root from the released dataset:

python -m tools.eval_recon_smpl22 \
    --checkpoint checkpoints/tokenizer/split_branch_sem.pt \
    --manifest data/test.txt --recordings-root data/recordings \
    --out recon_smpl22.json

Text-to-motion generation and the prediction benchmark live in SeMoCo-Generator.

License

Apache-2.0. Evaluation gates involving SMPL-X require the SMPL-X model, distributed separately under its own license.

Citation

@article{semoco,
  title   = {SeMoCo: A Semantic-First Motion Codec for Motion Language Modeling},
  author  = {OMEGA-team},
  journal = {arXiv preprint},
  year    = {2026}
}