AvatarMoE: Decomposing Non-Rigid Deformation
with Part-Aware Experts for 3DGS Avatars

Computers & Graphics 137 (2026) · Article 104597
Hyeri Yang1, Junyoung Hong2, Shinwoong Kim3, Kyungjae Lee1
1School of Artificial Intelligence, Yong In University
2Department of Artificial Intelligence, Yonsei University
3Department of Artificial Intelligence Software, Yong In University
Paper arXiv Video Code BibTeX
AvatarMoE overview: weak to robust out-of-distribution poses via learned dynamic weights

A single global network averages every body part together, causing tearing and stretching in complex poses. AvatarMoE instead assigns each part its own expert, producing robust deformations on challenging out-of-distribution poses.

Abstract

While 3D Gaussian Splatting (3DGS) has enabled high-quality, real-time animatable avatars, current methods suffer from a key limitation: they model non-rigid deformations with a single global network, which leads to unnatural artifacts such as tearing and stretching in complex poses. To overcome this, we propose AvatarMoE (part-based Mixture of Experts), a framework that decomposes the human body into joint-centric local regions, each governed by a specialized expert network. We introduce three core contributions: (1) a pose-aware Gaussian Mixture Model-based gating network that dynamically allocates expert influence according to the input pose; (2) a memory-efficient hybrid expert architecture with a shared HashGrid encoder and lightweight decoders; and (3) a novel Intra-Expert Coherence Loss that regularizes each expert's deformation field to prevent local inconsistencies. Extensive experiments show that AvatarMoE significantly reduces such artifacts and produces more robust, visually plausible avatars than previous methods, especially for out-of-distribution (OOD) poses.

Method

We reframe non-rigid deformation as modeling a collection of 24 joint-centric local regions corresponding to the SMPL skeleton, applying fine-grained deformation on a per-cluster basis through a Mixture of Experts architecture. The framework has three core components:

  1. Dynamic GMM gating. A pose-conditioned Gaussian Mixture Model assigns each 3D point to experts, combining a global pose prior with a local spatial likelihood. Expert influence is a learnable function of pose, rather than a fixed skinning weight.
  2. Hybrid expert network. A single shared multi-resolution HashGrid encoder feeds 24 lightweight MLP decoders, conditioned by local coordinates and per-joint pose features. This keeps memory bounded and reaches convergence in roughly half the baseline's iterations.
  3. Intra-Expert Coherence Loss. Minimizing the weighted variance of each expert's deformation field makes every body part move as a coherent unit, suppressing the flicker and tearing caused by routing fluctuation.
AvatarMoE pipeline architecture
Overview. A pose encoder produces a global feature and 24 per-joint features. The GMM gating predicts pose-aware weights for each expert; in parallel, shared HashGrid features augmented with local coordinates and the per-joint feature drive 24 expert decoders. The final non-rigid deformation is the weighted sum of expert offsets, applied before the rigid transformation and rendering.
Static SMPL gating weights versus dynamic GMM gating weights for two poses
Static vs. dynamic gating weights. We visualize the most influential joint for each point. Static SMPL skinning weights (left) stay fixed regardless of pose, giving an identical influence map for Pose A and Pose B. Our dynamic GMM gating (right) adapts each expert's influence region to the specific pose — note how the weights around the shoulder, arm, and knee shift as the arm moves forward.

Results

On out-of-distribution poses from AIST++ and AMASS, AvatarMoE produces smooth, coherent surfaces and avoids the tearing artifacts of global deformation methods. We show comparisons against two baselines: 3DGS-Avatar (subjects 377, 392, 393) and GART (subjects 386, 387, 394). In each clip the baseline is on the left and AvatarMoE on the right.

Want to see more results? Download the supplementary videos and images (.zip, Google Drive).

BibTeX

@article{yang2026avatarmoe,
  title   = {AvatarMoE: Decomposing Non-Rigid Deformation with Part-Aware Experts for 3DGS Avatars},
  author  = {Yang, Hyeri and Hong, Junyoung and Kim, Shinwoong and Lee, Kyungjae},
  journal = {Computers & Graphics},
  volume  = {137},
  pages   = {104597},
  year    = {2026},
  doi     = {10.1016/j.cag.2026.104597},
  publisher = {Elsevier}
}