Diffusion Priors for Ill-Posed Skeleton Reconstruction: When a Learned Prior Is Warranted

Source abstract

Reconstructing a 3D human skeleton from partial joint observations is an ill-posed inverse problem: when joints are missing, infinitely many anatomically distinct poses fit the observation, but most methods return a single reconstruction. We formulate single-frame reconstruction as a linear inverse problem, characterize the null space of the missing joints, and argue that the output should be a distribution over the feasible set rather than a point estimate. We model this distribution as a Bayesian posterior with a diffusion model as a learned prior. The question is not whether such a prior can reconstruct skeletons, but when it is warranted: a conjecture relates the error of linear interpolation to the curvature of the pose manifold, with a low-curvature limit in which interpolation is near-optimal. Measured on NTU RGB+D, the curvature is non-zero and intrinsic to individual motions but moderate, and the results are as follows: the prior outperforms nearest-neighbor averaging under scattered occlusion up to moderate severity, is outperformed by it under structural occlusion, and yields per-joint uncertainty that tracks the realized error (r≈0.7). The prior satisfies the kinematic constraints implicitly: explicit guidance improves bone-length fidelity but degrades accuracy, so the constraints serve the formulation and the analysis rather than the sampler.