[1912.06395v1] Neural Cages for Detail-Preserving 3D Deformations
We focus on the deformation quality produced by the predicted cages: hence, the cage geometry itself is not designed to be comparable to professionally-created cages for 3D artists
Abstract We propose a novel learnable representation for detailpreserving shape deformation. The goal of our method is to warp a source shape to match the general structure of a target shape, while preserving the surface details of the source. Our method extends a traditional cage-based deformation technique, where the source shape is enclosed by a coarse control mesh termed cage, and translations prescribed on the cage vertices are interpolated to any point on the source mesh via special weight functions. The use of this sparse cage scaffolding enables preserving surface details regardless of the shape’s intricacy and topology. Our key contribution is a novel neural network architecture for predicting deformations by controlling the cage. We incorporate a differentiable cage-based deformation module in our architecture, and train our network end-to-end. Our method can be trained with common collections of 3D models in an unsupervised fashion, without any cage-specific annotations. We demonstrate the utility of our method for synthesizing shape variations and deformation transfer.
‹Figure 2: Applications of our neural cage-based deformation method. Top: Complex source chairs (brown) deformed (blue) to match target chairs (green), while accurately preserving detail and style with nonhomogeneous changes that adapt different regions differently. No correspondences are used at any stage. Bottom: A cage-based deformation network trained on many posed humans (SMPL) can transfer novel poses of novel targets (SCAPE, skeleton, X-Bot, in green) to a very dissimilar robot of which only a single neutral pose is available. A few matching landmarks between the robot and a neutral SMPL human are required. Dense correspondences between SMPL humans are used only during training. (Introduction)Figure 3: An overview of our cage learning approach. A source Ss and a target St are encoded by the same point net encoder EPN into latent codes fs and ft, resp. The source shape’s code is decoded by an AtlasNetstyle decoder DAN c to a source cage Cs in the cage-prediction module Nc. Source and target codes are also concatenated and decoded by DAN d to create the cage’s deformation offset in the deformation-prediction module Nd. The offset is added to Cs to compute the cage deformation Cs→t that will deform the source to the target. Given a source cage and a shape, our novel MVC layer computes the mean value coordinates φCs (Ss). These coordinates and cage deformation Cs→t are then used by our novel cagebased deformation layer (CBD) to produce a deformed source cage Ss→t. (Method)Figure 4: Synthesizing variations of source shapes (brown), by deforming them to match targets (green). (Loss terms)Figure 5: Comparison of our method with other non-homogeneous deformation methods. Our method achieves superior detail preservation of the source shape in comparison to optimization-based  and learningbased [6,9,28] techniques, while still aligning aligning output to the target. (Loss terms)Figure 6: Comparison of our method with anisotropic scaling. Our method better matches corresponding semantic parts. (Stock amplification via deformation)Figure 7: Quantitative evaluation of our method vs alternative methods. Each point represents a method, embedded according to its average alignment error (Chamfer Distance) and distortion (∆CotLaplacian). Points near the bottom-left corners are better. (Stock amplification via deformation)Figure 8: We use our method to deform a 3D shape to match a real 2D image. We first use AtlasNet  to reconstruct a 3D proxy target. Despite the poor quality of the proxy, it still serves as a valid target for our network to generate a matching output preserving the fine details of the source. (Stock amplification via deformation)Figure 9: The deformation model, trained to deform a fixed source (left) to various articulations. (Deformation transfer)
pp__deriv == sum(phi_j**C p cagev'.)
Learning cage-based deformation
C_s == N_c * S_s + C_0, C_(s to t) == N_d(S_t, S_s) + C_s
L == alpha_MVC * L_MVC + L_align + alpha_shape * L_shapeFigure 10: Deformation transfer. We first learn the cage deformation space for a template shape (top left, brown) with known pose and body shape variations. Then, we annotate predefined landmarks on new characters in neutral poses (left column, rows 2-4, brown), thereby establishing sparse correspondences to the template. At test time, given novel target poses (top row, green) without known correspondences to the template, we transfer their poses to the other characters (blue). Figure ?? shows additional examples where the target pose also comes from a shape which is morphologically very different from the template (e.g. an anatomical skeleton). (Deformation transfer)Figure 11: Effect of LMVC. Given a source/target pair we set different values for αMVC and show predicted and deformed cages, as well as the final output. Higher regularization yields more conservative deformations. (Evaluation)Figure 12: We evaluate effect of different terms in shape preservation loss Lshape, note that all results also include Lsymm. (Evaluation)Figure 13: The effect of the source-cage prediction. We compare the prediction of Nc (middle column) with two alternative static options for cages (right column): using a spherical cage (top row) and using the single optimal prediction over the entire training set (bottom row). Our approach achieves better alignment with the target shape. (Evaluation)Figure 14: Rectilinearity errors. (Conclusion)›