[1909.12354v2] Realtime Simulation of Thin-Shell Deformable Materials using CNN-Based Mesh Embedding
We would like to explore new optimization methods to train the two networks in an end-to-end fashion while achieving good convergence behavior.

Abstract: We address the problem of accelerating thin-shell deformable object
simulations by dimension reduction. We present a new algorithm to embed a
high-dimensional configuration space of deformable objects in a low-dimensional
feature space, where the configurations of objects and feature points have
approximate one-to-one mapping. Our key technique is a graph-based
convolutional neural network (CNN) defined on meshes with arbitrary topologies
and a new mesh embedding approach based on physics-inspired loss term. We have
applied our approach to accelerate high-resolution thin shell simulations
corresponding to cloth-like materials, where the configuration space has tens
of thousands of degrees of freedom. We show that our physics-inspired embedding
approach leads to higher accuracy compared with prior mesh embedding methods.
Finally, we show that the temporal evolution of the mesh in the feature space
can also be learned using a recurrent neural network (RNN) leading to fully
learnable physics simulators. After training our learned simulator runs
$10-100\times$ faster and the accuracy is high enough for robot manipulation
tasks.

‹Fig. 1: Overview of our method: Each generated mesh (pm) is represented as vertices connected by edges. (a): We use a graph-based CNN where each convolutional layer is a local filter and the filter stencil is the 1-ring neighbor (red arrow). (b): We build an autoencoder using the filter-based convolutional layers. The decoder D mirrors the encoder E and both D, E use L convolutional layers and one fully connected layer. The input of E and the output of D are defined in the ACAP feature space, in which we define the reconstruction loss, Lrecon. We recover the vertex-level features, pm, using the function ACAP−1 , on which we define our PB-loss, Lphys, and vertex-level regularization, Lvert. The PB-loss can be formulated using two methods. (c): In the mass-spring model, the stretch resistance term is modeled as springs between each vertex and its 1-ring neighbors (blue) and the bend resistance term is modeled as springs between each vertex and its 2-ring neighbors (green). (d): FEM models the stretch resistance term as the linear elastic energy on each triangle and the bend resistance term as a quadratic penalty on the dihedral angle between each pair of neighboring triangles (yellow). (Introduction)Fig. 2: Three examples of cloth inverse kinematics with fixed vertices marked in red. Note that our method can synthesize detailed wrinkles and cloth-like deformations in unconstrained parts of the meshes (black box). (Cloth Inverse Kinematics)

(C_(m, i) = M x ACAP_(m, i) + M_N * sum() * ACAP_(m, j) / abs(N_i**1) + b,)

Fig. 3: Two examples of simulation sequence generation in our feature space. (a): 5 frames in the simulation of a cloth swinging down. (b): Synthesized simulation sequence. (c): Another example where two diagonal points are grasped. (d): Synthesized simulation sequence. (Accuracy of Learned Simulator for Robotic Cloth Manipulation)Fig. 4: We reproduce benchmarks from [26] where the robot is collaborating with human to manipulate a piece of cloth (a). We randomly perturb two grasp points on the left (gray arms) and the robot is controlling the other two grasp points (purple arms) using a visual-serving method to maintain the cloth at a target state, e.g., keeping the cloth flat (b), twisted (c), or bent (d). The red cloth is the groundtruth acquired by running the accurate FEM-based cloth simulator [38], which takes 3 hours. The difference between our result (blue) and the groundtruth is indistinguishable. (Accuracy of Learned Simulator for Robotic Cloth Manipulation)Fig. 5: A visualization of our two datasets. The SHEET dataset contains 4 simulation sequences, each with N = 2400 frames. (a,b): We generate the dataset by grasping two corners of the cloth (red dot) and moving the grasping points back and forth along the ±X/Y axes. (c): In two sequences of the SHEET dataset, we add a spherical obstacle to interact with the cloth. (d): The BALL dataset contains 4 simulation sequences, each with N = 500 frames. We generate the dataset by grasping the topmost vertex of the cloth ball (red dot) and moving the grasping point back and forth along the ±Z axes. (Results)Fig. 6: A feature space visualization for SHEET+[11] using t-SNE. (Discriminability of Feature Space)