[1910.11163] On the geometry of learning neural quantum states
Our main result is that the eigenvalues and eigenvectors of the quantum Fisher matrix reflect both the learning dynamics, which is unsurprising, as well as the intrinsic static phase information of the model under study which is rather surprising

Abstract: Combining insights from machine learning and quantum Monte Carlo, the
stochastic reconfiguration method with neural network Ansatz states is a
promising new direction for high precision ground state estimation of quantum
many body problems. At present, the method is heuristic, lacking a proper
theoretical foundation. We initiate a thorough analysis of the learning
landscape, and show that it reveals universal behavior reflecting a combination
of the underlying physics and of the learning dynamics. In particular, the
spectrum of the quantum Fisher matrix of complex restricted Boltzmann machine
states can dramatically change across a phase transition. In contrast to the
spectral properties of the quantum Fisher matrix, the actual weights of the
network at convergence do not reveal much information about the system or the
dynamics. Furthermore, we identify a new measure of correlation in the state by
analyzing entanglement the eigenvectors. We show that, generically, the
learning landscape modes with least entanglement have largest eigenvalue,
suggesting that correlations are encoded in large flat valleys of the learning
landscape, favoring stable representations of the ground state.

‹FIG. 1. Complex RBM consisting of one hidden and one visible layer. Visible, hidden biases, and weights are a ∈ CN , b ∈ CM , and w ∈ CN × CM , respectively. x, y are binary vectors of length n and m respectively. (Introduction)FIG. 2. Transverse field Ising model, variational ground state energy optimization using the stochastic reconfiguration method: (a) Rescaled energy as a function of epochs for different values of h ∈ [0.0, 0.6, 1.0, 1.4, 2.0]. The energy is rescaled to have 0 at the exact ground state energy and 1 at initialization. (b) Ordered eigenvalues of the quantum Fisher matrix (Eqn. (??)) at different epochs during the learning process. The spectrum exhibits universal behavior for the first ∼ 25 epochs. After that, the eigenvalues slowly approach a model dependent final profile (see main text). (c) The 500 largest eigenvalues after convergence for different values of h as well as for randomly initialized RBM (black curve). Color coding is the same as in (a). The two vertical gray lines indicate N = 28 and N(N + 1)/2 = 406. (d) Spectrum (blue) and entanglement in the eigenvectors (red) on log-log scale. The eigenvectors corresponding to the dominant eigenvalues have significantly reduced entanglement, especially in the ferromagnetic phase. (Spectral analysis of the quantum Fisher matrix)FIG. 3. (a) Eigenvalue distributions of the quantum Fisher matrix for coherent Gibbs states of two dimensional classical Ising model with the inverse temperature β. We used L × L lattice with L = 10, so N = 100. The number of hidden units M is given by the number of edges in the graph which is 180 (open boundary condition is used). The step is exactly located at N(N + 1)/2 = 5050. (b) The rank the quantum Fisher matrix and (c) the trace of the quantum Fisher matrix as a function of β. (Coherent Gibbs state of the two dimensional classical Ising model)FIG. 5. Epochs versus rescaled energies obtained from the RMSProp with different learning rates and the SR for the TFI. (Implication to optimization)

FIG. 4. (a) Rescaled energy as a function of epochs for the XXZ model with ∆ = −1.0, 0.0 and 1.0. (b) Dynamics of the spectrum of the quantum Fisher matrix during the learning. (c) Spectra of converged Fisher matrices. The same colors with (a) are used for ∆. Hyper-parameters η = 0.02 and = 0.001 are used for SR. (The XXZ model)FIG. 8. Numerical results of XXZ model with size N = 20 using exactly constructed wave function. (a) Normalized energy e E = (hEi − Eed)/(E0 − Eed) as a function of epochs. (b) Dynamics of the spectrum of the Fisher information matrix for different values of ∆. (c) Spectrum of converged Fisher information matrix. The same colors with (a) are used to indicate ∆. (Coherent Gibbs states for classical Ising models)FIG. 9. Rescaled energy e E as a function of epochs for TFI in the paramagnetic phase. (The XXZ model using exact wave functions)

FIG. 6. (a) Converges weights (a, b, w) for the TFI model with different values of h. The large rectangle shows the weights w, whereas the small strips show the biases a and b, which are much weaker in magnitude than the leading weights. (b) Real and imaginary parts of the quantum Fisher matrix after convergence for the TFI as well as randomly initialized RBM. Insets show the correlation between unary variables. The whole matrix is order N + M + NM = 2464 and the unary part is order N + M = 112. The covariance between visible units are small left bottom corner of the unary part. (quantum Fisher matrix of random RBM)FIG. 7. Normalized eigenvalues λi/N of the converged quantum Fisher matrix for the TFI with different values of h. The whole shapes of the distributions remain the same for different N. (Non-zero elements of Fisher information matrix)