From Sound to Metric Priors: A New Paradigm for Shape Generation

Abstract: Spectral and metric geometry are at the heart of various problems in computer vision, graphics, pattern recognition, and machine learning. Ultimately, the core reason for their success can be traced down to questions of stability and to the informativeness of the eigenvalues of certain operators. In this talk, I will discuss and show tangible examples of such properties and showcase some dramatic implications on a selection of notoriously hard problems in computer vision and graphics. First, I will address the question of whether one can recover the shape of a geometric object from its vibration frequencies (‘hear the shape of the drum’); while theoretically the answer to this question is negative, little is known about the practical possibility of using the spectrum for shape reconstruction and optimization. I will introduce a numerical procedure called isospectralization, as well as a data-driven variant, showing how this *practical* problem is solvable. Then, I will discuss the increasingly popular task of designing an effective generative model for deformable 3D shapes. I will demonstrate how injecting metric distortion priors into a simple geometric reconstruction loss can lead to the formation of a very informative latent space, which can be trained with extremely scarce data (less than 10 examples) and still yield competitive generation quality as well as aiding geometric disentanglement.

machine learningMathematics

Audience: researchers in the discipline

Workshop on Geometry and Machine Learning