BEGIN:VCALENDAR
VERSION:2.0
PRODID:researchseminars.org
CALSCALE:GREGORIAN
X-WR-CALNAME:researchseminars.org
BEGIN:VEVENT
SUMMARY:Nathan Kutz (University of Washington)
DTSTART;VALUE=DATE-TIME:20200701T150000Z
DTEND;VALUE=DATE-TIME:20200701T151000Z
DTSTAMP;VALUE=DATE-TIME:20200705T044310Z
UID:SciDL/1
DESCRIPTION:Title: Opening remarks\nby Nathan Kutz (University of Washingt
on) as part of Workshop on Scientific-Driven Deep Learning (SciDL)\n\nAbst
ract: TBA\n
END:VEVENT
BEGIN:VEVENT
SUMMARY:George Em Karniadakis (Brown University)
DTSTART;VALUE=DATE-TIME:20200701T151000Z
DTEND;VALUE=DATE-TIME:20200701T160000Z
DTSTAMP;VALUE=DATE-TIME:20200705T044310Z
UID:SciDL/2
DESCRIPTION:Title: DeepOnet: Learning nonlinear operators based on the uni
versal approximation theorem of operators\nby George Em Karniadakis (Brown
University) as part of Workshop on Scientific-Driven Deep Learning (SciDL
)\n\n\nAbstract\nIt is widely known that neural networks (NNs) are univers
al approximators of continuous functions\, however\, a less known but powe
rful result is that a NN with a single hidden layer can approximate accura
tely any nonlinear continuous operator. This universal approximation theor
em of operators is suggestive of the potential of NNs in learning from sca
ttered data any continuous operator or complex system. To realize this the
orem\, we design a new NN with small generalization error\, the deep opera
tor network (DeepONet)\, consisting of a NN for encoding the discrete inpu
t function space (branch net) and another NN for encoding the domain of th
e output functions (trunk net). We demonstrate that DeepONet can learn var
ious explicit operators\, e.g.\, integrals and fractional Laplacians\, as
well as implicit operators that represent deterministic and stochastic dif
ferential equations. We study\, in particular\, different formulations of
the input function space and its effect on the generalization error.\n
END:VEVENT
BEGIN:VEVENT
SUMMARY:Frank Noe (FU Berlin)
DTSTART;VALUE=DATE-TIME:20200701T160000Z
DTEND;VALUE=DATE-TIME:20200701T162500Z
DTSTAMP;VALUE=DATE-TIME:20200705T044310Z
UID:SciDL/3
DESCRIPTION:Title: PauliNet: Deep neural network solution of the electroni
c Schrödinger Equation\nby Frank Noe (FU Berlin) as part of Workshop on S
cientific-Driven Deep Learning (SciDL)\n\n\nAbstract\nThe electronic Schr
ödinger equation describes fundamental properties of molecules and materi
als\, but can only be solved analytically for the hydrogen atom. The numer
ically exact full configuration-interaction method is exponentially expens
ive in the number of electrons. Quantum Monte Carlo is a possible way out:
it scales well to large molecules\, can be parallelized\, and its accurac
y has\, as yet\, only been limited by the flexibility of the used wave fun
ction ansatz. Here we propose PauliNet\, a deep-learning wave function ans
atz that achieves nearly exact solutions of the electronic Schrödinger eq
uation. PauliNet has a multireference Hartree-Fock solution built in as a
baseline\, incorporates the physics of valid wave functions\, and is train
ed using variational quantum Monte Carlo (VMC). PauliNet outperforms compa
rable state-of-the-art VMC ansatzes for atoms\, diatomic molecules and a s
trongly-correlated hydrogen chain by a margin and is yet computationally e
fficient. We anticipate that thanks to the favourable scaling with system
size\, this method may become a new leading method for highly accurate ele
ctronic-strucutre calculations on medium-sized molecular systems.\n
END:VEVENT
BEGIN:VEVENT
SUMMARY:Alejandro Queiruga (Google\, LLC)
DTSTART;VALUE=DATE-TIME:20200701T162500Z
DTEND;VALUE=DATE-TIME:20200701T165000Z
DTSTAMP;VALUE=DATE-TIME:20200705T044310Z
UID:SciDL/4
DESCRIPTION:Title: Continuous-in-Depth Neural Networks through Interpretat
ion of Learned Dynamics\nby Alejandro Queiruga (Google\, LLC) as part of W
orkshop on Scientific-Driven Deep Learning (SciDL)\n\n\nAbstract\nData-dri
ven learning of dynamical systems is of interest to the scientific communi
ty\, which wants to recover information about the true physics from the di
scretized model\, and the machine learning community\, which wants to impr
ove model interpretability and performance. We present a refined interpret
ation of learned dynamical models by investigating canonical systems. Rece
nt ML literature draws a metaphor between residual components of neural ne
tworks and a forward Euler time integrator\, but we show that these compon
ents actually learn a more accurate integrator. We examine\, the harmonic
oscillator\, 1D wave equation\, and the pendulum in two forms\, using pure
ly linear models\, feed-forward shallow neural networks\, and neural netwo
rks embedded in time integrators. Each of the model configurations overfit
to a better operator than commonly understood\, confounding recovery of p
hysics and attempts to improve the algorithms. We show two analytical meth
ods for reconstructing underlying operators from linear systems. For the n
onlinear problems\, unmodified neural networks outperform the expected num
erical methods\, but do not allow for inspection or generalization. Embedd
ing the models in integrators such as RK4 improves performance and general
izability. However\, for the constrained pendulum\, the model is still bet
ter than excepted\, exhibiting better than expected stiffness-stability. W
e conclude by revisiting the components of neural networks where improveme
nts are suggested.\n
END:VEVENT
BEGIN:VEVENT
SUMMARY:Michael Muehlebach (UC Berkeley)
DTSTART;VALUE=DATE-TIME:20200701T165000Z
DTEND;VALUE=DATE-TIME:20200701T171500Z
DTSTAMP;VALUE=DATE-TIME:20200705T044310Z
UID:SciDL/5
DESCRIPTION:Title: Optimization with Momentum: Dynamical\, Control-Theoret
ic\, and Symplectic Perspectives\nby Michael Muehlebach (UC Berkeley) as p
art of Workshop on Scientific-Driven Deep Learning (SciDL)\n\n\nAbstract\n
My talk will focus on the analysis of accelerated first-order optimization
algorithms. I will show how the continuous dependence of the iterates wit
h respect to their initial condition can be exploited to characterize the
convergence rate. The result establishes criteria for accelerated converge
nce that are easily verifiable and applicable to a large class of first-or
der optimization algorithms. The analysis is not restricted to the convex
setting and unifies discrete-time and continuous-time models. It also rigo
rously explains why structure-preserving discretization schemes are import
ant for momentum-based algorithms.\n
END:VEVENT
BEGIN:VEVENT
SUMMARY:Tess Smidt (LBL)
DTSTART;VALUE=DATE-TIME:20200701T204000Z
DTEND;VALUE=DATE-TIME:20200701T210500Z
DTSTAMP;VALUE=DATE-TIME:20200705T044310Z
UID:SciDL/6
DESCRIPTION:Title: Euclidean Neural Networks for Emulating Ab Initio Calcu
lations and Generating Atomic Geometries\nby Tess Smidt (LBL) as part of W
orkshop on Scientific-Driven Deep Learning (SciDL)\n\n\nAbstract\nAtomic s
ystems (molecules\, crystals\, proteins\, nanoclusters\, etc.) are natural
ly represented by a set of coordinates in 3D space labeled by atom type. T
his is a challenging representation to use for neural networks because the
coordinates are sensitive to 3D rotations and translations and there is n
o canonical orientation or position for these systems. We present a genera
l neural network architecture that naturally handles 3D geometry and opera
tes on the scalar\, vector\, and tensor fields that characterize physical
systems. Our networks are locally equivariant to 3D rotations and translat
ions at every layer. In this talk\, we describe how the network achieves t
hese equivariances and demonstrate the capabilities of our network using s
imple tasks. We’ll also present examples of applying Euclidean networks
to applications in quantum chemistry and discuss techniques for using thes
e networks to encode and decode geometry.\n
END:VEVENT
BEGIN:VEVENT
SUMMARY:Michael P. Brenner (Harvard University)
DTSTART;VALUE=DATE-TIME:20200701T190000Z
DTEND;VALUE=DATE-TIME:20200701T195000Z
DTSTAMP;VALUE=DATE-TIME:20200705T044310Z
UID:SciDL/7
DESCRIPTION:Title: Machine Learning for Partial Differential Equations\nby
Michael P. Brenner (Harvard University) as part of Workshop on Scientific
-Driven Deep Learning (SciDL)\n\n\nAbstract\nI will discuss several ways i
n which machine learning can be used for solving and understanding the sol
utions of nonlinear partial differential equations. Most of the talk will
focus on learning discretizations for coarse graining the numerical soluti
ons of PDEs. I will start with examples in 1d\, and then move on to advect
ion/diffusion in a turbulent flow and then the Navier Stokes equation.\n
END:VEVENT
BEGIN:VEVENT
SUMMARY:Elizabeth Qian (MIT)
DTSTART;VALUE=DATE-TIME:20200701T195000Z
DTEND;VALUE=DATE-TIME:20200701T201500Z
DTSTAMP;VALUE=DATE-TIME:20200705T044310Z
UID:SciDL/8
DESCRIPTION:Title: Lift & Learn: Analyzable\, Generalizable Data-Driven Mo
dels for Nonlinear PDEs\nby Elizabeth Qian (MIT) as part of Workshop on Sc
ientific-Driven Deep Learning (SciDL)\n\n\nAbstract\nWe present Lift & Lea
rn\, a physics-informed method for learning low-dimensional models for non
linear PDEs. The method exploits knowledge of a system’s governing equat
ions to identify a coordinate transformation in which the system dynamics
have quadratic structure. This transformation is called a lifting map beca
use it often adds auxiliary variables to the system state. The lifting map
is applied to data obtained by evaluating a model for the original nonlin
ear system. This lifted data is projected onto its leading principal compo
nents\, and low-dimensional linear and quadratic matrix operators are fit
to the lifted reduced data using a least-squares operator inference proced
ure. Analysis of our method shows that the Lift & Learn models are able to
capture the system physics in the lifted coordinates at least as accurate
ly as traditional intrusive model reduction approaches. This preservation
of system physics makes the Lift & Learn models robust to changes in input
s. Numerical experiments on the FitzHugh-Nagumo neuron activation model an
d the compressible Euler equations demonstrate the generalizability of our
model.\n
END:VEVENT
BEGIN:VEVENT
SUMMARY:Lars Ruthotto (Emory University)
DTSTART;VALUE=DATE-TIME:20200701T201500Z
DTEND;VALUE=DATE-TIME:20200701T204000Z
DTSTAMP;VALUE=DATE-TIME:20200705T044310Z
UID:SciDL/9
DESCRIPTION:Title: Deep Neural Networks Motivated by PDEs\nby Lars Ruthott
o (Emory University) as part of Workshop on Scientific-Driven Deep Learnin
g (SciDL)\n\n\nAbstract\nOne of the most promising areas in artificial int
elligence is deep learning\, a form of machine learning that uses neural n
etworks containing many hidden layers. Recent success has led to breakthro
ughs in applications such as speech and image recognition. However\, more
theoretical insight is needed to create a rigorous scientific basis for de
signing and training deep neural networks\, increasing their scalability\,
and providing insight into their reasoning. This talk bridges the gap bet
ween partial differential equations (PDEs) and neural networks and present
s a new mathematical paradigm that simplifies designing\, training\, and a
nalyzing deep neural networks. It shows that training deep neural networks
can be cast as a dynamic optimal control problem similar to path-planning
and optimal mass transport. The talk outlines how this interpretation can
improve the effectiveness of deep neural networks. First\, the talk intro
duces new types of neural networks inspired by to parabolic\, hyperbolic\,
and reaction-diffusion PDEs. Second\, the talk outlines how to accelerate
training by exploiting reversibility properties of the underlying PDEs.\n
END:VEVENT
BEGIN:VEVENT
SUMMARY:Yasaman Bahri (Google Brain)
DTSTART;VALUE=DATE-TIME:20200701T171500Z
DTEND;VALUE=DATE-TIME:20200701T174000Z
DTSTAMP;VALUE=DATE-TIME:20200705T044310Z
UID:SciDL/10
DESCRIPTION:Title: Learning Dynamics of Wide\, Deep Neural Networks: Beyon
d the Limit of Infinite Width\nby Yasaman Bahri (Google Brain) as part of
Workshop on Scientific-Driven Deep Learning (SciDL)\n\n\nAbstract\nWhile m
any practical advancements in deep learning have been made in recent years
\, a scientific\, and ideally theoretical\, understanding of modern neural
networks is still in its infancy. At the heart of this would be to better
understand the learning dynamics of such systems. In a first step towards
tackling this problem\, one can try to identify limits that have theoreti
cal tractability and are potentially practically relevant. I’ll begin by
surveying our body of work that has investigated the infinite width limit
of deep networks. These results establish exact mappings between deep net
works and other\, existing machine learning methods (namely\, Gaussian pro
cesses and kernel methods) but with novel modifications to them that had n
ot been previously encountered. With these exact mappings in hand\, the na
tural question is to what extent they bear relevance to neural networks at
finite width. I’ll argue that the choice of learning rate is a crucial
factor in dynamics away from this limit and naturally classifies deep netw
orks into two classes separated by a sharp phase transition. This is eluci
dated in a class of solvable simple models we present\, which give quantit
ative predictions for the two phases. Quite remarkably\, we test these emp
irically in practical settings and find excellent agreement.\n
END:VEVENT
BEGIN:VEVENT
SUMMARY:Omri Azencot (UCLA)
DTSTART;VALUE=DATE-TIME:20200701T210500Z
DTEND;VALUE=DATE-TIME:20200701T213000Z
DTSTAMP;VALUE=DATE-TIME:20200705T044310Z
UID:SciDL/11
DESCRIPTION:Title: Robust Prediction of High-Dimensional Dynamical Systems
using Koopman Deep Networks\nby Omri Azencot (UCLA) as part of Workshop o
n Scientific-Driven Deep Learning (SciDL)\n\n\nAbstract\nWe present a new
deep learning approach for the analysis and processing of time series data
. At the core of our work is the Koopman operator which fully encodes a no
nlinear dynamical system. Unlike the majority of Koopman-based models\, we
consider dynamics for which the Koopman operator is invertible. We exploi
t the structure of these systems to design a novel Physically-Constrained
Learning (PCL) model that takes into account the inverse dynamics while pe
nalizing for inverse prediction. Our architecture is composed of an autoen
coder component and two Koopman layers for the dynamics and their inverse.
To motivate our network design\, we investigate the connection between in
vertible Koopman operators and pointwise maps\, and our analysis yields a
loss term which we employ in practice. To evaluate our work\, we consider
several challenging nonlinear systems including the pendulum\, fluid flows
on curved domains and real climate data. We compare our approach to sever
al baseline methods\, and we demonstrate that it yields the best results f
or long time predictions and in noisy settings.\n
END:VEVENT
END:VCALENDAR