BEGIN:VCALENDAR
VERSION:2.0
PRODID:researchseminars.org
CALSCALE:GREGORIAN
X-WR-CALNAME:researchseminars.org
BEGIN:VEVENT
SUMMARY:João Caldeira (Fermilab)
DTSTART:20200501T220000Z
DTEND:20200501T230000Z
DTSTAMP:20260423T035455Z
UID:CosmoConB/6
DESCRIPTION:Title: Comparing Methods of Uncertainty Quantification in Deep Learning Algorit
hms\nby João Caldeira (Fermilab) as part of CosmoConβ - Cosmology fr
om Home\n\n\nAbstract\nWe present a comparison of methods for uncertainty
quantification (UQ) in deep learning algorithms in the context of a simple
physical system. Three of the most common uncertainty quantification meth
ods – Bayesian Neural Networks (BNN)\, Concrete Dropout (CD)\, and Deep
Ensembles (DE) – are compared to the standard analytic error propagation
. We discuss this comparison in terms endemic to both machine learning ("e
pistemic" and "aleatoric") and the physical sciences ("statistical" and "s
ystematic"). The comparisons are presented in terms of simulated experimen
tal measurements of a single pendulum – a prototypical physical system f
or studying measurement and analysis techniques. Our results highlight som
e pitfalls that may occur when using these UQ methods. For example\, when
the variation of noise in the training set is small\, all methods predicte
d the same relative uncertainty independently of the inputs. This issue is
particularly hard to avoid in BNN. On the other hand\, when the test set
contains samples far from the training distribution\, we found that no met
hods sufficiently increased the uncertainties associated to their predicti
ons. This problem was particularly clear for CD. In light of these results
\, we make some recommendations for usage and interpretation of UQ methods
.\n\nVideo of talk:
https://www.youtube.com/watch?v=K3UYWQMP_Hs\n\nRelevant Paper: [2004.10710] Deeply Uncertain: Compari
ng Methods of Uncertainty Quantification in Deep Learning Algorithms\n
LOCATION:https://researchseminars.org/talk/CosmoConB/6/
END:VEVENT
END:VCALENDAR