Conformal hypothesis testing
Vladimir Vovk (University of London, U.K.)
Abstract: https://www.isibang.ac.in/~statmath/pcm2020/
Martingales in the foundations of statistics Date: March 15th, 2021 Time: 15:00-15:45 IST
Abstract: Traditional methods of testing statistical hypotheses have been developed for the batch setting, in the terminology of machine learning: given a batch of data, statisticians typically compute measures of disagreement, such as p-values or Bayes factors, between a null hypothesis and the data. An alternative that is popular in machine learning is the online setting, in which the items of data (observations ) keep arriving sequentially. In this introductory lecture I will explain the role of martingales, in the form of test martingales, in online hypothesis testing and discuss their applications in the foundations of probability and statistics.
Multiple hypothesis testing with e-values Date: March 15th, 2021 Time: 16:00-16:45 IST
Abstract: It is interesting that test martingales do not trivialize in the case of only one observation. In fact, they provide a useful alternative, sometimes called e-values, to the standard statistical notion of p-values. The most important mathematical advantage of e-values over p-values is that the average of e-values is always an e-value. This property is valuable in multiple hypothesis testing, which will be the topic of this lecture.
Conformal prediction Date: March 19th, 2021 Time: 15:00-15:45 IST
Abstract: Mainstream machine learning, despite its recent successes, has a serious drawback: while its state-of-the-art algorithms often produce excellent predictions, they do not provide measures of their accuracy and reliability that would be both practically useful and provably valid. On the other hand, such measures are commonplace in statistics. Conformal prediction adapts rank tests, popular in nonparametric statistics, to testing the IID assumption (the observations being independent and identically distributed), which is the standard assumption made in machine learning. This gives us practical measures, provably valid under the IID assumption, of the accuracy and reliability of predictions produced by traditional and recent machine-learning algorithms. In this lecture I will give a brief review of conformal prediction. Conformal hypothesis testing Date: March 19th, 2021 Time: 16:00-16:45 IST
Abstract: An interesting application of conformal prediction is the existence of exchangeability martingales, i.e., random processes that are test martingales under any exchangeable probability measure. In particular, they are martingales whenever the observations are IID. The topics of this last lecture in this series will be the construction of exchangeability martingales and their use for different kinds of change detection, including detecting a point at which the IID assumption becomes violated and detecting concept shift.
probability
Audience: advanced learners
Series comments: The link to zoom meeting can be found on the seminar's google calendar - www.isibang.ac.in/~d.yogesh/BPS.html
| Organizers: | D Yogeshwaran*, Sreekar Vadlamani |
| *contact for this listing |