[1909.12180v1] Towards neural networks that provably know when they don't know
In the future it would be interesting to use more powerful generative models for which one can also guarantee their behavior far away from the training data

Abstract: It has recently been shown that ReLU networks produce arbitrarily
over-confident predictions far away from the training data. Thus, ReLU networks
do not know when they don't know. However, this is a highly important property
in safety critical applications. In the context of out-of-distribution
detection (OOD) there have been a number of proposals to mitigate this problem
but none of them are able to make any mathematical guarantees. In this paper we
propose a new approach to OOD which overcomes both problems. Our approach can
be used with ReLU networks and provides provably low confidence predictions far
away from the training data as well as the first certificates for low
confidence predictions in a neighborhood of an out-distribution point. In the
experiments we show that state-of-the-art methods fail in this worst-case
setting whereas our model can guarantee its performance while retaining
state-of-the-art OOD performance.

hat(p(y / x, o)) == 1/M and hat(p(y / x, i)) == e**f_y(x)/(sum(e**f_k(x) )), y in {(1, hdots * M)}

Figure 1: Illustration on toy dataset: We show the color-coded confidence in the prediction (yellow indicates high confidence maxy p̂(y|x) ≈ 1, whereas dark purple regions indicate low confidence maxy p̂(y|x) ≈ 0.5) for a normal neural network (left) and our CCU neural network (right). The decision boundary is shown in white which is similar for both models. Our CCU-model retains high-confidence predictions in regions close to the training data, whereas far away from the training the CCU-model outputs close to uniform confidence. In contrast the normal neural network is over-confident everywhere except very close to the decision boundary. (Introduction)Figure 2: Adversarial Noise: We maximize the confidence of the OOD methods using PGD in the ball around a uniform noise sample (seed images, left) on which CCU is guaranteed by Corollary ?? to yield less than 1.1 1 M maximal confidence. For each OOD method we report the image with the highest confidence. Maha and MCD use scores where lower is more confident (indicated by ∗). If we do not find a sample that has higher confidence/lower score than the mean of the in-distribution, we highlight this in boldface. All other OOD methods fail on some dataset, see Table ?? for a quantitative version. Arguably ACET produces good samples on MNIST, but the sample is classified as a 5. ODIN at high temperatures always returns low confidence, so a value of 0.1 is not informative. (Experiments)Figure 3: Histograms of bounds: Certified radius in transformed space for different datasets. (Appendix Finding a the certifiable radius)›