{"cells":[{"cell_type":"markdown","metadata":{},"source":["# [1910.04241v1] Out-of-distribution Detection in Classifiers via Generation"]},{"cell_type":"markdown","metadata":{},"source":["Abstract By design, discriminatively trained neural network classifiers produce reliable predictions only for in-distribution samples. For their real-world deployments, detecting out-of-distribution (OOD) samples is essential. Assuming OOD to be outside the closed boundary of in-distribution, typical neural classifiers do not contain the knowledge of this boundary for OOD detection during inference. There have been recent approaches to instill this knowledge in classifiers by explicitly training the classifier with OOD samples close to the in-distribution boundary. However, these generated samples fail to cover the entire in-distribution boundary effectively, thereby resulting in a sub-optimal OOD detector. In this paper, we analyze the feasibility of such approaches by investigating the complexity of producing such “effective” OOD samples. We also propose a novel algorithm to generate such samples using a manifold learning network (e.g., variationalautoencoder) and then train an n+1 classifier for OOD detection, where the n + 1th class represents the OOD samples. We compare our approach against several recent classifier-based OOD detectors on MNIST and Fashion-MNIST datasets. Overall the proposed approach consistently performs better than the others."]},{"cell_type":"markdown","metadata":{},"source":["We find that only a few OOD samples are sufficient to guide the decision boundary of the classifier to be bounded around the in-distribution regions evidenced by their OOD detection results"]},{"cell_type":"markdown","metadata":{},"source":["![title](https://a2c.fyi/q/T65T9mEJnY/f0.png)"]},{"cell_type":"markdown","metadata":{},"source":["Figure 1 Figure 2 Figure 3: Decision boundaries change and become more bounded around in-distribution when a classifier is trained with an n + 1th class containing OOD samples. (a) Unbounded decision boundaries of a typical 4-class classifier. (b) A 5-class classifier trained with outlier samples ‘x’ forming the fifth class, that are close to in-distribution resulting in bounded decision boundaries around in-distribution. (Introduction)"]},{"cell_type":"markdown","metadata":{},"source":["![title](https://a2c.fyi/q/T65T9mEJnY/f1.png)"]},{"cell_type":"markdown","metadata":{},"source":["Figure 4: Type I Figure 5: Type II Figure 6: Categories of outliers that we generate: (a) Type I (yellow), which includes samples that are close to the data but outside the in-distribution sub-manifolds, and (b) Type II (black), which includes samples that lie on the in-distribution sub-manifolds and trace the in-distribution boundary; in-distribution clusters are represented through blue and red points. (Introduction)"]},{"cell_type":"markdown","metadata":{},"source":["![title](https://a2c.fyi/q/T65T9mEJnY/f2.png)"]},{"cell_type":"markdown","metadata":{},"source":["Figure 7: MNIST (Type I) Figure 8: MNIST (Type II) Figure 9: F-MNIST (Type I) Figure 10: F-MNIST (Type II) Figure 11: Generated outlier samples using the proposed method; Type I outliers typically modify the background pixels (normal components have the least variance), while Type II outliers modify the object pixels. (Out-of-distribution Sample Generation)"]},{"cell_type":"markdown","metadata":{},"source":["$$\\begin{equation}\nJ(x) = \\frac{\\partial g(z)}{\\partial z}\\bigg|_{z=h(x)} \n\\end{equation}$$"]},{"cell_type":"markdown","metadata":{},"source":["![title](https://a2c.fyi/q/T65T9mEJnY/f3.png)"]},{"cell_type":"markdown","metadata":{},"source":["Figure 12: 3d plot Figure 13: 2d projection Figure 14: Generated OOD samples using a joint training of a GAN and a confident-classifier. We observe that the generated OOD samples don’t cover the entire in-distribution boundary. (Generating OOD samples using a GAN vs Our approach)"]},{"cell_type":"markdown","metadata":{},"source":["![title](https://a2c.fyi/q/T65T9mEJnY/f4.png)"]},{"cell_type":"markdown","metadata":{},"source":["Figure 15 Figure 16 Figure 17 Figure 18: Generated boundary OOD samples using our approach. (a) 3d plot of in-distribution data with out-of-manifold boundary OOD samples. (b) 3d plot of in-distribution data with on-manifold boundary OOD samples. (c) 2d projection of in-distribution data with on-manifold boundary samples to show that they cover the in-distribution boundary on the manifold. (Generating OOD samples using a GAN vs Our approach)"]},{"cell_type":"markdown","metadata":{},"source":["$$\\begin{align}\n \\Tilde{x} = x + \\beta v(x)\n\\end{align}$$"]},{"cell_type":"markdown","metadata":{},"source":["![title](https://a2c.fyi/q/T65T9mEJnY/f5.png)"]},{"cell_type":"markdown","metadata":{},"source":["Figure 19: Encoder architecture Figure 20: Decoder architecture Figure 21: Classifier architecture (Experimental Architecture)"]}],"metadata":{"kernelspec":{"display_name":"Python 3","language":"python","name":"python3"}},"nbformat":4,"nbformat_minor":2}