Visual Out-of-Distribution Detection in Open-Set Noisy Environments

Abstract

The presence of noisy examples in the training set inevitably hampers the performance of out-of-distribution (OOD) detection. In this paper, we investigate a previously overlooked problem called OOD detection under asymmetric open-set noise, which is frequently encountered and significantly reduces the identifiability of OOD examples. We analyze the generating process of asymmetric open-set noise and observe the influential role of the confounding variable, entangling many open-set noisy examples with partial in-distribution (ID) examples referred to as hard-ID examples due to spurious-related characteristics. To address the issue of the confounding variable, we propose a novel method called Adversarial Confounder REmoving (ACRE) that utilizes progressive optimization with adversarial learning to curate three collections of potential examples (easy-ID, hard-ID, and open-set noisy) while simultaneously developing invariant representations and reducing spurious-related representations. Specifically, by obtaining easy-ID examples with minimal confounding effect, we learn invariant representations from ID examples that aid in identifying hard-ID and open-set noisy examples based on their similarity to the easy-ID set. By triplet adversarial learning, we achieve the joint minimization and maximization of distribution discrepancies across the three collections, enabling the dual elimination of the confounding variable. We also leverage potential open-set noisy examples to optimize a K+1-class classifier, further removing the confounding variable and inducing a tailored K+1-Guided scoring function. Theoretical analysis establishes the feasibility of ACRE, and extensive experiments demonstrate its effectiveness and generalization. Code is available at https://github.com/Anonymous-re-ssl/ACRE0.

1 Introduction

Ensuring the reliability of machine learning models during real-world deployments is crucial, and out-of-distribution (OOD) detection plays a vital role in achieving this goal (Fang et al., 2022; Nguyen et al., 2015; Yang et al., 2021b; Hell et al., 2021). Pioneering studies have demonstrated impressive performance when the test data belongs to classes that are not seen during training (Yang et al., 2021b; Hendrycks and Gimpel, 2017; Liang et al., 2017; Liu et al., 2020a; Sun et al., 2021a). In practice, this achievement is contingent upon the essential prerequisite that the labeled training in-distribution (ID) data are devoid of any data noise.

Fig. 1

Motivation illustration. a the problem of OOD detection under asymmetric open-set noise. b, c asymmetric open-set noise in the training set poses a greater detriment than the symmetric. AUROC and AUPR are metrics for OOD detection. The larger the AUROC and AUPR, the better the performance of the OOD detection

The presence of noisy examples in the training set inevitably hampers the effectiveness of OOD detection (Wu et al., 2021). While training-time OOD detection methods make adjustments to the model training process, the inclusion of noisy examples in the training set can disrupt classification boundaries (Hendrycks et al., 2018; Chen et al., 2021; He et al., 2022a). Conversely, test-time OOD detection methods heavily rely on trained models and design an OOD scoring function for identifying unseen classes without access to the training data (Hendrycks and Gimpel, 2017; Liang et al., 2017; Liu et al., 2020a). Accordingly, their performance is irreversibly compromised when the noisy datasets interfere with the prediction uncertainties of trained models, particularly if the noisy dataset contains open-set noisy examples that closely resemble the upcoming OOD examples.

To address the challenge of OOD detection in the presence of noisy training examples, a few training-time OOD detection methods have been proposed (Wu et al., 2021; Yu and Aizawa, 2020; Wei et al., 2021). These methods primarily focus on addressing closed-set noise (e.g., label noise) and symmetric open-set noise. Symmetric open-set noise occurs when examples from unobserved classes are distributed across ID classes in a random manner without considering the characteristics of the examples. We argue that this problem can be effectively resolved by combining existing techniques in noisy label learning and OOD detection (Wu et al., 2021; Wei et al., 2021). Pioneering studies have also shown that symmetric open-set noise can even provide benefits for OOD detection  (Hendrycks et al., 2018; Ming et al., 2022). Our experiments show that existing methods are limited in handling OOD detection in scenarios with previously overlooked asymmetric open-set noise. Asymmetric open-set noise frequently occurs when OOD classes are distributed towards the ID classes that share spurious-related characteristics with them, as depicted in Fig. 1a. Compared with symmetric open-set noise, the presence of asymmetric open-set noise significantly decreases the OOD detection performance by reducing the identifiability of OOD examples, as depicted in Fig. 1b and c.

We delve into the data-generating process of asymmetric open-set noise using structural causal models (Pearl, 2009). We observe that the confounding variable plays a significant role in entangling many open-set noisy examples with a subset of ID examples, referred to as “hard-ID examples”, due to spurious-related characteristics. For instance, consider an ID dataset that includes the shoe class while the open-set classes encompass zebra-related classes in Fig. 1a. Due to the presence of zebra-like stripes on a subset of shoe examples, the data collection process (e.g., from web sources) often leads to misclassifying examples from the zebra class as belonging to the shoe class (Han et al., 2022a). In this scenario, the shoe examples with zebra stripes represent hard-ID examples, while the other shoes constitute easy-ID examples. The zebra stripes act as a potentially confounding variable, reducing the identifiability of OOD examples, especially when the OOD data comprise zebra-related classes.

In this paper, we introduce Adversarial Confounder REmoving (ACRE) as a solution to eliminating the confounding variable. ACRE employs progressive optimization with adversarial learning to curate three distinct collections of potential examples: easy-ID, hard-ID, and open-set noisy. Inspired by domain-invariant representation learning (Nguyen et al., 2021; Han et al., 2022b; Wang et al., 2022a; Jang et al., 2022), this approach allows us to simultaneously develop invariant representations from ID examples while reducing spurious-related representations from open-set noisy examples. To begin, we can efficiently identify the easy-ID examples with minimal effect from confounding variable by small loss criteria (Han et al., 2018; Jiang et al., 2018). From these examples, we learn invariant representations that help differentiate between hard-ID and open-set noisy examples based on their similarity to the easy-ID set. Simultaneously, we employ triplet adversarial learning to facilitate the minimization and maximization of distribution discrepancies across the three collections, successfully achieving the dual elimination of the confounding variable. In addition, we leverage potential open-set noisy examples to optimize a K+1-class classifier. This process aids in removing the confounding variable and induces a tailored K+1-Guided scoring function. We give a theoretical analysis to verify the feasibility of triplet adversarial learning in confounder removal.

In the field of computer vision, the concept of Open-World Visual Recognition has emerged as a critical area of research. Its aim is to develop resilient systems capable of handling real-world scenarios, where out-of-distribution (OOD) data is common. Our research focuses on Visual Out-of-Distribution Detection in Open-Set Noisy Environments, addressing the urgent need for algorithms that can effectively identify OOD instances within complex real-world visual data. By introducing innovative methodologies and insights to the field of Open-World Visual Recognition, our work aims to advance the frontier of visual recognition systems towards greater adaptability and robustness, to cope with dynamic and uncertain environments.

Our contributions can be summarized as follows:

• We introduce and investigate the problem of OOD detection under asymmetric open-set noise, which accommodates a variety of real-world applications but is unexplored.

• We propose a novel method called ACRE that employs an adversarial learning approach to remove the confounding variable between open-set noisy examples and hard-ID examples, resulting in improved OOD detection performance.

• Theoretical analysis and empirical results demonstrate the feasibility and effectiveness of ACRE on real-world datasets, and ACRE can pave a solid baseline for future studies.

2 Related Work

2.1 Out-of-Distribution Detection

The ability to distinguish between in-distribution (ID) and out-of-distribution (OOD) data is a fundamental concern for deploying machine learning models in real-world applications. OOD detection methods can be broadly classified into two main categories: classification-based methods (Hendrycks and Gimpel, 2017; Liang et al., 2017; Lee et al., 2018; Liu et al., 2020a; Gomes et al., 2022; Sun et al., 2022a; Ming and Li, 2023; Yang et al., 2023), density-based methods (Ren et al., 2019; Xiao et al., 2020; Morningstar et al., 2021; Zhou and Levine, 2021; Jiang et al., 2021; Zhang et al., 2021). Classification-based methods for detecting out-of-distribution (OOD) data involve modeling the conditional distribution of the in-distribution (ID) data, and then designing a scoring function to measure the uncertainty of test data. Density-based methods model the ID distribution using probabilistic models and consider test data in low-density regions as OOD data. Density-based OOD detection methods can be difficult to train and optimize, often yielding inferior performance compared to classification-based methods (Yang et al., 2021b). Therefore, in this paper, we focus on classification-based methods. Within this category, there are two main branches of research: testing-time methods (Hendrycks and Gimpel, 2017; Liang et al., 2017; Lee et al., 2018; Liu et al., 2020a; Wang et al., 2022b; Zhu et al., 2022; Song et al., 2022a; He et al., 2024b) and training-time methods (Ming et al., 2023; Ming and Li, 2023; Yang et al., 2023; Du et al., 2023; He et al., 2024ba). Test-time methods are easy to use without modifying the training procedure and objective (Yang et al., 2021b).

Unlike testing-time methods, training-time methods aim to mitigate overconfident predictions for OOD data during the training period. According to whether the OOD-supervised signals are used in the training process, training-time methods can be categorized into OOD-free and OOD-needed methods. The representatives of OOD-free methods are Wei et al. (2022), Lin et al. (2021). Wei et al. (2022) decoupled the influence of logits’ norm from the training procedure by incorporating LogitNorm into the cross-entropy loss. Lin et al. (2021) exploited intermediate classifier outputs for dynamic and efficient OOD inference. The OOD-needed methods aim to calibrate the model by OOD-supervised signals, which are from auxiliary OOD datasets (Hendrycks et al., 2018; Liu et al., 2020a; Chen et al., 2021; Ming et al., 2022; Wang et al., 2023), unlabeled data He et al. (2022b), Yu and Aizawa (2019), Yang et al. (2021a), Zhou et al. (2021), Katz-Samuels et al. (2022), He et al. (2024ba), or synthetic virtual OOD data (Du et al., 2022; Tang et al., 2021; Tack et al., 2020; He et al., 2022a; Du et al., 2023).

Nevertheless, the OOD detection methods commonly used in representative works assume an impeccable learning environment in which the labeled ID data is noise-free. In real-world applications, however, this assumption is often unattainable, which can severely compromise the robustness of these methods. There are only three pioneering works (Yu and Aizawa, 2020; Wei et al., 2021; Wu et al., 2021) that leverage label cleaning, geometric structure or injects open-set auxiliary data to enhance OOD detection. Although these methods have addressed the problem of OOD detection under noisy environments, the open-set noise they consider is symmetric. According to Hendrycks et al. (2018), Ming et al. (2022), Wang et al. (2023), we can know that symmetric open-set noise is helpful for OOD detection. However, in many real-world scenarios, open-set noise is not symmetric (random) but rather asymmetric (dependent). According to Fig. 1 in main body, we can know that asymmetric open-set noise is harmful to OOD detection severely. Our paper aims to address the issue of OOD detection under asymmetric open-set noise, which is a highly challenging and valuable problem that has received relatively little attention.

2.2 Learning from Noisy Labels

Previous works on learning from noisy labels can be classified into three categories: label-based, sample-based, and loss-based methods. Early methods focused on correcting corrupted labels by estimating the noise transition matrix Patrini et al. (2017), Goldberger and Ben-Reuven (2017), but this approach is challenging due to the difficulty in accurately estimating the matrix. Sample-based methods (Han et al., 2018; Wei et al., 2020; Yao et al., 2023) aim to select representative samples for training, while loss-based methods (Reed et al., 2014; Zhang and Sabuncu, 2018) focus on using robust loss functions to improve model performance. However, these methods are only designed for close-set noise in the training set. More recent works (Wang et al., 2018; Sun et al., 2020; Yu and Aizawa, 2020; Sachdeva et al., 2021; Li et al., 2020; Yao et al., 2021; Li et al., 2021; Xia et al., 2022; Sun et al., 2022b; Wei et al., 2021; Wan et al., 2024) propose to handle both in-distribution (IND) and out-of-distribution (OOD) noise in training datasets. However, these approaches are not directly applicable for detecting OOD data at test time. Combining them with existing OOD detection methods may not yield satisfactory performance (Wu et al., 2021).

3 Methodology

In this section, we first introduce the learning set-up. Then, we point out the key insight by the structural causal model. According to the insight, we present adversarial confounder removing.

3.1 Introduction of Different Noise Type

Table 1 Classification of different noise types

We introduce a classification of different noise types, as shown in Table 1. Then, we introduce the different noise types in detailed as follows:

• Symmetric Close-Set Noise: The noise belongs to one of the known categories, but the label is incorrect. Moreover, the distribution of noise is uniform, meaning that samples from every category have the same probability of being incorrectly labeled as another category. This noise model does not favor any specific mislabeling pattern, and is therefore considered “symmetric."

• Asymmetric Close-Set Noise: The noise belongs to one of the known categories, but the label is incorrect. Moreover, the distribution of noise is uneven, with samples from certain categories more likely to be mislabeled as specific other categories. This type of noise usually occurs between categories that are similar or easily confused with one another, for instance, mislabeling a known wolf category as a known dog category is more common than mislabeling it as a known cat category.

• Symmetric Open-Set Noise: The noisy samples do not belong to any known category in the training set, and these samples are evenly distributed across known categories.

• Asymmetric Open-Set Noise: The noisy samples do not belong to any known category in the training set, but these samples have an uneven probability of being misclassified into specific known categories. For example, in an animal classification task, there may be samples from new animal categories not included in the training set, and these samples are more likely to be classified into specific categories that resemble them in appearance or ecological characteristics.

3.2 Problem Set-Up

We consider a noisy training set $\mathcal {D}_{in}^{train}=\left\{ \left( \textbf{x}_{k}, \tilde{\textbf{y}}_{k}\right) \right\} _{k=1}^{n}$ where $\textbf{x}_{k} \in \mathcal {X}$, $\tilde{\textbf{y}}_{k} \in \mathcal {Y}$, $\mathcal {X}$ denotes the input space, $\mathcal {Y}$ denotes the ID label space, $\mathcal {Y}=\{1,2,\dots ,K\}$, K denotes the number of ID classes, and n denotes the number of examples in $\mathcal {D}_{in}^{train}$. We assume the examples from ID classes are clean. In our setting of OOD detection under asymmetric open-set noise, $\mathcal {D}_{in}^{train}$ contains two types of example: (1) ID example $\textbf{x}_{i}$ whose assigned label $\textbf{y}_i$ is the same as the ground-truth label $\textbf{y}_{k}^{*}$ and $\textbf{y}_{k}^{*} \in \mathcal {Y}$; (2) Asymmetric open-set noise example $\textbf{x}_{o}$ whose assigned label $\tilde{\textbf{y}}_{k}$ does not equal to the ground-truth label $\textbf{y}_{k}^{*}$, $\tilde{\textbf{y}}_{k} \in \mathcal {Y}$ but $\textbf{y}_{k}^{*} \notin \mathcal {Y}$, and $\tilde{\textbf{y}}_{k}$ is assigned based on similarity to ID classes.

Let $\mathcal {D}^{test}$ denote the test set, which consists of ID test set $\mathcal {D}_{in}^{test}$ and OOD test set $\mathcal {D}_{out}^{test}$. The example $\textbf{x}_{it}$ in $\mathcal {D}_{in}^{test}$ is from the ID classes. The example $\textbf{x}_{ot}$ in $\mathcal {D}_{out}^{test}$ is from the unknown classes. The goal of OOD detection is to define a decision function $\mathcal {F}$ such that for a given test input $\textbf{x} \in \mathcal {D}^{test}$,

$$\begin{aligned} \mathcal {F}(\textbf{x})= {\left\{ \begin{array}{ll} 0 &{} \text { if } \textbf{x} \in \mathcal {D}_{out}^{test}\,, \\ 1 &{} \text { if } \textbf{x} \in \mathcal {D}_{in}^{test} \,, \end{array}\right. } \end{aligned}$$

(1)

where $\mathcal {F}(\textbf{x})=1$ means that $\textbf{x}$ is ID data and $\mathcal {F}(\textbf{x})=0$ means that $\textbf{x}$ is OOD data.

3.3 Problem Analysis

We reveal a generating process of the training set by Fig. 2, where the shaded variables are unobservable, and the unshaded variables are observable. The left graph in Fig. 2 and Eq. (2) reveal the generating process of ID data.

$$\begin{aligned} \textbf{z}_a \sim p_{\textbf{z}_a}, \textbf{z}_b \sim p_{\textbf{z}_b}, \textbf{x}_i=g\left( \textbf{z}_a, \textbf{z}_b\right) \,. \end{aligned}$$

(2)

In the generating process of ID data, we assume that ID data $\textbf{x}_i\in \mathcal {X}$ is generated by latent variable $\textbf{z} \in \mathcal {Z} \subseteq \mathbb {R}^m$ through a function $g: \mathcal {Z} \rightarrow$ $\mathcal {X}$. We partition latent variable $\textbf{z}$ into two variables $\textbf{z}=[\textbf{z}_a, \textbf{z}_b]$. $\textbf{z}_a$ denotes the private variable which only $\textbf{x}_i$ owns. $\textbf{z}_b$ denotes the confounding variable which both $\textbf{x}_i$ and $\textbf{x}_o$ own simultaneously. Further, we assume that $\textbf{y}_i$ is generated by the private variable $\textbf{z}_a$ and the confounding variable $\textbf{z}_b$.

Fig. 2

The training data generating process: The gray shade of nodes indicates that the variables are unobservable

The right graph in Fig. 2 and Eq. (3) reveal the generating process of asymmetric open-set noise.

$$\begin{aligned} \textbf{z}_b \sim p_{\textbf{z}_b}, \textbf{z}_c \sim p_{\textbf{z}_c}, \textbf{x}_o=g\left( \textbf{z}_b, \textbf{z}_c\right) \,. \end{aligned}$$

(3)

In the generating process of asymmetric open-set noise, we assume that open-set data $\textbf{x}_o \in \mathcal {X}$ is generated by latent variables $\textbf{z}_b$ and $\textbf{z}_c$. $\textbf{z}_c$ denotes the private variable which only $\textbf{x}_o$ owns. $\tilde{\textbf{y}}_o$ denotes the noisy label of open-set data, and $\tilde{\textbf{y}}_o$ is generated due to the biased influence $\textbf{z}_b \rightarrow \textbf{y}$.

During the inference stage, when the tested OOD example $\textbf{x}_{ot}$ in $D^{test}_{out}$ contains $\textbf{z}_b$, $\textbf{x}_{ot}$ is likely to be identified as ID example, resulting in poor performance of OOD detection. The existence of $\textbf{z}_b$ is the essential reason for the performance decline of OOD detection. A pivotal insight is to remove confounding variable $\textbf{z}_b$ to increase the separability of ID and OOD data, thus improving the performance of OOD detection.

Fig. 3

The t-SNE visualization

We visualized the t-SNE graph of CIFAR-10 ID data contaminated with asymmetric open-set noise from CIFAR-100. In Fig. 3, the points labeled from ‘0’ to ‘9’ represent the features of ID examples, while the points labeled as ‘10’ correspond to the features of asymmetric open-set noise examples. We can see that the ID data and open-set noise are difficult to separate due to their spurious-related features, acting as a confounding variable. This reduces the separability of ID and OOD data, leading to poor OOD detection performance. Based on Fig. 3, we also find that certain ID examples (at the edge) can be well separated from open-set examples, with low influence from confounding variable $\textbf{z}_b$. These are easy-ID examples ($\textbf{x}_e$), almost generated by $\textbf{z}_a$. The remaining ID examples ($\textbf{x}_h$) are hard-ID examples, generated by both $\textbf{z}_a$ and $\textbf{z}_b$. To address the confounding between $\textbf{x}_h$ and $\textbf{x}_o$, we propose adversarial confounder removing, which uses adversarial learning on $\textbf{x}_e$, $\textbf{x}_h$, and $\textbf{x}_o$ to remove $\textbf{z}_b$.

3.4 Adversarial Confounder Removing

Adversarial Confounder REmoving (ACRE) includes three components: 1) a triplet estimation module to obtain $\textbf{x}_e$, $\textbf{x}_h$, and $\textbf{x}_o$; 2) a triplet adversarial learning module that uses adversarial learning on $\textbf{x}_e$, $\textbf{x}_h$, and $\textbf{x}_o$ to remove $\textbf{z}_b$; and 3) a K+1-Guided scoring function to detect OOD data. The network consists of three subnetworks: 1) feature extractor G; 2) two-head classifier (including a K-class classifier E and a K+1-class classifier C); and 3) discriminator D.

Our method includes a pre-training phase and a training phase. The pre-training phase primarily focuses on obtaining the initial feature extractor G, a K-classifier head E, and a K+1 classifier head C. The training phase mainly includes three components: triplet estimation, discriminative learning, and adversarial learning. The triplet estimation component provides uncertainty estimates and continuously updated triplets for the next two modules. The discriminative learning and adversarial learning components utilize the estimated triplets for adversarial training to remove confounding factors, thereby improving the identifiability of OOD data. Detailed optimization workflow can be seen in Fig. 4 and Algorithm 1.

Algorithm 1

Adversarial Confounder REmoving (ACRE).

Fig. 4

The overview of ACRE for OOD detection under asymmetric open-set noise

3.4.1 Triplet Estimation

In this part, we first select easy-ID examples based on small loss criteria. Then we propose $\omega (\textbf{x})$ estimation to progressively identify potential hard-ID examples and open-set noisy examples, where $\omega (\textbf{x})$ denotes the probability that the example $\textbf{x}$ belongs to the ID classes.

Easy-ID examples selection. We identify the easy-ID examples by the small loss criteria (Han et al., 2018; Jiang et al., 2018):

$$\begin{aligned} \hat{D}_{E}&=\{(\textbf{x}_k, \tilde{\textbf{y}}_k)| (\textbf{x}_k, \tilde{\textbf{y}}_k) \in \mathcal {D}_{in}^{train}, \bar{\ell }(\textbf{x}_k)<\zeta \} ,\nonumber \\ b\bar{\ell }(\textbf{x}_k)&=-\frac{1}{T_1} \sum _{t=1}^{T_1} \log \left( \frac{e^{E_{\tilde{\textbf{y}}_k}(G(\textbf{x}_k))}}{\sum _{j=1}^{K}e^{E_j(G(\textbf{x}_k))}}\right) \,, \end{aligned}$$

(4)

where $T_1$ denotes the number of epoch to select easy-ID data, $\zeta$ is the pre-defined threshold, and $E_j(\cdot )$ denotes the j-th logit from classifier E. After obtaining the easy-ID data set $\hat{D}_{E}$, we use $\hat{D}_{R}$ to denote the remaining set, which contains both hard-ID examples and open-set noisy examples.

Progressive estimation of $\omega (\textbf{x})$. The estimation of $\omega (\textbf{x})$ contains two stages: the pre-training and updating stage. The updating of $\omega (\textbf{x})$ is progressive and is performed simultaneously with Triplet Adversarial Learning (see Sect. 3.4.2). Details optimization workflow can be seen in Fig. 4.

During the pre-training stage, we utilize all examples in $\mathcal {D}_{in}^{train}$ to pre-train the feature extractor G and two-head classifier. In the optimization process, we utilize $\mathcal {D}_{in}^{train}$ to optimize E and utilize $\hat{D}_{E}$ to optimize C. The pre-training optimization objective $\mathcal {L}_{pre}$ is defined by

$$\begin{aligned} \mathcal {L}_{pre}&=-\frac{1}{n} \sum _{\left( \textbf{x}_k, \tilde{\textbf{y}}_{k}\right) \in \mathcal {D}_{in}^{train}} \log \left( \frac{e^{E_{\tilde{\textbf{y}}_{k}} (G({\textbf{x}_{k}}))}}{\sum _{j=1}^{K} e^{E_j(G({\textbf{x}_{k}}))}}\right) \nonumber \\&\quad -\frac{1}{n_e} \sum _{\left( \textbf{x}, y\right) \in \hat{D}_E} \log \left( \frac{e^{C_{y}(G(\textbf{x}))}}{\sum _{j=1}^{K+1}e^{C_j(G(\textbf{x}))}}\right) \,, \end{aligned}$$

(5)

where $n_e$ denotes the number of examples in $\hat{D}_E$. After the pre-trained stage by minimizing $\mathcal {L}_{pre}$, we obtain pre-trained G, E, and C. Then, based the pre-trained G, E, and C, we estimate $\omega (\textbf{x})$ by

$$\begin{aligned} \omega (\textbf{x}) = (1-\tau ) \cdot f^e(E(G(\textbf{x}))) + \tau \cdot (1-f^c(C(G(\textbf{x}))) )\,, \end{aligned}$$

(6)

where $f^e$ denotes maximum softmax probability (MSP) (Hendrycks and Gimpel, 2017), $f^c$ denotes the K+1-th softmax probability, and $\tau$ is a harmonization factor. Our experiments verify that using two different classifiers can improve the estimation of $\omega (\textbf{x})$ because, at the initial stage, E trained with all examples outperforms C, E can primarily guide C to constantly self-growth and self-renewal trained with easy-ID examples.

During the updating stage, we begin by preliminarily splitting $\hat{D}_{R}$ into hard-ID set $\hat{D}_H$ and open-set noisy set $\hat{D}_O$ by $\omega \hat{D}_{R}$ and $(1-\omega )\hat{D}_{R}$, respectively. Let $n_h$ and $n_o$ denote the number of examples in $\hat{D}_H$ and $\hat{D}_O$. Then, to improve the estimation of $\omega (\textbf{x})$, we rectify the label of $\hat{D}_O$ to $K+1$ and fine-tune the two-head classifier with $\hat{D}_H$ and $\hat{D}_O$ by $\mathcal {L}_{C}$, which is defined by

$$\begin{aligned} \mathcal {L}_{C}&= -\frac{\lambda _2}{n_h+n_e} \sum _{\left( \textbf{x}, y\right) \in \hat{D}_H \cup \hat{D}_E} \log \left( \frac{e^{C_y(G(\textbf{x}))}}{\sum _{j=1}^{K+1}e^{C_j(G(\textbf{x}))}}\right) \nonumber \\&\quad -\frac{\lambda _3}{n_o}\sum _{\textbf{x}\in \hat{D}_O} \log \left( \frac{e^{C_{K+1}(G(\textbf{x}))}}{\sum _{j=1}^{K+1} e^{C_j(G(\textbf{x}))}}\right) \,, \end{aligned}$$

(7)

where $\lambda _2$ and $\lambda _3$ denote coefficients. The first item in Eq. (7) optimizes the first K outputs of C, and the second item in Eq. (7) optimizes the K+1-th output of C. Then, we update $\omega (\textbf{x})$ by optimizing the second item in Eq. (6) with fine-tuned G and C. Besides updating $\omega (\textbf{x})$, another advantage of the K+1-class classifier is that it can be utilized to design an OOD scoring function (see Sect. 3.4.3).

3.4.2 Triplet Adversarial Learning

The objective of this section is to remove the confounding variable through adversarial learning over the estimated triplet, comprising both the discriminative and adversarial stages.

During the discriminative stage, we introduce a triplet discriminator D as follows,

$$\begin{aligned} D(G(\textbf{x})) = [D_0(G(\textbf{x})), D_1(G(\textbf{x})), D_2(G(\textbf{x}))]\,, \end{aligned}$$

(8)

where $D_0(\cdot )$, $D_1(\cdot )$, and $D_2(\cdot )$ represent the probability that example $\textbf{x}$ belongs to easy-ID subset, hard-ID subset, and open-set subset, respectively. Given these three dimensions outputted by D, we propose the triplet discrimination loss $\mathcal {L}_D$, which is defined by

$$\begin{aligned} \mathcal {L}_D (x)&= \frac{1}{n_e}\sum _{\textbf{x} \in \hat{D}_E}[ -\log {D_{0}}(G(\textbf{x}))] \nonumber \\&\quad + \frac{1}{n_h}\sum _{\textbf{x}\in \hat{D}_H}[ -\log {D_{1}}(G(\textbf{x}))] \nonumber \\&\quad + \frac{1}{n_o}\sum _{\textbf{x}\in \hat{D}_O} [ -\log {D_{2}}(G(\textbf{x}))] \,. \end{aligned}$$

(9)

By fixing G and optimizing D with $\mathcal {L}_D$, we can obtain an optimal discriminator $D^*$, which can identify which of the three triplets the example comes from.

During the adversarial stage, our approach tackles the removal of the confounding variable by learning invariant representations across easy-ID and hard-ID examples and minimizing the spurious-related representations between hard-ID examples and open-set noisy examples. However, designing an effective adversarial loss remains a critical challenge, as prior adversarial methods typically employ two-dimensional discriminators (Gui et al., 2023; Ganin et al., 2016). In contrast, our method utilizes a three-dimensional discriminator. To address this issue, we propose the OOD-aware triplet adversarial loss $\mathcal {L}_{OTA}$:

$$\begin{aligned} \mathcal {L}_{OTA} (x)&= \frac{1}{n_e}\sum _{\textbf{x} \in \hat{D}_E}[ -\log {D_{1}}(G(\textbf{x}))] \nonumber \\&\quad + \frac{1}{n_h}\sum _{\textbf{x}\in \hat{D}_H}[ -\log {D_{0}}(G(\textbf{x}))] \nonumber \\&\quad + \frac{1}{n_o}\sum _{\textbf{x}\in \hat{D}_O} [ -\log {D_{2}}(G(\textbf{x}))] \,. \end{aligned}$$

(10)

Adversarial optimization. Based on Eq. (9) and Eq. (10), we define the bi-level optimization by

$$\begin{aligned}&\min _{G} \frac{1}{n_e}\sum _{\textbf{x}\in \hat{D}_E}[ -\log {D_{1}^{*}}(G(\textbf{x}))] \nonumber \\&\quad + \frac{1}{n_h}\sum _{\textbf{x}\in \hat{D}_H}[ -\log {D_{0}^{*}}(G(\textbf{x}))] \nonumber \\&\quad + \frac{1}{n_o}\sum _{\textbf{x}\in \hat{D}_O} [ -\log {D_{2}^{*}}(G(\textbf{x}))]\,, \nonumber \\&\quad s.t. D^{*}=\arg \min _{D} \frac{1}{n_e}\sum _{\textbf{x} \in \hat{D}_E}[ -\log {D_{0}}(G(\textbf{x}))] \nonumber \\&\quad + \frac{1}{n_h}\sum _{\textbf{x}\in \hat{D}_H}[ -\log {D_{1}}(G(\textbf{x}))] \nonumber \\&\quad + \frac{1}{n_o}\sum _{\textbf{x}\in \hat{D}_O} [ -\log {D_{2}}(G(\textbf{x}))]\,, \end{aligned}$$

(11)

The first two items in Eq. (11) achieve the minimization of distribution discrepancies across easy-ID and hard-ID data to learn invariant representations to remove $\textbf{z}_b$. The last two items achieve the maximization of distribution discrepancies across hard-ID data and open-set noisy data, enabling the dual elimination of spurious-related representations to remove $\textbf{z}_b$. Triplet adversarial learning increases the distribution discrepancies across hard-ID data and open-set noisy data, which contributes to better estimation of $\omega (\textbf{x})$ and curates three collections of potential examples more reliably.

Although confounding variable $\textbf{z}_b$ can be removed by optimizing Eq. (11), the classifier still outputs an overconfident prediction to OOD data. The reason is that, during the training stage, ID example $(\textbf{x}_i,\textbf{y}_i)$ and asymmetric open-set example $(\textbf{x}_o,\tilde{\textbf{y}}_o)$ are used for optimizing classifier. Classifier is constantly learning spurious influence from $\textbf{x}_o$ to $\tilde{\textbf{y}}_o$. Although $\textbf{x}_o$ is only generated by $\textbf{z}_c$ after removing $\textbf{z}_b$, classifier also learns another spurious influence: $\textbf{z}_c \rightarrow \tilde{\textbf{y}}_o$. During the inference stage, when the tested OOD example $\textbf{x}_{ot}$ in $D^{test}_{out}$ contains $\textbf{z}_c$, $\textbf{x}_{ot}$ is likely to be identified as ID example, resulting in poor performance of OOD detection. To remove the spurious influence $\textbf{z}_c \rightarrow \tilde{\textbf{y}}_o$, we propose to rectify the label of $\textbf{x}_o$ to $K+1$, and optimate model by connecting Eq. (11) with Eq. (7), which helps learn a correct influence $\textbf{z}_c \rightarrow K+1$, improve the classifier’s separability of ID and OOD data, and enhance the estimation of ID probability $\omega (\textbf{x})$.

Therefore, considering Eqs. (11) and (7), we define the final optimization formally by

$$\begin{aligned} \min _{G, C} \lambda _1 \mathcal {L}_{OTA} + \mathcal {L}_{C}, \quad s.t.\quad D^{*} = \arg \min _{D} \mathcal {L}_D\,, \end{aligned}$$

(12)

where $\lambda _1$ are coefficient.

3.4.3 OOD Detection in Testing Stage

During the test stage, we define the K+1-Guided scoring function $S(\textbf{x})$ from classifier C by

$$\begin{aligned} S(\textbf{x}) = \frac{e^{C_{K+1}(G(\textbf{x}))}}{\sum _{j=1}^{K+1} e^{C_j(G(\textbf{x}))}}\,, \end{aligned}$$

(13)

where $C_j$ denotes the j-th logit from C. For OOD detection, one can exercise the thresholding mechanism to distinguish between ID and OOD data by

$$\begin{aligned} G_{\gamma }\left( \textbf{x}\right) = {\left\{ \begin{array}{ll} \text { OOD } &{} S\left( \textbf{x} \right) \ge \gamma \,,\\ \text { ID } &{} S\left( \textbf{x}\right) <\gamma \,. \end{array}\right. } \end{aligned}$$

(14)

The threshold $\gamma$ is chosen so that a high fraction of ID data (e.g., 95%) is correctly classified (Sun et al., 2021b). All the used notations can be seen in Table 2.

Table 2 The summary of all the used notations

4 Theoretical Analysis

According to inner-level optimization in Eq. (11), we minimize $\mathcal {L}_D$ loss to find optimal $D^{*}$ with a fixed G. The output of $D^{*}$ is $D^*(z)=\left[ \frac{P_E(z)}{3 P_{a v g}(z)}, \frac{P_{H}(z)}{3 P_{a v g}(z)}, \frac{P_{O}(z)}{3 P_{a v g}(z)}\right]$, where $P_{a v g}(z) = (P_E(z) + P_{H}(z) +P_{O}(z))/3$, z denotes the output from G. $P_E(z)$, $P_H(z)$, and $P_O(z)$ are the feature distributions of easy-ID data, hard-ID data, and open-set data, respectively. We optimize G given $D^{*}$ by minimizing $\mathcal {L}_{OTA}$ loss. Then we can obtain the following Theorem.

Theorem 1

(Proof in Appendix) $\mathcal {L}_{OTA}$ loss can be expressed as,

$$\begin{aligned} \mathcal {L}_{OTA}&= KL\left( P_H \Vert P_{avg}\right) + 3KL\left( P_{avg} \Vert P_{H}\right) \nonumber \\&\quad + KL\left( P_E \Vert P_{avg}\right) + 3KL \left( P_{avg} \Vert P_{E}\right) \nonumber \\&\quad - KL\left( P_O \Vert P_{avg}\right) + O_{EH} + 5 \log 3 \,, \end{aligned}$$

(15)

where

$$\begin{aligned} O_{EH} = \int _z\left( P_O(z)\log \frac{P_H(z)}{3P_{avg}(z)} + P_O(z)\log \frac{P_E(z)}{3P_{avg}(z)}\right) d z\,. \end{aligned}$$

(16)

By minimizing $\mathcal {L}_{OTA}$ loss, the confounding variable can be removed (Fig. 5).

Fig. 5

The force analysis of optimizing $\mathcal {L}_{OTA}$

Remark We analyze $KL\left( P_H \Vert P_{avg}\right) + 3KL\left( P_{avg} \Vert P_{H}\right) + KL\left( P_E \Vert P_{avg}\right) + 3KL\left( P_{avg} \Vert P_{E}\right) - KL\left( P_O \Vert P_{avg}\right)$ by the analysis of forces in the field of physics. Since the KL divergence is asymmetric, it can be viewed as a force approximately. We use $F_{ea}, F_{ha}, F_{ah}, F_{ae}$ to denote $KL\left( P_E \Vert P_{avg}\right)$, $KL\left( P_H \Vert P_{avg}\right)$, $KL\left( P_{avg} \Vert P_{H}\right)$, $KL\left( P_{avg} \Vert P_{E}\right)$, respectively. $\mathcal {E}, \mathcal {H}, \mathcal {O}, \mathcal {A}$ denote $P_E, P_H, P_O, P_A$, located at the three vertices and the center of the triangle, respectively. $F_{aeh}$ denotes the resultant force, and its direction represents the direction $\mathcal {A}$ moves. $F_{ha}$ and $F_{ea}$ will keep $\mathcal {E}$ and $\mathcal {H}$ moving closer to $\mathcal {A}$. By optimizing $\mathcal {L}_{OTA}$, $KL\left( P_E \Vert P_{avg}\right)$, $KL\left( P_H \Vert P_{avg}\right)$, $KL\left( P_{avg} \Vert P_{H}\right)$, $\left( P_{avg} \Vert P_{E}\right)$ will keep decreasing until $P_E \approx P_H \approx P_{avg}$. We use $F_a$ denote $- KL\left( P_O \Vert P_{avg}\right)$. Minimizing $\mathcal {L}_{OTA}$ increases $KL\left( P_O \Vert \right. \left. P_{avg}\right)$, resulting in $\mathcal {O}$ constantly moving away from $\mathcal {A}$. $D_{AO}$ denotes the distance of $\mathcal {A}$ and $\mathcal {O}$ in the optimal G. Moreover, minimizing $\mathcal {L}_{OTA}$ will decrease $O_{EH}$ and the output of the open-set data on $D_0$ and $D_1$. $P_E \approx P_H \approx P_{avg}$ dictates that we learn invariant representations across $P_E$ and $P_H$, and $\mathcal {O}$ constantly moving away from $\mathcal {A}$ dictates that we minimize the spurious-related representations, successfully removing confounding variable.

Table 3 Dataset configurations for open-set noisy environments

5 Experiments

To validate the effectiveness of ACRE, we conduct a comprehensive performance evaluation, comparing it against state-of-the-art methods.

5.1 Setup

Following Wu et al. (2021), we choose CIFAR-10 and CIFAR-100 as the ID benchmark datasets, and choose CIFAR-100, TinyImageNet (Deng et al., 2009), and Places365 (Zhou et al., 2017) as the OOD benchmark datasets. Taking the ID dataset CIFAR-10 and OOD dataset CIFAR-100 as an example (abbreviated as CIFAR-10 &CIFAR-100), the generation process of the noisy training set $\mathcal {D}_{in}^{train}$ can be described in three steps. Firstly, we train a supervised model on CIFAR-10 using cross-entropy loss. Secondly, a certain percentage of open-set examples from CIFAR-100 are randomly selected, and their pseudo-labels are predicted using the trained supervised model. Lastly, the open-set examples with pseudo-labels are integrated into the training set. This procedure approximates the actual noise generation process. Table 3 shows the dataset configurations for open-set noisy environments.

We compare our proposed ACRE with state-of-the-art methods: MSP (Hendrycks and Gimpel, 2017), ODIN (Liang et al., 2017), Mahalanobis (Lee et al., 2018), Energy (Liu et al., 2020a), GradNorm (Huang et al., 2021), RankFeat (Song et al., 2022b), LogitNorm (Wei et al., 2022), NGC (Wu et al., 2021), and ODNL (Wei et al., 2021).

Similar to Liu et al. (2020b), we measure the following metrics for OOD detection: (1) the false positive rate (FPR95) of OOD examples when the true positive rate of ID examples is at 95%; (2) the area under the receiver operating characteristic curve (AUROC); and (3) the area under the precision-recall curve (AUPR).

We implement our algorithm in PyTorch using the standard ResNet-50 backbone. The model is trained for 4,000 iterations with ACRE. Mini-batch SGD is employed with a weight decay of 5e-4 and Nesterov momentum of 0.9. The learning rate is 0.002 and follows a cosine annealing schedule.

Table 4 The effectiveness of ACRE on the ID datasets CIFAR-10 and OOD datasets CIFAR-100, TinyImageNet, and Places365.

Table 5 OOD detection results of ACRE and comparison with competitive baselines on the ID datasets CIFAR-100

Table 6 The effectiveness of ACRE on the ID datasets CIFAR-10 and OOD datasets CIFAR-100, TinyImageNet, and Places365 under K+1-Guided score

5.2 Results

Table 4 presents the results obtained on the CIFAR-10 in-distribution (ID) dataset, along with three out-of-distribution (OOD) datasets: CIFAR-100, TinyImageNet, and Places365. ACRE demonstrates superior performance compared to all test-time OOD detection methods, as shown in Table 4. For instance, when compared to RankFeat, ACRE exhibits an 8.68% improvement in AUROC, 7.03% improvement in AUPR, and 27.72% improvement in FPR95 on the CIFAR-100 OOD dataset. Similarly, in comparison to Energy, ACRE achieves an 15.29% improvement in AUROC, 9.98% improvement in AUPR, and 48.99% improvement in FPR95 on the TinyImageNet OOD dataset, demonstrating the effectiveness of confounding variable removing. These results also demonstrate that test-time OOD detection methods are vulnerable to asymmetric open-set noise. Moreover, the results indicate that CIFAR-100 and TinyImageNet are closer to CIFAR-10 than Places365. Nevertheless, ACRE consistently achieves the best performance across all evaluation metrics, regardless of the proximity of the OOD dataset, thereby confirming its robustness.

Table 7 The effectiveness of ACRE on larger-scale datasets

Table 5 presents the results obtained on the CIFAR-100 in-distribution (ID) dataset, along with two out-of-distribution (OOD) datasets: TinyImageNet and Places365. ACRE surpasses all training-time OOD detection methods, including LogitNorm, NGC, and ODNL, as indicated in Table 5. For instance, compared to LogitNorm, ACRE achieves an 18.68% improvement in AUROC, 25.68% improvement in AUPR, and 41.03% improvement in FPR95 on the TinyImageNet OOD dataset. NGC and ODNL are designed to address scenarios where the ID training set contains close-set noise and symmetric open-set noise. In comparison, ACRE achieves 25.69% and 39.27% improvement in FPR95 on the TinyImageNet OOD dataset, respectively, outperforming NGC and ODNL. Notably, the test-time OOD detection method MSP exhibits a higher AUROC by 0.61% and 3.32% compared to NGC and ODNL, respectively, on the Places365 OOD dataset. These results emphasize the limitations of combining noisy label learning and OOD detection techniques in effectively handling OOD detection under asymmetric open-set noise Table 6.

5.3 More Results on Larger-Scale Datasets

To validate the effectiveness and generalizability of ACRE, we also conduct experiments on larger-scale datasets TinyImageNet and ImageNet. For TinyImageNet, we use TinyImageNet as the in-distribution (ID) data and obtain out-of-distribution (OOD) data from CIFAR-100. For ImageNet, we treat the first 100 classes (ImageNet-100) as the ID classes and the classes with index of 100 to 200 (ImageNet-100-200) as OOD classes. The specific experimental results are shown in Table 7. According to Table 7, ACRE comprehensively surpasses all the baseline methods, which demonstrates the effectiveness of our approach in addressing out-of-distribution (OOD) detection under asymmetric open-set noise. ACRE exhibits strong performance on both standard-scale datasets and larger-scale datasets, which also reflects ACRE’s good generalization ability.

5.4 Analyses

We analyze the individual strengths of three components: the adversarial loss $\mathcal {L}_{OTA}$, ID probability $\omega (\textbf{x})$ estimation through the two-head classifier, and the K+1-Guided scoring function.

Table 8 The verification of the effectiveness of $\mathcal {L}_{OTA}$ and K+1-Guided score

Effectiveness of $\mathcal {L}_{OTA}$. Table 9 reports the results of with/without adversarial loss $\mathcal {L}_{OTA}$. For instance, when CIFAR-10 is the ID dataset and Places365 is the OOD dataset, the removal of $\mathcal {L}_{OTA}$ leads to a 5.99% decrease in AUPR and a 24.84% increase in FPR95. These findings demonstrate the importance of removing the confounding variable and $\mathcal {L}_{OTA}$ effectively achieve that.

Effectiveness of $\omega (\textbf{x})$ estimation by two-head classifier. Table 8 presents the results of ACRE with different $\omega (\textbf{x})$ estimation approaches: only using E to estimate $\omega (\textbf{x})$, only using C to estimate $\omega (\textbf{x})$, using both E and C to estimate $\omega (\textbf{x})$. Comparing the results, solely relying on E leads to a decline in OOD detection performance, with approximately 1.50% decrease in AUROC, 10.53% decrease in AUPR, and 4.10% increase in FPR95. Similarly, using only C also results in a decline in OOD detection performance, with 0.60% decrease in AUROC, 3.73% decrease in AUPR, and 1.96% increase in FPR95. These findings validate the effectiveness of $\omega (\textbf{x})$ estimation by the two-head classifier (Fig. 6).

Fig. 6

A comparative analysis with methods specifically addressing open-set noise

Table 9 The verification of the effectiveness of $\mathcal {L}_{OTA}$ and K+1-Guided score

Effectiveness of the K+1-Guided scoring function. Table 9 reports the results obtained with and without the K+1-Guided scoring function. It reveals that replacing the K+1-Guided scoring function in ACRE with the MSP score leads to a considerable decrease in performance. Specifically, the average AUROC experiences a drop of 9.74%, the average AUPR declines by 16.93%, and the average FPR95 increases by 33.52%. Our findings emphasize the significant advantages of our approach, as it not only successfully eliminates the confounding variable but also enables the training of a K+1-class classifier C, which yields a novel OOD score. Remarkably, this new OOD score proves to be effective for detecting OOD instances under the presence of asymmetric open-set noise.

Effectiveness of ACRE to eliminate confounding factors. To verify that our method can eliminate confounding factors, we fixed the score as K+1-Guided score and then tested the separability between open-set data and ID (In-Distribution) data with and without the use of our ACRE. The results are shown in Table 6. According to Table 6, we can clearly see that after using ACRE, the separability between ID and OOD (Out-Of-Distribution) data significantly increases, indicating that our adversarial training to remove interfering factors is effective. Since confounding factors are key obstacles to the separability between ID and OOD, the improvement in separability after using ACRE also indicates that the confounding factors have been successfully removed by ACRE. To further prove that our method can increase the separability of ID and OOD data by removing confounder, we add t-SNE visualization in Fig. 7. According to Fig. 7, ACRE improves the separability between ID and OOD data, while reducing the variance among OOD data.

Fig. 7

Comparison of t-SNE plots without and with ACRE under CIFAR-10 &Places365

Fig. 8

An analysis of $\lambda _1$, $\lambda _{2}$, $\lambda _{3}$, and $\tau$ under different values

Table 10 An analysis of the choice of $\zeta$

5.5 Comparative Analysis with Methods Specifically Addressing Open-Set Noise

To further validate the effectiveness of ACRE, we compare ACRE with methods specifically addressing open-set noise. We compare with ONL (Wang et al., 2018), ODNL (Wei et al., 2021), NGC (Wu et al., 2021), and our ACRE. ONL (Wang et al., 2018) detects open-set noise and learns deep discriminative features in an iterative fashion. However, it primarily targets symmetric open-set noise. Symmetric open-set noise does not impair the performance of out-of-distribution detection. In our paper, we investigate asymmetric open-set noise, which is detrimental to out-of-distribution detection. ODNL (Wei et al., 2021) mitigates the problem of label noise by incorporating symmetric open-set noise. Unlike it, we address asymmetric open-set noise and focus on the task of out-of-distribution (OOD) detection. NGC (Wu et al., 2021) investigates the classification problem in scenarios mixed with closed-set noise and symmetric open-set noise. We integrate it with the OOD detection method as one of the baselines for analysis. The experimental results are shown in Fig. 6. According to Fig. 6, our method has a clear advantage compared to baseline methods involving open-set noise. Conventional solutions for symmetric open-set noise are not suitable for direct application in asymmetric open-set noise scenarios, which further validates the high value and significance of researching OOD detection in the context of asymmetric open-set noise.

5.6 Sensitivity of Hyperparameters

Our method has four most important hyperparameters, including $\tau$, $\lambda _1$, $\lambda _{2}$ and $\lambda _{3}$. Then, we analyze the sensitivity of them in detail. The sensitivity experiments about $\lambda _1$, $\lambda _{2}$ and $\lambda _{3}$ are based on ID dataset CIFAR-10 with OOD dataset CIFAR-100. The sensitivity experiment about $\tau$ is based on the ID dataset CIFAR-10 with the OOD dataset Places365.

The sensitivity of $\tau$. $\tau$ is the harmonic factor in Eq. (6), which balances the outputs by E and C. Results can be seen in Fig. 8d. Figure 8d shows that choosing a proper value of $\tau$ is critical for the performance of OOD detection to a certain extent.

Table 11 The K+1-Guided score with (without) ACRE

Fig. 9

AUROC(%) of ACRE and compared baselines with different noisy ratios

The sensitivity of $\lambda _1$, $\lambda _{2}$, and $\lambda _{3}$. $\lambda _1$ in Eq. (12) acts to balance the OOD-aware triplet adversarial loss $\mathcal {L}_{OTA}$ which is aimed to remove the confounding variable. $\lambda _{2}$ and $\lambda _{3}$ in Eq. (7) act to balance the optimization of the first K outputs of C and the $K+1$-th output of C. The impact of selecting appropriate values for $\lambda _1$, $\lambda _{2}$, and $\lambda _{3}$ are crucial for achieving optimal performance in OOD detection, as highlighted in Fig. 8a, b, and c, respectively. Specifically, when the ID dataset is CIFAR-10 and the OOD dataset is CIFAR-100, varying values of $\lambda _1$, $\lambda _{2}$, and $\lambda _{3}$ lead to significant differences in the results, with a 1.02% difference of AUROC in $\lambda _1$, a 2.41% difference of AUROC in $\lambda _{2}$, and a 2.33% difference of AUROC in $\lambda _{3}$ observed. These findings underscore the importance of carefully choosing the value of $\lambda _1$, $\lambda _{2}$, and $\lambda _{3}$ when training the model by our proposed ACRE.

The sensitivity of $\zeta$. $\zeta$ is a hype-parameter to select easy ID samples. Table 10 reflects the sensitivity of our method to $\zeta$. According to Table 10, it is clear that our method is quite sensitive to the choice of $\zeta$. When $\zeta$ changes from 0.1 to 0.3, the AUROC decreases by 3.12%. A larger $\zeta$ means a larger selected easy-ID data set, but it introduces more potential open-set noise. Therefore, we should choose a smaller $\zeta$ value to ensure that the selected easy-ID data set contains less open-set noise.

5.7 Generalization of ACRE

Generalization to unseen OOD datasets. In the previous experimental setup, $\textbf{x}_{o}$ in $\mathcal {D}_{in}^{train}$ and $\textbf{x}_{ot}$ in $\mathcal {D}_{out}^{test}$ are from the identical distribution, which follows the setting of Wu et al. (2021), Yu and Aizawa (2020), Zhou et al. (2021). To demonstrate the generalization capability of ACRE, we evaluate its performance on unseen OOD datasets, that is $\textbf{x}_{o}$ and $\textbf{x}_{ot}$ are not from the identical distribution. In our experiments, during the training phase, we utilize CIFAR-10 &CIFAR-100 and CIFAR-100 &TinyImageNet. However, during the inference phase, we evaluate the model on six unseen (new) OOD datasets, including iSUN, Places365, Texture, SVHN, LSUN-C, and LSUN-R. The results are presented in Table 11. These results highlight that removing the confounding variable effectively mitigates the detrimental effects of asymmetric open-set noise and enhances its capability to detect unseen OOD data. For instance, under CIFAR-100 &TinyImageNet, ACRE achieves an average AUROC gain of 11.93%, an average AUPR gain of 13.87%, and an average FPR95 gain of 26.13%. These compelling results substantiate the generalization ability of ACRE.

Generalization to different noisy ratios. To assess the generalization capability of ACRE, we conduct experiments on CIFAR-10 &CIFAR-100 with varying ratios of asymmetric open-set noise: 30%, 50%, and 70%. The results are depicted in Fig. 9. The findings can be summarized as follows. Firstly, as the ratio of asymmetric open-set noise increases, the performance of OOD detection declines across all methods, confirming the adverse impact of asymmetric open-set noise on OOD detection. Secondly, ACRE consistently outperforms all methods by a significant margin, affirming the effectiveness of removing the confounding variable as a strategy to mitigate the negative effects of asymmetric open-set noise on OOD detection. Lastly, ACRE demonstrates superior stability compared to the baselines, exhibiting reduced susceptibility to the influence of asymmetric open-set noise. These observations validate the robustness and generalization capability of ACRE.

6 Conclusion

In this paper, we investigated a previously overlooked problem in detecting OOD examples under the presence of asymmetric open-set noise. Despite its broad applications in the real world, this problem presents significant challenges. To address this problem, we proposed Adversarial Confounder REmoving (ACRE) that introduces triplet estimation and triplet adversarial learning to remove the confounding variable between open-set noisy examples and hard-ID examples. Our method is substantiated by rigorous theoretical analysis and compelling empirical results, highlighting its feasibility and effectiveness. We believe that ACRE establishes a solid foundation for future studies.

The primary limitation of the proposed method lies in its ability to handle solely asymmetric open-set noise, where the ground-truth label exists outside the ID label space, while the assigned label tends to align with the known class that exhibits similar spurious-related characteristics. In future research, we aim to tackle the challenge of OOD detection under multiple noisy environments.

Appendix A: The Proof of Theorem 1

Proof

First, we fix the feature extractor G, and minimize the distribution discrimination loss $\mathcal {L}_D$.

$$\begin{aligned} \min _{D} \mathcal {L}_D (x) =&\mathbb {E}_{P_E(x)}[ -\log D_0(G(x))]\nonumber \\&\quad + \mathbb {E}_{P_H(x)} [ -\log D_1(G(x))]\nonumber \\&\quad \quad + \mathbb {E}_{P_O(x)} [ -\log D_2(G(x))] \nonumber \\ =&-\int _{x \sim P_E(x)} \log D_0(G(x)) d x \nonumber \\&\quad {-} \int _{x \sim P_H(x)} \log D_{1}(G(x)) d x\nonumber \\&\quad - \int _{x \sim P_O(x)} \log D_{2}(G(x)) d x \nonumber \\ =&{-}\int _{z \sim P_E(z)} \log D_0(z) d z \nonumber \\ {}&\quad - \int _{z \sim P_H(z)} \log D_{1}(z) d z\nonumber \\&\quad - \int _{z \sim P_O(z)} \log D_{2}(z) d z \nonumber \\ =&\int _z\left( -P_E(z) \log D_0(z)- P_H(z) \log D_{1}(z)\right. \nonumber \\&\quad \left. - P_O(z) \log D_{2}(z)\right) d z \end{aligned}$$

(A1)

$D_0(z) + D_1(z) + D_2(z) = 1$ for all z. Therefore, we transform the above optimization problem into an optimization problem with constraints as follows:

$$\begin{aligned}&\min _{D} -P_E(z) \log D_0(z)- P_H(z) \log D_{1}(z)\nonumber \\&\quad - P_O(z) \log D_{2}(z) \nonumber \\&s.t. \quad D_0(z) + D_1(z) + D_2(z) = 1 \end{aligned}$$

(A2)

To solve the optimization problem with constraints, we use the Lagrange multiplier method.

$$\begin{aligned} \min _{D} \tilde{\mathcal {L}}_D&:= -P_E(z) \log D_0(z) -P_H(z) \log D_{1}(z)\nonumber \\&\quad - P_O(z) \log D_{2}(z) \nonumber \\&\quad + v(D_0(z) + D_1(z) + D_2(z) - 1)\, \end{aligned}$$

(A3)

where v denotes the Lagrange variable.

We compute the derivative of $\tilde{\mathcal {L}}_D$ with respect to D and v as follows:

$$\begin{aligned}&\frac{\partial \tilde{\mathcal {L}}_D}{\partial D_0(z)} =\frac{-P_E(z)}{D_0(z)}+v=0 \quad \Leftrightarrow \quad D_0(z) =\frac{P_E(z)}{v} \nonumber \\&\frac{\partial \tilde{\mathcal {L}}_D}{\partial D_{1}(z)} =\frac{- P_H(z)}{D_{1}(z)}+v=0 \quad \Leftrightarrow \quad D_{1}(z)=\frac{P_H(z)}{v} \nonumber \\&\frac{\partial \tilde{\mathcal {L}}_D}{\partial D_{2}(z)} =\frac{- P_O(z)}{D_{2}(z)}+v=0 \quad \Leftrightarrow \quad D_{2}(z)=\frac{P_O(z)}{v} \nonumber \\&\frac{\partial \tilde{\mathcal {L}}_D}{\partial v}=D_0(z) +D_{1}(z)+D_{2}(z)-1=0 \nonumber \\&\quad \Leftrightarrow \quad D_0(z)+D_{1}(z)+D_{2}(z)=1 \end{aligned}$$

(A4)

According to the above equations, we can know

$$\begin{aligned} D_0(z)+D_{1}(z)+D_{2}(z) = \frac{P_E(z)}{v} + \frac{P_H(z)}{v} + \frac{P_O(z)}{v} = 1\,, \end{aligned}$$

(A5)

where

$$\begin{aligned} v = P_E(z) + P_H(z) + P_O(z) = 3P_{avg}\,. \end{aligned}$$

(A6)

Thus, we obtain optimal $D^{*}$ as

$$\begin{aligned} D^{*}(z)&= [D^{*}_0(z), D^{*}_1(z), D^{*}_2(z)] \nonumber \\&= \left[ \frac{P_E(z)}{3P_{avg}(z)}, \frac{P_H(z)}{3P_{avg}(z)}, \frac{P_O(z)}{3P_{avg}(z)}\right] \,. \end{aligned}$$

(A7)

Then, during optimizing G through minimizing $\mathcal {L}_{OTA}$, we fix D with $D^*$.

$$\begin{aligned}&\min _{G} \mathcal {L}_{OTA} (x) = \mathbb {E}_{P_E (x)}[ -\log D_1^{*}(G(x))] \nonumber \\&\qquad + \mathbb {E}_{P_H(x)}[ -\log D_0^{*}(G(x))] + \mathbb {E}_{P_O(x)} [ -\log D_2^{*}(G(x))] \nonumber \\&\quad =\int _z\left( -P_E(z) \log D_1^*(z) - P_H(z) \log D_{0}^*(z)\right. \nonumber \\&\qquad \left. -P_O(z) \log D_{2}^*(z)\right) d z \nonumber \\&\quad =\int _z\left( -P_E(z) \log \frac{P_H(z)}{3P_{avg}(z)} - P_H(z) \log \frac{P_E(z)}{3P_{avg}(z)}\right. \nonumber \\&\qquad \left. -P_O(z) \log \frac{P_O(z)}{3P_{avg}(z)}\right) d z \nonumber \\&\quad =\int _z\left( -P_E(z) \log \frac{P_H(z)}{3P_{avg}(z)} - P_H(z) \log \frac{P_E(z)}{3P_{avg}(z)}\right. \nonumber \\&\qquad \left. -P_O(z) \log \frac{P_O(z)}{3P_{avg}(z)}\right) d z \nonumber \\&\quad =\int _z\left( (P_H(z)+P_O(z)-3P_{avg}) \log \frac{P_H(z)}{3P_{avg}(z)}\right. \nonumber \\&\qquad \left. +(P_E(z)+P_O(z)-3P_{avg}) \log \frac{P_E(z)}{3P_{avg}(z)}\right. \nonumber \\&\qquad \left. -P_O(z) \log \frac{P_O(z)}{3P_{avg}(z)}\right) d z \nonumber \\&\quad =\int _z\left( P_H(z)\log \frac{P_H(z)}{3P_{avg}(z)} +P_O(z)\log \frac{P_H(z)}{3P_{avg}(z)}\right. \nonumber \\&\qquad \left. -3P_{avg}\log \frac{P_H(z)}{3P_{avg}(z)} + P_E(z)\log \frac{P_E(z)}{3P_{avg}(z)}\right. \nonumber \\&\qquad \left. +P_O(z)\log \frac{P_E(z)}{3P_{avg}(z)} -3P_{avg}\log \frac{P_E(z)}{3P_{avg}(z)} \right. \nonumber \\&\qquad \left. -P_O(z) \log \frac{P_O(z)}{3P_{avg}(z)}\right) d z \nonumber \\&\quad {=}KL\left( P_H \Vert 3P_{avg}\right) {+} KL\left( 3P_{avg} \Vert P_{H}\right) {+} KL\left( P_E \Vert 3P_{avg}\right) \nonumber \\&\qquad + KL\left( 3P_{avg} \Vert P_{E}\right) - KL\left( P_O \Vert 3P_{avg}\right) \nonumber \\&\qquad + \int _z\left( P_O(z)\log \frac{P_H(z)}{3P_{avg}(z)} + P_O(z)\log \frac{P_E(z)}{3P_{avg}(z)}\right) d z\nonumber \\&\quad =KL\left( P_H \Vert P_{avg}\right) + 3KL\left( P_{avg} \Vert P_{H}\right) + KL\left( P_E \Vert P_{avg}\right) \nonumber \\&\qquad + 3KL\left( P_{avg} \Vert P_{E}\right) - KL\left( P_O \Vert P_{avg}\right) + 5 \log 3\nonumber \\&\qquad + \int _z\left( P_O(z)\log \frac{P_H(z)}{3P_{avg}(z)} + P_O(z)\log \frac{P_E(z)}{3P_{avg}(z)}\right) d z \nonumber \\&\quad = KL\left( P_H \Vert P_{avg}\right) + 3KL\left( P_{avg} \Vert P_{H}\right) + KL\left( P_E \Vert P_{avg}\right) \nonumber \\&\qquad + 3KL\left( P_{avg} \Vert P_{E}\right) - KL\left( P_O \Vert P_{avg}\right) \nonumber \\&\qquad + O_{EH} + 5 \log 3\,, \end{aligned}$$

(A8)

where $O_{EH}$ denotes $\int _z\left( P_O(z)\log \frac{P_H(z)}{3P_{avg}(z)}+ P_O(z) \log \right. \left. \frac{P_E(z)}{3P_{avg}(z)}\right) d z$ for convenience. Then, we analyze $KL\left( P_H \Vert P_{avg}\right) + 3KL \left( P_{avg} \Vert P_{H}\right) + KL\left( P_E \Vert P_{avg}\right) + 3KL\left( P_{avg} \Vert P_{E}\right) - KL \left( P_O \Vert P_{avg}\right)$ by the analysis of forces in the field of physics. Since the KL dispersion is asymmetric, it can be viewed as a force approximately. As shown in Fig. 5, we use $F_{ea}, F_{ha}, F_{ah}, F_{ae}$ to denote $KL\left( P_E \Vert P_{avg}\right)$, $KL\left( P_H \Vert P_{avg}\right)$, $KL\left( P_{avg} \Vert P_{H}\right)$, $\left( P_{avg} \Vert P_{E}\right)$, respectively. $\mathcal {E}, \mathcal {H}, \mathcal {O}, \mathcal {A}$ denote $P_E, P_H, P_O, P_A$, located at the three vertices and the center of the triangle, respectively. $F_{aeh}$ denotes the resultant force, and its direction represents the direction $\mathcal {A}$ moves. $F_{ha}$ and $F_{ea}$ will keep $\mathcal {E}$ and $\mathcal {H}$ moving closer to $\mathcal {A}$. By optimizing $\mathcal {L}_{OTA}$, $KL\left( P_E \Vert P_{avg}\right)$, $KL\left( P_H \Vert P_{avg}\right)$, $KL\left( P_{avg} \Vert P_{H}\right)$, $\left( P_{avg} \Vert P_{E}\right)$ will keep decreasing until $P_E \approx P_H \approx P_{avg}$. We use $F_a$ denote $- KL\left( P_O \Vert P_{avg}\right)$. Minimizing $\mathcal {L}_{OTA}$ increases $KL\left( P_O \Vert P_{avg}\right)$, resulting in $\mathcal {O}$ constantly moving away from $\mathcal {A}$. $D_{AO}$ denotes the distance of $\mathcal {A}$ and $\mathcal {O}$ in the optimal G. Moreover, minimizing $\mathcal {L}_{OTA}$ will decrease $O_{EH}$ and the output of the open-set data on $D_0$ and $D_1$, contributing to enhancing the separability of ID and OOD distribution as well. $\square$