A Polarimetric Scattering Characteristics-Guided Adversarial Learning Approach for Unsupervised PolSAR Image Classification

Abstract

Highly accurate supervised deep learning-based classifiers for polarimetric synthetic aperture radar (PolSAR) images require large amounts of data with manual annotations. Unfortunately, the complex echo imaging mechanism results in a high labeling cost for PolSAR images. Extracting and transferring knowledge to utilize the existing labeled data to the fullest extent is a viable approach in such circumstances. To this end, we are introducing unsupervised deep adversarial domain adaptation (ADA) into PolSAR image classification for the first time. In contrast to the standard learning paradigm, in this study, the deep learning model is trained on labeled data from a source domain and unlabeled data from a related but distinct target domain. The purpose of this is to extract domain-invariant features and generalize them to the target domain. Although the feature transferability of ADA methods can be ensured through adversarial training to align the feature distributions of source and target domains, improving feature discriminability remains a crucial issue. In this paper, we propose a novel polarimetric scattering characteristics-guided adversarial network (PSCAN) for unsupervised PolSAR image classification. Compared with classical ADA methods, we designed an auxiliary task for PSCAN based on the polarimetric scattering characteristics-guided pseudo-label construction. This approach utilizes the rich information contained in the PolSAR data itself, without the need for expensive manual annotations or complex automatic labeling mechanisms. During the training of PSCAN, the auxiliary task receives category semantic information from pseudo-labels and helps promote the discriminability of the learned domain-invariant features, thereby enabling the model to have a better target prediction function. The effectiveness of the proposed method was demonstrated using data captured with different PolSAR systems in the San Francisco and Qingdao areas. Experimental results show that the proposed method can obtain satisfactory unsupervised classification results.

1. Introduction

Benefiting from its ability to operate in all-weather and day-and-night conditions [1], polarimetric synthetic aperture radar (PolSAR) image land cover classification [2] provides a rich source of information about the earth’s surface and can be used in various fields such as agriculture (to classify different crops and monitor their growth stages) and oceanology (to map the distribution of sea ice and detect oil spills in the ocean) [3,4,5]. PolSAR classification algorithms have developed rapidly over the past two decades. However, the complexity and diversity of natural terrain makes it difficult to completely describe observation targets using existing PolSAR image representations, which brings a challenge to the land cover classification of PolSAR images.

Feature engineering and classifier design are the two primary means of improving PolSAR image classification. Driven by the basic scattering models [6], the polarimetric target decomposition technique [7] emerged, resulting in a wide range of polarimetric feature extraction methods [8,9,10]. Enhanced features can emphasize the essential attributes of land cover types, making them more recognizable. The development of PolSAR classifiers can be categorized into two paths: supervised and unsupervised. In supervised algorithms [11,12,13], ground truth maps with manual annotations are required to prepare the training samples. Then, the similarity or margin between predictions and true labels needs to be measured to learn the prediction function. Supervised methods have high accuracy when the training and testing data are drawn from the same distribution. However, their drawbacks are evident: the algorithm fails when the ground truth is unavailable or inadequate. In contrast, unsupervised approaches are more appealing because they can be applied to a wider range of problems including those without labeled training data. Classical unsupervised PolSAR classification primarily follows two methods. One approach is based on clustering methods, such as fuzzy C-means [14], Wishart clustering [15], and spectral clustering [16]. The other focus is feature engineering, which aims to construct features with physical meanings based on polarimetric scattering mechanisms, providing discriminative information for land cover types [17]. Among them, the most renowned method is the $H/\alpha$ classifier [18], which utilizes the scattering entropy and mean alpha angle to implement unsupervised classification. However, a drawback is that only a limited set of classical unsupervised methods preserve the category semantics in the classification results based on the comprehension of the scattering mechanism. Therefore, most of such methods require manual inferential prediction since no information is received from the true labels.

The application of convolutional neural networks (CNNs) in PolSAR image classification has produced remarkable outcomes [19]. The majority of these studies have concentrated on designing advanced network architectures for supervised learning. Although a few studies have tackled the challenge of few-shot or semisupervised PolSAR classification [20,21], to date, unsupervised CNN-based approaches have not been explored, which is the primary focus of this paper.

For unsupervised classification problems, unsupervised domain adaptation (DA) presents a viable solution [22]. In this context, DA involves two similar but not identical data domains, namely source and target. There are two differences between these domains. First, data in the source domain is labeled, while data in the target domain is unlabeled. Second, the two domains have different data distributions. Unsupervised DA is suitable for those applications in which machine learning algorithms are trained using labeled source data and unlabeled target data to ensure the good generalization performance for the testing target data [23,24]. Currently, the most prevalent technique for DA is to enhance feature transferability by aligning the feature distributions of the source and target domains, extracting domain-invariant features, and sharing the source classifier. Several excellent studies have been developed based on this approach, including using CNNs as feature encoders [24]. They have used statistical criteria [25,26,27] or adversarial loss [28,29] to achieve feature alignment between source and target. Despite these achievements, some studies have found that since the discriminability of features has not been fully taken into account, the discriminability cannot be guaranteed when the feature transferability is improved [30], and the source-specified classifiers may not generalize well on the target domain. Pseudo-labeling is an effective approach for addressing this issue. It enhances feature discriminability by aligning class-specific features from both the source and target domains [31,32]. However, the construction of pseudo-labels is a critical issue. The methods for generating pseudo-labels related to optical images do not consider the polarimetric scattering mechanism of PolSAR images. This is the main focus of our research.

Inspired by previous works, this paper introduces and applies unsupervised deep adversarial domain adaptation (ADA) for PolSAR image classification. To the best of our knowledge, this is the first work that investigates the effectiveness of ADA for unsupervised CNN-based PolSAR classification. Additionally, a novel polarimetric scattering characteristics-guided adversarial network (PSCAN) has been developed to leverage the valuable information present in PolSAR images, with the aim of enhancing the generalization performance by improving the feature discriminability. The key contributions of this paper can be summarized as follows:

–

We propose the integration of deep ADA into the PolSAR community and extend CNN-based PolSAR classifiers to the field of unsupervised classification. Compared to those of traditional unsupervised PolSAR classification methods, the deep features extracted by CNNs yield better a decoupling performance. Furthermore, the transfer of knowledge enables ADA to make predictions without requiring manual inference, making it more suitable for application scenarios.

–

We describe the design of an auxiliary task and incorporate it into the standard adversarial adaptation network to capture category semantic information from the polarimetric scattering characteristics-guided pseudo-labels. This enhances the learning of the class-wise correlations between the source and target domains.

–

We propose a novel unsupervised ADA method to address the issue of insufficient discriminability exhibited by domain-invariant features obtained through the existing methods. Compared with the related methods [31,32], our proposed method utilizes the polarimetric scattering characteristics of PolSAR images, i.e., Cloude–Pottier decomposition [18,33], to construct pseudo-labels, which avoids the complex and inefficient automatic labeling mechanisms in optical image-related methods, so it is very simple and efficient.

We conduct extensive comparative experiments on eight transfer tasks using five PolSAR datasets from the San Francisco and Qingdao areas. The experimental results validate the effectiveness of our proposed method.

2. Related Work

2.1. Convolutional Neural Network

As one of the most prominent techniques in deep learning, CNNs [34] have achieved state-of-the-art results across a variety of tasks [35]. Currently, CNN architectures are considered mature and are often used for end-to-end feature extraction and classification (or other tasks). Compared to traditional hand-crafted features and kernel methods, automated feature engineering with CNNs is more powerful [36]. The capability to effectively use big data endows CNNs with a distinct advantage over many small-sample algorithms.

In recent years, the emergence of deep learning and CNNs have significantly improved the performance of PolSAR image classification. Zhou et al. were one of the first to use CNNs for PolSAR image classification [19]. They proposed a normalized six-dimensional real vector representation of PolSAR data and designed a four-layer CNN architecture. Since then, various CNN-based PolSAR classification studies have emerged, which have improved the performance of PolSAR image classification by utilizing novel input forms [37,38,39], complex-valued operations [40], feature selection [41,42,43], 3D convolution [44,45], neural architecture search [46], and generative learning [47,48]. While remarkable fully supervised results have been achieved [49] and some efforts have been made to reduce the need for large amounts of labeled training samples [20,21], almost all existing works still depend on annotations. Two main reasons make it necessary to develop unsupervised CNN-based PolSAR classification methods: First, the labeling cost of PolSAR images is relatively high. Second, the transform of sensors, imaging modes, and conditions can significantly affect PolSAR images, which causes the independent and identically distributed assumption of the supervised algorithm, i.e., training and testing data drawn from the same distribution, to fail in its testing phase.

2.2. Unsupervised Domain Adaptation

DA is a popular method of transfer learning [23] and a classical problem in machine learning. In the context of these methods, unsupervised DA holds greater research value than does supervised DA due to its wider applicability and stronger irreplaceability. This is because it does not require labeled data from the target domain. Measuring the similarity between distributions and matching different domains are critical aspects of this problem. Several classical unsupervised DA methods attempted to find combinations of hand-crafted features and distance metrics to reduce the domain discrepancy [50,51]. With the rise of deep learning, some studies have combined deep features and statistical criterion-based metrics, such as maximum mean discrepancy (MMD) [25], correlation alignment (CORAL) [26], and KL-divergence [52]. Another group of deep feature-based approaches employ adversarial loss [53] to reduce domain discrepancy. These methods learn domain-invariant features through a zero-sum game between the feature encoder and the domain discriminator [28,29], thereby enabling the source-specified classifier to achieve good generalization performance on the target domain.

Unsupervised DA holds great potential for PolSAR image classification, as it aligns with the application scenario of PolSAR classification. CNN-based PolSAR classifiers are known to require a significant number of labeled training samples, which is a challenging task in PolSAR images due to the high labeling cost. Additionally, the generalization of CNNs in PolSAR classification is also difficult because transforms of sensors, imaging modes, and conditions may cause considerable changes in PolSAR images. As a result, the independent and identically distributed assumption of the algorithm is likely to be violated, leading to test failure.

3. Methods

This section will provide implementation details. First, we will present the initial representation of the PolSAR image data. Next, we will introduce the problem definition of unsupervised DA. Finally, we will provide a detailed introduction to each component of the proposed method.

As depicted in Figure 1, the proposed PSCAN model consists four parts, namely the feature encoder, classifier, domain discriminator, and auxiliary task. During training, the adaptation network is trained using labeled samples from the source domain and unlabeled samples from the target domain. It is worth noting that the classifier update relies only on the information obtained from the source domain, whereas the rest parts leverage information from both the source and target domains. The proposed PSCAN uses the domain discriminator to enhance the transferability and the auxiliary task to improve the discriminability of features. The encoder learns domain-invariant and discriminative features, enabling the source-specific classifier to be shared by the target domain. After training, the validity is tested using samples from the target domain only, and the whole map classification results are obtained based on the trained model.

3.1. Representation of PolSAR Data

The study starts by acquiring two-dimensional image data using radar echo imaging. Each individual pixel in PolSAR images can be represented by the following polarimetric coherency matrix:

$$T=\left[\begin{array}{ccc}{T}_{11}& T_{12}& T_{13}\\ T_{21}& T_{22}& T_{23}\\ T_{31}& T_{32}& T_{33}\end{array}\right].$$

(1)

The matrix T is a Hermitian matrix, with complex-valued elements except for the diagonal ones. In numerous research works, the upper triangular elements of matrix T have been used as the initial PolSAR data representation.

Apart from matrix T, it is also feasible to use polarimetric features as the input [37]. Specifically, to formulate roll-invariant features, the matrix T can be parameterized using eigenvalue–eigenvector-based methods [18] as follows:

$$T=U\left[\begin{array}{ccc}{\lambda }_1& 0& 0\\ 0& {\lambda }_2& 0\\ 0& 0& {\lambda }_3\end{array}\right]U^H.$$

(2)

Three polarimetric features, including the entropy H, mean alpha angle $\overline{\alpha }$, and anisotropy $Ani$ can be formulated as follows:

$$H=-\sum _{n=1}^3{P}_n{log}_3{P}_n,\overline{\alpha }=\sum _{n=1}^3{P}_n{\alpha }_n,Ani=\frac{{\lambda }_2-{\lambda }_3}{{\lambda }_2+{\lambda }_3},$$

(3)

where $P_n={\lambda }_n/({\lambda }_1+{\lambda }_2+{\lambda }_3)$, and $\alpha$ is related to the matrix U. Another roll-invariant feature, i.e., total backscattering power SPAN, can be defined as following:

$$\mathrm{SPAN}=\langle \mid S_{\mathrm{HH}}{\mid }^2\rangle +2\langle \mid S_{\mathrm{HV}}{\mid }^2\rangle +\langle \mid S_{\mathrm{VV}}{\mid }^2\rangle .$$

(4)

In this article, we adopt a 16-dimensional PolSAR initial representation, which consists of the real part, imaginary part, and modulus of the polarimetric coherency matrix, along with entropy, mean alpha angle, anisotropy, and total backscattering power. Each dimension of the feature is normalized to the range of $[0,1]$ to enhance the training efficiency. We apply min–max normalization to achieve this, while excluding the maximum $1\%$ and the minimum $1\%$ of the values in each feature to avoid the influence of outliers in PolSAR images.

3.2. Problem Definition

We first present the problem definition before delving into the proposed method. Similar to those of many CNN-based methods [19,37,42], the classification steps of our approach can be summarized as follows: First, we extract pixels and their small neighborhoods to generate the training and testing sets. Then, we train the model using the training samples and subsequently feed the testing samples into the pretrained model to obtain the predictions. However, our paper addresses the task of unsupervised DA, which introduces a significant difference. Let $X\times Y$ denote the input space, where X represents the input and Y denotes the label. The labeled samples from the source domain can be expressed as $X^s=\{(x_1^s,y_1^s),\cdots ,(x_{n_s}^s,y_{n_s}^s)\}$, and the unlabeled samples from the target domain can be expressed as $X^t=\{x_1^t,\cdots ,x_{n_t}^t\}$. For supervised methods, their training and testing sets are derived from $X^s$. On the other hand, our training set is a combination of $X^s$ and $X^t$, whereas the testing set only comprises samples from $X^t$. Consequently, our objective is to learn a classifier based on the source and target domains with the least empirical risk in the target domain.

3.3. Feature Encoder and Classifier

The feature encoder and classifier form the foundation for performing land cover classification of each pixel in the PolSAR image. Given that the input pixel-centric PolSAR image patch has a relatively small spatial size, a classical convolutional architecture is adequate to construct effective features. Figure 2 displays the detailed network architecture that we utilized.

The feature extraction process involves three blocks, which consist of cascaded convolutional layers, ReLU activation function ($f\left(x\right)=max(0,x)$), batch normalization (BN) [54], and pooling layers. Two max-pooling layers are employed to expand the receptive field. At the end of the network, a global average pooling layer is utilized to generate the features, and a linear classification layer is employed to derive the predictions. The feature encoder is denoted as $f(·)$, and the classifier is denoted as $c(·)$. Training them can be achieved by optimizing the cross-entropy loss:

$$\underset{f,c}{min}{\mathcal{L}}_{\mathrm{cls}}(f,c,X)=-\frac{1}{n}\sum _{i=1}^n\sum _{k=1}^K\mathbb{I}(k=y_i)·logc\left(f\left(x_i\right)\right),$$

(5)

where $\mathbb{I}$ is the indicator function, n is the number of samples, and K is the number of categories. For the task of unsupervised DA, the variables in (5) should be re-formulized as ${\mathcal{L}}_{\mathrm{cls}}(f,c,X^s)$ because samples must be labeled when back propagating their cross-entropy loss errors. Therefore, although samples from both source and target domains can participate in the forward propagation to obtain corresponding features and predictions, $f(·)$ and $c(·)$ can only receive the update information from the source domain.

3.4. Domain Discriminator

As demonstrated in the previous subsection, utilizing only the feature encoder and classifier can lead to the production of source-specific features and predictions, which is not conducive for unsupervised DA. To address this issue, we introduce a domain discriminator that provides update information from the target domain to $f(·)$, enabling the extraction of domain-invariant features.

The concept of the domain discriminator draws heavily from the framework of generative adversarial networks (GANs) [53]. In GANs, the generator and discriminator engage in a zero-sum game to ensure that the distribution of generated data matches that of real data. Ganin et al. extended this problem from generation to unsupervised DA [28], where they introduced the domain discriminator, denoted as $d(·)$, to distinguish whether the sample is from the source or target domain. The optimization problem for $f(·)$ and $d(·)$ can be formulated as follows:

$$\underset{d}{min}\underset{f}{max}{\mathcal{L}}_{\mathrm{adv}}(f,d,X^s,X^t)=-\frac{1}{n_s}\sum _{i=1}^{n_s}logd\left(f\left(x^s\right)\right)-\frac{1}{n_t}\sum _{i=1}^{n_t}log(1-d\left(f\left(x^t\right)\right)).$$

(6)

In most cases, the min–max objective function described above should be solved iteratively [29,53]. Once the optimal $d^{*}$ has been obtained, the optimization of f can be used to minimize the JS-divergence between $f\left(X^s\right)$ and $f\left(X^t\right)$, aligning the feature distributions of the source and target domains. However, the iterative nature of this approach can be cumbersome, and to address this issue, Ganin et al. proposed the gradient reversal layer. This layer enables the training of the domain discriminator to support standard backpropagation by acting as an identity layer during forward propagation and reversing the value of the gradient during backpropagation. With the gradient reversal layer denoted as $r(·)$, (6) can be expressed as follows:

$$\underset{f,d}{min}{\mathcal{L}}_{\mathrm{adv}}(f,d,X^s,X^t)=-\frac{1}{n_s}\sum _{i=1}^{n_s}logd\left(r\left(f\left(x^s\right)\right)\right)-\frac{1}{n_t}\sum _{i=1}^{n_t}log(1-d\left(r\left(f\left(x^t\right)\right)\right)).$$

(7)

In this paper, the domain discriminator $d(·)$ is defined as a three-layer fully connected network with 64 hidden nodes. Outputs of the first two hidden layers are activated by ReLU and BN, and that of the last hidden layer is activated by sigmoid.

3.5. Pseudo-Label and Auxiliary Task

Benefiting from the domain discriminator, feature transferability can be ensured. In the context of ADA methods, a critical issue has been identified. Despite the adversarial loss leading to domain confusion, the source-specific classifier still struggles to generalize to the target domain. This issue is referred to as insufficient feature discriminability [30,31,55] or negative transfer [32]. The root cause of this problem is that even though the feature distributions of the source and target domains have been aligned, the feature extraction process fails to acquire any category semantic information related to the target domain. As a result, the discriminability of the obtained features cannot be guaranteed.

Recent works that focus on optical image domain adaptation and classification have demonstrated that pseudo-labeling is an effective way for improving feature discriminability. Some studies have employed the source-specific classifier to generate pseudo-labels for the target data [32,56]. Others, such as Kang et al., have used clustering operations to address this issue [31]. In general, these methods assign pseudo-labels to the samples from both the source and target domains based on specific criteria, with the aim of aligning the features class-wise and refining their discriminative structures. Although pseudo-labeling improves the feature discriminability, its construction is quite challenging [57,58]. We conclude that an effective pseudo-labeling method must satisfy the following three requirements:

–

The pseudo-labels need to be arbitrary to the data source. Specifically, if a region in the source domain is labeled as a certain class, the same region in the target domain should also be labeled as the same class in order to ensure correct alignment of class-specific features.

–

The pseudo-labeling approach must exhibit broad generalizability. To achieve this, the labeling process must be automated, without any manual intervention. Additionally, the method should produce the expected results for all types of data, not just specific data types. The combination of these two aspects ensures the applicability of the algorithm.

–

The pseudo-labels must strive for maximum accuracy. While pseudo-labeling can be a powerful tool, the introduction of labeling errors can significantly reduce classification accuracy. Therefore, it is important to minimize label noise in order to achieve optimal performance.

Based on the aforementioned requirements, it is evident that constructing an ideal pseudo-label requires not only unsupervised classification but also the integration of semantic knowledge associated with the relevant categories. The conventional optical image related pseudo-labeling methods cannot fulfill this requirement as they solely provide intensity information on RGB channels and hence do not establish a corresponding relationship between the intensity and land cover types. Consequently, the pseudo-labels generated by such methods are typically initialized randomly and subsequently refined during the training process to improve accuracy. However, this iterative approach is not only potentially counterproductive in the early stages but also increases the complexity of the algorithm.

In contrast, due to the special imaging mechanism of PolSAR system, the expert knowledge of human researchers on scattering mechanisms allows different land cover types in PolSAR images to be described with distinct polarimetric scattering characteristics, enabling unsupervised classification with category semantics. As PolSAR images contain more information than does the RGB format, utilizing the characteristics derived from PolSAR’s unique information is a more sensible approach than is relying on the pseudo-label generation mechanisms in optical image-related methods, as previously mentioned.

Building upon the aforementioned analysis, we highlight the benefits of PolSAR images, whereby the polarimetric scattering characteristics are employed to guide the construction of pseudo-labels and are followed by the development of a customized algorithm. These two components serve to enhance the features discriminability by leveraging the unique properties of PolSAR image data, thereby facilitating the generalization of the deep adaptation model. In the following contents, we elaborate upon the comprehensive design and implementation.

Pseudo-Label Construction: This part outlines an efficient approach for generating pseudo-labels for both source and target domains. The process can be divided into two stages. First, we conduct unsupervised classification based on the guidance of polarimetric scattering characteristics. Next, we refine the initial classification results to obtain superior pseudo-labels. During the first stage, we assign categories to samples based on their scattering mechanisms, with the assigned categories being consistent with the annotations in the source domain. The latter stage employs a clustering method based on the Wishart distance metric. Given the advancements in PolSAR image target decomposition and classification, these methods have become relatively mature.

Given that the objective of this study is to investigate the integration of polarimetric information into ADA methods, rather than improving the feature extraction process, we employ the Cloude–Pottier target decomposition method [18,33], which is widely adopted in the PolSAR community, to acquire polarimetric scattering characteristics. This enables us to obtain unsupervised classification results and guide the construction of pseudo-labels. Figure 3 displays the Cloude–Pottier decomposition result of PolSAR image data acquired with RadarSat-2, demonstrating the representation of the three decomposition parameters on different land cover types.

Cloude and Pottier [18] represented the random scattering mechanism in PolSAR images using two decomposition parameters—entropy and mean alpha angle—defined in (3) and obtained via eigenvalue decomposition of the polarimetric coherency matrix. Entropy describes the degree of randomness in scattering. For areas with a smooth surface such as water, surface scattering dominates, resulting in low randomness. For man-made areas, the scattering mechanisms are mixed, leading to medium entropy. In vegetation areas, multiple scattering mechanisms result in high scattering randomness. The value of the mean alpha angle ranges from 0 to 90 degrees, representing the continuous change of the main scattering mechanism of the land cover from surface scattering to double-bounce scattering and reaching a medium value in areas described as volume scattering. Pottier introduced anisotropy A to characterize the importance of two nondominant scattering mechanisms, which can be used as a supplement to distinguish the scattering characteristics of different land cover types when the value of entropy is high [33]. As depicted in Figure 3, the results of Cloude–Pottier decomposition enable the rough identification of the three basic land cover types in the area: water, man-made, and vegetation. By combining entropy and the mean alpha angle, the $H/\alpha$ classification hyperplane was proposed [18], as shown in Figure 4.

The polarimetric scattering characteristics are partitioned into nine regions, $Z_1$ to $Z_9$, by a two-dimensional classification hyperplane based on predetermined thresholds for entropy and alpha angle. As the scattering characteristics of the surface cannot be discerned when $H>0.9$, the high entropy surface scattering region $Z_3$ is not an effective region on the classification hyperplane. To further enhance this approach, a classification scheme that incorporates anisotropy has been proposed [33], and the classification hyperplanes and results are presented in Figure 5.

The $H/A/\alpha$ method can be used to obtain the classification results of fundamental land cover types. Furthermore, more precise types can be achieved by employing other decomposition methods [9,10]. These methods not only allow for the identification of most land cover types in PolSAR images, but also offer a considerable degree of flexibility and generality. This is because the classification boundaries are determined based on the extracted physical information from the microwave scattering observations. Therefore, the pseudo-labels constructed using these methods are insensitive to data changes and independent of the data source. It should be noted that the selection and configuration of unsupervised classification methods are highly adaptable. Various target decomposition methods [8], hyperplane combinations [59], and threshold settings [60] can be utilized to construct pseudo-labels with the guidance of polarimetric scattering characteristics.

Up to this point, the construction of pseudo-labels has been accomplished. In order to further enhance the accuracy, we take inspiration from some existing studies [61,62,63] and compute the average value of the polarimetric coherency matrix for the pixels in each category obtained from the initial stage results. These average values are then employed as clustering centers for the Wishart distance-based classifier, which is used to reclassify all pixels. This process is iterated until the algorithm converges. The purpose of this approach is to refine and adjust the classification boundaries that were previously established by means of thresholds, thereby avoiding the occurrence of ambiguous classification results. Specifically, the center $T^m$ of the mth category can be obtained by calculating the sample mean:

$$T^m=\frac{1}{N_m}\sum _{i=1}^{N_m}{T}_i^m$$

(8)

where $N_m$ represents the total number of pixels contained in this category, and $T_i^m$ is the polarimetric coherency matrix of the ith pixel in the mth category. The distance measurement of classification is defined as follows:

$$d_{\mathrm{W}}(T,T^m)=ln\left|T^m\right|+\mathrm{Tr}\left({\left(T^m\right)}^{-1}T\right)$$

(9)

where $\mathrm{Tr}(·)$ represents the trace operation of matrix. Based on the distance value from each center, each sample should be assigned to the category of the nearest cluster center. When a sample is assigned to the ith category, the following inequality holds:

$$d_{\mathrm{W}}(T,T^i)\le d_{\mathrm{W}}(T,T^j),\forall j\in \{1,\cdots ,K\}\Vert j\ne i.$$

(10)

Once all the sample points have been partitioned, the center of each category should be recalculated, and the above process should be repeated until convergence. This method is a fusion of polarimetric scattering characteristics and Wishart clustering. The former yields the initial classification results with category semantic information based on the distinct scattering mechanisms, while the latter takes into account the data distribution of the polarimetric coherency matrix. The combination not only preserves the category semantics in the unsupervised classification results, but also optimizes the classification boundaries to enhance the accuracy of the pseudo-labels. As a result of the aforementioned two steps, each pixel in the source and target images can be assigned a unique label $\widehat{y}$. Therefore, the source dataset $X^s$ and the target dataset $X^t$ can be represented as $X^s=\{(x_1^s,y_1^s,{\widehat{y}}_1^s),\cdots ,(x_{n_s}^s,y_{n_s}^s,{\widehat{y}}_{n_s}^s)\}$ and $X^t=\{(x_1^t,{\widehat{y}}_1^t),\cdots ,(x_{n_t}^t,{\widehat{y}}_{n_t}^t)\}$, respectively. Compared to pseudo-labels in related methods of optical image classification, the use of polarimetric scattering characteristics in PolSAR image classification can directly obtain pseudo-labels with a certain degree of accuracy. This eliminates the need for random initialization and the iterative refinement during the model training, making it significantly advantageous in terms of implementation difficulty and algorithmic complexity.

Utilization: Similar to previous works [31,32,56], we utilize class-wise alignment based on the obtained polarimetric scattering characteristics and constructed pseudo-labels. This is a straightforward approach to enhance the feature discriminability using the concept of metric learning, which functions to maximize interclass distance and minimize intraclass distance. Hence, we can formulate the following optimization problem:

$$\underset{f}{min}{\mathcal{L}}_{\mathrm{metric}}(f,X^s,X^t)=\sum _{i,j}\mathbb{I}({\widehat{y}}_i^s={\widehat{y}}_j^t)·\mathcal{S}(f\left(x_i^s\right),f\left(x_j^t\right))-\sum _{i,j}\mathbb{I}({\widehat{y}}_i^s\ne {\widehat{y}}_j^t)\mathcal{S}(f\left(x_j^s\right),f\left(x_j^t\right)),$$

(11)

where $\mathcal{S}(·)$ is a similarity measurement. The specific definition of ${\mathcal{L}}_{\mathrm{metric}}(·)$ can take many forms [64,65,66]. In the proposed PSCAN, we adopt a more direct approach, i.e., optimizing the cross-entropy loss:

$$\underset{f,c_{\mathrm{aux}}}{min}{\mathcal{L}}_{\mathrm{psc}}(f,c_{\mathrm{aux}},X^s,X^t)={\mathcal{L}}_{\mathrm{cls}}(f,c_{\mathrm{aux}},X^s)+{\mathcal{L}}_{\mathrm{cls}}(f,c_{\mathrm{aux}},X^t),$$

(12)

where $c_{\mathrm{aux}}(·)$ denotes the classifier for auxiliary task, which is designed to receive the category semantic information from polarimetric scattering characteristics-guided pseudo-labels. Equation (12) is also a good choice because it balances complexity and effectiveness. Compared with (11), the cross-entropy loss is undoubtedly more convenient, owing to its wide range of applications and ease of implementation. Additionally, it captures the interclass distance, which is the most important ability in improving feature discriminability.

Based on the above introduction, the objective function of PSCAN can be formulated as follows:

$$\underset{f,c}{min}{\mathcal{L}}_{\mathrm{cls}}(f,c,X^s)+{\mathcal{L}}_{\mathrm{adv}}(f,d,X^s,X^t)+\alpha {\mathcal{L}}_{\mathrm{psc}}(f,c_{\mathrm{aux}},X^s,X^t),$$

(13)

where $\alpha$ is the trade-off hyperparameter. $f,c,d,c_{\mathrm{aux}}$ are optimization variables, but only $f,c$ are finally required. From (13), it is evident that the input data should initially be mapped into a latent space by the feature encoder f. Subsequently, the feature distributions of the source and target domains can be aligned with the precondition of ensuring discriminability by minimizing ${\mathcal{L}}_{\mathrm{adv}}$ and ${\mathcal{L}}_{\mathrm{psc}}$. Finally, the classifier c can be learned from the source and generalized to the target.

The three items in (13) correspond to different objectives of the proposed PSCAN, which can be seen as an end-to-end multitask deep adaptation model. The following are specific analyses of these tasks:

The item ${\mathcal{L}}_{\mathrm{cls}}$ corresponds to the task of predicting the manual annotations of source domain data, whose definition is same as the standard cross-entropy loss function, as shown in (5). Note that with the learning rate of the optimizer as $\eta$, the parameter c corresponding to the classifier in the model will be updated as follows:

$$c=c-\eta \frac{\partial {\mathcal{L}}_{\mathrm{cls}}(f,c,x^s,y^s)}{\partial c}.$$

(14)

It is evident that the classifier training solely depends on the samples from the source domain. Therefore, the trained classifier, which can generalize well on the source domain based on the feature extractor f, forms the basis for generalization on the target domain during testing.

The term ${\mathcal{L}}_{\mathrm{adv}}$ corresponds to the task of predicting the data source, as defined in (7). With the features extracted from the source and target data, the parameter d of the domain discriminator can be obtained by maximizing its prediction accuracy:

$$d=d-\eta \frac{\partial {\mathcal{L}}_{\mathrm{adv}}(f,d,x^s,x^t)}{\partial d}.$$

(15)

It can be observed from Equation (15) that the training of the domain discriminator does not require any manual annotations, as the data source serves as the supervisory information. The domain discriminator is employed to counteract the feature extractor, whereby it predicts the data source and minimizes the probability of prediction error. Meanwhile, the feature extractor aims to extract features that can deceive the domain discriminator; that is, it maximizes the probability of prediction error of the domain discriminator and aligns the feature distributions of the source and target domains. Therefore, the domain discriminator forms the foundation of extracting domain-invariant features.

The item ${\mathcal{L}}_{\mathrm{psc}}$ corresponds to the task of predicting pseudo-labels, as defined in (12). The update of parameter $c_{\mathrm{aux}}$ in the auxiliary task can be expressed as follows:

$$c_{\mathrm{aux}}=c_{\mathrm{aux}}-\eta \frac{\partial {\mathcal{L}}_{\mathrm{psc}}(f,c_{\mathrm{aux}},x^s,{\widehat{y}}^s,x^t,{\widehat{y}}^t)}{\partial c_{\mathrm{aux}}}.$$

(16)

The auxiliary task in PSCAN is designed to extract category semantic information from pseudo-labels using supervised learning. The pseudo-labels guide the auxiliary task to mine the category semantic information. Since the discriminability of source-specific feature extraction can be ensured by learning with manual annotations, and the true and pseudo-labels share the same category space, learning pseudo-labels of source and target domains can also lead to the discriminability of target features.

4. Experiments

4.1. Study Area and Data Sources

We have selected the Golden Gate Bridge area and its surrounding regions in San Francisco, CA, USA, as our research objects. This study area comprises both urban and suburban areas, making it ideal for evaluating the generalization performance of the proposed method. Our experiments involve PolSAR image data captured with three different sensors. These datasets were obtained from the Institute of Electronics and Telecommunications of Rennes (https://ietr-lab.univ-rennes1.fr/polsarpro-bio/san-francisco/ (accessed on 21 February 2023)), and the ground maps used for reference were labeled by the Intelligent Perception and Image Understanding Laboratory [67].

ALOS-2 San Francisco: This dataset was captured by the L-band satellite-bone PolSAR system ALOS-2 in March 2015. The spatial resolution is 18 m. The size of the image is $2784\times 2900$. Figure 6a is the Pauli RGB map of this dataset, and Figure 6b shows the ground truth map.

GF-3 San Francisco: This dataset was captured by the C-band satellite-bone PolSAR system Gaofen 3 (GF-3) in September 2017. The spatial resolution is 8 m. The size of the image is $2304\times 1600$. Figure 6c is the Pauli RGB map of this dataset, and Figure 6d shows the ground truth map.

RS-2 San Francisco: This dataset was captured by the C-band satellite-bone PolSAR system RadarSat-2 (RS-2) in April 2008. The spatial resolution is 8 m. The size of the image is $1380\times 1800$. Figure 6e is the Pauli RGB map of this dataset, and Figure 6f shows the ground truth map.

In addition, we used PolSAR images from GF-3 and RS-2 near the Jiaozhou Bay cross-sea bridge in Qingdao, Shandong, China. The details are as follows.

GF-3 Qingdao: This dataset was captured with the GF-3 in November 2017. The spatial resolution is 4.73 m. The size of the image is $2100\times 1600$. Figure 7a is the Pauli RGB map of this dataset, and Figure 7b shows the ground truth map.

RS-2 Qingdao: This dataset was captured with the RS-2 in September 2009. The spatial resolution is 8 m. The size of the image is $1300\times 1200$. Figure 7c is the Pauli RGB map of this dataset, and Figure 7d shows the ground truth map.

4.2. Experimental Setup

Comparison methods: To validate the significance of the proposed method, both traditional learning methods and deep DA methods were chosen for comparison. Specifically, for traditional learning methods, we chose transfer component analysis (TCA) [50], joint distribution alignment (JDA) [68], balanced distribution adaptation (BDA) [69], and transfer joint matching (TJM) [70]. For deep DA methods, we chose deep adaptation network (DAN) [25], deep correlation alignment (DCORAL) [71], joint adaptation network (JAN) [27], domain adversarial neural network (DANN) [28], and conditional domain adversarial network (CDAN) [72].

Experiment settings: Before the training start, we prepared the pixel-centric PolSAR image patch with the size of $15\times 15\times 16$ for each dataset. Then the training and testing sets were obtained based on the ground truth maps in Figure 6. It needs to be pointed out that there is no overlap between the training and testing sets. When a dataset acts as the source domain, the label of its training set will be preserved; when it acts as the target domain, the label of the training set will be ignored. Therefore, six transfer tasks could be constructed based on the three datasets.

The prediction of the traditional learning methods is performed using the 1-nearest neighbor classifier based on feature alignment. The bandwidth of Gaussian kernels in multikernel MMD has a range of $\{2^{-3},\cdots ,2^2\}$. To ensure fairness of comparison, the model parameters of all deep DA methods were shared. We used the Adam optimizer for the optimization of deep methods and iterated the training for 150 epochs with a batch size of 256. The learning rate was set to a small value of $0.0001$ for the stability of the adversarial training, and the learning rate of the feature encoder was set to $0.1$ of that of the other parts for better convergence [73].

To evaluate the performance of involved adaptation methods, overall accuracy (OA), average accuracy (AA), and kappa coefficient (KC) were chosen as the evaluation criteria. The higher the values of OA, AA, and KC are, the better the model is.

4.3. Comparison of Results

Based on the above-mentioned settings, we conducted experiments on the existing transfer tasks to assess the effectiveness of the proposed method. The total experimental results are shown in

Table 1. Comparison of the overall accuracy (%) on all transfer tasks. The A, G, and R in the first column represent the ALOS-2, GF-3, and RS-2 datasets, respectively, and the Avg means average of the performance of each task. With A→G taken as an example, the ALOS-2 is the source domain and GF-3 is the target domain. The best results are highlighted in bold.

	San Francisco							Qingdao
Method	A→G	A→R	G→A	G→R	R→A	R→G	Avg	G→R	R→G	Avg	Total
Source-only	58.40	79.91	35.22	64.85	73.20	40.77	58.73	61.02	57.37	59.20	58.96
TCA	68.68	81.47	68.29	76.58	75.97	75.16	74.36	87.62	64.68	76.15	75.25
JDA	71.35	82.12	71.36	74.24	79.58	78.06	76.12	88.56	84.43	86.50	81.31
BDA	70.53	81.75	72.87	73.49	78.85	77.81	75.88	91.06	84.22	87.64	81.76
TJM	73.83	80.90	74.28	77.55	77.52	79.22	77.22	93.64	74.87	84.25	80.74
DAN	81.41	93.13	85.21	86.38	91.23	86.67	87.34	92.86	91.97	92.42	89.88
JAN	77.73	93.06	85.96	86.56	91.40	86.69	86.90	92.91	91.52	92.22	89.56
DCORAL	77.56	94.43	83.22	92.38	84.96	79.17	85.29	94.42	90.98	92.70	88.99
DANN	80.45	93.57	91.77	91.67	92.31	85.15	89.15	96.23	90.25	93.24	91.20
CDAN	78.72	94.33	90.82	92.28	92.24	82.36	88.46	95.36	92.38	93.87	91.16
PSCAN	82.14	93.51	91.72	92.86	93.23	88.48	90.32	97.64	94.49	96.07	93.19

, which contains the OA of the involved methods across all transfer tasks.

The results demonstrate that the proposed PSCAN achieved the best performance in four out of six transfer tasks on the San Francisco area, with a suboptimal performance in one of the remaining two tasks. Therefore, as shown in the Avg column, the proposed method exhibited the highest overall accuracy across all transfer tasks. In comparison with traditional learning methods, there is a noticeable performance gap in the test results of deep DA methods, highlighting the significance of deep feature adaptation. Additionally, the performance of statistical criterion-based methods in deep DA is not as impressive as that of the adversarial loss-based ones, which is in line with current understanding. Moreover, the proposed PSCAN improves the overall accuracy by $1.17\%$ as compared to the second-best method DANN on the San Francisco area and by $2.20\%$ as compared to the second-best method CDAN on the Qingdao area. Considering that the models of both methods are identical, except for the incorporation of the auxiliary task in PSCAN, we can attribute the improvement in performance to the utilization of the polarimetric scattering characteristics. This observation substantiates the potential of the proposed method in achieving effective knowledge transfer, leading to satisfactory unsupervised classification results.

Table 2. Comparison of experimental results (%) on the San Francisco area when the source domain is set to ALOS-2. The C1 to C5 refer to different categories: Water, Vegetation, High-Density Urban, Low-Density Urban, and Developed.

	ALOS-2 → GF-3							ALOS-2 → RS-2
Method	C1	C2	C3	C4	C5	AA	KC	C1	C2	C3	C4	C5	AA	KC
Source-only	8.96	96.72	92.78	48.63	45.37	58.49	47.81	85.88	79.25	16.23	99.36	95.38	75.22	74.34
TCA	87.47	93.85	84.34	40.65	40.72	69.41	60.69	98.94	94.29	83.96	57.52	67.04	80.35	76.72
JDA	99.12	97.49	78.07	44.23	42.85	72.35	64.10	99.94	95.30	77.49	71.50	59.63	80.77	77.51
BDA	99.12	96.60	77.22	45.94	37.99	71.37	63.04	99.82	95.11	70.62	76.17	60.75	80.49	77.06
TJM	99.39	93.20	74.22	49.47	58.82	75.02	67.19	97.71	93.29	68.38	70.71	70.06	80.03	76.02
DAN	99.72	87.45	76.57	73.63	77.92	83.06	76.32	97.31	88.93	95.46	92.92	93.93	93.71	91.32
JAN	62.35	93.52	95.67	74.51	61.70	77.55	71.67	96.36	89.86	93.95	93.44	93.53	93.43	91.21
DCORAL	99.66	94.67	95.82	58.99	55.74	80.98	71.84	99.89	89.56	95.04	94.11	96.12	94.94	92.95
DANN	95.30	96.88	91.32	81.35	32.58	79.49	75.00	99.55	87.55	97.95	92.08	95.28	94.48	91.89
CDAN	82.40	92.59	96.74	65.11	68.55	81.08	73.21	99.84	90.01	96.22	92.76	95.67	94.90	92.83
PSCAN	99.67	94.78	94.71	75.89	49.77	82.96	77.29	99.97	91.52	94.34	90.91	92.27	93.80	91.78

,

Table 3. Comparison of experimental results (%) on the San Francisco area when the source domain is set to GF-3.

	GF-3 → ALOS-2							GF-3 → RS-2
Method	C1	C2	C3	C4	C5	AA	KC	C1	C2	C3	C4	C5	AA	KC
Source-only	3.42	96.47	88.01	46.84	5.51	48.05	25.70	83.61	93.34	98.26	45.62	3.45	64.86	56.05
TCA	55.05	86.19	48.20	94.84	61.38	69.13	60.21	93.25	80.00	45.73	89.01	73.01	76.20	70.66
JDA	56.81	72.81	63.85	94.65	67.54	71.13	63.95	84.10	83.07	51.38	88.15	61.39	73.62	67.70
BDA	59.00	80.29	68.95	93.19	62.75	72.83	65.90	82.57	83.39	52.28	86.75	59.08	72.81	66.75
TJM	73.63	83.60	61.77	90.71	63.43	74.63	67.74	92.37	81.69	48.75	88.75	74.52	77.22	71.88
DAN	83.97	78.89	96.49	85.54	81.38	85.25	80.96	98.24	75.71	88.01	85.33	90.38	87.53	82.94
JAN	85.50	77.18	96.96	86.19	82.38	85.64	81.90	97.80	72.46	91.43	86.34	93.25	88.25	83.20
DCORAL	81.70	90.79	97.17	89.54	64.11	84.66	78.64	98.90	89.94	97.07	90.50	88.53	92.99	90.37
DANN	97.74	73.39	97.68	94.95	82.76	89.30	89.28	99.74	89.08	98.04	89.37	85.85	92.42	89.47
CDAN	98.61	80.92	96.53	90.27	77.86	88.84	88.05	99.16	83.65	97.23	92.34	94.90	93.46	90.28
PSCAN	99.28	77.55	93.10	95.27	80.82	89.20	89.20	98.78	86.86	93.64	93.67	94.45	93.48	90.97

,

Table 4. Comparison of experimental results (%) on the San Francisco area when the source domain is set to RS-2.

	RS-2 → ALOS-2							RS-2 → GF-3
Method	C1	C2	C3	C4	C5	AA	KC	C1	C2	C3	C4	C5	AA	KC
Source-only	59.69	94.40	88.30	88.26	58.70	77.87	66.88	0.17	94.85	97.46	14.33	12.23	43.81	27.76
TCA	70.48	95.47	67.30	90.02	60.35	76.72	69.92	99.73	96.54	75.69	59.09	47.77	75.77	68.81
JDA	77.68	91.01	76.80	86.08	68.28	79.97	74.40	100.00	95.94	72.02	64.69	61.34	78.80	72.48
BDA	77.26	91.87	76.13	84.43	66.97	79.33	73.49	100.00	96.24	71.34	64.38	60.81	78.56	72.16
TJM	78.38	95.18	66.97	87.49	63.66	78.34	71.86	100.00	95.53	61.73	73.00	69.46	79.94	73.91
DAN	96.00	88.20	94.79	94.31	78.09	90.28	88.61	99.71	90.33	86.93	83.62	75.70	87.26	82.94
JAN	96.80	88.93	94.29	94.32	77.51	90.37	88.83	99.72	90.41	86.88	83.68	75.66	87.27	82.96
DCORAL	80.42	91.01	96.49	91.16	74.29	86.67	80.83	99.62	93.85	94.95	63.41	58.46	82.06	73.77
DANN	98.26	88.85	94.35	96.04	77.86	91.07	90.01	97.31	96.53	88.93	82.55	60.99	85.26	81.01
CDAN	98.58	87.71	97.14	91.37	80.06	90.97	89.93	96.15	96.93	91.39	80.40	45.89	82.15	77.46
PSCAN	99.67	83.22	94.35	93.71	85.46	91.28	91.16	97.81	93.41	81.97	89.09	78.88	88.23	85.20

and

Table 5. Comparison of experimental results (%) on the Qingdao area. The C1 to C3 refer to different respective categories: Urban, Vegetation, and Water.

	GF-3 → RS-2					RS-2 → GF-3
Method	C1	C2	C3	AA	KC	C1	C2	C3	AA	KC
Source-only	85.53	0.00	100.00	61.84	41.90	100.00	0.00	85.45	61.82	37.40
TCA	92.73	69.55	100.00	87.43	81.38	87.52	63.64	37.39	62.85	45.41
JDA	92.56	72.98	99.71	88.42	82.74	85.21	85.18	82.32	84.23	76.14
BDA	92.15	81.45	99.52	91.04	86.53	84.76	86.95	79.39	83.70	75.75
TJM	92.73	88.49	99.90	93.71	90.44	83.07	64.01	81.08	76.05	61.88
DAN	90.03	88.89	99.81	92.91	89.29	97.46	89.66	88.76	91.96	87.87
JAN	92.37	86.69	99.91	92.99	89.37	93.95	84.93	97.59	92.15	87.21
DCORAL	90.51	99.36	93.19	94.35	91.62	89.69	85.55	99.77	91.67	86.39
DANN	90.37	98.26	99.96	96.20	94.34	81.04	91.31	99.46	90.60	85.23
CDAN	86.00	99.88	100.00	95.29	93.04	85.76	93.74	98.19	92.56	88.46
PSCAN	93.71	99.29	99.86	97.62	96.46	89.62	95.11	99.27	94.66	91.67

present the performance of each method in the transfer task under a specific source domain, including the accuracy of each category, AA and KC. The corresponding whole map classification results can be seen from Figure 8, Figure 9, Figure 10 and Figure 11. Based on these results, we provide the following analysis.

In the case of the source domain being ALOS-2, we present a detailed comparison of the experimental results of the involved methods in Table 2, and the whole map classification results are illustrated in Figure 8. For the first task, i.e., ALOS-2→GF-3, the proposed PSCAN achieved the best testing results. As indicated in Table 2, the proposed method shows a certain degree of improvement compared to most of the compared methods. Specifically, the OA and KC are $0.73\%$ and $0.97$ higher, respectively, than is the suboptimal DAN method, and the whole map classification result of PSCAN is significantly closer to the ground truth map. For the task of ALOS-2→RS-2, all deep DA methods demonstrate good classification performance, and the variation between the proposed method and the model with the highest accuracy is approximately $1\%$.

A comprehensive comparison of experimental results for the respective methods when the source domain is set to GF-3 is presented in Table 3, and the whole map classification results are illustrated in Figure 9. The highest level of accuracy during testing was attained by the proposed PSCAN when the target domain was assigned as RS-2. This same conclusion can be drawn from Figure 9, where PSCAN demonstrates superior classification results in the marked areas. For the other transfer task, PSCAN yields suboptimal testing results but it is only about $0.1\%$ behind the optimal result in both AA and KC. The addition of the auxiliary task improves the classification accuracy of the baseline for Vegetation and Low-Density Urban in the first task as well as Low-Density Urban and Developed in the second task.

A detailed comparison of experimental results for the involved methods when the source domain is set to RS-2 is presented in Table 4, while the whole map classification results are illustrated in Figure 10. PSCAN outperformed the previous two methods in this experimental setting, achieving the best results in both transfer tasks. Specifically, for the RS-2→GF-3 task, the proposed PSCAN achieved about a $2\%$ increase in both AA and KC metrics. The conclusions derived from the whole map classification results are consistent with those drawn from the testing results. As indicated in Figure 10a, the performance of PSCAN in the two marked regions is significantly better than other methods. Although Figure 10b shows that the classification performance of PSCAN for water is slightly inferior, it exhibits the best classification performance for urban areas. Moreover, the addition of the auxiliary task improves the accuracy for Low-Density Urban in the first task and Developed in the both tasks.

We present the detailed experimental results on the Qingdao area in Table 5, following a similar approach as that for the San Francisco area. The Qingdao area contains only two datasets, resulting in two transfer tasks: GF-3 as the source domain and RS-2 as the target domain, or RS-2 as the source domain and GF-3 as the target domain. Our results show that the classification performance of the method without adaptation is unsatisfactory and has the lowest classification accuracy among all methods, regardless of the source and target domains. However, when RS-2 is set as the target domain, the traditional UDA methods perform well, and both BDA and TJM methods have similar classification accuracy to the deep UDA methods. However, when GF-3 is used as the target domain, the performance of these methods decreases significantly, which reflects the limitations of using shallow features. In contrast, the performance of deep UDA methods remains relatively stable in both transfer tasks, and the performance of three ADA-based methods is higher. In comparison with other methods, the proposed method achieves the most outstanding classification performance. When RS-2 is the target domain, the proposed method’s evaluation criteria are $1.41\%$, $1.42\%$, and $2.12$ higher than those of the second-best DANN method. With GF-3 as the target domain, the proposed method’s evaluation criteria are also $2.11\%$, $2.10\%$, and $3.21$ higher than those of the second-best CDAN method, which clearly demonstrates its advantages. It achieves the optimal or suboptimal accuracy in the classification of Urban, Vegetation in both tasks. In summary, the proposed method results in an increase of about $2\%$ in OA and AA, and the performance improves by more than $30\%$ compared to that before adaptation.

Figure 11 shows the whole map classification results on the Qingdao area. Similar to the previous ones, the meanings of the results of each row and the circles are given in the caption. From the results, it can be seen that the proposed method achieves the best classification results on both tasks. The conclusions reflected in the classification maps are consistent with the quantitative results, which can verify the effectiveness of the proposed PSCAN.

4.4. Effect of Hyperparameter

As shown in (13), the value of hyperparameter $\alpha$ reflects how much the model pays attention to the auxiliary task in the process of adversarial training. Using the Qingdao datasets, we analyzed the influence of $\alpha$ on the model performance. The experimental results are shown in Figure 12.

According to the experimental settings, we set the value of $\alpha$ within $[0.05,0.15,\cdots ,$$0.95,1.0]$. From the experimental results, it can be seen that when GF-3 is the source domain and RS-2 is the target domain, the classification performance is poor when the $\alpha$ is around $0.85$, and the OA changes slightly between $96.5\%$ and $97.5\%$ in the other times. When RS-2 is source and GF-3 is target, the performance of PSCAN is poor when the value of $\alpha$ is between $0.5$ and $0.75$, and the change of OA is relatively stable when using other values.

4.5. Feature Visualization Results

We used the San Francisco datasets, taking ALOS-2 as the source and GF-3 as the target to train the network. Then, principal component analysis (PCA) was used to reduce the dimension of features extracted before and after the adaption to realize the feature visualization so as to verify the effectiveness of the proposed method in improving the transferability and discriminability of features.

Feature distributions of the data under two categories of Water and Low-Density Urban with low classification accuracy are compared in Figure 13, where the red scatter represents the source data and the blue is the target data.

The experiment used the features corresponding to the first three principal components extracted by PCA to form a three-dimensional space. Compared with the feature visualization result after adaptation, the scatter points in the result before adaptation were more distant from each other, and their spatial distribution was more dispersed, which shows that the proposed method can effectively align the feature distribution of source and target domains, and extract features with transferability.

Then, we compared the feature distribution of all categories of target data before and after adaptation. By reducing the dimension of features extracted from the target domain to two and observing the distance between categories in the two-dimensional plane, the discriminability of the target domain features can be inferred.

As shown in Figure 14a, the samples of Water and Vegetation, High-Density Urban and Low-Density Urban, and Vegetation and Developed have a certain degree of overlap in spatial distribution. In contrast, the result shown in Figure 14b have better feature discriminability, which is reflected in the samples of the Water category being far away from the ones of other categories the fewer overlapping areas between High-Density Urban and Low-Density Urban and between Vegetation and Developed than the result before adaptation. These results prove the effectiveness of the proposed method in improving the discriminability of target features.

4.6. Ablation Experiments

The auxiliary task for receiving the guidance from polarimetric scattering characteristics and the domain discriminator for aligning feature distributions are two important components as well as the keys to improvement. To verify their effectiveness, ablation experiments were conducted. It could be observed that the performance differences of DAN, DANN, and PSCAN can be used to verify this issue. When the hyperparameters of these models were set the same, the DAN could be seen as lacking the domain discriminator version of DANN, while the DANN could be seen as lacking the auxiliary task version of the proposed PSCAN.

Specific comparison results on the two areas are summarized in Figure 15, where the meaning of coordinate axis is consistent with that in Table 1. It can be seen that in the eight transfer tasks, the performance of the model with polarimetric scattering characteristics is better than or at least equal to that of the model without auxiliary task. In addition, the domain discriminator also plays a positive role in improving the accuracy in most cases. These observations show that the performance improvements should be attributed to the addition of the domain discriminator and our proposed auxiliary task, which ensure the transferability and discriminability of features, respectively. In addition, it also shows that the proposed method has the potential to improve the adaptive networks in a plug-and-play manner.

5. Conclusions

This paper presents a novel adversarial learning approach for the unsupervised classification of PolSAR images. Its innovation is reflected in the following aspects: First, the proposed method builds upon the unexplored deep unsupervised DA studies while taking into account the specific characteristics of PolSAR images. Second, given that PolSAR data contains unique information, we designed an auxiliary task to capture the polarimetric scattering characteristics and construct the pseudo-labels using a deep ADA model. This enables us to align the features of the source and target domains while ensuring both feature transferability and discriminability. Third, we propose a PSCAN method by explicitly modeling the feature confusion process. Experimental results demonstrate that our approach can extract domain-invariant and discriminative features and improve the performance on the target domain classification task.

Our experiments utilized datasets from three distinct PolSAR systems covering the San Francisco and Qingdao areas. The results demonstrate that our proposed method achieves superior unsupervised classification performance across most transfer tasks. Furthermore, we discuss the novelty, particularity, and applicable conditions of our approach.

While some progress has been achieved, shortcomings still exist, which are mainly reflected in the fact that the uncertainty in the process of pseudo-label construction has not received sufficient attention. In future work, we aim to address this issue through selective pseudo-labeling. Moreover, several unresolved issues in the PolSAR community, including improving the utilization of polarimetric information in the PolSAR data, exploring multisource-based adaptation techniques, and tackling domain generalization problems are all issues we are considering.