NeuroImage 268 (2023) 119863

Contents lists available at ScienceDirect

NeuroImage
journal homepage: www.elsevier.com/locate/neuroimage

DomainATM: Domain adaptation toolbox for medical data analysis

Hao Guan, Mingxia Liu∗
The Department of Radiology and Biomedical Research Imaging Center, University of North Carolina at Chapel Hill, Chapel Hill, NC 27599, USA

a r t i c l e i n f o a b s t r a c t

Keywords: Domain adaptation (DA) is an important technique for modern machine learning-based medical data analysis,
Domain adaptation which aims at reducing distribution differences between different medical datasets. A proper domain adaptation
Medical image analysis method can significantly enhance the statistical power by pooling data acquired from multiple sites/centers. To
Medical image processing toolbox
this end, we have developed the Domain Adaptation Toolbox for Medical data analysis (DomainATM) – an open-
Open source software
source software package designed for fast facilitation and easy customization of domain adaptation methods for
medical data analysis. The DomainATM is implemented in MATLAB with a user-friendly graphical interface, and
it consists of a collection of popular data adaptation algorithms that have been extensively applied to medical
image analysis and computer vision. With DomainATM, researchers are able to facilitate fast feature-level and
image-level adaptation, visualization and performance evaluation of different adaptation methods for medical
data analysis. More importantly, the DomainATM enables the users to develop and test their own adaptation
methods through scripting, greatly enhancing its utility and extensibility. An overview characteristic and usage
of DomainATM is presented and illustrated with three example experiments, demonstrating its effectiveness,
simplicity, and flexibility. The software, source code, and manual are available online.

1. Introduction gained considerable attention and increasing interest recently (Guan and
Liu, 2022; Valverde et al., 2021). Briefly, domain adaptation can be de-
Medical data analysis is nowadays being boosted by modern statisti- fined as follows. Let  ×  represent the joint feature space of samples
cal analysis tools, i.e., machine learning (Barragán-Montero et al., 2021; and their corresponding category labels. A source domain  and a target
Deo, 2015; Erickson et al., 2017; Fatima et al., 2017; Rajkomar et al., domain  are defined on the joint feature space, with different distri-
2019). Classic machine learning typically assumes that training dataset butions 𝐏𝐒 and 𝐏𝐓 , respectively. Suppose there are 𝑛𝑠 samples (subjects)
(source domain) and test dataset (target domain) follow an independent with or without category labels in the source domain, as well as 𝑛𝑡 sam-
but identical distribution (Valiant, 1984). In real-world practice, how- ples in the target domain without category labels. Then the problem
ever, this assumption can hardly hold due to the well-known “domain is how to reduce the distribution differences/variability between source
shift” problem (Kondrateva et al., 2021; Pooch et al., 2020; Quiñonero- and target domains so as to increase the performance of down-streaming
Candela et al., 2009). In medical imaging, domain shift or data het- tasks such as classification or segmentation.
erogeneity is widespread and caused by different scanning parameters Many domain adaptation methods have been proposed or utilized
(i.e., between-scanner variability) and subject populations in multiple in the field of medical data analysis which shows tremendous appli-
imaging sites. It may increase the test error along with the distribution cability. Most solutions, however, are implemented independently for
difference between training and test data (Ben-David et al., 2007; Tor- very specific scenarios or target applications. Researchers often need
ralba and Efros, 2011). Thus the domain shift/difference may greatly to re-implement an algorithm or do methodological tailoring. The dif-
degrade statistical power of multi-site/multi-center studies and hinder ferences in implementation will often cause inconsistent experiment
the building of effective machine learning models. and analysis results. There is a lack of a unified platform for exten-
For handling the domain shift problem among datasets and enhanc- sive comparison of different domain adaptation methods, helping avoid
ing the generalization ability of machine learning models, domain adap- hand-crafted re-implementation for specific medical data analysis re-
tation has gradually come under the spotlight of the research commu- search. Thus a software toolbox that provides a platform of different
nity (Csurka, 2017; Kouw and Loog, 2019; Patel et al., 2015; Wang and adaptation methods is quite beneficial and necessary for researchers
Deng, 2018; Wilson and Cook, 2020; Zhang et al., 2020; Zou et al., to compare, evaluate and select the proper method for their research
2020). In the field of medical data analysis, domain adaptation has project.

∗
Corresponding author.
E-mail address: [email protected] (M. Liu).

https://doi.org/10.1016/j.neuroimage.2023.119863.
Received 31 October 2022; Received in revised form 11 December 2022; Accepted 3 January 2023
Available online 5 January 2023.
1053-8119/© 2023 The Author(s). Published by Elsevier Inc. This is an open access article under the CC BY-NC-ND license
(http://creativecommons.org/licenses/by-nc-nd/4.0/)
H. Guan and M. Liu NeuroImage 268 (2023) 119863

with user-defined statistical properties that can simulate domain shift. A

dataset is in the format of 𝑀 × 𝑁 matrix, where 𝑀 denotes the number
of samples while 𝑁 represents the feature dimension. 2) The algorithm
module contains the implementations of different domain adaptation
methods. All these adaptation algorithms have uniform input/output pa-
rameter formats. Users can easily add their self-defined algorithms into
the toolbox with the same input/output format. By default, several rep-
resentative methods which have been widely used in medical data anal-
ysis are included in the DomainATM. These methods can be categorized
into feature-level adaptation methods and image-level adaptation meth-
ods. Besides, inspired by the design philosophy of fast facilitation, most
of the algorithms included in the toolbox can run in real time and output
results in seconds. 3) The evaluation module assesses the performance
of different adaptation methods. For feature-level adaptation methods,
Fig. 1. Illustration of the “domain shift” phenomenon (Quiñonero-
we employ two evaluation metrics, including: domain-level classifica-
Candela et al., 2009) (top row) and the fundamental of domain adaptation
tion accuracy and domain distribution distance. For image-level adap-
(distribution of source and target samples before and after adaptation).
tation methods, we use three evaluation metrics, including correlation
coefficient (CC), peak signal-noise ratio (PSNR) and mean square er-
An important issue for medical imaging researchers is the fast fa- ror (MSE). The DomainATM provides visualization functions to visual-
cilitation of domain adaptation algorithms. Due to privacy protection ize the data distribution (or images) before and after adaptation which
issues, many real-world medical data sets are not accessible or with helps investigate and understand the performance of different domain
restrictions. Using synthetic data which is able to simulate the “do- adaptation algorithms.
main shift” phenomenon in a machine learning setting will greatly boost The DomainATM is implemented in MATLAB (originally imple-
the efficiency. Another limitation is the complexity of certain domain mented in MATLAB 2021b on Windows 10, MATLAB 2019 or more ad-
adaptation methods. Time-consuming model training and exhaustive pa- vanced versions are all good for it). Through test, DomainATM can be
rameter tuning will be rather inconvenient, especially for researchers run on Windows, Mac OS and Linux systems. It can be easily used with
without high-level programming skills. Thus, fast facilitation of domain a graphical-user-interface (GUI), as shown in Fig. 3. The hardware plat-
adaptation methods with real-time visualization for performance check form can be a CPU-based PC (originally developed on Intel i-7 PC with
is beneficial for medical data analysis. 16 GB memory), which does not require much computation or memory
We also observe that in medical imaging image-level domain adap- resources. For advanced users, DomainATM provides an interface for
tation is an important topic (Guan and Liu, 2022). For example, MRIs writing MATLAB scripts to implement self-defined domain adaptation
acquired from different scanners may negatively influence the analysis methods. The software, manual and source code for DomainATM are
result (Lee et al., 2019; Wittens et al., 2021). This has become the con- accessible online1 .
cern of many radiologists and neuroscientists. Thus incorporating both
feature-level and image-level adaptation methods into one platform is 3. Toolbox workflow
beneficial for related medical imaging research.
In light of these motivations, we develop the Domain Adaptation 3.1. Creating/loading data
Toolbox for Medical data analysis (DomainATM) – a software package
that offers a platform for simulating, evaluating and developing differ- The DomainATM can work for both feature-level adaptation and
ent domain adaptation algorithms for medical data analysis. The tool- image-level adaptation. These two key modules in the toolbox are inde-
box is designed with a major principle that it could help researchers do pendent of each other. With respect to the input of feature-level adap-
fast facilitation of adaptation methods. Besides real-world medical data, tation, the toolbox accepts data in standard MATLAB .mat file format.
synthetic data with user-defined statistical properties can be generated Each row represents an observation (subject or sample) while every col-
quickly for real-time simulation. Both feature-level and image-level do- umn represents a feature. Existing real-world medical datasets (in .mat
main adaptation algorithms are included in the software package with format) can be directly imported and loaded into the toolbox for process-
a graphical-user-interface (GUI). The running results will be automat- ing. In addition, the users can create a synthetic dataset. After assigning
ically saved which can be further analyzed by the evaluation module the sample number, mean value and covariance matrix, the toolbox can
of the toolbox. All the algorithms have consistent input/output formats automatically generate a synthetic dataset following a normal distri-
under which the users can define their own adaptation algorithms and bution. After loading the real/synthetic data, their distribution will be
add them to the DomainATM freely. Thus the toolbox has good flexibil- automatically displayed in the toolbox. Both the real-world and created
ity and scalability. datasets are stored in the “data” subfolder of the toolbox.
This paper is organized as follows. In Section 2, we introduce the For image-level adaptation, the toolbox currently accepts 3D volu-
characteristics of DomainATM, including its overall structure, key fea- metric data (in .nii format). All the data will be converted to inner-built
tures and functions. In Section 3, the workflow of DomainATM for the data in MATLAB. After loading the volumetric data, a middle slice (in
facilitation of domain adaptation is described. In Section 4, representa- axial view) will be automatically shown. Note that the “Create Dataset”
tive domain adaptation methods that have been included in the toolbox module currently only generates data for feature-level domain adapta-
are presented. In Sections 5 and 6, experiments for both feature-level tion.
and image-level adaptation are conducted to illustrate the application
of the toolbox. This paper is concluded in Section 7.
3.2. Selecting domain adaptation algorithms

2. Toolbox overview/characteristics
After loading the data, the following procedure is to select, configure,
and run the domain adaptation methods. Most adaptation methods have
The main structure of the DomainATM is illustrated in Fig. 2. Cur-
several hyper-parameters to be set. Users can tune them according to the
rently, the toolbox consists of three modules. 1) The data module is
responsible for loading and generating datasets. It can directly load an
1
existing medical dataset (in .mat data file) or create synthetic datasets https://www.mingxia.web.unc.edu/domainatm/

2
H. Guan and M. Liu NeuroImage 268 (2023) 119863

Fig. 2. Illustration of workflow of the DomainATM software. The DomainATM consists of three major components: 1) the data module loads or creates the datasets;
2) the algorithm module conducts feature-level or image-level domain adaptation and saves the results; and 3) the evaluation module assesses the adaptation
performance according to specific metrics. DA: Domain Adaptation.

All the files are named with the corresponding adaptation method with
time stamp.

3.3. Evaluating data adaptation performance

After running the adaptation methods and getting the results, per-
formance evaluation can be conducted for the methods. For feature-
level adaptation, we use distribution difference and domain-level classi-
fication accuracy as two metrics to assess the adaptation performance.
For image-level adaptation, we adopt correlation coefficient (CC), peak
signal-to-noise ratio (PSNR) and mean-square error (MSE) to evaluate the
adaptation result. More details about these evaluation metrics will be
elaborated in the experiment section.

3.4. Visualization of data adaptation results

Besides quantitative evaluation, result visualization is useful for

qualitative analysis. The DomainATM provides visualization functions
that help users better understand domain adaptation for medical data.
For feature-level adaption, the feature distribution (in 2D space) be-
fore and after adaptation can be visualized. High-dimensional features
will be mapped to 2D feature space via t-SNE (Van der Maaten and
Hinton, 2008). For image-level adaptation, the adapted source image,
Fig. 3. Graphical-User-Interface (GUI) of DomainATM. the original source and target images can be viewed using the tool-
box. After the adapted images have been saved in the “evaluation” sub-
folder, they can also be visually inspected by other medical imaging
software.
specific tasks. Otherwise, default settings of these methods will be used.
After configuration, the users can run the algorithms. All the built-in 3.5. Extension: Adding self-defined data adaptation algorithm
methods provided by the toolbox are simple, easy to use, and can run
in real time within 5 seconds (on a PC with an Intel i-7 CPU, 16 GB In some tasks of medical data analysis, users might need to develop
memory). their own domain adaptation methods. The DomainATM supports self-
After running the adaptation methods, the results will be automat- defined algorithms for task-specific usage. The users can write a MAT-
ically saved in the “evaluation” subfolder of the toolbox. For feature- LAB script to define and implement their algorithms. The input/output
level adaptation, the original source/target data, and the adapted format of the self-defined functions has to be consistent with other built-
source/target data will be saved (in .mat data format). For image-level in adaptation methods. When adding an new algorithm, the self-defined
adaptation, the adapted source images (target image is used as the ref- script should be put in the “algorithms_feat” (feature-level) or the “al-
erence image and will not be changed) will be saved (in .nii format). gorithms_img” (image-level) subfolders in the toolbox. One can sim-

3
H. Guan and M. Liu NeuroImage 268 (2023) 119863

ply run and analyze their methods like the other built-in ones through 4.1.9. Scatter Component Analysis (SCA)
GUI. In this algorithm (Ghifary et al., 2016), original features are firstly
projected to a reproducing kernel Hilbert space. Domain adaptation is
4. Algorithms then conducted through an optimization formulation, including max-
imizing the class separability, maximizing the data separability, and
In this section, we briefly introduce the algorithms for feature-level minimizing domain mismatch. Category labels of the source domain are
and image-level data adaptation in DomainATM. More details can be used during adaptation. The key parameter is the dimension of the trans-
found in the online manual. formed space.

4.1. Feature-level data adaptation algorithm 4.1.10. Information-Theoretical Learning (ITL)

In this algorithm (Shi and Sha, 2012), an optimal feature space is
4.1.1. Baseline learned through jointly maximizing domain similarity and minimizing
No feature-level domain adaptation is utilized. Both source and tar- the expected classification error on target samples. Source category la-
get data are kept in their original distributions (in the feature space). bels are required. The key hyper-parameters include subspace dimen-
sion and regularization parameter.
4.1.2. Subspace Alignment (SA)
In this algorithm (Fernando et al., 2013), the source and target med-
4.2. Image-level data adaptation algorithm
ical data are represented by subspaces in terms of eigenvectors. The
source data are projected to the target domain through a transforma-
4.2.1. Baseline
tion matrix. No category labels of source domain are needed. The key
For two medical images acquired by different scanners/sites, no do-
hyper-parameter is the dimension of the shared subspace.
main adaptation is facilitated in this method. Instead, the homogene-
4.1.3. Correlation Alignment (CORAL) ity/heterogeneity of the paired original images is directly compared in
In this algorithm (Sun et al., 2016), domain shift/difference is mini- terms of certain evaluation metrics.
mized by aligning the second-order statistics (e.g., covariance) of source
and target distributions. No category label information and hyper- 4.2.2. Histogram Matching (HM)
parameters are required for this method. This method transforms source image to make its histogram matches
the histogram of the target image (Shinohara et al., 2014). After adapta-
4.1.4. Transfer Component Analysis (TCA) tion, the intensity distributions of the source and target images become
In this algorithm (Pan et al., 2010), a subspace shared by the source closer.
and target domain is searched in a reproducing kernel Hilbert space by
minimizing the maximum mean discrepancy (MMD) distance. No source 4.2.3. Spectrum Swapping-based Image-level MRI Harmonization (SSIMH)
category labels are demanded. The key hyper-parameters are the kernel In this method (Guan et al., 2022), the source and target images are
type and subspace dimension. firstly transformed into the frequency domain (e.g., through Discrete Co-
sine Transform). Then, part of the low-frequency region of source im-
4.1.5. Optimal Transport (OT) age is replaced by the corresponding low-frequency area of the target
In this algorithm (Guan et al., 2021b), the samples in the source image. Finally, the source image in the revised frequency domain is in-
domain are projected into the target domain while keeping their con- verted back to the spatial domain to get the adapted image. The key
ditional distributions. The projection is facilitated through minimiza- hyper-parameter of this method is the threshold which defines the low-
tion of Wasserstein distance between the two distributions. No category frequency region that is swapped between source and target images. In
labels of the source domain are used. The key hyper-parameter is the the toolbox, the default value is set to 3.
regularization coefficient. The image-level domain adaptation methods work well in two dif-
ferent settings. (1) One-to-one image harmonization: Given a source im-
4.1.6. Joint Distribution Adaptation (JDA) age and a target/reference image, one can select a specific algorithm to
In this algorithm (Long et al., 2013), maximum mean discrepancy adapt the source image to the target image space. (2) Batch image har-
(MMD) is adopted to measure domain distribution differences, and is monization: Given multiple source images and a target image, we can
integrated into Principal Component Analysis (PCA) to build a represen- adapt all source images to target image space via batch harmonization.
tation that is robust to domain shift. Source category labels are needed in
this algorithm. The key hyper-parameters include kernel type, subspace 5. Empirical evaluation of feature-level data adaptation
dimension and regularization parameter. algorithms in DomainATM

4.1.7. Transfer Joint Matching (TJM)

5.1. Evaluation metric
In this algorithm (Long et al., 2014), feature matching and instance
reweighting strategies are combined to reduce domain shift. Minimiza-
For feature-level adaptation methods, we adopt the metrics that eval-
tion of maximum mean discrepancy (MMD) and 𝑙2,1 norm sparsity
uate the distribution changes before and after the adaptation process.
penalty on source data are integrated into PCA to construct domain-
Specifically, we use the following three methods/metrics for adaptation
invariant features. Category labels of source domain are required. The
performance evaluation.
key hyper-parameters include kernel type, subspace dimension and reg-
ularization parameter. • Distribution difference. We adopt maximum mean discrepancy
(MMD) to measure the data distribution differences between the
4.1.8. Geodesic Flow Kernel (GFK) source and target domains before and after domain adaptation. As a
In this algorithm (Gong et al., 2012), the source and target data are popular metric, the maximum mean discrepancy (MMD) has been
embedded into the Grassmann manifolds, and the geodesic flows be- widely used in domain adaptation research (Kumagai and Iwata,
tween them are used to model domain shift. Domain adaptation is con- 2019; Long et al., 2013; 2014; Pan et al., 2010; Wang et al., 2021;
ducted by projecting the data into several domain-invariant subspaces Yan et al., 2017), defined as follows:
on the geodesic flow. Source category labels can be either used or not.
‖ ‖2
The key hyper-parameter is the subspace dimension. 𝐌𝐌𝐃2𝑘 = ‖𝐄𝑝 [𝜙(𝐱𝑠 )] − 𝐄𝑞 [𝜙(𝐱𝑡 )]‖ (1)
‖ ‖𝑘

4
H. Guan and M. Liu NeuroImage 268 (2023) 119863

Fig. 4. Distribution of the synthetic data (baseline) and adapted data by nine different domain adaptation methods in the DomainATM toolbox. ( positive source
samples; positive target samples; negative source samples; negative target samples).

where 𝑘 denotes the Reproducing Kernel Hilbert Space endowed Table 1

⟨ ⟩
with a kernel function 𝑘, and 𝑘(𝐱𝑠 , 𝐱𝑡 ) = 𝜙(𝐱𝑠 ), 𝜙(𝐱𝑡 ) . If the MMD Domain classification accuracy (%) using different classifiers on the synthetic
distance of source and target domains gets lower after adaptation, it dataset. (SVM: support vector machine; RF: random forest).
indicates the data distribution difference becomes smaller. Method Baseline SA CORAL OT TCA TJM JDA GFK SCA ITL
• Domain classification. Suppose an equal number of samples are
SVM 85 47 80 35 77 40 41 39 60 41
sampled from the source and target domains, respectively. These RF 85 51 82 24 60 47 37 41 78 59
samples are assigned with domain labels, i.e., the source samples are
labeled as “1” while target samples are assigned with the label “0”.
A domain discriminator/classifier is applied to all samples for distin-
5.2.2. Distribution difference
guishing which samples come from the source domain and which
The data distribution differences (in terms of maximum mean dis-
ones are from the target domain. The classification result is used
crepancy) of the source and target domains after domain adaptation are
to assess domain shift/difference. A high domain classification ac-
shown in Fig. 5. The result of the Baseline method shows the original
curacy indicates that the source and target samples can be easily
distribution of the source and target domain without any adaptation
distinguished, which means the domain shift is large. In contrast, if
processing. From Fig. 5, we can observe that domain adaptation can re-
the domain classification accuracy drops down after the adaptation
duce the distribution differences between the original source and target
processing, it indicates the domain adaptation algorithm works be-
domains.
cause it makes the two domains get closer and become more difficult
to distinguish.
5.2.3. Domain-level classification
We conduct domain-level classification on the source and target data.
5.2. Experiment 1: Adaptation on synthetic dataset A domain classifier (we use a k-nearest neighbors classifier) is trained
with source data (with the label “1”) and target data (with the label
We first conduct experiments on synthetic datasets using Do- “0”). Source and target data are combined together and shuffled. In the
mainATM. Users can set the statistical properties of the synthetic data experiments, we use 60% of the entire data samples for training the do-
freely using DomainATM, and thus, can conduct fast test of different do- main classifier while 40% are for test. The result of domain classification
main adaptation methods, which is helpful for understanding the char- accuracy is shown in Fig. 6.
acteristics of different methods and avoiding the access restrictions of We also use another two classifiers, i.e., support vector machine
many real-world medical datasets. Specifically, we generate two do- (SVM) and random forest (RF) for domain-level classification. For the
mains by Gaussian distributions. Each domain has two classes, with 30 SVM, we use a linear kernel and the penalty parameter C is set to 1.
positive samples and 30 negative ones, respectively. For the source do- For the RF, 50 decision trees are used for the ensemble classification.
main , the means of positive and negative samples are [0, 0] and [0, These settings are also used for the other experiments. Their domain-
1], while their covariance matrices are [0.2, 0; 0, 0.2] and [0.1, 0; 0, level classification results are shown in Table 1.
0.1]. For the target domain  , the means of positive and negative sam- From Fig. 6 and Table 1, it can be seen that the domain classification
ples are [1, -0.5] and [1, 0.2], while their covariance matrices are [0.2, accuracy drops after domain adaptation even different classifiers are
0; 0, 0.2] and [0.1, 0; 0, 0.1]. used. This implies that source and target data become more difficult
to be distinguished, i.e., domain adaptation makes their distributions
5.2.1. Data distribution visualization become more similar than in the original space.
The distributions of the original data and the adapted data by differ-
ent methods are visualized in Fig. 4. From the visualization result, differ- 5.3. Experiment 2: Adaptation for Alzheimer’s disease analysis on ADNI
ent domain adaptation methods can reduce the distributions of source
and target samples to certain extent. For example, the optimal transport We conduct experiments on the Alzheimer’s Disease Neuroimaging
adaptation (OT) can project the source data into the target domain, and Initiative (ADNI) dataset (Jack Jr et al., 2008). The dataset consists of
make the source distribution quite similar to the target domain. T1-weighted MRI data for Alzheimer’s disease (AD) analysis. We use

5
H. Guan and M. Liu NeuroImage 268 (2023) 119863

Fig. 5. Synthetic data distribution differences in terms

of maximum mean discrepancy before (baseline) and
after domain adaptation using nine feature-level adap-
tation methods.

Fig. 6. Synthetic data distribution differences in terms

of domain-level classification accuracy on the syn-
thetic dataset before (baseline) and after domain adap-
tation using nine feature-level adaptation methods.

two subsets of ADNI, i.e., ADNI-1 (100 subjects with 1.5T T1-weighted Table 2
structural MRIs) and ADNI-2 (100 subjects with 3.0T T1-weighted struc- Domain classification accuracy (%) using different classifiers on the ADNI-1 and
tural MRIs) as the source and target domains, respectively, to test the ADNI-2 datasets. (SVM: support vector machine; RF: random forest).
domain adaptation algorithms using DomainATM. ADNI-1 contains 50 Method Baseline SA CORAL OT TCA TJM JDA GFK SCA ITL
patients with Alzheimer’s disease (AD) (positive samples) and 50 nor-
SVM 79 43 79 43 77 43 43 43 43 43
mal control (NC) subjects (negative samples). ADNI-2 has 50 CE sub- RF 85 64 85 52 80 63 58 60 50 57
jects and 50 NC subjects. All the MRIs have been processed through a
standard pipeline, including skull stripping, intensity correction, regis-
tration and re-sampling. Regions-of-interest (ROIs) features which are
5.3.3. Domain-level classification
defined on 90 regions in the Anatomical Automatic Labeling (AAL) at-
We facilitate domain-level classification on the source data, i.e.,
las (Tzourio-Mazoyer et al., 2002) are used to represent each subject.
ADNI-1, and target data, i.e., ADNI-2. A domain classifier (k-nearest
The 90-dimensional features denote the gray matter volumes in each
neighbors classifier) is trained with source data (with the label “1”) and
brain region.
target data (with the label “0”). Source and target data are combined
together and shuffled. 60% of the entire data are adopted for training
5.3.1. Distribution visualization while 40% for testing. The result of domain-level classification is illus-
The distributions of original ADNI-1 and ADNI-2 data (in feature trated in Fig. 9. Another two classifiers, including support vector ma-
space) and the adapted data by different methods are visualized in Fig 7. chine (SVM) and random forest (RF) are also adopted for domain-level
From the visualization results, the original source and target data have a classification, and the result is listed in Table 2. From Fig. 9 and Table 2,
relatively clear boundary. After domain adaptation, the domain bound- we can see that the domain classification accuracy drops after domain
aries become blurred, and the distribution of source and target domains adaptation despite the different types of domain classifiers. This indi-
gets closer to each other. cates that the adapted source and target data get more difficult to be
correctly classified, i.e., domain adaptation is effective in reducing their
distribution differences.
5.3.2. Distribution distance
The distribution differences (in terms of maximum mean discrep-
ancy) of the source data, i.e., ADNI-1, and target data, i.e., ADNI-2, after 5.4. Experiment 3: Domain adaptation for autism analysis on ABIDE
domain adaptation are shown in Fig. 8. The baseline illustrates the origi-
nal distribution of the source and target domain without any adaptation We conduct experiments on the Autism Brain Imaging Data Exchange
processing. From the result, it can be observed that domain adaptation (ABIDE) dataset (Di Martino et al., 2014). This database consists of
is able to reduce the distribution differences between the original source resting-state functional MRI (fMRI) data for Autism analysis. We use two
and target domains. sites from the ABIDE project, i.e., NYU (184 subjects) and UM (145 sub-

6
H. Guan and M. Liu NeuroImage 268 (2023) 119863

Fig. 7. Distribution of the original ADNI data (baseline) and adapted data by nine feature-level domain adaptation methods in the DomainATM toolbox. ( positive
source samples; positive target samples; negative source samples; negative target samples).

Fig. 8. Data distribution differences in terms of

maximum mean discrepancy on ADNI-1 and ADNI-
2 before (baseline) and after domain adaptation op-
erations.

Fig. 9. Data distribution differences in terms of

domain-level classification accuracy on ADNI-1
and ADNI-2 before (baseline) and after domain
adaptation operations.

jects) as the source and target domains, respectively, to test the domain correction, nuisance signal regression, temporal filtering, and registra-
adaptation algorithms using the DomainATM. The NYU site consists of tion. The mean time series of 116 regions-of-interest (ROIs) defined by
79 positive samples (autism patients) and 105 negative samples (normal the Anatomical Automatic Labeling (AAL) atlas (Tzourio-Mazoyer et al.,
controls). These fMRIs are acquired by a 3 Tesla Allegra scanner. The 2002) are extracted. Then, a 116 × 116 symmetrical resting-state func-
UM site includes 68 positive samples (autism patients) and 77 negative tional connectivity (FC) matrix is generated for each subject, with each
samples (normal controls). These fMRIs are acquired using a 3 Tesla element representing the Pearson correlation coefficient between a pair
GE scanner located at the UM Functional MRI Laboratory. All the fM- of ROI signals. We extract the node betweenness centrality (Rubinov and
RIs go through a standard pipeline, including slice-timing and motion Sporns, 2010) based on the FC matrix to represent each subject/sample.

7
H. Guan and M. Liu NeuroImage 268 (2023) 119863

Fig. 10. Distribution of the original ABIDE data (baseline) and adapted data by nine feature-level domain adaptation methods in the proposed DomainATM toolbox.
( positive source samples; positive target samples; negative source samples; negative target samples).

Fig. 11. Data distribution differences of two sites

of ABIDE in terms of maximum mean discrepancy
before (baseline) and after domain adaptation us-
ing nine feature-level adaptation methods.

5.4.1. Distribution visualization Table 3

The original distributions of two sites in ABIDE (in feature space) and Domain classification accuracy (%) using different classifiers on two sites of
the adapted data by different methods are visualized in Fig. 10. From ABIDE dataset. (SVM: support vector machine; RF: random forest).
the visualization result, it can be observed that the boundary between Method Baseline SA CORAL OT TCA TJM JDA GFK SCA ITL
original source and target data is relatively clear. After the domain adap-
SVM 69 55 67 55 68 55 55 55 55 65
tation processing, the domain boundaries become blurred, and the dis- RF 66 55 64 31 66 48 43 50 65 63
tributions of source and target domain get similar to each other.

5.4.2. Distribution distance result is shown in Table 3 From the results, the domain classification
The data distribution differences (in terms of MMD) of the source accuracy gets worse after domain adaptation processing regardless of
NYU domain and target UM domain after domain adaptation are shown what domain classifiers have been used. This indicates that the adapted
in Fig. 11. The baseline is the original distribution of the source and source and target data become more difficult to be discriminated, i.e.,
target domain without any adaptation processing. The result shows that using domain adaptation has successfully reduced their distribution dif-
the distribution differences become smaller after adaptation processing ferences.
by different algorithms.
5.5. Discussion
5.4.3. Domain-level classification
We facilitate domain-level classification on the source data, i.e., In the above experiments, we use two quantitative metrics, i.e., MMD
NYU, and target data, i.e., UM. A domain classifier (k-nearest neigh- and domain classification accuracy, to evaluate the performance of dif-
bors classifier) is trained with source data (with the label “1”) and tar- ferent domain adaptation methods in DomainATM. The MMD is a direct
get data (with the label “0”). Source and target data are combined to- assessment metric because it is directly calculated based on the statis-
gether and shuffled. 60% of the entire data are adopted for training tical properties of source and target domains (datasets). Generally, if
while 40% for test. The result of domain-level classification accuracy is method A achieves a smaller MMD than method B, then A is supposed
illustrated in Fig. 12. We also use support vector machine (SVM) and to be better. Domain classification accuracy is an indirect metric because
random forest (RF) to conduct the domain-level classification, and the it relies on a specific domain classifier. But it can also reflect the adap-

8
H. Guan and M. Liu NeuroImage 268 (2023) 119863

Fig. 12. Data distribution differences in terms of

domain-level classification accuracy on two sites of
ABIDE before (baseline) and after domain adapta-
tion using nine feature-level adaptation methods.

Table 4 Specifically, we adopt the following three metrics for image-level adap-
Running time (in terms of seconds) of nine domain adaptation algorithms in tation performance evaluation.
DomainATM on three datasets.
• Correlation Coefficient (CC). Denote the source and target images
Method SA CORAL OT TCA TJM JDA GFK SCA ITL as 𝑠 and 𝑡 . After adaptation, we get 𝑠′ . For performance assess-
Synthetic 0.09 0.05 1.28 0.06 0.19 0.85 0.09 1.04 0.06 ment, if the correlation coefficient of 𝑠′ and 𝑡 is higher than 𝑠 and
ADNI 0.05 0.01 2.78 0.04 0.21 0.92 0.09 1.13 0.13 𝑡 , it indicates the corresponding adaptation algorithm works.
ABIDE 0.03 0.01 6.07 0.07 0.26 0.92 0.09 1.74 0.25 • Peak Signal-to-Noise Ratio (PSNR). If the peak signal-to-noise ra-
tio of 𝑠′ and 𝑡 is higher than 𝑠 and 𝑡 , it indicates the adaptation
algorithm works.
tation performance since confusing a classifier is difficult. If method A • Mean-Squared Error (MSE). If the mean-squared error of 𝑠′ and 𝑡
achieves a smaller domain classification accuracy than method B, then is smaller than 𝑠 and 𝑡 , it indicates the adaptation algorithms are
A is supposed to be better. Based on the experimental results, we have effective.
the following empirical findings.
6.2. Materials and settings
• The CORAL, TCA and SCA algorithms have relatively worse domain
adaptation performance than the other methods. They get signifi- Phantom data of five traveling subjects with T1-weighted (T1-w)
cantly higher MMD values and domain classification accuracy than structural MRIs from the ABCD dataset (Volkow et al., 2018) are used for
the others. performance evaluation. Phantom-1 is scanned by GE and Philips scan-
• The OT algorithm achieves the overall best performance among ners, respectively. Phantom-2 and Phantom-3 are acquired by Siemens
these adaptation methods. It generally produces the smallest MMD and GE scanners, respectively. Phantom-4 and Phantom-5 are scanned
value and domain classification accuracy in all these three experi- by Philips and Siemens scanners, respectively. The protocols of the GE,
ments. Philips and Siemens scanners are consistent. These phantoms are used to
• On the ADNI dataset, the TJM, JDA, GFK and ITL have comparable test the performance of image-level domain adaptation methods in han-
performance. They get similar domain classification accuracy and dling domain shift caused by different scanners. All these 3D MRIs are
low MMD. On the ABIDE dataset, the algorithm ITL is worse than raw data in the NIfTI file format. We do not perform any pre-processing
the others. such as skull-stripping, registration or segmentation before image-level
• Most algorithms are effective in significantly reducing the MMD adaptation. During adaptation, the intensity of each image is normal-
value. By contrast, the domain classification accuracy is more dif- ized to the range of [0, 1]. For these volumetric images which contain
ficult to reduce. This implies that it is challenging to confuse or de- multiple slices, the adaptation is facilitated on each slice, then the per-
ceive a domain classifier with certain domain adaptation methods. formance is calculated as an average metric value for all the slices within
Thus, domain classification accuracy is a rigorous metric to assess an image (volume).
the robustness of an adaptation algorithm.
6.3. Result
We also conduct statistical testing for performance comparison in
terms of domain classification accuracy. Specifically, we compute the We conduct image-level domain adaptation on these five phantom
p-values via paired sample t-test between each adaptation method and structural MRI data, and the adaptation results in terms of the three met-
the baseline. The p-values are smaller than 0.05, indicating that their rics are shown in Table 5. From the result, it can be observed that image-
differences are significant. In addition, we calculate the running time of level domain adaptation methods can generally achieve higher scores of
each domain adaptation algorithm for each dataset on a PC with an Intel correlation coefficient (CC) and peak signal-to-noise ratio (PSNR) and
i-7 CPU and 16 GB memory. The comparison result is listed in Table 4. smaller mean square error (MSE). In some cases (e.g., GE → Philips), the
Histogram Matching (HM) does not perform very well in terms of PSNR
and MSE. Overall, the result indicates that image-level adaptation meth-
6. Empirical evaluation of image-level data adaptation ods are useful in reducing the distribution shift between images caused
algorithms in DomainATM by different scanners.

6.1. Evaluation metrics 6.4. Visual inspection

For image-level adaptation methods, we adopt the metrics that eval- To further investigate the effectiveness of image-level domain adap-
uate the image similarity/dissimilarity before and after adaptation. tation, we do visual inspections of the MRIs that are adapted to dif-

9
H. Guan and M. Liu NeuroImage 268 (2023) 119863

Table 5
Results of three image-level domain adaptation methods on T1-weighted MRIs of five traveling phantom subjects acquired by three
different scanners from the ABCD dataset.

Source Domain→Target Method CC PSNR MSE

Domain (Subject ID)

GE→Siemens (Phantom-2, Baseline 0.4889±0.0081 21.4143±2.8718 0.0080±0.0049

Phantom-3) HM 0.5642±0.0395 22.3131±2.5975 0.0064±0.0037
SSIMH 0.5935±0.0221 22.7624±2.5310 0.0057±0.0032
Philips→Siemens Baseline 0.5408±0.0194 18.7578±0.8847 0.0135±0.0028
(Phantom-4, Phantom-5) HM 0.5495±0.0388 18.7477±1.1303 0.0135±0.0035
SSIMH 0.6098±0.0269 20.1269±1.8421 0.0101±0.0042
GE→Philips (Phantom-1) Baseline 0.4682 21.3915 0.0073
HM 0.5108 21.2482 0.0075
SSIMH 0.5570 22.6421 0.0054

Fig. 13. Image-level domain adaptation via the Spectrum Swapping-based Image-level MRI Harmonization (SSIMH) method (Guan et al., 2022) for T1-weighted
(T1-w) MRIs acquired by different scanners. Domain shift caused by the use of different scanners can be partly reduced by image-level adaptation via SSIMH.

ferent scanner styles. We divide the phantom MRIs into three groups 7. Conclusion and future work
in terms of the scanners. Then we adapt MRIs acquired by one scan-
ner to the styles of MRIs scanned by other scanners. We use the SSIMH Domain adaptation has become an important topic in the field of
method (Guan et al., 2022) in DomainATM to perform image-level adap- medical data analysis. In this paper, we develop a Domain Adaptation
tation. Fig. 13 shows the results of three different MRIs and their corre- Toolbox for Medical data analysis (DomainATM), aiming to help re-
sponding adapted images to different scanner styles. From the result, we searchers facilitate fast domain adaptation for medical data acquired
have the following two observations. 1) Different scanners (i.e., Siemens, from different sites/scanners. The DomainATM is easy to use, efficient
Philips and GE) have a significant impact on the MRIs, which can cause to run, and most importantly, it is able to do both feature-level and
the domain shift. 2) The image-level domain adaptation method is ef- image-level adaptation. In addition, users can add their own domain
fective in harmonizing the source image to the target image (reference adaptation algorithms into the toolbox, making it flexible and extensi-
image) and reducing the domain shift caused by different scanners. ble. Experiments on both synthetic and real-world medical datasets have

10
H. Guan and M. Liu NeuroImage 268 (2023) 119863

been conducted to show the usage and effectiveness of DomainATM. Di Martino, A., Yan, C.G., Li, Q., Denio, E., et al., 2014. The autism brain imaging data ex-
We hope the toolbox can provide more convenience and benefit for re- change: towards a large-scale evaluation of the intrinsic brain architecture in autism.
Mol. Psychiatry 19 (6), 659–667.
searchers to do domain adaptation research in medical data analysis. Erickson, B.J., Korfiatis, P., Akkus, Z., Kline, T.L., 2017. Machine learning for medical
There are several potential future works to further enrich and ex- imaging. Radiographics 37 (2), 505.
tend the DomainATM. First, for the sake of fast and easy facilitation of Fatima, M., Pasha, M., et al., 2017. Survey of machine learning algorithms for disease
diagnostic. Journal of Intelligent Learning Systems and Applications 9 (01), 1.
domain adaptation in medical imaging data, we only include machine Fernando, B., Habrard, A., Sebban, M., Tuytelaars, T., 2013. Unsupervised visual domain
learning methods in the current version, without considering deep learn- adaptation using subspace alignment. In: Proceedings of the IEEE International Con-
ing methods that often require large computation resources. In the fu- ference on Computer Vision, pp. 2960–2967.
Ghifary, M., Balduzzi, D., Kleijn, W.B., Zhang, M., 2016. Scatter component analysis: a uni-
ture, we plan to develop another version of the toolbox to include deep
fied framework for domain adaptation and domain generalization. IEEE Trans Pattern
learning methods (such as various GANs (Sinha et al., 2021; Yi et al., Anal Mach Intell 39 (7), 1414–1430.
2019) and CNNs (Guan et al., 2021a; Tibrewala et al., 2020)). Second, Gong, B., Shi, Y., Sha, F., Grauman, K., 2012. Geodesic flow kernel for unsupervised do-
main adaptation. In: 2012 IEEE Conference on Computer Vision and Pattern Recog-
the current evaluation metrics merely reflect domain differences, lack-
nition. IEEE, pp. 2066–2073.
ing the ability to further analyze practical applications (e.g., to what Guan, H., Liu, M., 2022. Domain adaptation for medical image analysis: asurvey. IEEE
extent Dice scores in a segmentation application varies before and after Trans. Biomed. Eng. 69 (3), 1173–1185.
domain adaptation). We will address this issue to enrich the toolbox in Guan, H., Liu, S., Lin, W., Yap, P.-T., Liu, M., 2022. Fast image-level MRI harmonization via
spectrum analysis. International Workshop on Machine Learning in Medical Imaging.
the future. Besides, we plan to further improve the graphic user inter- Springer.
face to enable users to set and tune the hyper-parameters of each domain Guan, H., Liu, Y., Yang, E., Yap, P.-T., Shen, D., Liu, M., 2021. Multi-site MRI harmoniza-
adaptation method in a more convenient manner. tion via attention-guided deep domain adaptation for brain disorder identification.
Med Image Anal 71, 102076.
Guan, H., Wang, L., Liu, M., 2021. Multi-source domain adaptation via optimal transport
Data Availability for brain dementia identification. In: 2021 IEEE 18th International Symposium on
Biomedical Imaging (ISBI). IEEE, pp. 1514–1517.
Jack Jr, C.R., Bernstein, M.A., Fox, N.C., et al., 2008. The Alzheimer’s Disease Neuroimag-
The ADNI dataset used in this study can be accessed via the ing Initiative (ADNI): MRI methods. J. Magn. Reson. Imaging 27 (4), 685–691.
following link http://www.ida.loni.usc.edu/login.jsp?project=ADNI& Kondrateva, E., Pominova, M., Popova, E., Sharaev, M., Bernstein, A., Burnaev, E., 2021.
page=HOME. An access application should be permitted firstly. Domain shift in computer vision models for MRI data analysis: An overview. In: Thir-
teenth International Conference on Machine Vision, Vol. 11605. SPIE, pp. 126–133.
The ABIDE dataset used in this study can be accessed via Kouw, W.M., Loog, M., 2019. A review of domain adaptation without target labels. IEEE
the following link http://www.fcon_1000.projects.nitrc.org/indi/ Trans Pattern Anal Mach Intell 43 (3), 766–785.
abide/abide_I.html. An access application should be permitted firstly. Kumagai, A., Iwata, T., 2019. Unsupervised domain adaptation by matching distributions
based on the maximum mean discrepancy via unilateral transformations. In: Proceed-
For review convenience, we included the processed datasets (in
ings of the AAAI Conference on Artificial Intelligence, Vol. 33, pp. 4106–4113.
terms of 2D features) in the submitted source code of this paper which Lee, H., Nakamura, K., Narayanan, S., Brown, R.A., Arnold, D.L., Initiative, A.D.N., et al.,
can be found in the data folder. 2019. Estimating and accounting for the effect of MRI scanner changes on longitudinal
whole-brain volume change measurements. Neuroimage 184, 555–565.
Long, M., Wang, J., Ding, G., Sun, J., Yu, P.S., 2013. Transfer feature learning with joint
Code Availability Statement distribution adaptation. In: Proceedings of the IEEE International Conference on Com-
puter Vision, pp. 2200–2207.
The software and code of the DomainATM toolbox as well as Long, M., Wang, J., Ding, G., Sun, J., Yu, P.S., 2014. Transfer joint matching for unsuper-
vised domain adaptation. In: Proceedings of the IEEE Conference on Computer Vision
the manual have been submitted at the time of paper submis- and Pattern Recognition, pp. 1410–1417.
sion. These materials can be also found at the following link: Van der Maaten, L., Hinton, G., 2008. Visualizing data using t-SNE. Journal of Machine
https://www.mingxia.web.unc.edu/domainatm/ Learning Research 9 (11).
Pan, S.J., Tsang, I.W., Kwok, J.T., Yang, Q., 2010. Domain adaptation via transfer com-
ponent analysis. IEEE Trans. Neural Networks 22 (2), 199–210.
Data availability Patel, V.M., Gopalan, R., Li, R., Chellappa, R., 2015. Visual domain adaptation: a survey
of recent advances. IEEE Signal Process Mag 32 (3), 53–69.
Data will be made available on request. Pooch, E.H., Ballester, P., Barros, R.C., 2020. Can we trust deep learning based diagno-
sis? The impact of domain shift in chest radiograph classification. In: International
Workshop on Thoracic Image Analysis. Springer, pp. 74–83.
Quiñonero-Candela, J., Sugiyama, M., Lawrence, N.D., Schwaighofer, A., 2009. Dataset
Credit authorship contribution statement Shift in Machine Learning. MIT Press.
Rajkomar, A., Dean, J., Kohane, I., 2019. Machine learning in medicine. N top N. Engl. J.
Med. 380 (14), 1347–1358.
Hao Guan: Conceptualization, Methodology, Software, Writing – Rubinov, M., Sporns, O., 2010. Complex network measures of brain connectivity: uses and
original draft. Mingxia Liu: Conceptualization, Validation, Writing – interpretations. Neuroimage 52 (3), 1059–1069.
Shi, Y., Sha, F., 2012. Information-theoretical learning of discriminative clusters for un-
review & editing, Supervision. supervised domain adaptation. In: Proceedings of the 29th International Conference
on International Conference on Machine Learning, pp. 1275–1282.
Acknowledgment Shinohara, R.T., et al., 2014. Statistical normalization techniques for magnetic resonance
imaging. NeuroImage: Clinical 6, 9–19.
Sinha, S., Thomopoulos, S.I., Lam, P., Muir, A., Thompson, P.M., 2021. Alzheimer’s disease
This work was supported in part by NIH grants (Nos. AG073297 and classification accuracy is improved by MRI harmonization based on attention-guided
AG041721). generative adversarial networks. In: 17th International Symposium on Medical Infor-
mation Processing and Analysis, Vol. 12088. SPIE, pp. 180–189.
Sun, B., Feng, J., Saenko, K., 2016. Return of frustratingly easy domain adaptation. In:
Supplementary material Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 30.
Tibrewala, R., Ozhinsky, E., Shah, R., Flament, I., Crossley, K., Srinivasan, R., Souza, R.,
Link, T.M., Pedoia, V., Majumdar, S., 2020. Computer-aided detection AI reduces in-
Supplementary material associated with this article can be found, in
terreader variability in grading hip abnormalities with MRI. J. Magn. Reson. Imaging
the online version, at doi:10.1016/j.neuroimage.2023.119863 52 (4), 1163–1172.
Torralba, A., Efros, A.A., 2011. Unbiased look at dataset bias. In: Proceedings of the
References IEEE Conference on Computer Vision and Pattern Recognition. IEEE, pp. 1521–
1528.
Barragán-Montero, et al., 2021. Artificial intelligence and machine learning for medical Tzourio-Mazoyer, N., et al., 2002. Automated anatomical labeling of activations in SPM
imaging: a technology review. Physica Med. 83, 242–256. using a macroscopic anatomical parcellation of the MNI MRI single-subject brain.
Ben-David, S., Blitzer, J., Crammer, K., Pereira, F., et al., 2007. Analysis of representations Neuroimage 15 (1), 273–289.
for domain adaptation. MIT; 1998, pp. 137–144. Valiant, L.G., 1984. A theory of the learnable. Commun ACM 27 (11), 1134–1142.
Csurka, G., 2017. A comprehensive survey on domain adaptation for visual applications. Valverde, J.M., Imani, V., Abdollahzadeh, A., De Feo, R., Prakash, M., Ciszek, R., Tohka, J.,
In: Domain Adaptation in Computer Vision Applications. Springer, pp. 1–35. 2021. Transfer learning in magnetic resonance brain imaging: a systematic review.
Deo, R.C., 2015. Machine learning in medicine. Circulation 132 (20), 1920–1930. Journal of Imaging 7 (4), 66.

11
H. Guan and M. Liu NeuroImage 268 (2023) 119863

Volkow, N.D., et al., 2018. The conception of the ABCD study: from substance use to a Yan, H., Ding, Y., Li, P., Wang, Q., Xu, Y., Zuo, W., 2017. Mind the class weight
broad NIH collaboration. Dev Cogn Neurosci 32, 4–7. bias: Weighted maximum mean discrepancy for unsupervised domain adaptation. In:
Wang, M., Deng, W., 2018. Deep visual domain adaptation: a survey. Neurocomputing Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition,
312, 135–153. pp. 2272–2281.
Wang, W., Li, H., Ding, Z., Nie, F., Chen, J., Dong, X., Wang, Z., 2021. Rethinking maxi- Yi, X., Walia, E., Babyn, P., 2019. Generative adversarial network in medical imaging: a
mum mean discrepancy for visual domain adaptation. IEEE Trans Neural Netw Learn review. Med Image Anal 58, 101552.
Syst. Zhang, J., Chao, H., Yan, P., 2020. Robustified domain adaptation. arXiv: 2011.09563
Wilson, G., Cook, D.J., 2020. A survey of unsupervised deep domain adaptation. ACM Zou, D., Zhu, Q., Yan, P., 2020. Unsupervised domain adaptation with dual-scheme fu-
Transactions on Intelligent Systems and Technology (TIST) 11 (5), 1–46. sion network for medical image segmentation. In: International Joint Conference on
Wittens, M.M.J., et al., 2021. Inter-and intra-scanner variability of automated brain vol- Artificial Intelligence, pp. 3291–3298.
umetry on three magnetic resonance imaging systems in Alzheimer’s disease and con-
trols. Front Aging Neurosci 13.

12