Auto-DeepLab:

Hierarchical Neural Architecture Search for Semantic Image Segmentation

Chenxi Liu1∗, Liang-Chieh Chen2 , Florian Schroff2 , Hartwig Adam2 , Wei Hua2 ,
Alan Yuille1 , Li Fei-Fei3
1
Johns Hopkins University 2 Google 3 Stanford University
arXiv:1901.02985v1 [cs.CV] 10 Jan 2019

Abstract Auto Search

Model Cell Network Dataset Days Task
Recently, Neural Architecture Search (NAS) has success- ResNet [25] 7 7 - - Cls
fully identified neural network architectures that exceed DenseNet [31] 7 7 - - Cls
DeepLabv3+ [11] 7 7 - - Seg
human designed ones on large-scale image classification
problems. In this paper, we study NAS for semantic image NASNet [92] 3 7 CIFAR-10 2000 Cls
AmoebaNet [61] 3 7 CIFAR-10 2000 Cls
segmentation, an important computer vision task that as- PNASNet [47] 3 7 CIFAR-10 150 Cls
signs a semantic label to every pixel in an image. Existing DARTS [49] 3 7 CIFAR-10 4 Cls
works often focus on searching the repeatable cell struc- DPC [6] 3 7 Cityscapes 2600 Seg
ture, while hand-designing the outer network structure that Auto-DeepLab 3 3 Cityscapes 3 Seg
controls the spatial resolution changes. This choice sim-
plifies the search space, but becomes increasingly problem- Table 1: Comparing our work against other CNN architec-
atic for dense image prediction which exhibits a lot more tures with two-level hierarchy. The main differences in-
network level architectural variations. Therefore, we pro- clude: (1) we directly search CNN architecture for semantic
pose to search the network level structure in addition to segmentation, (2) we search the network level architecture
the cell level structure, which forms a hierarchical archi- as well as the cell level one, and (3) our efficient search only
tecture search space. We present a network level search requires 3 P100 GPU days.
space that includes many popular designs, and develop a
formulation that allows efficient gradient-based architec-
ture search (3 P100 GPU days on Cityscapes images). We tizing AI, there has been significant interest in designing
demonstrate the effectiveness of the proposed method on the neural network architectures automatically, instead of rely-
challenging Cityscapes, PASCAL VOC 2012, and ADE20K ing heavily on expert experience and knowledge. Impor-
datasets. Without any ImageNet pretraining, our architec- tantly, in the past year, Neural Architecture Search (NAS)
ture searched specifically for semantic image segmentation has successfully identified architectures that exceed human-
attains state-of-the-art performance. designed architectures on large-scale image classification
problems [92, 47, 61].
1. Introduction Image classification is a good starting point for NAS,
because it is the most fundamental and well-studied high-
Deep neural networks have been proved successful
level recognition task. In addition, there exists benchmark
across a large variety of artificial intelligence tasks, includ-
datasets (e.g., CIFAR-10) with relatively small images, re-
ing image recognition [38, 25], speech recognition [27],
sulting in less computation and faster training. However,
machine translation [72, 80] etc. While better optimiz-
image classification should not be the end point for NAS,
ers [36] and better normalization techniques [32, 79] cer-
and the current success shows promise to extend into more
tainly played an important role, a lot of the progress comes
demanding domains. In this paper, we study Neural Archi-
from the design of neural network architectures. In com-
tecture Search for semantic image segmentation, an impor-
puter vision, this holds true for both image classification
tant computer vision task that assigns a label like “person”
[38, 71, 74, 75, 73, 25, 84, 31, 30] and dense image predic-
or “bicycle” to each pixel in the input image.
tion [16, 51, 7, 63, 56, 55].
Naively porting ideas from image classification would
More recently, in the spirit of AutoML and democra-
not suffice for semantic segmentation. In image classifica-
∗ Work done while an intern at Google. tion, NAS typically applies transfer learning from low res-

1
olution images to high resolution images [92], whereas op- ADE20K, our best model outperforms several state-of-the-
timal architectures for semantic segmentation must inher- art models [89, 44, 81, 87, 82] while using strictly less data
ently operate on high resolution imagery. This suggests the for pretraining.
need for: (1) a more relaxed and general search space to To summarize, the contribution of our paper is four-fold:
capture the architectural variations brought by the higher
• Ours is one of the first attempts to extend NAS beyond
resolution, and (2) a more efficient architecture search tech-
image classification to dense image prediction.
nique as higher resolution requires heavier computation.
We notice that modern CNN designs [25, 84, 31] usu- • We propose a network level architecture search space
ally follow a two-level hierarchy, where the outer network that augments and complements the much-studied cell
level controls the spatial resolution changes, and the inner level one, and consider the more challenging joint
cell level governs the specific layer-wise computations. The search of network level and cell level architectures.
vast majority of current works on NAS [92, 47, 61, 59, 49] • We develop a differentiable, continuous formulation
follow this two-level hierarchical design, but only automat- that conducts the two-level hierarchical architecture
ically search the inner cell level while hand-designing the search efficiently in 3 GPU days.
outer network level. This limited search space becomes
problematic for dense image prediction, which is sensitive • Without ImageNet pretraining, our model significantly
to the spatial resolution changes. Therefore in our work, outperforms FRRN-B and GridNet, and attains com-
we propose a trellis-like network level search space that parable performance with other ImageNet-pretrained
augments the commonly-used cell level search space first state-of-the-art models on Cityscapes. On PASCAL
proposed in [92] to form a hierarchical architecture search VOC 2012 and ADE20K, our best model also outper-
space. Our goal is to jointly learn a good combination of forms several state-of-the-art models.
repeatable cell structure and network structure specifically
for semantic image segmentation. 2. Related Work
In terms of the architecture search method, reinforce- Semantic Image Segmentation Convolutional neural
ment learning [91, 92] and evolutionary algorithms [62, 61] networks [42] deployed in a fully convolutional manner
tend to be computationally intensive even on the low resolu- (FCNs [67, 51]) have achieved remarkable performance
tion CIFAR-10 dataset, therefore probably not suitable for on several semantic segmentation benchmarks. Within the
semantic image segmentation. We draw inspiration from state-of-the-art systems, there are two essential compo-
the differentiable formulation of NAS [68, 49], and de- nents: multi-scale context module and neural network de-
velop a continuous relaxation of the discrete architectures sign. It has been known that context information is cru-
that exactly matches the hierarchical architecture search cial for pixel labeling tasks [26, 69, 37, 39, 16, 54, 14, 10].
space. The hierarchical architecture search is conducted via Therefore, PSPNet [87] performs spatial pyramid pooling
stochastic gradient descent. When the search terminates, [21, 41, 24] at several grid scales (including image-level
the best cell architecture is decoded greedily, and the best pooling [50]), while DeepLab [8, 9] applies several par-
network architecture is decoded efficiently using the Viterbi allel atrous convolution [28, 20, 67, 57, 7] with different
algorithm. We directly search architecture on 321×321 im- rates. On the other hand, the improvement of neural net-
age crops from Cityscapes [13]. The search is very efficient work design has significantly driven the performance from
and only takes about 3 days on one P100 GPU. AlexNet [38], VGG [71], Inception [32, 75, 73], ResNet
We report experimental results on multiple semantic seg- [25] to more recent architectures, such as Wide ResNet [85],
mentation benchmarks, including Cityscapes [13], PAS- ResNeXt [84], DenseNet [31] and Xception [12]. In addi-
CAL VOC 2012 [15], and ADE20K [89]. Without Ima- tion to adopting those networks as backbones for semantic
geNet [64] pretraining, our best model significantly out- segmentation, one could employ the encoder-decoder struc-
performs FRRN-B [60] by 8.6% and GridNet [17] by tures [63, 2, 55, 44, 60, 58, 33, 78, 18, 11, 86, 82] which ef-
10.9% on Cityscapes test set, and performs comparably ficiently captures the long-range context information while
with other ImageNet-pretrained state-of-the-art models [81, keeping the detailed object boundaries. Nevertheless, most
87, 4, 11, 6] when also exploiting the coarse annotations of the models require initialization from the ImageNet [64]
on Cityscapes. Notably, our best model (without pretrain- pretrained checkpoints except FRRN [60] and GridNet [17]
ing) attains the same performance as DeepLabv3+ [11] for the task of semantic segmentation. Specifically, FRRN
(with pretraining) while being 2.23 times faster in Multi- [60] employs a two-stream system, where full-resolution in-
Adds. Additionally, our light-weight model attains the per- formation is carried in one stream and context information
formance only 1.2% lower than DeepLabv3+ [11], while in the other pooling stream. GridNet, building on top of a
requiring 76.7% fewer parameters and being 4.65 times similar idea, contains multiple streams with different reso-
faster in Multi-Adds. Finally, on PASCAL VOC 2012 and lutions. In this work, we apply neural architecture search
for network backbones specific for semantic segmentation. mid Pooling (ASPP) module using random search, whereas
We further show state-of-the-art performance without Im- we focus on searching the much more fundamental network
ageNet pretraining, and significantly outperforms FRRN backbone architecture using more advanced and more effi-
[60] and GridNet [17] on Cityscapes [13]. cient search methods.

Neural Architecture Search Method Neural Architec- 3. Architecture Search Space

ture Search aims at automatically designing neural network
architectures, hence minimizing human hours and efforts. This section describes our two-level hierarchical archi-
While some works [22, 34, 91, 49] search RNN cells for tecture search space. For the inner cell level (Sec. 3.1), we
language tasks, more works search good CNN architectures reuse the one adopted in [92, 47, 61, 49] to keep consistent
for image classification. with previous works. For the outer network level (Sec. 3.2),
Several papers used reinforcement learning (either pol- we propose a novel search space based on observation and
icy gradients [91, 92, 5, 76] or Q-learning [3, 88]) to train summarization of many popular designs.
a recurrent neural network that represents a policy to gen-
erate a sequence of symbols specifying the CNN architec- 3.1. Cell Level Search Space
ture. An alternative to RL is to use evolutionary algorithms We define a cell to be a small fully convolutional module,
(EA), that “evolves” architectures by mutating the best ar- typically repeated multiple times to form the entire neural
chitectures found so far [62, 83, 53, 48, 61]. However, these network. More specifically, a cell is a directed acyclic graph
RL and EA methods tend to require massive computation consisting of B blocks.
during the search, usually thousands of GPU days. PNAS Each block is a two-branch structure, mapping from 2
[47] proposed a progressive search strategy that markedly input tensors to 1 output tensor. Block i in cell l may be
reduced the search cost while maintaining the quality of the specified using a 5-tuple (I1 , I2 , O1 , O2 , C), where I1 , I2 ∈
searched architecture. NAO [52] embedded architectures Iil are selections of input tensors, O1 , O2 ∈ O are selections
into a latent space and performed optimization before de- of layer types applied to the corresponding input tensor, and
coding. Additionally, several works [59, 49, 1] utilized ar- C ∈ C is the method used to combine the individual outputs
chitectural sharing among sampled models instead of train- of the two branches to form this block’s output tensor, Hil .
ing each of them individually, thereby further reduced the The cell’s output tensor H l is simply the concatenation of
search cost. Our work follows the differentiable NAS for- the blocks’ output tensors {H1l , . . . , HB l
}.
mulation [68, 49] and extends it into the more general hier- l
The set of possible input tensors, Ii , consists of the out-
archical setting. put of the previous cell H l−1 , the output of the previous-
previous cell H l−2 , and previous blocks’ output in the cur-
Neural Architecture Search Space Earlier papers, e.g., rent cell {H1l , . . . , Hil }. Therefore, as we add more blocks
[91, 62], tried to directly construct the entire network. How- in the cell, the next block has more choices as potential
ever, more recent papers [92, 47, 61, 59, 49] have shifted to source of input.
searching the repeatable cell structure, while keeping the The set of possible layer types, O, consists of the follow-
outer network level structure fixed by hand. First proposed ing 8 operators, all prevalent in modern CNNs:
in [92], this strategy is likely inspired by the two-level hier-
archy commonly used in modern CNNs. • 3 × 3 depthwise-separable conv • 3 × 3 average pooling
• 5 × 5 depthwise-separable conv • 3 × 3 max pooling
Our work still uses this cell level search space to keep
• 3 × 3 atrous conv with rate 2 • skip connection
consistent with previous works. Yet one of our contributions
• 5 × 5 atrous conv with rate 2 • no connection (zero)
is to propose a new, general-purpose network level search
space, since we wish to jointly search across this two-level For the set of possible combination operators C, we sim-
hierarchy. Our network level search space shares a similar ply let element-wise addition to be the only choice.
outlook as [66], but the important difference is that [66] kept
3.2. Network Level Search Space
the entire “fabrics” with no intention to alter the architec-
ture, whereas we associate an explicit weight for each con- In the image classification NAS framework pioneered by
nection and focus on decoding a single discrete structure. [92], once a cell structure is found, the entire network is
In addition, [66] was evaluated on segmenting face images constructed using a pre-defined pattern. Therefore the net-
into 3 classes [35], whereas our models are evaluated on work level was not part of the architecture search, hence its
large-scale segmentation datasets such as Cityscapes [13], search space has never been proposed nor designed.
PASCAL VOC 2012 [15], and ADE20K [89]. This pre-defined pattern is simple and straightforward: a
The most similar work to ours is [6], which also studied number of “normal cells” (cells that keep the spatial reso-
NAS for semantic image segmentation. However, [6] fo- lution of the feature tensor) are separated equally by insert-
cused on searching the much smaller Atrous Spatial Pyra- ing “reduction cells” (cells that divide the spatial resolution
Downsample\Layer 1 2 3 4 5 …… L-1 L ... co
s
H1l
nc
at
1

s
2 H2l

PP
AS

concat
4 s
Hl-2 ... s
Hl-1 s
H3l s
Hl

PP
AS
8
s
H4l

PP
AS
16

co
s
H5l

n
PP
AS

ca
32

t
...

Figure 1: Left: Our network level search space with L = 12. Gray nodes represent the fixed “stem” layers, and a path along
the blue nodes represents a candidate network level architecture. Right: During the search, each cell is a densely connected
structure as described in Sec. 4.1.1. Every yellow arrow is associated with the set of values αj→i . The three arrows after
concat are associated with β ls →s , βs→s
l l
, β2s→s respectively, as described in Sec. 4.1.2. Best viewed in color.
2

……
Downsample\Layer 1 2 3 4 5 L-1 L more network level variations [9, 56, 55].
1
Among the various network architectures for dense im-
2 age prediction, we notice two principles that are consistent:
PP
AS

4
• The spatial resolution of the next layer is either twice
PP
AS

8
as large, or twice as small, or remains the same.
PP
AS

16
PP
AS

32
• The smallest spatial resolution is downsampled by 32.
(a) Network level architecture used in DeepLabv3 [9].
Following these common practices, we propose the follow-
Downsample\Layer 1 2 3 4 5 …… L-1 L
ing network level search space. The beginning of the net-
1
work is a two-layer “stem” structure that each reduces the
2
spatial resolution by a factor of 2. After that, there are a
4 total of L layers with unknown spatial resolutions, with the
8 maximum being downsampled by 4 and the minimum being
16
downsampled by 32. Since each layer may differ in spatial
resolution by at most 2, the first layer after the stem could
32
only be either downsampled by 4 or 8. We illustrate our net-
(b) Network level architecture used in Conv-Deconv [56]. work level search space in Fig. 1. Our goal is then to find a
Downsample\Layer 1 2 3 4 5 …… L-1 L
good path in this L-layer trellis.
1 In Fig. 2 we show that our search space is general enough
2
to cover many popular designs. In the future, we have plans
to relax this search space even further to include U-net ar-
4
chitectures [63, 45, 70], where layer l may receive input
8
from one more layer preceding l in addition to l − 1.
16
We reiterate that our work searches the network level ar-
32 chitecture in addition to the cell level architecture. There-
fore our search space is strictly more challenging and
(c) Network level architecture used in Stacked Hourglass [55]. general-purpose than previous works.
Figure 2: Our network level search space is general and
includes various existing designs. 4. Methods
We begin by introducing a continuous relaxation of
the (exponentially many) discrete architectures that ex-
by 2 and multiply the number of filters by 2). This keep- actly matches the hierarchical architecture search described
downsampling strategy is reasonable in the image classifi- above. We then discuss how to perform architecture search
cation case, but in dense image prediction it is also impor- via optimization, and how to decode back a discrete archi-
tant to keep high spatial resolution, and as a result there are tecture after the search terminates.
4.1. Continuous Relaxation of Architectures also implemented as softmax.
Eq. (6) shows how the continuous relaxations of the two-
4.1.1 Cell Architecture
level hierarchy are weaved together. In particular, β con-
We reuse the continuous relaxation described in [49]. Every trols the outer network level, hence depends on the spatial
block’s output tensor Hil is connected to all hidden states in size and layer index. Each scalar in β governs an entire set
Iil : X of α, yet α specifies the same architecture that depends on
Hil = Oj→i (Hjl ) (1) neither spatial size nor layer index.
Hjl ∈Iil As illustrated in Fig. 1, Atrous Spatial Pyramid Pooling
(ASPP) modules are attached to each spatial resolution at
In addition, we approximate each Oj→i with its continuous the L-th layer (atrous rates are adjusted accordingly). Their
relaxation Ōj→i , defined as: outputs are bilinear upsampled to the original resolution be-
X fore summed to produce the prediction.
Ōj→i (Hjl ) = k
αj→i Ok (Hjl ) (2)
O k ∈O 4.2. Optimization
where The advantage of introducing this continuous relaxation
|O| is that the scalars controlling the connection strength be-
X
k
αj→i =1 ∀i, j (3) tween different hidden states are now part of the differen-
k=1
tiable computation graph. Therefore they can be optimized
k efficiently using gradient descent. We adopt the first-order
αj→i ≥0 ∀i, j, k (4)
approximation in [49], and partition the training data into
k two disjoint sets trainA and trainB. The optimization alter-
In other words, αj→i are normalized scalars associated with
k nates between:
each operator O ∈ O, easily implemented as softmax.
Recall from Sec. 3.1 that H l−1 and H l−2 are al- 1. Update network weights w by ∇w LtrainA (w, α, β)
ways included in Iil , and that H l is the concatenation of
{H1l , . . . , HB
l
}. Together with Eq. (1) and Eq. (2), the cell 2. Update architecture α, β by ∇α,β LtrainB (w, α, β)
level update may be summarized as:
where the loss function L is the cross entropy calculated on
H l = Cell(H l−1 , H l−2 ; α) (5) the semantic segmentation mini-batch.

4.1.2 Network Architecture 4.3. Decoding Discrete Architectures

Within a cell, all tensors are of the same spatial size, which Cell Architecture Following [49], we decode the dis-
enables the (weighted) sum in Eq. (1) and Eq. (2). How- crete cell architecture by first retaining the 2 strongest pre-
ever, as clearly illustrated in Fig. 1, tensors may take dif- decessors for each block (with the strength from hidden
k
ferent sizes in the network level. Therefore in order to set state j to hidden state i being maxk,Ok 6=zero αj→i ; recall
up the continuous relaxation, each layer l will have at most from Sec. 3.1 that “zero” means “no connection”), and then
4 hidden states {4 H l , 8 H l , 16 H l , 32 H l }, with the upper left choose the most likely operator by taking the argmax.
superscript indicating the spatial resolution. Network Architecture Eq. (7) essentially states that the
We design the network level continuous relaxation to ex- “outgoing probability” at each of the blue nodes in Fig. 1
actly match the search space described in Sec. 3.2. We as- sums to 1. In fact, the β values can be interpreted as
sociated a scalar with each gray arrow in Fig. 1, and the the “transition probability” between different “states” (spa-
network level update is: tial resolution) across different “time steps” (layer number).
s s Quite intuitively, our goal is to find the path with the “max-
H l =β l2s →s Cell( 2 H l−1 , s H l−2 ; α)
imum probability” from start to end. This path can be de-
l
+ βs→s Cell(s H l−1 , s H l−2 ; α) coded efficiently using the classic Viterbi algorithm, as in
l
+ β2s→s Cell(2s H l−1 , s H l−2 ; α) (6) our implementation.

where s = 4, 8, 16, 32 and l = 1, 2, . . . , L. The scalars β 5. Experimental Results

are normalized such that
Herein, we report our architecture search implementa-
l l l tion details as well as the search results. We then report
βs→ s
2
+ βs→s + βs→2s =1 ∀s, l (7)
l l l semantic segmentation results on benchmark datasets with
βs→ s
2
≥0 βs→s ≥0 βs→2s ≥0 ∀s, l (8) our best found architecture.
Downsample\Layer 1 2 3 4 5 …… L-1 L atr
5x5

1 +
sep
3x3 sep
5x5
2
+
sep
sep 5x5

PP
AS
4 3x3

concat
l-2 l-1
H ... H + Hl
sep
atr

PP
AS
8 3x3
5x5
+
sep

PP
AS
16 atr
5x5
3x3
+

PP
AS
32 sep
3x3

Figure 3: Best network and cell architecture found by our Hierarchical Neural Architecture Search. Gray dashed arrows show
the connection with maximum β at each node. atr: atrous convolution. sep: depthwise-separable convolution.

5.1. Architecture Search Implementation Details

We consider a total of L = 12 layers in the network, and
B = 5 blocks in a cell. The network level search space has
2.9 × 104 unique paths, and the number of cell structures
is 5.6 × 1014 . So the size of the joint, hierarchical search
space is in the order of 1019 .
We follow the common practice of doubling the number
of filters when halving the height and width of feature ten-
sor. Every blue node in Fig. 1 with downsample rate s has
B × F × s output filters, where F is the filter multiplier
controlling the model capacity. We set F = 8 during the Figure 4: Validation accuracy during 40 epochs of architec-
architecture search. A stride 2 convolution is used for all ture search optimization across 10 random trials.
s
2 → s connections, both to reduce spatial size and dou-
ble the number of filters. Bilinear upsampling followed by
1 × 1 convolution is used for all 2s → s connections, both α, β after 20 epochs. The entire architecture search opti-
to increase spatial size and halve the number of filters. mization takes about 3 days on one P100 GPU. Fig. 4 shows
The Atrous Spatial Pyramid Pooling module used in [9] that the validation accuracy steadily improves throughout
has 5 branches: one 1 × 1 convolution, three 3 × 3 convo- this process. We also tried searching for longer epochs (60,
lution with various atrous rates, and pooled image feature. 80, 100), but did not observe benefit.
During the search, we simplify ASPP to have 3 branches Fig. 3 visualizes the best architecture found. In terms of
instead of 5 by only using one 3 × 3 convolution with atrous network level architecture, higher resolution is preferred at
rate 96s . The number of filters produced by each ASPP
both beginning (stays at downsample by 4 for longer) and
branch is still B × F × s. end (ends at downsample by 8). We also show the strongest
We conduct architecture search on the Cityscapes dataset outgoing connection at each node using gray dashed arrows.
[13] for semantic image segmentation. More specifically, We observe a general tendency to downsample in the first
we use 321 × 321 random image crops from half-resolution 3/4 layers and upsample in the last 1/4 layers. In terms of
(512 × 1024) images in the train fine set. We randomly cell level architecture, the conjunction of atrous convolution
select half of the images in train fine as trainA, and the other and depthwise-separable convolution is often used, suggest-
half as trainB (see Sec. 4.2). ing that the importance of context has been learned. Note
The architecture search optimization is conducted for a that atrous convolution is rarely found to be useful in cells
total of 40 epochs. The batch size is 2 due to GPU memory for image classification1 .
constraint. When learning network weights w, we use SGD 5.2. Semantic Segmentation Results
optimizer with momentum 0.9, cosine learning rate that de-
cays from 0.025 to 0.001, and weight decay 0.0003. When We evaluate the performance of our found best archi-
learning the architecture encoding α, β, we use Adam opti- tecture on Cityscapes [13], PASCAL VOC 2012 [15], and
mizer [36] with learning rate 0.003 and weight decay 0.001. ADE20K [89] datasets.
We empirically found that if α, β are optimized from the be- 1 Among NASNet-{A, B, C}, PNASNet-{1, 2, 3, 4, 5}, AmoebaNet-
ginning when w are not well trained, the architecture tends {A, B, C}, ENAS, DARTS, atrous convolution was used only once in
to fall into bad local optima. Therefore we start optimizing AmoebaNet-B reduction cell.
 Method ImageNet F Multi-Adds Params mIOU (%) found best network architecture augmented with the ASPP
Auto-DeepLab-S 20 333.25B 10.15M 79.74 module [8, 9], and our decoder is the same as the one in
Auto-DeepLab-M 32 460.93B 21.62M 80.04
Auto-DeepLab-L 48 695.03B 44.42M 80.33
DeepLabv3+ which recovers the boundary information by
exploiting the low-level features that have downsample rate
FRRN-A [60] - - 17.76M 65.7
FRRN-B [60] - - 24.78M - 4. Additionally, we redesign the “stem” structure with three
DeepLabv3+ [11] 3 - 1551.05B 43.48M 79.55 3 × 3 convolutions (with stride 2 in the first and third con-
volutions). The first two convolutions have 64 filters while
Table 2: Cityscapes validation set results with different the third convolution has 128 filters. This “stem” has been
Auto-DeepLab model variants. F : the filter multiplier con- shown to be effective for segmentation in [87, 77].
trolling the model capacity. All our models are trained from
scratch and with single-scale input during inference.
5.2.1 Cityscapes
Method itr-500K itr-1M itr-1.5M SDP mIOU (%) Cityscapes [13] contains high quality pixel-level annota-
Auto-DeepLab-S 3 75.20 tions of 5000 images with size 1024 × 2048 (2975, 500,
Auto-DeepLab-S 3 77.09
Auto-DeepLab-S 3 78.00
and 1525 for the training, validation, and test sets respec-
Auto-DeepLab-S 3 3 79.74 tively) and about 20000 coarsely annotated training images.
Following the evaluation protocol [13], 19 semantic labels
Table 3: Cityscapes validation set results. We experi- are used for evaluation without considering the void label.
ment with the effect of adopting different training iterations In Tab. 2, we report the Cityscapes validation set results.
(500K, 1M, and 1.5M iterations) and the Scheduled Drop Similar to MobileNets [29, 65], we adjust the model capac-
Path method (SDP). All models are trained from scratch. ity by changing the filter multiplier F . As shown in the
table, higher model capacity leads to better performance at
Method ImageNet Coarse mIOU (%) the cost of slower speed (indicated by larger Multi-Adds).
In Tab. 3, we show that increasing the training iterations
FRRN-A [60] 63.0
from 500K to 1.5M iterations improves the performance
GridNet [17] 69.5
FRRN-B [60] 71.8 by 2.8%, when employing our light-weight model vari-
ant, Auto-DeepLab-S. Additionally, adopting the Scheduled
Auto-DeepLab-S 79.9 Drop Path [40, 92] further improves the performance by
Auto-DeepLab-L 80.4
1.74%, reaching 79.74% on Cityscapes validation set.
Auto-DeepLab-S 3 80.9 We then report the test set results in Tab. 4. Without
Auto-DeepLab-L 3 82.1 any pretraining, our best model (Auto-DeepLab-L) signif-
ResNet-38 [81] 3 3 80.6 icantly outperforms FRNN-B [60] by 8.6% and GridNet
PSPNet [87] 3 3 81.2 [17] by 10.9%. With extra coarse annotations, our model
Mapillary [4] 3 3 82.0 Auto-DeepLab-L, without pretraining on ImageNet [64],
DeepLabv3+ [11] 3 3 82.1 achieves the test set performance of 82.1%, outperform-
DPC [6] 3 3 82.7
ing PSPNet [87] and Mapillary [4], and attains the same
DRN CRL Coarse [90] 3 3 82.8
performance as DeepLabv3+ [11] while requiring 55.2%
fewer Mutli-Adds computations. Notably, our light-weight
Table 4: Cityscapes test set results with multi-scale inputs
model variant, Auto-DeepLab-S, attains 80.9% on the test
during inference. ImageNet: Models pretrained on Ima-
set, comparable to PSPNet, while using merely 10.15M pa-
geNet. Coarse: Models exploit coarse annotations.
rameters and 333.25B Multi-Adds.

We follow the same training protocol in [9, 11]. In brief, 5.2.2 PASCAL VOC 2012
during training we adopt a polynomial learning rate sched-
ule [50] with initial learning rate 0.05, and large crop size PASCAL VOC 2012 [15] contains 20 foreground object
(e.g., 769 × 769 on Cityscapes, and 513 × 513 on PAS- classes and one background class. We augment the original
CAL VOC 2012 and resized ADE20K images). Batch nor- dataset with the extra annotations provided by [23], result-
malization parameters [32] are fine-tuned during training. ing in 10582 (train aug) training images.
The models are trained from scratch with 1.5M iterations In Tab. 5, we report our validation set results. Our best
on Cityscapes, 1.5M iterations on PASCAL VOC 2012, and model, Auto-DeepLab-L, with single scale inference sig-
4M iterations on ADE20K, respectively. nificantly outperforms [19] by 20.36%. Additionally, for
We adopt the simple encoder-decoder structure similar to all our model variants, adopting multi-scale inference im-
DeepLabv3+ [11]. Specifically, our encoder consists of our proves the performance by about 1%. Further pretraining
 Method MS COCO mIOU (%) Method ImageNet COCO mIOU (%)
DropBlock [19] 53.4 Auto-DeepLab-S 3 82.5
Auto-DeepLab-M 3 84.1
Auto-DeepLab-S 71.68 Auto-DeepLab-L 3 85.6
Auto-DeepLab-S 3 72.54
RefineNet [44] 3 3 84.2
Auto-DeepLab-M 72.78 ResNet-38 [81] 3 3 84.9
Auto-DeepLab-M 3 73.69 PSPNet [87] 3 3 85.4
Auto-DeepLab-L 73.76 DeepLabv3+ [11] 3 3 87.8
Auto-DeepLab-L 3 75.26 MSCI [43] 3 3 88.0

Auto-DeepLab-S 3 78.31 Table 6: PASCAL VOC 2012 test set results. Our Auto-
Auto-DeepLab-S 3 3 80.27 DeepLab-L attains comparable performance with many
Auto-DeepLab-M 3 79.78 state-of-the-art models which are pretrained on both Ima-
Auto-DeepLab-M 3 3 80.73 geNet and COCO datasets. We refer readers to the official
leader-board for other state-of-the-art models.
Auto-DeepLab-L 3 80.75
Auto-DeepLab-L 3 3 82.04 Method ImageNet mIOU (%) Pixel-Acc (%) Avg (%)
Auto-DeepLab-S 40.69 80.60 60.65
Auto-DeepLab-M 42.19 81.09 61.64
Table 5: PASCAL VOC 2012 validation set results. We ex- Auto-DeepLab-L 43.98 81.72 62.85
periment with the effect of adopting multi-scale inference CascadeNet (VGG-16) [89] 3 34.90 74.52 54.71
(MS) and COCO-pretrained checkpoints (COCO). With- RefineNet (ResNet-152) [44] 3 40.70 - -
UPerNet (ResNet-101) [82] † 3 42.66 81.01 61.84
out any pretraining, our best model (Auto-DeepLab-L) out- PSPNet (ResNet-152) [87] 3 43.51 81.38 62.45
PSPNet (ResNet-269) [87] 3 44.94 81.69 63.32
performs DropBlock by 20.36%. All our models are not DeepLabv3+ (Xception-65) [11] † 3 45.65 82.52 64.09
pretrained with ImageNet images.
Table 7: ADE20K validation set results. We employ multi-
scale inputs during inference. †: Results are obtained from
our models on COCO [46] for 4M iterations improves the
their up-to-date model zoo websites respectively. Ima-
performance significantly.
geNet: Models pretrained on ImageNet. Avg: Average of
Finally, we report the PASCAL VOC 2012 test set re-
mIOU and Pixel-Accuracy.
sult with our COCO-pretrained model variants in Tab. 6.
As shown in the table, our best model attains the perfor-
mance of 85.6% on the test set, outperforming RefineNet tion to dense image prediction problems. Instead of fixating
[44] and PSPNet [87]. Our model is lagged behind the top- on the cell level, we acknowledge the importance of spa-
performing DeepLabv3+ [11] with Xception-65 as network tial resolution changes, and embrace the architectural vari-
backbone by 2.2%. We think that PASCAL VOC 2012 ations by incorporating the network level into the search
dataset is too small to train models from scratch and pre- space. We also develop a differentiable formulation that
training on ImageNet is still beneficial in this case. allows efficient (about 1000× faster than DPC [6]) archi-
tecture search over our two-level hierarchical search space.
5.2.3 ADE20K The result of the search, Auto-DeepLab, is evaluated by
training on benchmark semantic segmentation datasets from
ADE20K [89] has 150 semantic classes and high quality
scratch. On Cityscapes, Auto-DeepLab significantly out-
annotations of 20000 training images and 2000 validation
performs the previous state-of-the-art by 8.6%, and per-
images. In our experiments, the images are all resized so
forms comparably with ImageNet-pretrained top models
that the longer side is 513 during training.
when exploiting the coarse annotations. On PASCAL VOC
In Tab. 7, we report our validation set results. Our mod-
2012 and ADE20K, Auto-DeepLab also outperforms sev-
els outperform some state-of-the-art models, including Re-
eral ImageNet-pretrained state-of-the-art models.
fineNet [44], UPerNet [82], and PSPNet (ResNet-152) [87];
There are many possible directions for future work.
however, without any ImageNet [64] pretraining, our per-
Within the current framework, related applications such
formance is lagged behind the latest work of [11].
as object detection should be plausible; we could also try
6. Conclusion untying the cell architecture α across different layers (cf .
[76]) with little computation overhead. Beyond the current
In this paper, we present one of the first attempts to ex- framework, a more general and relaxed network level search
tend Neural Architecture Search beyond image classifica- space should be beneficial (cf . Sec. 3.2).
Figure 5: Visualization results on Cityscapes val set. Our failure mode is shown in the last row where our model confuses
with some difficult semantic classes such as person and rider.

Figure 6: Visualization results on ADE20K validation set. Our failure mode is shown in the last row where our model could
not segment very fine-grained objects (e.g., chair legs) and confuse with some difficult semantic classes (e.g., floor and rug).

Acknowledgments We thank Sergey Ioffe for the valu- [2] V. Badrinarayanan, A. Kendall, and R. Cipolla. Segnet: A
able feedback about the draft; Cloud AI and Mobile Vision deep convolutional encoder-decoder architecture for image
team for support. segmentation. arXiv:1511.00561, 2015. 2
[3] B. Baker, O. Gupta, N. Naik, and R. Raskar. Designing neu-
ral network architectures using reinforcement learning. In
References ICLR, 2017. 3
[4] S. R. Bulò, L. Porzi, and P. Kontschieder. In-place acti-
[1] K. Ahmed and L. Torresani. Maskconnect: Connectivity vated batchnorm for memory-optimized training of dnns. In
learning by gradient descent. In ECCV, 2018. 3 CVPR, 2018. 2, 7
 [5] H. Cai, T. Chen, W. Zhang, Y. Yu, and J. Wang. Efficient ar- [23] B. Hariharan, P. Arbeláez, L. Bourdev, S. Maji, and J. Malik.
chitecture search by network transformation. In AAAI, 2018. Semantic contours from inverse detectors. In ICCV, 2011. 7
3 [24] K. He, X. Zhang, S. Ren, and J. Sun. Spatial pyramid pooling
[6] L.-C. Chen, M. D. Collins, Y. Zhu, G. Papandreou, B. Zoph, in deep convolutional networks for visual recognition. In
F. Schroff, H. Adam, and J. Shlens. Searching for effi- ECCV, 2014. 2
cient multi-scale architectures for dense image prediction. In [25] K. He, X. Zhang, S. Ren, and J. Sun. Deep residual learning
NIPS, 2018. 1, 2, 3, 7, 8 for image recognition. In CVPR, 2016. 1, 2
[7] L.-C. Chen, G. Papandreou, I. Kokkinos, K. Murphy, and [26] X. He, R. S. Zemel, and M. Carreira-Perpindn. Multiscale
A. L. Yuille. Semantic image segmentation with deep con- conditional random fields for image labeling. In CVPR,
volutional nets and fully connected crfs. In ICLR, 2015. 1, 2004. 2
2 [27] G. Hinton, L. Deng, D. Yu, G. E. Dahl, A.-r. Mohamed,
[8] L.-C. Chen, G. Papandreou, I. Kokkinos, K. Murphy, and N. Jaitly, A. Senior, V. Vanhoucke, P. Nguyen, T. N. Sainath,
A. L. Yuille. Deeplab: Semantic image segmentation with et al. Deep neural networks for acoustic modeling in speech
deep convolutional nets, atrous convolution, and fully con- recognition: The shared views of four research groups. IEEE
nected crfs. TPAMI, 2017. 2, 7 Signal Processing Magazine, 29(6):82–97, 2012. 1
[9] L.-C. Chen, G. Papandreou, F. Schroff, and H. Adam. Re- [28] M. Holschneider, R. Kronland-Martinet, J. Morlet, and
thinking atrous convolution for semantic image segmenta- P. Tchamitchian. A real-time algorithm for signal analy-
tion. arXiv:1706.05587, 2017. 2, 4, 6, 7 sis with the help of the wavelet transform. In Wavelets:
[10] L.-C. Chen, Y. Yang, J. Wang, W. Xu, and A. L. Yuille. At- Time-Frequency Methods and Phase Space, pages 289–297.
tention to scale: Scale-aware semantic image segmentation. Springer, 1989. 2
In CVPR, 2016. 2 [29] A. G. Howard, M. Zhu, B. Chen, D. Kalenichenko, W. Wang,
[11] L.-C. Chen, Y. Zhu, G. Papandreou, F. Schroff, and H. Adam. T. Weyand, M. Andreetto, and H. Adam. Mobilenets: Effi-
Encoder-decoder with atrous separable convolution for se- cient convolutional neural networks for mobile vision appli-
mantic image segmentation. In ECCV, 2018. 1, 2, 7, 8 cations. arXiv:1704.04861, 2017. 7
[30] J. Hu, L. Shen, and G. Sun. Squeeze-and-excitation net-
[12] F. Chollet. Xception: Deep learning with depthwise separa-
works. In CVPR, 2018. 1
ble convolutions. In CVPR, 2017. 2
[31] G. Huang, Z. Liu, L. van der Maaten, and K. Q. Weinberger.
[13] M. Cordts, M. Omran, S. Ramos, T. Rehfeld, M. Enzweiler,
Densely connected convolutional networks. In CVPR, 2017.
R. Benenson, U. Franke, S. Roth, and B. Schiele. The
1, 2
cityscapes dataset for semantic urban scene understanding.
[32] S. Ioffe and C. Szegedy. Batch normalization: accelerating
In CVPR, 2016. 2, 3, 6, 7
deep network training by reducing internal covariate shift. In
[14] J. Dai, K. He, and J. Sun. Convolutional feature masking for
ICML, 2015. 1, 2, 7
joint object and stuff segmentation. In CVPR, 2015. 2
[33] M. A. Islam, M. Rochan, N. D. Bruce, and Y. Wang. Gated
[15] M. Everingham, S. M. A. Eslami, L. V. Gool, C. K. I. feedback refinement network for dense image labeling. In
Williams, J. Winn, and A. Zisserman. The pascal visual ob- CVPR, 2017. 2
ject classes challenge a retrospective. IJCV, 2014. 2, 3, 6,
[34] R. Jozefowicz, W. Zaremba, and I. Sutskever. An empiri-
7
cal exploration of recurrent network architectures. In ICML,
[16] C. Farabet, C. Couprie, L. Najman, and Y. LeCun. Learning 2015. 3
hierarchical features for scene labeling. PAMI, 2013. 1, 2 [35] A. Kae, K. Sohn, H. Lee, and E. Learned-Miller. Augment-
[17] D. Fourure, R. Emonet, E. Fromont, D. Muselet, ing crfs with boltzmann machine shape priors for image la-
A. Tremeau, and C. Wolf. Residual conv-deconv grid net- beling. In CVPR, 2013. 3
work for semantic segmentation. In BMVC, 2017. 2, 3, 7 [36] D. P. Kingma and J. Ba. Adam: A method for stochastic
[18] J. Fu, J. Liu, Y. Wang, and H. Lu. Stacked deconvolutional optimization. In ICLR, 2015. 1, 6
network for semantic segmentation. arXiv:1708.04943, [37] P. Kohli, P. H. Torr, et al. Robust higher order potentials for
2017. 2 enforcing label consistency. IJCV, 82(3):302–324, 2009. 2
[19] G. Ghiasi, T.-Y. Lin, and Q. V. Le. Dropblock: A regulariza- [38] A. Krizhevsky, I. Sutskever, and G. E. Hinton. Imagenet
tion method for convolutional networks. In NIPS, 2018. 7, classification with deep convolutional neural networks. In
8 NIPS, 2012. 1, 2
[20] A. Giusti, D. Ciresan, J. Masci, L. Gambardella, and [39] L. Ladicky, C. Russell, P. Kohli, and P. H. Torr. Associa-
J. Schmidhuber. Fast image scanning with deep max-pooling tive hierarchical crfs for object class image segmentation. In
convolutional neural networks. In ICIP, 2013. 2 ICCV, 2009. 2
[21] K. Grauman and T. Darrell. The pyramid match kernel: [40] G. Larsson, M. Maire, and G. Shakhnarovich. Fractalnet:
Discriminative classification with sets of image features. In Ultra-deep neural networks without residuals. In ICLR,
ICCV, 2005. 2 2017. 7
[22] K. Greff, R. K. Srivastava, J. Koutnı́k, B. R. Steunebrink, [41] S. Lazebnik, C. Schmid, and J. Ponce. Beyond bags of
and J. Schmidhuber. Lstm: A search space odyssey. features: Spatial pyramid matching for recognizing natural
arXiv:1503.04069, 2015. 3 scene categories. In CVPR, 2006. 2
[42] Y. LeCun, B. Boser, J. S. Denker, D. Henderson, R. E. [62] E. Real, S. Moore, A. Selle, S. Saxena, Y. L. Suematsu,
Howard, W. Hubbard, and L. D. Jackel. Backpropagation J. Tan, Q. Le, and A. Kurakin. Large-scale evolution of im-
applied to handwritten zip code recognition. Neural compu- age classifiers. In ICML, 2017. 2, 3
tation, 1(4):541–551, 1989. 2 [63] O. Ronneberger, P. Fischer, and T. Brox. U-net: Convolu-
[43] D. Lin, Y. Ji, D. Lischinski, D. Cohen-Or, and H. Huang. tional networks for biomedical image segmentation. In MIC-
Multi-scale context intertwining for semantic segmentation. CAI, 2015. 1, 2, 4
In ECCV, 2018. 8 [64] O. Russakovsky, J. Deng, H. Su, J. Krause, S. Satheesh,
[44] G. Lin, A. Milan, C. Shen, and I. Reid. Refinenet: Multi- S. Ma, Z. Huang, A. Karpathy, A. Khosla, M. Bernstein,
path refinement networks with identity mappings for high- A. C. Berg, and L. Fei-Fei. ImageNet Large Scale Visual
resolution semantic segmentation. In CVPR, 2017. 2, 8 Recognition Challenge. IJCV, 2015. 2, 7, 8
[45] T.-Y. Lin, P. Dollár, R. Girshick, K. He, B. Hariharan, and [65] M. Sandler, A. Howard, M. Zhu, A. Zhmoginov, and L.-C.
S. Belongie. Feature pyramid networks for object detection. Chen. Mobilenetv2: Inverted residuals and linear bottle-
In CVPR, 2017. 4 necks. In CVPR, 2018. 7
[46] T.-Y. Lin et al. Microsoft coco: Common objects in context. [66] S. Saxena and J. Verbeek. Convolutional neural fabrics. In
In ECCV, 2014. 8 NIPS, 2016. 3
[47] C. Liu, B. Zoph, M. Neumann, J. Shlens, W. Hua, L.-J. Li, [67] P. Sermanet, D. Eigen, X. Zhang, M. Mathieu, R. Fergus,
L. Fei-Fei, A. Yuille, J. Huang, and K. Murphy. Progressive and Y. LeCun. Overfeat: Integrated recognition, localization
neural architecture search. In ECCV, 2018. 1, 2, 3 and detection using convolutional networks. In ICLR, 2014.
[48] H. Liu, K. Simonyan, O. Vinyals, C. Fernando, and 2
K. Kavukcuoglu. Hierarchical representations for efficient [68] R. Shin, C. Packer, and D. Song. Differentiable neural net-
architecture search. In ICLR, 2018. 3 work architecture search. In ICLR Workshop, 2018. 2, 3
[49] H. Liu, K. Simonyan, and Y. Yang. Darts: Differentiable [69] J. Shotton, J. Winn, C. Rother, and A. Criminisi. Textonboost
architecture search. arXiv:1806.09055, 2018. 1, 2, 3, 5 for image understanding: Multi-class object recognition and
[50] W. Liu, A. Rabinovich, and A. C. Berg. Parsenet: Looking segmentation by jointly modeling texture, layout, and con-
wider to see better. arXiv:1506.04579, 2015. 2, 7 text. IJCV, 2009. 2
[51] J. Long, E. Shelhamer, and T. Darrell. Fully convolutional [70] A. Shrivastava, R. Sukthankar, J. Malik, and A. Gupta. Be-
networks for semantic segmentation. In CVPR, 2015. 1, 2 yond skip connections: Top-down modulation for object de-
[52] R. Luo, F. Tian, T. Qin, and T.-Y. Liu. Neural architecture tection. arXiv:1612.06851, 2016. 4
optimization. In NIPS, 2018. 3 [71] K. Simonyan and A. Zisserman. Very deep convolutional
[53] R. Miikkulainen, J. Liang, E. Meyerson, A. Rawal, networks for large-scale image recognition. In ICLR, 2015.
D. Fink, O. Francon, B. Raju, H. Shahrzad, A. Navruzyan, 1, 2
N. Duffy, and B. Hodjat. Evolving deep neural networks. [72] I. Sutskever, O. Vinyals, and Q. V. Le. Sequence to sequence
arXiv:1703.00548, 2017. 3 learning with neural networks. In NIPS, 2014. 1
[54] M. Mostajabi, P. Yadollahpour, and G. Shakhnarovich. Feed- [73] C. Szegedy, S. Ioffe, V. Vanhoucke, and A. A. Alemi.
forward semantic segmentation with zoom-out features. In Inception-v4, inception-resnet and the impact of residual
CVPR, 2015. 2 connections on learning. In AAAI, 2017. 1, 2
[55] A. Newell, K. Yang, and J. Deng. Stacked hourglass net- [74] C. Szegedy, W. Liu, Y. Jia, P. Sermanet, S. Reed,
works for human pose estimation. In ECCV, 2016. 1, 2, D. Anguelov, D. Erhan, V. Vanhoucke, and A. Rabinovich.
4 Going deeper with convolutions. In CVPR, 2015. 1
[56] H. Noh, S. Hong, and B. Han. Learning deconvolution net- [75] C. Szegedy, V. Vanhoucke, S. Ioffe, J. Shlens, and Z. Wojna.
work for semantic segmentation. In ICCV, 2015. 1, 4 Rethinking the inception architecture for computer vision. In
[57] G. Papandreou, I. Kokkinos, and P.-A. Savalle. Modeling CVPR, 2016. 1, 2
local and global deformations in deep learning: Epitomic [76] M. Tan, B. Chen, R. Pang, V. Vasudevan, and Q. V. Le.
convolution, multiple instance learning, and sliding window Mnasnet: Platform-aware neural architecture search for mo-
detection. In CVPR, 2015. 2 bile. arXiv:1807.11626, 2018. 3, 8
[58] C. Peng, X. Zhang, G. Yu, G. Luo, and J. Sun. Large kernel [77] P. Wang, P. Chen, Y. Yuan, D. Liu, Z. Huang, X. Hou, and
matters–improve semantic segmentation by global convolu- G. Cottrell. Understanding convolution for semantic seg-
tional network. In CVPR, 2017. 2 mentation. In WACV, 2018. 7
[59] H. Pham, M. Y. Guan, B. Zoph, Q. V. Le, and J. Dean. Ef- [78] Z. Wojna, V. Ferrari, S. Guadarrama, N. Silberman, L.-C.
ficient neural architecture search via parameter sharing. In Chen, A. Fathi, and J. Uijlings. The devil is in the decoder.
ICML, 2018. 2, 3 In BMVC, 2017. 2
[60] T. Pohlen, A. Hermans, M. Mathias, and B. Leibe. Full- [79] Y. Wu and K. He. Group normalization. In ECCV, 2018. 1
resolution residual networks for semantic segmentation in [80] Y. Wu, M. Schuster, Z. Chen, Q. V. Le, M. Norouzi,
street scenes. In CVPR, 2017. 2, 3, 7 W. Macherey, M. Krikun, Y. Cao, Q. Gao, K. Macherey,
[61] E. Real, A. Aggarwal, Y. Huang, and Q. V. Le. Reg- et al. Google’s neural machine translation system: Bridg-
ularized evolution for image classifier architecture search. ing the gap between human and machine translation.
arXiv:1802.01548, 2018. 1, 2, 3 arXiv:1609.08144, 2016. 1
[81] Z. Wu, C. Shen, and A. van den Hengel. Wider or
deeper: Revisiting the resnet model for visual recognition.
arXiv:1611.10080, 2016. 2, 7, 8
[82] T. Xiao, Y. Liu, B. Zhou, Y. Jiang, and J. Sun. Unified per-
ceptual parsing for scene understanding. In ECCV, 2018. 2,
8
[83] L. Xie and A. Yuille. Genetic cnn. In ICCV, 2017. 3
[84] S. Xie, R. Girshick, P. Dollr, Z. Tu, and K. He. Aggregated
residual transformations for deep neural networks. In CVPR,
2017. 1, 2
[85] S. Zagoruyko and N. Komodakis. Wide residual networks.
In BMVC, 2016. 2
[86] Z. Zhang, X. Zhang, C. Peng, D. Cheng, and J. Sun. Ex-
fuse: Enhancing feature fusion for semantic segmentation.
In ECCV, 2018. 2
[87] H. Zhao, J. Shi, X. Qi, X. Wang, and J. Jia. Pyramid scene
parsing network. In CVPR, 2017. 2, 7, 8
[88] Z. Zhong, J. Yan, W. Wu, J. Shao, and C.-L. Liu. Practical
block-wise neural network architecture generation. In CVPR,
2018. 3
[89] B. Zhou, H. Zhao, X. Puig, S. Fidler, A. Barriuso, and A. Tor-
ralba. Scene parsing through ade20k dataset. In CVPR, 2017.
2, 3, 6, 8
[90] Y. Zhuang, F. Yang, L. Tao, C. Ma, Z. Zhang, Y. Li, H. Jia,
X. Xie, and W. Gao. Dense relation network: Learning con-
sistent and context-aware representation for semantic image
segmentation. In ICIP, 2018. 7
[91] B. Zoph and Q. V. Le. Neural architecture search with rein-
forcement learning. In ICLR, 2017. 2, 3
[92] B. Zoph, V. Vasudevan, J. Shlens, and Q. V. Le. Learning
transferable architectures for scalable image recognition. In
CVPR, 2018. 1, 2, 3, 7