A Review of Research on Road Feature Extraction

through Remote Sensing Images Based on Deep

Learning Algorithms

Jyoti Pruthi Sunita Dhingra

2023 3rd International Conference on Innovative Sustainable Computational Technologies (CISCT) | 979-8-3503-0336-0/23/$31.00 ©2023 IEEE | DOI: 10.1109/CISCT57197.2023.10351299

Computer Science and Engineering Computer Science and Engineering

Maharishi Dayanand University, Rohtak, Haryana, India Maharishi Dayanand University, Rohtak, Haryana, India
[email protected] [email protected]

Abstract — Road feature extraction from remote sensing adopt the road extraction methods, many obstacles such as
images is one of the most challenging research which is mainly occlusion, shadow of objects, and complex backgrounds
performed in the spatial system. The road feature extraction become obstacles to achieving accuracy in images. This
may be utilized for intelligent transportation, smart cities, study area has been attracting the attention of most of the
autonomous driving, urban planning, and emergency researchers. Many of the methods related to road extraction
management. Many researchers have studied road feature
extraction by applying machine learning algorithms. However,
are based on deep learning algorithms. The study on road
no literature indicates a comprehensive survey pertaining to extraction by researchers have increased drastically since
the higher accuracy of the road feature extraction in the last 2020. This is shown in Fig 1 which recites a graph based on
many years. This paper is focused on comprehensive survey of the number of publications. The data in this figure is taken
the road feature extraction that uses algorithms of deep from science direct by using keywords as road extraction
learning for higher accuracy in the road feature extraction. from remote sensing images. Considering the amount of
This paper first indicates a tree structure that extends into four research that already has been done, a thorough analysis of
subtasks such as road centerline detection, road surface published work is necessary to identify difficulties and
segmentation, road edge detection and a combination thereof. forecast emerging trends.
Subsequently, methodological level classification is analyzed in
respect of all said four categories present in various kinds of
literature published in recent years. Further, this paper also
indicates the datasets and metrics which are used commonly.

Keywords— Remote sensing, Road extraction, Deep

learning, Road surface, Road centerline, Road edge.

I. INTRODUCTION
In spatial geographic information databases, man-made
objects such as buildings, bridges, roads, farmlands, etc. are
the most important elements. Apart from this, road
development plays a vital role for the socioeconomic
prosperity of urban area. The satellites launched for space
exploration are being used for earth observation for the
betterment of mankind. The United States propelled the first
earth resources observation satellite in 1972 which is further
renamed as landsat. The remote sensing imaging captures an
area's physical properties and surrounding ecosystem by the Fig 1 shows a number of publications in the years from 2012 to 2023.
satellite-mounted sensors to accurately record and analyze
global development. Nowadays, research focuses on the In this review paper, a systematic review of road extraction
extraction of road features with the help of remote sensing methodologies is used for sensing remote images based on
images. The road feature extraction can be utilized in a deep learning This review paper is divided into the
variety of societal contexts, including emergency geological following sections. Section 2 gives a brief introduction to
disaster response, automatic driving, intelligent road extraction techniques. Section 3 presents the extensive
transportation, traffic management, road network update, literature. The dataset that is suitable to use for this kind of
LULC mapping, Urban planning, and road health condition research is described in Section 4. Section 5 describes the
identification. The traditional methods used for getting the evaluation metrics, and the overall study is concluded in
required road information are either manual or GPS-based Section 6.
methods which are costly and laborious. Therefore, remote II. OVERVIEW
sensing images are used for automatic road extraction for
improving the economy and road extraction. In order to Road feature extraction are divided into three subtasks,
that are defined as a road centreline, road surface extraction

Authorized licensed use limited to: Jain University. Downloaded on February 12,2025 at 10:42:03 UTC from IEEE Xplore. Restrictions apply.
and road edge detection, as shown in Fig 2. The Fig. 2
illustrates the blue colour area, green colour area, and red
colour area. The blue colour area represents road surface
extraction, the green colour represents road edge detection,
and the red colour area represents road centerline. There are
few researchers who have worked on road extraction in one
of the said subtasks and other few researchers have worked
simultaneously on the remaining two or three subtasks out
of said total subtasks.

Fig. 2 illustrates different tasks of road extraction.

Road extraction techniques that are in use are categorized

into conventional and deep learning methods. Accordingly,
deep learning models are mainly drawn intention since there
are high-level distinct features associated with the deep
learning techniques. Figure 3 depicts the different deep
learning techniques which have been utilized by the
researchers in their respective literature for different kind of
Fig. 3 illustrates classification of road feature extraction.
road feature extraction.
III. LITERATURE SURVEY IV. DATASET
Mnih et al. is a first researcher who has introduced a In this section, the most commonly utilized and publicly
convolution neural network based road extraction from accessible data sets are briefly introduced. The said data set
aerial images. Afterwards, various researcher started has been utilized by different researchers for road feature
working on CNN based methods. For gathering image extraction from remote sensing images.
information at multiple levels, the Deep CNNs with end-to-
end training generate hierarchical features. Patch- based • The Massachusetts roads dataset
CNN implemented by a finite state machine for road pattern The above illustrated dataset contains 1171 images of 1m
recognition is proposed by Wang et al. [4]. After patch- resolution. This dataset is divided into three parts described
based CNN, the semantic segmentation network framework as: a first part includes 1108 training images, a second part
based on fully convolutional network (FCN) has been used includes 14 validation images, and a third part includes 49
for road extraction. The semantic segmentation networks are test images.
emerging exponentially in computer vision. The architecture
like Segnet, Unet are based on encoder-decoder architecture, • The DeepGlobe dataset
such as U-Net, SegNet. These architectures have achieved The above illustrated dataset is obtained from the
significant progress in recent years for semantic DeepGlobe 2018 road extraction challenge. In this dataset,
segmentation of images. Table 1 recites consolidated data of the images have 0.5m resolution with the size of 1024 ×
research works in deep learning for automatic extraction of 1024 pixels. There are 8570 images that further splits into
road feature that have been done by various researchers in 6626 images for training purpose, 1243 images for
the past few years. validation purpose, and 1101 images for testing purpose.

Authorized licensed use limited to: Jain University. Downloaded on February 12,2025 at 10:42:03 UTC from IEEE Xplore. Restrictions apply.
Table 1: Literature Survey and F1 of 94.64 on
S. Author Technique Dataset Findings Google Earth
No and year 10 Yan, J., CNN and Aerial IOU of 84.4 and F1
1 Guo, Q. One-class Deepglobe Completeness of et al. GNN based KITTI of 91.4 on aerial
et al. and the Massachuse 36.21 ,correctness of (2022) Bavaria KITTI, IOU of
(2020) random tts 54.00 on deepglobe, [10] Wuhan 83.13 and f1 of
[1] forest and 41.96 , 34.55 on Shaoxing 90.75 on bavaria and
classifier massachusetts. IOU of 76.34 anf F1
2 Chen, Z., Adaboost LRSNY, IoU of 0.8825 on of 86.55 on wuhan,
et based Shaoshan , LRSNY dataset, IOU of 79.3 anf f1
al.(2021) muliple Massachuse Shaoshan dataset of 88.45 on
[2] lightweight tts achieved IoU score shaoxing
Unet of 0.75 and 11 Yang, Z., TransRoad Deepglobe IOU of 70.06 and F1
Massachusetts Road et al. Net(Transf of 81.34 on
dataset achieved IoU (2022) ormer Deepglobe
score as high as [11] based
0.6477 CNN)
3 Wang, S., Inner Massachuse F1 of 84.6 on 12 Guan, H., RoadCapsF IOU of 92.3 and f1
et Convolutio tts Massachusetts et al. PN(CNN) Massachuse of 96.6 on
al.(2021) n Integrated (2022) tts massachusetts, IOU
[3] and [12] Google of 90.3 and f1 of
Conditional Earth 94.6 on google
Random earth.
Fields CNN
4 Zhang, Transforme Massachuse IoU of 65.36% on • LRSNY dataset
Z., et al. r based tts, Massachusetts The above illustrated dataset is a large road segmentation
(2022) CNN DeepGlobe dataset and 56.74%
[4] on DeepGlobe dataset. The said dataset has been taken from Optical
dataset Remote Sensing Images of New York. The said dataset is
5 Li, A et Semantic ISPRS OA of 92.80 % ,F1 publicly open. In this dataset, there are total 1368 images
al. (2022) Boundary Potsdam of 93.01% on with 0.5m resolution which further splits into 716 images
[5] awareness and the potsdam dataset and
network(SB Vaihingen OA of 90.59%, F1 for training purpose, 220 images for validation purpose, 432
ANet) of 90.34% on images for testing purpose These images have size 256 ×
Vaihingen dataset 256 pixels.
6 Yuan, G., Scribble The CHN6- F1 score of 0.833,
et al. annotation CUG , iou of 0.862 on
(2022) and Deepglobe , Massachusetts, f1 • Shaoshan dataset
[6] adversarial Massachuse score of 0.814, iou The above illustrated dataset includes 14,580 images for
learning tts, of 0.776 on training purpose. The said images are extracted with 0.5m
Ottawa deepglobe,f1 score resolution and having size 256 × 256. Further, the said
dataset of 0.813, iou of
0.850 on dataset contains 456 images for testing purpose. Due to
chn6 and f1 score of copyright protection, the said dataset is not publicly
0.937, iou 0.869 on available. This dataset has images which relates to
ottawa Shaoshan, China.
7 Jiang, X., Pyramidal Deepglobe The precision,
et al. deformable LRSNY recall, IoU, and F1-
(2022, vision Google score evaluation is • The ISPRS Potsdam Challenge dataset
Septembe transformer Earth Road 0.9644, 0.9558, The above illustrated dataset includes aerial images. The
r) [7] s based Extraction 0.9232, and 0.9601 said dataset includes 38 tiles of 6000 × 6000 pixels and also
CNN on GE- Road
dataset.For the includes 24 tiles for training purposes. Further, the dataset
LRSNY dataset,the includes 14 tiles with spatial 5cm resolutions (RGB channel)
values are 0.9466, for testing purposes.
0.9503, 0.9019, and
0.9484 and on
DeepGlobe dataset, • The Vaihingen Challenge dataset
the values are The above illustrated dataset includes aerial images. The
0.9343, 0.8936, said dataset includes 33 tile of 2100 × 2100 pixels along
0.8408, and 0.9135
was achieved.
with 9 cm spatial resolution (IR-RG channels).
8 Luo,J., et Bi-direction Deepglobe F1 score of 80.30
al. (2022) transformer Massachuse and IOU of 67.09 on • The CHN6-CUG road dataset
[8] network tts Deepglobe, F1 score The above illustrated dataset includes 3680 training images
based CNN Google of 79.55 and IOU of
Earth 66.04 on
and 903 test images along with 0.5 m per pixel spatial
Massachusetts, F1 resolution and having size 512 × 512. The said images
score of 93.67 and showcase cities of China, for example, Beijing, Wuhan,
IOU of 88.03 on Shenzhen and other areas taken from Google Earth.
Google Earth
9 Abdollah Recurrent Deepglobe IOU of 82.72 and F1
i, et residual Massachuse of 89.33 on • The Ottawa dataset
al.(2022) CNN tts massachusetts, IOU The above dataset includes 21 urban areas in Ottawa,
[9] ,boundary Google of 86.59 and F1 of Canada along with 0.21 m per pixel spatial resolution. The
learning Earth 92.78 on deeplglobe
and IOU of 89.85
said dataset includes data from 14 regions for testing

Authorized licensed use limited to: Jain University. Downloaded on February 12,2025 at 10:42:03 UTC from IEEE Xplore. Restrictions apply.
purposes, from 1 region for validation purposes and from 6 The definition of F1 − score is (4), it is harmonic mean of
regions for testing purposes. completeness and correctness.

• Google Earth road extraction dataset (GE-Road) F1-score = (2∗Completeness ∗ Correctness) /

(Completeness + Correctness) (4)
The above illustrated GE-Road dataset 20,000 images along
with a spatial resolution in a range from 0.3m to 0.6m and • TOPO metric
having size 800 × 800 pixels. The said images are taken
from Google earth engine. It measures the distance from several seed locations in
ground truth graph and estimated graph then compares it in
V. METRICS terms of precision and recall.

This section describes the metrics used by different • SP metric

researchers in the literature for the evaluation of road
extraction method. Some researchers have worked on 2D The SP metric randomly selects two points from the true and
image and used recall, precision, IOU while other working estimated road networks, and it then determines if the length
on 3D images used completeness, correctness metrics. There of the shortest trip between them is the same in either
are only few researchers now that are using APLS metrics network. This process needs to repeat using new random
for evaluation. points, and after the percentages of feasible, too short, too
Following is the metrics that are being used by different lengthy, and accurate paths have converged, they are
researchers in their research work. recorded. Rarely is the SP metric replicable.

• Completeness • CRR metric

The definition of completeness is mentioned in equation (1), This metric estimates without discontinuity the ratio of road
where true positive (TP) denotes correctly detected road segments.
segments, false positive(FP) denotes incorrect parts, and
false negative (FN) denotes unextracted fragments. • CONN metric
Completeness is the percentage of the reference network
This metric is used to assess the connectivity similarity
that is contained within the buffer around the extracted road.
between ground truth graph and estimated graph.
The concept of completeness is defined in equation (1),
where false negative (FN) represents unextracted pieces,
• APLS metric
true positive (TP) indicates correctly recognized road
segments, and false positive (FP) indicates incorrect This metric is used to measure the average path length
sections. The percentage of the reference network that is similarity between ground truth graph and estimated graph.
contained in the buffer surrounding the extracted route is The APLS finds the number of unique paths between two
known as completeness. It is used in road extraction from points in both the graph then calculate the length of the
3D image and metrics used in 2D image is recall which is shortest path between points. This way it calculates the
same as completeness. average of the shortest length between points in both the
graph, and compares it.
Completeness = TP / (TP + FN) (1)
VI. CONCLUSION
• Correctness
This paper presents a comprehensive review of deep
The definition of correctness is in equation (2), it represents
learning methodology used in road feature extraction. This
the percentage of extracted road that is contained within
review is categorized into four different groups of road
buffer around the reference network. It is used in road
feature extraction, for example, road edges, road centerline,
extraction from 3D image and metrics used in 2D image is
road surface and combination of this from remote sensing
precision which is same as correctness.
images. During this survey, it has been observed that most
of the researchers that have worked on deep learning
Correctness = TP / (TP + FP) (2)
techniques that are related to road surface extraction only
and deep learning techniques that have given good results
• Quality belongs to semantic segmentation. However, in this area, the
The definition of quality is in equation (3), it measures the researchers have been still facing problems due to different
goodness of final result. It is used in road extraction from kinds of images present in every dataset, shadows on the
3D image and metrics used in 2D image is IOU which is road, and the unavailability of labelled data. This paper
same as quality. presents the different types of datasets on which researchers
have worked for road feature extraction and also presents
Quality = TP / (TP + FP + FN) (3) different metrics that are used for the evaluation of the
methodology adopted for road feature extraction. Based on
literature survey discussed in the paper, it has been
• F1-score
concluded that presenting many new ways to road semantic

Authorized licensed use limited to: Jain University. Downloaded on February 12,2025 at 10:42:03 UTC from IEEE Xplore. Restrictions apply.
segmentation is significant, and research on various
proposed techniques using cutting-edge technology is
expanding. Further, research is needed to develop detailed
approaches with high precision in other category of road
feature extraction.
REFERENCES

[1] Guo, Q. and Wang, Z., "A self-supervised learning framework for
road centerline extraction from high-resolution remote sensing
images", IEEE Journal of Selected Topics in Applied Earth
Observations and Remote Sensing, vol. 13, pp.4451-4461, 2020.
[2] Chen, Z., Wang, C., Li, J., Fan, W., Du, J. and Zhong, B.,
"Adaboost-like End-to-End multiple lightweight U-nets for road
extraction from optical remote sensing images", International
Journal of Applied Earth Observation and Geoinformation, vol. 100,
p.102341, 2021.
[3] Wang, S., Mu, X., Yang, D., He, H. and Zhao, P., "Road extraction
from remote sensing images using the inner convolution integrated
encoder-decoder network and directional conditional random
fields", Remote Sensing, vol. 13, no. 3, p.465, 2021.
[4] Zhang, Z., Miao, C., Liu, C.A. and Tian, Q., "DCS-TransUperNet:
Road segmentation network based on CSwin transformer with dual
resolution", Applied Sciences, vol. 12, no. 7, p.3511, 2022.
[5] Li, A., Jiao, L., Zhu, H., Li, L. and Liu, F., "Multitask semantic
boundary awareness network for remote sensing image
segmentation", IEEE Transactions on Geoscience and Remote
Sensing, vol. 60, pp.1-14, 2021.
[6] Yuan, G., Li, J., Liu, X. and Yang, Z., "Weakly supervised road
network extraction for remote sensing image based scribble
annotation and adversarial learning", Journal of King Saud
University-Computer and Information Sciences, vol. 34, no. 9,
pp.7184-7199, 2022.
[7] Jiang, X., Li, Y., Jiang, T., Xie, J., Wu, Y., Cai, Q., Jiang, J., Xu, J.
and Zhang, H., "RoadFormer: Pyramidal deformable vision
transformers for road network extraction with remote sensing
images", International Journal of Applied Earth Observation and
Geoinformation, vol. 113, p.102987, 2022.
[8] Luo, L., Wang, J.X., Chen, S.B., Tang, J. and Luo, B., "BDTNet:
Road extraction by bi-direction transformer from remote sensing
images," IEEE Geoscience and Remote Sensing Letters, vol. 19,
pp.1-5, 2022.
[9] Abdollahi, A., Pradhan, B. and Alamri, A., "SC-RoadDeepNet: A
new shape and connectivity-preserving road extraction deep
learning-based network from remote sensing data", IEEE
Transactions on Geoscience and Remote Sensing, vol. 60, pp.1-15,
2022.
[10] Yan, J., Ji, S. and Wei, Y., "A combination of convolutional and
graph neural networks for regularized road surface extraction", IEEE
transactions on geoscience and remote sensing, vol. 60, pp.1-13,
2022.
[11] Yang, Z., Zhou, D., Yang, Y., Zhang, J. and Chen, Z.,
"TransRoadNet: A novel road extraction method for remote sensing
images via combining high-level semantic feature and
context", IEEE Geoscience and Remote Sensing Letters, vol. 19,
pp.1-5, 2022.
[12] Guan, H., Yu, Y., Li, D. and Wang, H., "RoadCapsFPN: Capsule
feature pyramid network for road extraction from VHR optical
remote sensing imagery", IEEE Transactions on Intelligent
Transportation Systems, vol. 23, no. 8, pp.11041-11051, 2023.

Authorized licensed use limited to: Jain University. Downloaded on February 12,2025 at 10:42:03 UTC from IEEE Xplore. Restrictions apply.