Wang et al.

Journal of Cloud Computing: Advances, Systems Journal of Cloud Computing:

and Applications (2020) 9:27
https://doi.org/10.1186/s13677-020-00172-z Advances, Systems and Applications

RESEARCH Open Access

CNN based lane detection with instance

segmentation in edge-cloud computing
Wei Wang1 , Hui Lin2 and Junshu Wang3,4*

Abstract
At present, the number of vehicle owners is increasing, and the cars with autonomous driving functions have
attracted more and more attention. The lane detection combined with cloud computing can effectively solve the
drawbacks of traditional lane detection relying on feature extraction and high definition, but it also faces the problem
of excessive calculation. At the same time, cloud data processing combined with edge computing can effectively
reduce the computing load of the central nodes. The traditional lane detection method is improved, and the current
popular convolutional neural network (CNN) is used to build a dual model based on instance segmentation. In the
image acquisition and processing processes, the distributed computing architecture provided by edge-cloud
computing is used to improve data processing efficiency. The lane fitting process generates a variable matrix to
achieve effective detection in the scenario of slope change, which improves the real-time performance of lane
detection. The method proposed in this paper has achieved good recognition results for lanes in different scenarios,
and the lane recognition efficiency is much better than other lane recognition models.
Keywords: Double layer network, Lane line detection, Edge computing

Introduction lane detection technology is mainly based on image pro-

With the advent of autonomous driving technology, peo- cessing algorithms to extract the features of lane lines,
ple can largely get rid of the safety problems caused reduce the image channels, perform gray processing on
by daily manual driving. Therefore, self-driving cars are the original image, and then use Canny algorithm or Sobel
sought after by many automobile consumers. In recent algorithm to edge the grayed image, extract some fea-
years, many researchers from academic institutions and tures of the acquired image, and then perform lane line
industries have engaged in autonomous driving technol- fitting after extracting the lane. Common lane fitting mod-
ogy and these researches have promoted the development els include the cubic polynomial, the spline curve, and the
of image processing and computer vision technology. As arc curve [1–3]. At the same time, the quality of fitting
a key part of the automatic driving system, lane detec- is improved. Bertozzi [4] usually uses an inverse perspec-
tion technology is meaningful. At present, the difficulty tive transform to convert the image into a bird’s-eye view,
in lane detection is how to deal with lane detection accu- and finally performs lane tracking and other operations.
racy and real-time issues at the same time, so we need to Recently, deep learning techniques have been used for
improve the accuracy and efficiency of lane recognition lane detection and can effectively detect the lane images.
between traditional and neural network-based lane recog- Based on large-scale data training, the probability of lane
nition methods. The traditional computer vision-based detection errors can be greatly reduced and detection
accuracy can be improved.
*Correspondence: [email protected] Cloud computing, as a kind of distributed comput-
3
Key Laboratory for Virtual Geographic Environment, Ministry of Education,
Nanjing Normal University, Nanjing, China
ing paradigm, can decompose large-scale data into sub-
4
Jiangsu Center for Collaborative Innovation in Geographical Information modules through the network center, allocate them to a
Resource Development and Application, Nanjing, China system composed of multiple servers for processing and
Full list of author information is available at the end of the article

© The Author(s). 2020 Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which
permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit
to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The
images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated
otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended
use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the
copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.
Wang et al. Journal of Cloud Computing: Advances, Systems and Applications (2020) 9:27 Page 2 of 10

analysis, and finally feed the calculation results to the with lane image recognition, the real-time performance of
central node [5, 6]. Cloud computing system has a new lane image detection is greatly improved, and the delay is
resource scheduling method, and the use of the cloud reduced.
computing system can effectively reduce the system com- 2) Based on convolutional neural network and semantic
puting load and improve the data processing efficiency segmentation, the lane line problem in multi-lane scenes
with current computer systems, which is helpful to lane is effectively identified, and when vehicles change lanes,
detection [7, 8]. At the same time, a dynamic resource allo- the current lane scene can be dynamically identified.
cation method for intensive data in the cloud, which has 3) During the lane fitting process, the fixed inverse per-
greatly improved the fault tolerance of cloud computing spective transformation matrix is changed. The neural
has been proposed [9]. These are the factors that cloud network was used to dynamically generate the inverse per-
computing has become the current mainstream frame- spective transformation matrix, which effectively solved
work [10]. Based on the advantages of cloud computing the problem of lane line deviation of the fixed inverse per-
technology in large-scale data processing, we decided to spective transformation matrix fitting under the gradient
use cloud computing technology to improve data process- road scene.
ing efficiency in lane detection. The remainder of this paper is organized as fol-
Edge computing refers to a computing platform that is lows: In “Related work” section, we review the peer
close to the object or data source and can integrate core research and work. “Prior knowledge” section describes
capabilities such as networking, computing, and applica- a priori knowledge of lane detection, including the two
tions [11]. It has a very prominent advantage in the field main frameworks of lane detection techniques in this
of intelligent transportation [12]. To achieve safe and reli- article. “Method” section describes the specific steps
able operation, smart cars need to process a large amount of lane detection. The experimental environment and
of data collected by multiple sensor in real times [13]. results analysis are included in “Experimental evaluation”
Through edge computing, data processing will be made section. Finally, we conclude our work in “Conclusion”
closer to the source [9]. Compared with cloud comput- section.
ing, centralizing data processing in the cloud can greatly
reduce latency time and effectively improving the data Related work
processing efficiency of smart cars. In terms of secu- The current methods for lane detection can be divided
rity and privacy protection, edge computing solves the into three main categories: road-based models, road-
risk of transferring private data to the cloud and leak- based features, and neural network-based models. The
ing user privacy under the cloud computing model, which detection method based on the road model mainly
can effectively protect data and privacy security using abstracts the lane lines into geometric shapes such as
its distributed and mobile features [14–16]. A data pri- straight lines, curves, parabolas, and splines, and uses dif-
vacy optimization scheme that supports edge computing ferent two-dimensional or three-dimensional models to
[17, 18], which further strengthens the data security of determine each model parameter.
edge computing has been proposed and we have com- Wang et al. [20] first located the initial area of the lane
bined this scheme in the lane detection process in order line, and then converted the lane line detection problem
to further improve the efficiency of data processing. into the problem of determining the spline model in the
However, it is difficult for cloud computing or edge initial area. Tan et al. [21] proposed a robust curve lane
computing to play a prominent role in lane recognition detection method, which uses the improved river method
of smart cars. Current lanes processed by convolutional to search for feature points in the far field of view, guided
neural networks face the problem of low real-time per- by a straight line detected in the near field or the curve of
formance, especially in complex scenarios [19]. Currently, the last frame, and can connect dotted lane marks or fuzzy
few researchers use cloud computing or edge comput- lane marks. Shin et al. [22] used parallel features with con-
ing to analyze lane lines, while only use cloud computing stant distance between the left and right lane lines in the
needs to upload a large amount of data to the cloud for top view, and use parallel lines to detect and track lane
processing, which increases the load on the network and is lines. The above method has high accuracy for detect-
difficult to obtain better results under the circumstances ing a specific lane line, but needs to select an appropriate
of the Internet. Therefore, we combine cloud computing model and parameters. One model is often difficult to
and edge computing to process lane data, which greatly cope with multiple road scenarios, and its generalization
improves the real-time performance of lane detection ability and real-time performance are poor. The detec-
during the driving process of smart cars. tion method based on road features uses lane line and
The main contributions of this paper are as follows: environment differences to extract lane line features for
1) Inspired by cloud computing and edge computing, recognition and segmentation. The main features include
adopt more efficient data processing methods to deal lane line edges, color, and geometric features. Yoo et al.
Wang et al. Journal of Cloud Computing: Advances, Systems and Applications (2020) 9:27 Page 3 of 10

[23] proposed a gradient-enhanced conversion method for has greatly improved the robustness and accuracy com-
robust lane detection, which converts RGB space images pared with traditional detection methods, but in order to
into grayscale images and combines Canny’s algorithm train the network, a huge data set is required as support,
to generate large gradients at the boundaries. Son et al. and the requirements for hardware facilities are corre-
[24] realized lane line detection by extracting white and spondingly increased. Xu et al. [35, 36] proposed a video
yellow lane lines respectively. Jung et al. [25] proposed surveillance resource method when studying the Internet
an effective method for reliably detecting lanes based of Vehicles (IoV), which supports edge computing and can
on spatiotemporal images, which improved the detec- effectively use edge computing to solve the problem of
tion accuracy. Niu et al. [26] proposed a lane detection lane image acquisition and training requirements and high
method with two-stage feature extraction in order to solve network bandwidth, we combine this method in the data
the problem of robustness caused by the inconsistency of processing process to distribute computing power from
lighting and background clutter, where each lane has two the central node to devices with image acquisition and
boundaries. The above-mentioned detection methods are processing capabilities.
easy to implement, have low complexity, and can obtain At the same time, we choose to convert the lane line
high real-time performance, but are extremely suscepti- detection problem into an instance segmentation prob-
ble to environmental influences. In rainy days or under lem, and combines edge computing to design a two-
insufficient light, misjudgment of lane lines is prone to instance segmentation branch network, where the lane
occur. segmentation branch outputs the background or lane line,
With the continuous development of science and tech- and the lane embedding branch separates the lane lines
nology and the continuous improvement of hardware obtained by the segmentation branch into different lane
equipment, neural network-based detection methods examples to improve the accuracy of lane detection. At the
have made breakthrough progress. Using deep learning to same time, a custom network is designed, and the training
detect lane lines can ensure good recognition accuracy in data is marginalized to enhance the real-time nature of the
most scenarios [27]. Insteading of relying on highly spe- data processing. A perspective transformation matrix is
cialized manual features and heuristics to identify lane proposed to solve the problem of lane detection for classic
breaks in traditional lane detection methods, target fea- neural networks when the road plane changes and helps
tures under deep learning can automatically learn and us improve the real-time and accurate performance of the
modify parameters during the training process. network used in this paper.
Liu et al. [28] used the mobile edge computing frame-
work to greatly reduce the time loss of learning. Qi et al. Prior knowledge
[29, 30] significantly improved the data processing rate Lane prediction
through the method of distributed location and multiple Lane detection usually requires the use of relevant algo-
data sources. John et al. [31] used the learned CNN model rithms to extract the pixel features of the lane line,
to predict the lane line position in the image, especially in and then the appropriate pixel fitting algorithm is used
the case of occlusion and missing, CNN can extract robust to complete the lane detection. Traditional lane detec-
features from the road image, and then train an additional tion uses Canny edge extraction algorithm or Sobel edge
tree-like regression model directly from the features esti- extraction algorithm to obtain lane line candidate points
mated by lane line position. Kim et al. [32] used serial and use Hough transform for lane feature detection, but
end-to-end transfer learning to directly estimate and sep- most operations are based on manual feature extraction
arate the left and right lanes, and collects a dataset that [37, 38]. The latest research is based on deep neural
includes multiple road conditions to train a CNN model. networks to make dense predictions instead of artificial
Pan et al. [33] proposed a spatial CNN (SCNN) and the feature extraction.
convolution of the extended layer structure in the fea- In order to enable the model to adapt to more road
ture map is a slice structure convolution. In this way, the scenes, by analyzing the structure of classic convolutional
information transfer between pixels is performed between neural networks and semantic segmentation, we use an
the rows and columns of each layer, which is suitable for improved two-branch network and custom function net-
strong spatial relationships, but long continuous-shaped work to convert the lane line detection problem into an
structures or large targets lack apparent clues in this instance segmentation problem detection lane line, where
method, such as lane lines, walls, and pillars. Zhang et each lane line forms its own instance. In this paper, we use
al. [34] applied deep learning-based convolutional neural a multi-branch network with two branches and the two-
networks to instance segmentation of monocular vision to branch network contains a lane segmented branch and a
achieve segmentation of different objects in actual scenes, lane embedded branch that can be trained end-to-end.
and achieved better segmentation results, but less robust. Lane segmentation branches output backgrounds or lane
The use of deep learning methods for lane line detection lines, and lane embedding branches further decompose
Wang et al. Journal of Cloud Computing: Advances, Systems and Applications (2020) 9:27 Page 4 of 10

the divided lane pixels into different lane proportions. By lane points near the horizon to be projected to infin-
dividing the lane detection problem into the above two ity, which affects lane line simulation total accuracy. In
tasks, the function of lane segmentation can be fully uti- order to solve this situation, we apply an inverse per-
lized without having to assign different classes to different spective transformation to the image before fitting the
lanes. Lane embedding branches trained with the clus- curve, and uses a loss function customized for the lane
tering loss function assign lanes to each pixel from the fitting problem to optimize it. A custom function net-
lane segmentation branch, ignoring background pixels. By work is used to generate a transformable matrix and
doing so, the problem of lane change is solved, a vari- transform the lane pixels, then use curve fitting polyno-
able number of lanes can be handled, and the practice of mials to perform pixel fitting on the converted pixels, and
detecting a fixed lane line is improved. finally convert the fitted pixels into the input image. The
At the same time, combined with edge computing, advantage of this network is that the detection algorithm
a blockchain-based edge offloading method is used to can fit the pixels of distant lanes with good robustness
ensure real-timeness and integrity in the data transmis- when the road surface changes, and better adapt to lane
sion process [39, 40]. For real-time processing and feed- changes.
back, edge computing, as a new computing paradigm, has The overall framework of the model is shown in Fig. 1.
a better ability to solve some problems such as high trans- The camera on the car collects the image and transmits it
mission delays, high bandwidth expenditures, and privacy to the cloud data processing center. After the image is sub-
leaks. jected to binary segmentation and embedded segmenta-
tion, the clustering operation is performed and combined
Lane fitting with the transformable matrix generated by the custom
To estimate the lane instance, determine which pixel network to generate the final lane detection image.
belongs to which lane, we need to convert each of them
into a parameter description. To this end, we use a Method
widely used fitting algorithm. At present, the widely used Binary segmentation
lane fitting models are mainly cubic polynomials, spline In lane detection, in order to save computing resources,
curves or arc curves. Inverse perspective transforma- we binarize the image. The segmented branch of the
tion is used to transform the image into a “bird’s-eye two-branch network outputs a binary segmentation map,
view” to improve the quality of the fitting while main- which divides the lane line and non-lane line parts. In this
taining the computational efficiency, and then adopt the paper, all ground-level car-to-car points are connected
method of curve fitting [41]. The fit lines in the “bird’s- together to form the connecting line of each lane to avoid
eye view” can be re-projected into the original image situations where lane lines are unpredictable due to lane
through an inverse transformation matrix. Generally, the line degradation or road obstacles blocking the lane lines.
inverse perspective transform calculates the transforma- At the same time, due to the high imbalance between lane
tion matrix on a single image and can remain fixed, but if and background, we use bounded inverse class weighting
the road plane changes, the fix is no longer valid, causing for lane pixel segmentation.

Fig. 1 Model framework

Wang et al. Journal of Cloud Computing: Advances, Systems and Applications (2020) 9:27 Page 5 of 10

Lane instance embedding segmentation represents the threshold of the variance term, δd repre-
The lane loss embedding branch function of the dual sents the threshold of the distance term, and the total loss
branch network is used to output the pixel embeddings L = Lvar + Ldist . After the network converges, the lane
of each lane, so that the pixel embeddings belonging pixel embeddings will be clustered together so that the
to the same lane are clustered together, and the pixel distance between each cluster is greater than δd and the
embeddings belonging to different lanes have the largest radius of each cluster is less than δv .
distance. In the way, the pixel embeddings of the same In Fig. 2a and b are the lane images collected, and the
lane will come together to form a unique cluster for each images generated by the example embedding segmenta-
lane and this is achieved by introducing two terms, one tion are (c) and (d), respectively.
is the variance term (Lvar ), which applies a pulling force
to each embedding and tends to be closer to the aver- Clustering
age embedding of the lane. The other is the distance term For a large number of unlabeled data sets, the data set is
(Ldist ), which clusters the centers push away from each divided according to the inherent similarity of the data,
other. The pull force is active only when the distance from and the data set is divided into different categories with
the embedding point to the cluster center is greater than inherent differences. The similarity between the data in
δv . The thrust force is active only when the distance from the same category is large and the similarity between the
the embedding point to the cluster center is less than δd . data in different categories is small. In order to improve
⎧ the robustness of lane line detection in different envi-
⎪ 1 C 1 NC
⎨ Lvar = C c=1 NC i=1 [ μc − μi − δv ]+
2
ronments, it is necessary to perform cluster analysis on
C
Lsum = CA,CB=1 [ δd − μCA − μCB ]+ 2 (1) the lane line pixels, embed the pixels of the same lane
⎪
⎩L 1
dist = L
C(C−1) sum
line together, and distinguish the pixels of different lane
lines. In the future, clustering can ensure the recogni-
In (1), C is the number of lane lines, NC is the num- tion accuracy under the conditions of insufficient lighting
ber of elements in C, μi is the lane pixel embedding, μc conditions and poor traffic conditions.
is the average embedding of C, “ μc − μi ” represents In this paper, we complete clustering by iteration, com-
the average embedding and pixel embedding distance, and bining the clustering algorithm with the loss function. In
CA and CB represent two lane line, μCA − μCB repre- (1), by setting 6δv < δd , we take a random lane with a
sents the average embedding distance between CA lane radius of 2δv and its surrounding threshold to select the
line and CB lane line, [ δd −  μCA − μCB  −δv ]2+ means ones that belong to the same lane all embedded. Repeat
0 and δd −  μCA − μCB  has the largest positive num- the above operation until all lanes are embedded and
ber value, [ μc − μi − δv ]2+ represents the maximum value assigned to one lane. In order to avoid choosing the dis-
of a positive number between 0 and μc − μi − δv , δv tance from the outlier to the threshold, we first use the

a b

c d

Fig. 2 Lane instance embedding segmentation graph

Wang et al. Journal of Cloud Computing: Advances, Systems and Applications (2020) 9:27 Page 6 of 10

mean shift to approach the center of the cluster, and then transformation, P(Pi =[ Xi , Yi , 1]T ∈ P, i = 1, 2, . . .) rep-
sets the threshold. resents the pixels of real lane pixels, Xi , Yi represents the
horizontal and vertical coordinates of pixels of real lanes,
Pixel fitting Xi , Yi represents the horizontal and vertical coordinates
The output of the two-branch network is a set of pixels for of lane pixel points after inverse perspective transforma-
each lane line. It is not ideal to fit a polynomial with these tion. In order to obtain the position of the lane x at a
pixels in the input image space, because a higher-order given y position, the real lane pixel extraction point Pi =
polynomial is needed to process the curve lane. Several [ −, yi , 1]T is transformed into a transformed lane pixel
commonly used fitting methods are RANSAC algorithm, point Pi =[ −, yi , 1]T after perspective transformation ,
Bezier curve fitting, curve fitting based on spline interpo- predict lane pixel points and coordinates Pi∗ = (x∗ 
i , yi )
lation, and polynomial fitting based on least squares. In 
at each yi position. A third-order polynomial is used to
this paper, the image is first transformed by inverse per- fit the predicted lane pixel point Pi∗ by the least square
spective and projected into the “bird’s eye view", where method. Equations (4) and (5) are third-order polynomials
the lanes are parallel to each other, so the curve lane can and least squares formulas, respectively.
be fitted by a second to third order polynomial. How-
ever, in these cases, the fixed transformation matrix H f y = αy2 + βy + γ (4)
is calculated only once and applied to all images. This
results in errors in the case of ground-level changes, where −1
the vanishing point projected to infinity moves up or w = YTY Y T x (5)
down as shown. To solve this problem, this paper trains
In (4) and (5), α, β, and γ are the given optimal param-
a neural network with a custom loss function. The net-
eters, T is the transpose of the matrix, w =[ α, β, γ ]T is
work is optimized end-to-end to predict the parameters
the least square method to represent the letter, x =
of the perspective transformation matrix H. In the per-
[ x1 , x2 , x3 , · · · xn ]T represents the lane coordinate point P
spective transformation matrix H, the transformed lane
abscissa combination, y =[ y1 , y2 , y3 , · · · yn ]T represents
points can be optimally fitted with second or third order
the transformable parameter matrix ⎡ transform lane
⎤ pixel
polynomials. The prediction is based on the input image,
y2
1 y  1
1
allowing the network to adjust the projection parameters
point P ordinate combination, Y = ⎣ · · · · · · · · · ⎦ is the
when the ground plane changes, so that the lane line fit 
y2
n y1 1
is still correct. In (2), the transformation matrix H has 6
required matrix in the least squares formula. Fitting the
degrees of freedom, of which six variables a-f represent 6
predicted lane pixels Pi∗ =[ x∗  T
i , yi , 1] perspective trans-
degrees of freedom parameters. The purpose of placing 0
formation to the road lane image to obtain lane pixels Pi∗ at
is to enforce the constraint and make the horizontal line
different lane positions. The cluster loss function is used
remains horizontal under the transformation.
to segment the lane line instance into a pixel map, and
⎡ ⎤
a b c the predicted pixels of the fitted lane are projected into
H = ⎣0 d e ⎦ (2) the road lane image through perspective transformation
0 f 1 to obtain the final detection result image, and the perspec-
tive transformation formula of the cluster loss function is
The network architecture of the custom function net- shown in (6).
work remains small in size and consists of a continu-
ous block of 3 × 3 convolutional layers, Batchnorm, and Pi∗ = H −1 Pi∗ (6)
ReLU activation functions, and the largest pooling layer to In (6), Pi∗ =[ x∗i , yi , 1]T is a lane pixel projected into the
reduce the network size, and finally add 2 fully connected input road lane image, Pi∗ =[ x∗  T
i , yi , 1] is the predicted
layers. Before a pixel fits a lane line, a custom function −1
lane pixel point, and H represents the perspective trans-
network output transformable matrix is used to transform formation matrix. In order to train the output of the
the lane pixels. The inverse perspective transformation is custom function network H, which is the most suitable
performed on the road image to a new top view to gener- transformation matrix H for pixel fitting polynomials, the
ate pixels in the top view lane after the inverse perspective following loss function is constructed. Since lane fitting
transformation and the inverse perspective transforma- is done by using the least squares method, the loss func-
tion formula is shown in (3). tion is differentiable and the loss function of the custom
function network is shown in (7).
P = HP (3)
1  ∗
N
In (3), P (Pi =[ Xi , Yi , 1]T ∈ P , i = 1, 2, . . .) represents Loss = (xi − xi )2 (7)
the transformed lane line pixels after inverse perspective N
i=1
Wang et al. Journal of Cloud Computing: Advances, Systems and Applications (2020) 9:27 Page 7 of 10

Experimental evaluation process, the input image is regularized and randomly

Data set cropped and randomly rotated. The size of the obtained
This article uses the Tusimple dataset as a training and pictures is uniformly set to 256 × 512, and the batch size
testing source, which includes lane line data at different is set to 6 so that 6 pictures can input at the same time
times during the day, and also includes situations with 2 for training. The gradient descent algorithm optimizes
lanes, 3 lanes and 4 lanes. the training of network parameters. The optimizer has a
For the lane segmentation task, the final segmentation momentum parameter of 0.9, a weight decay of 1 × 10−4 ,
result needs an evaluation index to measure the perfor- and a learning rate update strategy of exponential decay:
mance of the test. We choose F-Measure to measure  
the model’s ability to predict the lane line because the iter power
lr = lrinit × 1 − (10)
F-Measure evaluation index can comprehensively con- N
sider the two measures of precision and recall in the
In (10), lrinit is the initial learning rate, lrinit = 0.0005, N
experiment. All pixels of the lane line are defined as pos-
is the total number of iterations for training, set to 80010,
itive samples, which are denoted by P and background.
iter is the current number of iterations, and power is the
A pixel is a negative sample, denoted as N. It is deter-
attenuation coefficient, set to 0.9. We select some images
mined whether the pixel prediction is correct by judging
from the Tusimple data set for testing. By comparing with
whether the predicted image result and each pixel of the
the traditional lane detection algorithms and the currently
real label are equal. Formulas for evaluating image predic-
used deep learning algorithms, it is found that the lane
tion results using the F-Measure index are shown in (8)
detection algorithm used in this paper has excellent per-
and (9).
formance in scenarios such as inadequate lighting, shadow
TP TP occlusion, missing lane lines and curved lanes.
Precision = , Recall = (8)
TP + FP TP + FN By using the model in this paper to detect under differ-
ent road complex environments, we can get the following
(β 2 + 1)PR detection results with Table 1. We found that other mod-
Fβ = (9)
β 2P + R els can achieve correspondingly good results when dealing
Precision reflects the accuracy of the model and is with lanes in some scenes, but it is difficult to take into
expressed as the ratio of the number of correct predic- account all the scenes, while our proposed method is
tions to the total number of predictions. Recall reflects superior to other models in various scenarios, which is a
the recall rate of the model and is a measure of coverage, benefit for lane recognition technology.
expressed as the ratio of the number of targets detected in At the same time, we obtained the time information of
the data to the total number of targets that actually exist. the relevant model when processing the lane images. Dur-
TP represents the number of positive instances detected ing the lane instance segmentation and clustering process,
correctly, FP represents the number of instances detected each frame image took 16.6ms and the FPS reached 60.2.
as positive cases, FN represents the number of instances During the lane fitting process, each frame took 2.4ms and
detected as negative cases and β is an adjustable parame- the FPS reached 416.7. The total test efficiency reached
ter which role is to highlight the proportion of Recall and 52.6 frames per second and achieved a good processing
we set it to 0.8, which helps reduce the damage caused by rate.
misrecognizing lane areas. In Fig. 3a and b are the lane images collected in a poorly
lit environment, (c) and (d) are the lane detection results.
Experimental results It can be seen that the lane line detection model used in
The experimental environment of this article is Ubuntu this paper can still achieve better detection results under
16.04 operating system, using NVIDIA GEFORCE 920M poor lighting conditions.
graphics processor to implement all code based on Ten- In Fig. 4a and b are the lane images collected in the road
sorflow deep learning framework. During the training line degradation scene, (c) and (d) are the corresponding

Table 1 Comparison of test results of different models

Models Insufficient light Shadow occlusion Missing lane line Curve lane line Normal road
VGG-FCN 0.9369 0.9311 0.8837 0.9259 0.9308
SCNN 0.9612 0.9713 0.9412 0.9315 0.9671
U-Net 0.9510 0.9613 0.9126 0.9168 0.9647
Tradition 0.9213 0.9122 0.8635 0.8874 0.9171
Ours 0.9662 0.9785 0.9588 0.9410 0.9757
Wang et al. Journal of Cloud Computing: Advances, Systems and Applications (2020) 9:27 Page 8 of 10

a b

c d

Fig. 3 Lane detection under poor lighting conditions

detection results. It can be seen that even in some cases lines well in the case of obstacles and shadow, and has
the lane line has been degraded or blurred, the lane line strong robustness.
can still be accurately detected under the deep training
lane network model, which proves that the network model Conclusion
used in this paper has better robustness. In this paper, the lane detection algorithm is carefully
In Fig. 5a and b are lane images collected in a scene studied. The lane detection method based on traditional
with shadows and obstacles, and (c) and (d) are corre- image processing and the lane detection method based
sponding detection results. It can be found that the lane on deep learning are analyzed and compared to solve
detection model used in this paper can still detect lane the problem of lane detection under complex conditions

a b

c d

Fig. 4 Lane line detection under lane line edge degradation

Wang et al. Journal of Cloud Computing: Advances, Systems and Applications (2020) 9:27 Page 9 of 10

a b

c d

Fig. 5 Detection of lane lines under shadows and obstacles

such as shadows and obstacles. In order to improve the Competing interests

efficiency of central computing, a two-branch training The authors declare that they have no competing interests.
network and a customized training network based on Author details
semantic segmentation are proposed, and the combina- 1 Neusoft Corporation, Shenyang, China. 2 Nanjing University of Information

tion of vehicle edge calculation and actual engineering can Science and Technology, China No.219 Ningliu Road, 210044 Nanjing, China.
3 Key Laboratory for Virtual Geographic Environment, Ministry of Education,
improve the effect of lane line detection network. Nanjing Normal University, Nanjing, China. 4 Jiangsu Center for Collaborative
In the inverse perspective transformation, the use of Innovation in Geographical Information Resource Development and
a fixed transformation matrix will cause errors when Application, Nanjing, China.
ground changes, which will cause the vanishing point pro- Received: 4 March 2020 Accepted: 22 April 2020
jected to infinity to move up or down. We trained a neural
network with a custom loss function that can dynamically
predict the parameter values of the transformable matrix. References
1. Smuda P, Schweiger R, Neumann H, Ritter W (2006) Multiple cue data
The transformed lane points are fitted with second or
fusion with particle filters for road course detection in vision systems. In:
third-order polynomials. The prediction is based on the 2006 IEEE Intelligent Vehicles Symposium. IEEE. pp 400–405. https://doi.
input image, allowing the network to adjust the projection org/10.1109/ivs.2006.1689661
2. Aly M (2008) Real time detection of lane markers in urban streets. In: 2008
parameters when the ground plane changes, making the
IEEE Intelligent Vehicles Symposium. IEEE. pp 7–12. https://doi.org/10.
model excellent stickiness. The final tests show that the 1109/ivs.2008.4621152
model in our paper has better performance in scenarios 3. Teng W, Wang Y, Yu B, Liu J (2020) Icciu: A new real-time lane-level
positioning algorithm. IEEE Access. https://doi.org/10.1109/access.2020.
such as insufficient lighting and lane line degradation.
2970503
4. Bertozzi M, Broggi A (1996) Real-time lane and obstacle detection on the
Authors’ contributions
gold system. In: Proceedings of Conference on Intelligent Vehicles. IEEE.
Wei Wang, Hui Lin, Junshu Wang conceived and designed the study. Wei
pp 213–218. https://doi.org/10.1109/ivs.1996.566380
Wang, Hui Lin and Junshu Wang performed the simulations. Wei Wang, Hui
5. Xue C, Lin C, Hu J (2019) Scalability analysis of request scheduling in cloud
Lin and Junshu Wang wrote the paper. All authors reviewed and edited the
computing. Tsinghua Sci Technol 24(3):249–261
manuscript. All authors read and approved the final manuscript.
6. Shen D, Luo J, Dong F, Zhang J (2019) Virtco: joint coflow scheduling and
virtual machine placement in cloud data centers. Tsinghua Sci Technol
Funding 24(5):630–644
This research is supported by National Natural Science Foundation of China 7. Xu X, He C, Xu Z, Qi L, Wan S, Bhuiyan M (2019) Joint optimization of
under Grant No. 41671055, Jiangsu Natural Science Foundation of China offloading utility and privacy for edge computing enabled IoT. IEEE
under Grant No. BK20171037, the Program of Natural Science Research of Internet of Things J. https://doi.org/10.1109/jiot.2019.2944007
Jiangsu colleges and universities under Grant No.17KJB170010. 8. Xu X, Zhang X, Khan M, Dou W, Xue S, Yu S (2017) A balanced virtual
machine scheduling method for energy-performance trade-offs in
Availability of data and materials cyber-physical cloud systems. Futur Gener Comput Syst
The raw data required to reproduce these findings cannot be shared at this 9. Xu X, Mo R, Dai F, Lin W, Wan S, Dou W (2019) Dynamic resource
time as the data also forms part of an ongoing study. provisioning with fault tolerance for data-intensive meteorological
Wang et al. Journal of Cloud Computing: Advances, Systems and Applications (2020) 9:27 Page 10 of 10

workflows in cloud. IEEE Trans Ind Inform. https://doi.org/10.1109/tii.2019. 33. Pan X, Shi J, Luo P, Wang X, Tang X (2018) Spatial as deep: Spatial cnn for
2959258 traffic scene understanding. In: Thirty-Second AAAI Conference on
10. Yang S, Wu J, Shan Y, Yu Y, Zhang S (2019) A novel vision-based Artificial Intelligence
framework for real-time lane detection and tracking. Technical report. 34. Zhang Z, Schwing AG, Fidler S, Urtasun R (2015) Monocular object
SAE Technical Paper. https://doi.org/10.4271/2019-01-0690 instance segmentation and depth ordering with cnns. In: Proceedings of
11. Xu X, Zhang X, Gao H, Xue Y, Qi L, Dou W (2020) Become: the IEEE International Conference on Computer Vision. pp 2614–2622.
Blockchain-enabled computation offloading for iot in mobile edge https://doi.org/10.1109/iccv.2015.300
computing. IEEE Trans Ind Inform 16(6):4187–4195 35. Xu X, Fu S, Yuan Y, Luo Y, Qi L, Lin W, Dou W (2019) Multiobjective
12. Ganin AA, Mersky AC, Jin AS, Kitsak M, Keisler JM, Linkov I (2019) Resilience computation offloading for workflow management in cloudlet-based
in intelligent transportation systems (its). Transp Res Part C Emerg mobile cloud using nsga-ii. Comput Intell 35(3):476–495
Technol 100:318–329 36. Xu X, Liu X, Xu Z, Dai F, Zhang X, Qi L (2019) Trust-oriented iot service
13. Kuo C, Lu Y, Yang S (2019) On the image sensor processing for lane placement for smart cities in edge computing. IEEE Internet Things J.
detection and control in vehicle lane keeping systems. Sensors 19(7):1665 https://doi.org/10.1109/jiot.2019.2959124
14. Xu Y, Qi L, Dou W, Yu J (2017) Privacy-preserving and scalable service 37. Aminuddin NS (2020) A new approach to highway lane detection by
recommendation based on simhash in a distributed cloud environment. using hough transform technique. J Inf Commun Technol 16(2):244–260
Complexity 2017. https://doi.org/10.1155/2017/3437854 38. Wang J, Kong B, Mei T, Wei H (2019) Lane detection algorithm based on
15. Xu X, Liu Q, Luo Y, Peng K, Zhang X, Meng S, Qi L (2019) A computation temporal–spatial information matching and fusion. CAAI Trans Intell
offloading method over big data for iot-enabled cloud-edge computing. Technol 2(4):154–165
Futur Gener Comput Syst 95:522–533 39. Xu X, Liu Q, Zhang X, Zhang J, Qi L, Dou W (2019) A blockchain-powered
16. Chi X, Yan C, Wang H, Rafique W, Qi L Amplified locality-sensitive crowdsourcing method with privacy preservation in mobile environment.
hashing-based recommender systems with privacy protection. Concurr IEEE Trans Comput Soc Syst 6(6):1407–1419
Comput Pract Experience:5681. https://doi.org/10.1002/cpe.5681 40. Nguyen TNA, Phung SL, Bouzerdoum A (2020) Hybrid deep
17. Xu X, Chen Y, Zhang X, Liu Q, Liu X, Qi L (2019) A blockchain-based learning-gaussian process network for pedestrian lane detection in
computation offloading method for edge computing in 5g networks. unstructured scenes. IEEE Trans Neural Netw Learn Syst. https://doi.org/
Softw Pract Experience. https://doi.org/10.1002/spe.2749 10.1109/tnnls.2020.2966246
18. Xu X, Cao H, Geng Q, Liu X, Dai F, Wang C (2020) Dynamic resource 41. Barrera A, Guindel C, Beltrán J, García F (2020) Birdnet+: End-to-end 3d
provisioning for workflow scheduling under uncertainty in edge object detection in lidar bird’s eye view. arXiv preprint. arXiv:2003.04188
computing environment. Concurr Comput Pract Experience:5674.
https://doi.org/10.1002/cpe.5674 Publisher’s Note
19. Son Y, Lee ES, Kum D (2019) Robust multi-lane detection and tracking Springer Nature remains neutral with regard to jurisdictional claims in
using adaptive threshold and lane classification. Mach Vis Appl published maps and institutional affiliations.
30(1):111–124
20. Wang Y, Teoh EK, Shen D (2004) Lane detection and tracking using
b-snake. Image Vis Comput 22(4):269–280
21. Tan H, Zhou Y, Zhu Y, Yao D, Li K (2014) A novel curve lane detection
based on improved river flow and ransa. In: 17th International IEEE
Conference on Intelligent Transportation Systems (ITSC). IEEE.
pp 133–138. https://doi.org/10.1109/itsc.2014.6957679
22. Shin B-S, Tao J, Klette R (2015) A superparticle filter for lane detection.
Pattern Recogn 48(11):3333–3345
23. Yoo H, Yang U, Sohn K (2013) Gradient-enhancing conversion for
illumination-robust lane detection. IEEE Trans Intell Transp Syst
14(3):1083–1094
24. Son J, Yoo H, Kim S, Sohn K (2015) Real-time illumination invariant lane
detection for lane departure warning system. Expert Syst Appl
42(4):1816–1824
25. Jung S, Youn J, Sull S (2015) Efficient lane detection based on
spatiotemporal images. IEEE Trans Intell Transp Syst 17(1):289–295
26. Niu J, Lu J, Xu M, Lv P, Zhao X (2016) Robust lane detection using
two-stage feature extraction with curve fitting. Pattern Recogn
59:225–233
27. De Brabandere B, Van Gansbeke W, Neven D, Proesmans M, Van Gool L
(2019) End-to-end lane detection through differentiable least-squares
fitting. arXiv preprint. arXiv:1902.00293
28. Liu L, Chen X, Lu Z, Wang L, Wen X (2019) Mobile-edge computing
framework with data compression for wireless network in energy
internet. Tsinghua Sci Technol 24(3):271–280
29. Qi L, Zhang X, Dou W, Ni Q (2017) A distributed locality-sensitive
hashing-based approach for cloud service recommendation from
multi-source data. IEEE J Sel Areas Commun 35(11):2616–2624
30. Qi L, Zhang X, Dou W, Hu C, Yang C, Chen J (2018) A two-stage
locality-sensitive hashing based approach for privacy-preserving mobile
service recommendation in cross-platform edge environment. Futur
Gener Comput Syst 88:636–643
31. John V, Liu Z, Guo C, Mita S, Kidono K (2015) Real-time lane estimation
using deep features and extra trees regression. In: Image Video Technol.
Springer. pp 721–733. https://doi.org/10.1007/978-3-319-29451-3_57
32. Kim J, Park C (2017) End-to-end ego lane estimation based on sequential
transfer learning for self-driving cars. In: Proceedings of the IEEE
Conference on Computer Vision and Pattern Recognition Workshops.
pp 30–38. https://doi.org/10.1109/cvprw.2017.158