Advances in Remote Sensing Extraction of Urban Roads

As early as 1970s, the United States has begun the research of remote sensing image processing technology. In recent ten years, the research of road remote sensing extraction in China has also advanced by leaps and bounds. High resolution remote sensing images have been widely used in many fields, such as urban development planning, environmental monitoring and evaluation, and public announcement information services. The main application goal of remote sensing image is to extract the information of the object of interest, then identify it and complete the image understanding. Road is the most important and basic transportation mode of human beings, which provides a lot of support for the development of human civilization. road extraction is important for traffic management, including urban planning, road monitoring, GPS navigation, map updating, image registration, etc. extracting roads from high-resolution remote sensing satellite images is not only a challenging research direction, but also of great practical value.


Introduction
With the rapid development of remote sensing technology, we can obtain more image information more quickly, and how to deal with and interpret these massive image information quickly and accurately is an important problem. As the skeleton of the city, the urban road is also a very important basic geographic information. It plays an important role in the economic activities of the city. The rapid development of image processing technology makes it possible to extract the road based on remote sensing image automatically [1].

Basic Features of High Resolution Remote Sensing Image Roads
In high-resolution remote sensing images, roads have four basic features [2]

Geometric characteristics
The road is long and arranged in strips. Roads are usually long and curved. The width of the same road is almost constant, and the edges are parallel to each other. Generally speaking, the contrast between road and non-road features is very obvious, but in high-resolution remote sensing images, there may be green belts, cars, tall buildings on the road.

Topological features
Roads are not isolated, roads cross with other roads to form a road network, especially in cities, the road network is generally more dense.

Radiation characteristics
The same section is usually built with the same material, and the gray value of the road is the same. But roads in China, especially urban roads, are mostly obscured by vegetation and buildings.

Contextual features
Also known as background features. Roads in different places have different backgrounds. For example, rural roads are dominated by farmland and sparse small houses; urban roads have many vehicles, many green belts around the roads, and dense distribution of tall buildings, which also leads to a large number of shadows on urban roads.

Current Situation of Remote Sensing Image Road Extraction Based on Traditional Methods
The process of road extraction is to use a certain method to obtain road points, eliminate other interference, and connect road sections to the road network [3]. There are many road extraction algorithms. Different remote sensing image resolution and different imaging methods lead to the diversification of road extraction methods. Scholars at home and abroad put forward many algorithms for this problem [4].

Watershed Algorithm
Watershed algorithm is a segmentation method of mathematical morphology based on topology theory. In the process of image segmentation, the similarity between adjacent pixels is regarded as a very important reference basis, and the pixel points with similar position and gray value are connected to form a set with boundaries [5].
In 2008, Pan Tingting proposed a watershed algorithm based on the basic characteristics of remote sensing image road and the regional characteristics of watershed algorithm [6]. In this method, the remote sensing image is first denoised by filter, then the road information is extracted by watershed algorithm, and finally some small spots are removed from the above operation. During 2011, Yu Changhui and Yi Yaohua first used watershed algorithm to segment remote sensing images into small areas, then marked each small area, and finally extracted the road information on the image by object-based Markov random field (MRF) method [7]. In 2013, Cai Hongyue and Yao Guoqing proposed an improved watershed road extraction algorithm based on object-oriented and mathematical morphology theory. This method can use mathematical morphology to remove or correct bad results [8].As shown in figure 1.

Edge-based road extraction
Because different geographical locations and ground objects have different gray values in remote sensing images, this paper proposes an edge detection method based on different gray values. It can greatly reduce the amount of data, remove the information independent of the target, and only retain the structure of the image. The operators commonly used by researchers in edge extraction generally include: Sobel operator, Prewitt operator, Laplacian operator and Canny operator [9].
R.Gaetano, J.Zernbia and G.Scarpa were first proposed to use edge detection operators in remote sensing image segmentation Canny 2011 [10]. In this method, firstly, the edge detection operator is used to detect the edge information of remote sensing image.Secondly, the edge center line information is calculated by using Euclidean distance function.Finally, the road information of the processed remote sensing image is extracted by using the graph cutting theory. Zeng Faming, Yang Bo and Wu Dewen also used the Canny edge detection operator in 2013 to extract the road in the mining area from the remote sensing image [11]. In this method, the edge information of the image is first detected by the edge detection operator. Then match the image edge information to find the edge of the road, and finally locate and vectorize the road in the mining area. In 2014, Xu Nan and Zhou Shaoguang used the direction template to detect the edge point information, then divided the image into blocks, and then used the search method to extract the sub-lines of the image after the block [12]. By 2016, Tan Yuan, Huang Huixian and Xu Jianmin improved the Sobe operator according to the continuous and complete characteristics of the road edge and the bad detection effect of the traditional edge detection operator. The continuity of edge information detected by this improved operator is better than that without improvement [13].As shown in figure 2.

Texture analysis
Unlike pixel-based color features, texture features represent the distribution of pixels in the region. In addition, texture feature is a global feature, which not only has good noise resistance, but also has rotation invariance [14].
In 2011, Wu Xuewen and Xu Hanqiu proposed a method to extract urban roads with high resolution remote sensing images by combining the speed function in the fast moving method with the gradient feature and texture feature of the road in the remote sensing image [15]. Zhou Shaoguang, Chen Chao, He Chunxiao proposed a road extraction method based on two shape priors using Graph Cuts computation to fuse them in 2013 [16]. In this method, the texture feature and input image are first extracted as the segmentation eigenvalue, then the road point map is extracted by Graph Cuts, and then the initial segment graph of the road is obtained by the method of straight line matching. Finally, the original segment of the road is extrapolated to obtain the complete remote sensing image road.As shown in figure 3.

Line segment matching method
W.Z.Shi, C.Q.ZhuW proposed line segment matching method for road extraction on high resolution remote sensing satellite images in 2002, which is a quasi-automated method [17]. This method is based on the characteristics of high resolution remote sensing images, the basic knowledge of roads and mathematical techniques. does not need to pre-select seed points as most semi-automatic method extraction methods do, although it needs to set some parameters. In 2004, Zhu Changqing, Wang Yaoge and Ma Qiuhe proposed a method to extract gray morphology based on mathematical morphology to facilitate the extraction of roads from high-resolution remote sensing images [18]. In this method, the image segmentation is carried out on the gray image through the morphological features to obtain the road network contour, and then the road network of the remote sensing image is obtained by using the line segment feature matching method to extract the road on the segmented image.

Current Situation of Remote Sensing Image Road Extraction Based on Deep Learning
Deep learning combines abstract high-level features by extracting simple low-level features of multiple layers, so that information such as categories and features of images can be obtained. It is developed by artificial neural network.The simplest artificial neural network consists of a data input layer, an output layer, and a hidden layer [19]. Deep neural network is a neural network containing multiple hidden layers. For deep learning, the number of layers of neural network is from dozens to hundreds, and some ResNet even reach more than 1000 layers [20]. In depth learning, the data set is trained, a loss function is set up, and the weights and deviations of the neural network are continuously calculated and updated to optimize the loss function and improve the prediction accuracy of the network. After continuous development, deep learning has been applied to face recognition, license plate recognition and other aspects. It is not superior, it has changed people's daily life and improved the efficiency of social production. Because of more and more research on depth learning, researchers in remote sensing field also apply depth learning to remote sensing image processing.
In 2015, Wang Gang, Chen Jinyong and Gao Feng et al .designed a deep neural network to solve the problem of target detection of remote sensing images with high score No .1 [21] . They accurately detected playgrounds, airports and other infrastructure on remote sensing images of High Score One. It is proved that the application of neural network in remote sensing image target detection is feasible.
In 2017, Pei Lei, Li Guoqing and Zeng Yi proposed a hierarchical convolution feature in image registration by studying the main processes of image registration and point transformation [22]. First, the convolution neural network is used to adaptively extract the hierarchical convolution features of feature points, and then the filter is used to analyze their correlation to see their correlation and calculate the location of feature points. Finally, the registration feature points are used to calculate the changed parameters for remote sensing image stitching. Compared with the traditional scale invariant feature transformation method, when convolution neural network is introduced into image registration, the accuracy and robustness of remote sensing image stitching are higher.
In 2017, Han Jie, Li Shengyang and Zhang Tao applied to remote sensing image classification and urban development analysis because of the high accuracy of deep learning and the lower misjudgment rate than traditional methods [23]. Compared with other methods, the correct rate is much higher, which provides a technical reference for urban planning.
In 2017, solemnly, Zhang Jingdong and du Jianhua proposed to apply the deep neural network method to the study of ground collapse in order to solve the problem of poor efficiency of traditional ground collapse identification method [24]. The depth belief network is used to continuously adjust the parameters of the network model, and then compared with the traditional method, it is proved that the depth learning network is more efficient and the recognition accuracy is better.
The application of depth learning to remote sensing image processing has been very successful, but the road extraction of high resolution remote sensing images is still in the initial stage.
Asemi-automatic method was proposed to extract remote sensing images based on deep learning in G.Cheng, C.Yang, X.Yao , 2015 [25]. They use deep convolutional neural networks to train a large number of road remote sensing images to obtain the directional features of each image, then use artificial methods to obtain the starting point of the road, and use trained neural networks to judge the direction of the road. Finally, the road is extracted from the image by finite state machine. P.K.Li, Y.Zang,C.Wang, etc . designed a neural network with only 3 layers of convolution layer and 1 layer of full connection layer to determine whether each pixel of a remote sensing image slice with a size of 32 x32 is a pixel of the road, and then use line integral convolution to post-process the results obtained by neural network judgment in 2017 [26].
G.L.Cheng, Y.Wang,S.B.Xu proposed a cascade end-to-end convolutional neural network in 2017, which solves the problem that some roads will be blocked by trees and cars when extracting the road centerline on high-resolution remote sensing images [27] . The network is divided into two parts, one is to carry out road detection, the other is cascaded with it to process the results of road detection and extract the center line of the road.
In 2018, Sun Zhenhui, Meng Qingyan, Sun Yunxiao and others proposed a neural network based on pulse coupled neural network to solve the influence of shadow on road extraction when high resolution remote sensing images are proposed [28]. First, it is necessary to eliminate the interference of shadow and water body in the near infrared band, then use neural network to segment the processed image, then use morphological method to eliminate the influence of buildings and vegetation on road extraction. Finally, the method of road extraction is extracted. This method has good effect on dealing with shadow interference, and the extracted urban road information is continuous and complete.
In 2019, Hong M, Guo J, Dai Y and others propose a new FMH model, main road information is extracted from high-resolution remote sensing images in urban areas [29]. Huang F, Yu Y, Feng T et al. considered that the potential of multi-source remote sensing data was not fully exploited, Low -level features are not effectively organized, IDE accuracy is low, Poor automation. In this paper, a new method of automatic extraction of impervious surface based on deep learning and multi-source remote sensing data is proposed, using the minimum noise fraction (MNF) method to de-noise multi-source data, extracting representative features of neighborhood windows using convolutional neural networks (CNN), trained ELM classifiers and MORspeech operations, Distinguish the city's impervious surface from other surfaces, Three methods are compared and selected [30].