Building 3D Realistic Modeling Based on Air-ground Multi-source Data Fusion

: Three-dimensional point cloud data is a new form of three-dimensional collection, which not only contains the geometric topology information of the object, but also has high simplicity and flexibility. In this paper, the air-ground multi-source data fusion technology is used to study the fine reconstruction of the 3D scene: based on the 3D laser scanning laser point cloud, the 3D spatial information of the ground visible objects is obtained, and the orthophoto obtained by the drone aerial photography is assisted , Obtain the three-dimensional space information of the top of the ground feature, and the ground three-dimensional laser scanner can quickly obtain the three-dimensional surface information of the building facade, ground, and trees. Due to the complex structure of the building and the occlusion of spatial objects, sub-station scanning is required when acquiring point cloud data. This article uses the Sino-German Energy Conservation Center Building of Shenyang Jianzhu University as the research area, using drone tilt photography technology and ground lidar technology to integrate. During the experiment, the field industry adopted the UAV image acquisition strategy of “automatic shooting of regular routes, supplemented by manual shooting of areas of interest”; in the field industry, the method of “manual coarse registration and ICP algorithm fine registration” The example results show that the ground 3D laser point cloud air-ground image fusion 3D modeling effect proposed in this paper is better and the quality is greatly improved, which makes up for the ground 3D laser scanning. In point cloud modeling, a large number of holes are insufficient due to occlusion and missing top information.


Introduction
As an emerging technology, 3D laser scanning can break through the limitations of traditional single-point measurement methods.It has the advantages of non-contact, high-efficiency, and high-precision acquisition of 3D point cloud data on the surface of the object to construct a true 3D digital model of the scanned object.Provide rich data sources for spatial information databases.This technology has been widely used in the protection and restoration of digital cultural heritage, smart cities, and building information extraction [1] .Using 3D laser scanning technology, you can go deep into the complex scene environment and space for scanning operations, and can directly realize the complete collection of various large, complex, irregular, standard or non-standard entities or real 3D data, and then Quickly reconstruct the three-dimensional model of the entity target and various drawing data such as line, surface, volume and space.Compared with traditional surveying and mapping technology, 3D laser scanning technology has obvious advantages, not only the efficiency of data collection is high, but also the accuracy of the 3D point cloud laser data collected is also high.In view of this, many scholars at home and abroad have carried out in-depth scientific research on how to quickly and effectively obtain spatial three-dimensional information and reconstruct its three-dimensional model.The commonly used methods of 3D model reconstruction are based on photogrammetry and based on 3D laser scanning technology.Among them, the use of a three-dimensional laser scanner to scan a building to obtain three-dimensional point cloud data can quickly rebuild a three-dimensional model of the building, but it cannot do anything about the roof of a taller building.Oblique photogrammetry technology is a new technology developed in the field of surveying, mapping and remote sensing in recent years.With this technology, images of different angles of ground objects can be quickly obtained, and more detailed side information of objects can be obtained [2] , and the data obtained can be more intuitive , Truly reflect the characteristics of actual features [3] .The UAV low-altitude photog-rammetric system has a series of advantages such as flexible maneuverability, small airspace limitation, high efficiency and speed, and low cost, which greatly supplements the shortcomings of traditional aerial survey systems and has significant advantages in obtaining high-resolution images in small areas.The use of drones can fully obtain the rich color and texture information of the building surface, especially the roof and other areas that cannot be scanned by the ground 3D laser scanner.At the same time, due to the development of image matching algorithms, images obtained by drones can generate colored 3D point clouds of buildings, making the 3D reconstruction of buildings more and more convenient and realistic.Therefore, the study of the reconstruction method of 3D building model based on UAV image is of practical significance.

Ancient building point cloud data acquisition
Three-dimensional laser scanning technology is a technical means to quickly obtain spatial three-dimensional information.The use of three-dimensional laser scanners can obtain massive amounts of spatial three-dimensional information with high efficiency, high quality and low cost.The collected data is processed.This technology is used in digital culture.Heritage protection and restoration, smart cities, architecture.Therefore, it has gradually become a common processing object in the fields of three-dimensional reconstruction [4] , medical research [5] , reverse research [6] , and cultural relics restoration [7] .The related point cloud processing technology has also become a hot topic in recent years.The ground 3D laser scanner can quickly obtain the 3D surface information of building facades, ground, trees, etc. Due to the complex structure of the building and the occlusion of spatial objects, sub-station scanning must be performed when point cloud data is acquired, and there must be more than 4 points with the same name between the two stations.This article relies on FARO Focus3D X330 phase-type 3D laser scanner.The layout of the control network of the survey area by the 3D laser scanner should meet the following requirements: (1) The network type of the control network should be appropriate.On the premise of ensuring complete acquisition of point cloud data of features and landforms, unnecessary station setting should be avoided.
(2) The overlapping part of the vision of each adjacent control point in the control network should reach more than 30%.
(3) The position of the control point in the control network should be appropriate, and should be set as far as possible unobstructed and within 50m from the building.(It is worth noting that there are some man-made uncontrollable factors, such as occlusion and light factors such as buildings or vegetation.In this case, the viewing distance or overlap can be adjusted artificially to ensure the integrity of the building) Control point The layout is shown in Figure 1.

Oblique photography data acquisition of ancient buildings
UAVs have the characteristics of low cost, flexible operation, little influence by weather, and suitable for aerial image acquisition in small areas.UAVs can better obtain the information on the top of the building, which can make up for the shortcomings that the ground 3D laser scanner cannot obtain the information on the top of the building.The images of the roof and walls are densely matched with PhotoScan software to generate dense point clouds, the aerial images are preprocessed, and the brightness, saturation, and contrast of the aerial images are adjusted at the same time.Generally speaking, there is no problem if the reprojection error is less than 1; otherwise, aerial triangulation needs to be repeated many times until the error is less than 1.However, in some detail areas, due to the occlusion of ground objects and poor image quality, the facades shot by 3D lasers will have distortions and voids, as shown in Figure 2. Therefore, it is necessary to supplement ground lidar data for this area of interest to compensate for the defects in this area.

Point cloud data processing
For point cloud data, use scene to splice, process, register, delete and export point clouds.Figure 4 is the exported point cloud xyz.format data, and Figure 5 is the exported point cloud model.After the point cloud processing is completed, the next step is to stitch the point clouds.Click the "Registration" option, the system will give priority to automatic registration, which can save the time for registration.Next, select "Manual registration", there are two ways to manually register: "Manual stitching and visual registration".Manual splicing is to splice the two site clouds based on the "same name point" of the overlapping part of the two site clouds.During the process, you need to manually click on the same name point, which requires a lot of work but is more accurate.Visual registration is based on the relative position of the stations on both sides, overlapping the overlapping parts of the stations on both sides, so as to achieve the purpose of splicing.This method has a small workload, but the error is sometimes too large.In the actual manual splicing process, the two methods should be combined with each other, whichever is better.The result of the point cloud data registration in this study is that the point cloud to point cloud error is 1 mm, the registration error between each station is ≤2 mm, and the confidence level is ≥50%, which meets the requirements of ancient building reconstruction

Oblique photography data processing
The tilted image data is subjected to geometric processing such as uniform light and color uniformity on the tilted data image before generating the model.In the aerial triangulation calculation, due to the large inclination of the ancient building tilt data, the large image distortion, the ununified scale, the large number of overlaps, and the need for multi-view processing, the aerial triangulation is different from the conventional digital aerial photogrammetry method.After preprocessing the image, it is necessary to perform aerial triangulation on the image data to determine the position and direction of the image when it was taken, that is, to determine the internal and external orientation elements of the image.The main task of aerial triangulation is to use aerial images to extract feature points and match feature points, and then connect the feature points to calculate external orientation elements and incorporate all images of the survey area into a unified object coordinate system.Through the dense matching of multiple visual images, it can be calculated to generate ultra-high-density point clouds based on real images of oblique photography, and to generate high-resolution real 3D models based on real images.

Multi-source data fusion technology
Data registration is to convert point sets in different spatial coordinate systems to a unified coordinate system, that is, to calculate the rotation transformation relationship of two spatial coordinate systems [8] .The spatial fusion of multi-source data refers to the unification of point clouds, images and other data obtained by different devices under the same coordinate system.Therefore, the conversion between coordinate systems is of great significance for multi-source data fusion processing.Three-dimensional laser scanner systems, aerial photogrammetry systems and camera shooting involve several more important coordinate systems, including world (absolute) coordinate systems, camera coordinate systems, scanner coordinate systems, image coordinate systems and imaging plane coordinate systems.The unification of the coordinate system is the basis for multi-source data fusion.Select the point with the same name, take one of the stations as the base station, and take the three translational parameters of three rotations, where X, Y, Z are the coordinates of the point cloud after point cloud registration, and x, y, z are the original coordinates.
In this paper, the method of manual coarse registration and automatic fine registration is used for point cloud registration.The precise registration uses the ICP point cloud matching algorithm, and the registration effect of the ICP algorithm has a great relationship with the initial position of the point set, so it cannot be directly used for the point cloud precise registration, and other methods need to be used for coarse registration.Better results can be obtained afterwards.The basic principle of ICP is: first obtain the feature points of two point sets, perform data matching according to the feature points, and set these matching points as imaginary corresponding points, then solve the motion parameters according to this correspondence relationship, and finally use these parameters for data Conversion [9][10] .In simple terms, the ICP algorithm is to minimize the objective function S2 through the corresponding points.The calculation formula is as follows Import the data.As shown in Figure 6, the coordinate system of the ground laser radar point cloud data is a relative coordinate system, and the ground laser point cloud data coordinate system needs to be converted to the WGS84 coordinate system to perform fusion modeling with the UAV point cloud data.

Fig 6 Data import
First, a manual registration method is used to register the ground scanning point cloud with the point cloud generated by the drone.As a result of manual point cloud registration (Figure 7), a rotation transformation matrix is obtained, and the transformation accuracy RMS is 1.756.This registration accuracy obviously cannot meet the requirements, and further fine registration is required.

Fig 7 Manual registration result
The ICP algorithm is used for fine registration.Because the sizes of the two point clouds are inconsistent, when using the ICP algorithm, the set overlap degree should be modified accordingly (Figure 8), set the overlap degree to 100%, and the obtained RMS is 0.23 .Figure 9 shows the final registration effect

Test results and analysis
After the calculation and fusion of the algorithm of this research, the missing data of the laser point cloud data model on the building roof, as well as the lacing and hole phenomenon in the inclined photographic data model are all improved.Finally, multi-source data fusion , Figure 9 shows the final registration effect.

Conclusion
This article discusses the 3D reconstruction technology of ancient buildings based on multi-source data fusion.
The test results show that, compared with single point cloud data modeling and air-ground image modeling, the ground 3D laser point cloud air-ground image fusion 3D modeling effect proposed in this paper is better, the quality is greatly improved, and the ground 3D Laser scanning point cloud modeling has a large number of holes due to occlusion and lack of top information.It also overcomes the shortcomings of low modeling accuracy for columns and small parts during image modeling.

Fig 2 .
Fig 2. Aerial image of the drone façade

Fig 3
Fig 3 Building model reconstruction process