Evaluation of the Accuracy of SRTM3 and ASTER GDEM in the Tibetan Plateau Mountain Ranges

Topographic data on The Tibetan Plateau (TP) terrain are fundamental for geoscientific research, but are difficult to obtain. The Shuttle Radar Topographic Mission (SRTM) Digital Elevation Model (DEM) and the Advanced Spaceborne Thermal Emission and Reflection Radiometer Global Digital Elevation Model (ASTER GDEM) are two commonly used GDEM data. Verifying the accuracy of the two dataset for the TP mountain areas provides a reference point for the application of both DEMs. For evaluating the elevation accuracy and topographic information, we used 8242 field measurements from Differential Global Positioning System (DGPS) points and DEM data generated from 1:100,000 topographic maps to examine the accuracy of ASTER GDEM V2 and SRTM3 V4.1 elevation results. The average RMSE for elevation differences between DGPS and ASTER GDEM across the study areas was 18.56m,while the average RMSE between DGPS and SRTM3 was 10.39m. The average RMSEs of ASTER GDEM and SRTM3 in glaciated areas were 8.55m and 5.87m, respectively. The vertical accuracy of SRTM3 is better than that of ASTER GDEM. The vertical accuracy of both DEMs do not vary with altitude, but is related to aspect and slope.


Introduction
Digital Elevation Model (DEM) produced based on remotely sensed data provide useful terrain data sources for Earth and environmental scientific research, particularly for the remote and vast Tibetan Plateau (TP). Two free global DEMs, one from the Shuttle Radar Topographic Mission (SRTM) and another from the Advanced Spaceborne Thermal Emission and Reflection Radiometer Global Digital Elevation Model (ASTER GDEM), cover the majority of the world's populated regions. They have been widely used in geology, geomorphology, hydrology, and glaciology [1][2][3][4]. Multiple studies have evaluated the accuracy of the data produced by SRTM and ASTER GDEM [5][6][7][8][9][10][11][12][13]. Several data have been adopted to assess their accuracy in practical use. The elevations measured by the Global Positioning System (GPS) have commonly been used as reference data to be compared to DEMs to assess their accuracies [13][14][15]. Topographic maps [6,8,12], the Ice, Cloud and Land Elevation Satellite (ICEsat) data [10,16,17], geodetic ground control points [7] or Light Detection and Ranging data [9,11] have also provided such reference data for assessing the accuracy of DEM data.
The ASTER GDEM and SRTM3 DEM provide valuable terrain information for TP, which lacks high resolution topographic data due to the difficulties in access for directly field survey. Researchers have already used these DEMs to monitor glacial changes in the TP [18], to analyse earthquakes [19], to model hydrological processes [20] and glacial lake outbursts [13], and to map glacial landscapes [21]. However, most of these studies did not assess the accuracy of the data used. Although some recharges evaluated the accuracy of these two DEMs on the TP [10,13], the study sites were constrained in two small areas and the sample numbers were limited for a proper assessment of the accuracy of the models over such broad area. Wan et al. and Gao et al. evaluated the accuracy of SRTM by using ICESat/GLAS data in the TP [17,22], however, there is a lack of verification of the accuracy of DEM data in glacier regions and assessment of topographic information. Therefore, in this paper, we assess the accuracies of ASTER GDEM V2 and SRTM3 V4.1 data by comparing them with DGPS measurements and the DEM generated from 1:100,000 topographic maps (DEM10), with the aim of furthering the use of DEMs on the TP. by vast basins and plateaus (Fig. 1), with the Himalaya in the south, the Kunlun Mountains and Qilian Mountains in the north, the Karakoram Mountains in the west, and the Hengduan Mountains in the east. These mountain ranges cover a significant area of the TP. A to F are the study areas.
GPS elevation data were collected in six, mostly mountainous, areas (A to F) (Fig. 1). Area A is located in the Qilian Mountains in northeastern TP and contains six sites with 1492 DGPS points. One of the sites A1 is named after Laolongwan, which is located in southeastern Qilian Mountains and has diverse landforms (Fig. 7). Area B is located in the Nyainqêntanglha Mountains in southeastern Tibet and contains 13 sites with 3941 GPS points. One of the sites B4 is named after Yangbajing, which is located in glacial area (Fig. 5). Area C is in the Himalaya on the southern border of Tibet and contains three sites with 1335 GPS points. Area D is in the Puruogangri Ice Field to the east of the Qiangtang Plateau and contains one site with 1397 DGPS points. Area E is in Gila in the centre of the Kailas Range and contains 56 GPS points. Finally, Area F is in the valley of the Midui Glacier in the eastern Himalaya and contains 21 GPS points ( Fig. 1).

SRTM3
In February 2000, SRTM DEM was released by the US National Aeronautics and Space Administration (NASA) and the National Geospatial-Intelligence Agency (NGA) in order to map the world in three dimensions. It covers >80% of the land surface of the Earth, from 60°N to 56°S [23]. SRTM3 DEM (SRTM3, hereafter) data with a 3″, ca. 90 m resolutions, are globally available. Global vertical accuracy for SRTM is ±16m at a 90% confidence level [24,25]. We used the latest version of the SRTM3 (ver. 4.1.) in this study.

ASTER GDEM
ASTER GDEM (GDEM, hereafter) is a global DEM dataset that was released in June 2009 by the Ministry of Economy, Trade and Industry of Japan (METI) and NASA. It covers 99% of the Earth's land surface from 83°N to 83°S. Its horizontal resolution is ca. 30m with a vertical accuracy of GDEM±20m [26]. In this study, we evaluated the latest version of ASTER GDEM (ver. 2), released in October 2011.

DEM derived from topographic maps
The map-based DEM (DEM10, hereafter) was generated from the 1:100,000-scale topographic map which was produced on the basis of 1974 aerial photogrammetry. The topographic maps were scanned into digitized images and a contour interval of 40m was applied. The digitized contours were then used to generate DEM data, also with a 40m resolution, which was then projected using the Albers projection and the Krasovsky 1940 ellipsoid. DEM10 was compared to the other two DEMs to determine the accuracy of their topographic information.

DGPS elevations
The DGPS points were obtained by using two or more portable THALES MobileMapper units. One was used as the reference station that provided differential correction. The other(s) were used to collect DGPS points for use as mobile stations. The horizontal accuracy of the DGPS points is within 1m, based on the User's Manual; estimation of the vertical accuracy of the points indicated that it is within 3m. Since the coordinates of the base station in the field are not absolutely accurate, we evaluated the vertical accuracy of the DGPS points by using DGPS to measure the elevation at the top of Xiangshan Hill, whose altitude is known. Thus, we calculated the vertical accuracy falling within a range of 10m when the coordinates of the base station were measured in field. A total of 8242 DGPS points were randomly extracted from all field measurements, the locations of which are indicated in Fig. 1. These DGPS elevation data were used to evaluate the vertical accuracies of SRTM3 and GDEM. Table 1 shows the characteristics of the four datasets for the study areas.  [27]. This difference can be ignored since we used DEM10 only to evaluate the topographic information contained in three DEMs, and thus did not pay close attention to its height. The elevations were extracted from GDEM and SRTM3 using the same coordinates as where the DGPS elevations were measured. The elevations retrieved from GDEM and SRTM3 are referred to as GDEM and SRTM3, respectively. Shaded reliefs were generated from these DEMs in order to compare their visual effects. Slope and aspect were computed from these DEMs.

Comparison of elevations between DEMs and DGPS
The elevation differences between the DEMs and the DGPS were used as reference data to evaluate errors in the vertical coordinates of the two DEMs ( Fig. 2 and Table 2). The following abbreviations are used hereafter: 'min.' for minimum vertical error, 'max.' for maximum vertical error, and 'mean' for mean vertical error. MAD refers to mean absolute deviation and RMSE refers to the root mean square error. The elevation differences between the two reference data and these two DEMs were computed and are presented in Fig. 2. Their correlations with elevation, aspect and slope are analysed ( Fig. 3 and 4, and Table 3). We are also interested in the performance of the two DEMs in glacier area as they are the two major topographic data for studying glaciers in the TP, so we selected a valley glacier in Yangbajing to further assess the performances of the two DEMs in glacier area ( Fig. 5  and 6). Given that the diversity of geomorphic features may influence accuracy, the DGPS data from Laolongwan, which has different landforms including floodplain, ridges and valleys with gentle and steep slopes, were also selected to evaluate how landform type affects the accuracy of the DEMs (Fig. 7 and Table 4).

Topographic information assessment
DEM10 was used as a reference to assess the topographic information from GDEM and SRTM3 based on visual comparison and slope analysis. Hill shaded images were produced from three DEMs and then displayed two-and three-dimensionally for visual comparison (Fig. 8). Slope is an important parameter of any DEM, and higher resolution DEMs tend to show higher slope values [28]. We classified the slopes into five levels of which the distributions were counted for the three DEMs. The mean slope was also calculated in order to analyse the topographic information provided by the DEMs (Table 5).

Assessment of vertical accuracy for GDEM and SRTM3
Elevation differences between DGPS and the two DEMs, SRTM3 and GDEM, both display a normal distribution (Fig. 2), and most of the differences fall within ±40m. The range of differences are smaller for SRTM3 than for GDEM. In most cases, the differences with SRTM3 are lower than those with GDEM. This implies that SRTM3 values are closer to DGPS values than are those rendered by GDEM. Five variables (count, min., max., mean and RMSE) were calculated for 25 sample sites within the six major study areas, as summarised in Table 2. The differences between GDEM and DGPS range from -102.99m to 165.73m, with an average value of 4.1m; RMSE values range from 7.65m to 40.09m, with an average of 18.6m. The differences between SRTM3 and DGPS range from -80.0m to 60.6m, with an average of -0.8m, and RMSE values range from 5.52m to 33.89m, with an average of 10.4m. The greatest differences occur on valley floors and sharp moraine ridges, due to the coarse cell size of the DEM, which obscure elevation changes in areas of high relief. Therefore, the accuracies of GDEM and SRTM3 are all within the nominal accuracies of such global models (20m for the GDEM and 16m for SRTM3), and SRTM3 has a higher vertical accuracy than GDEM, which is consistent with the findings of Gao et al. (2019) concerning SRTM3 and GDEM [17].

5.1.2The effects of topographic variables (elevation, slope and aspect) on the accuracy of DEMs
Some previous studies have demonstrated that topographic characteristics, such as elevation, slope and aspect, affect the accuracy of DEMs [14, 29 and 30]. However, our study doesn't show clear relationship between elevation differences and elevation (Fig. 3), indicating that elevation has little effect on vertical accuracy in the TP. This result is consistent with the ASTER GDEM validation report (2009) and the findings of Gorokhovich and Voustianiouk (2006) concerning SRTM. The statistical differences of the elevation differences for different surface slope classes highlight that surface slope is an important factor affecting the vertical accuracy of the DEMs (Table 3). Generally, RMSR and MAD of the elevation differences for both DEMs increase with slope steepness, indicating that the vertical error of the DEMs increases with slope steepness, and is less significant in SRTM3 than in GDEM. This result is consistent with that obtained by Racoviteanu et al. (2007) and Gao et al. (2019) [29,17]. All GDEM mean vertical errors are above zero, while most of the SRTM3 mean vertical errors are below zero. Thus suggests that the elevations obtained from GDEM are generally higher, while elevations obtained from SRTM3 are lower than DGPS measurements (Table 3). It's also noted that both RMSE and MAD are much larger for GDEM than for SRTM3, meaning smaller deviations of SRTM3 from DGPS than GDEM for all slope classes. Surface aspect is also an important factor controlling the vertical accuracy of the DEMs, as shown in Fig. 4. The RMSE of GDEM minus DGPS is smaller for N-S facing aspects and greater for those facing NW-SE. It ranges from 16.47m on N-facing aspects to 19.82m on those facing NE (Fig. 4(a)); these RMSE values are greater than those for SRTM3, which range from 8.87m on W-facing aspects to 12.20m on those facing SE ( Fig.  4(b)). The RMSE of the SRTM3 tends to be greater in aspects of N, NE, E and SE, and smaller in aspects of NW, W, SW and S. This is probably related to the flight direction of the shuttle radar. The orbital inclination of the space shuttle is 57° [31]. Radar shadows are formed on the back slope of the mountain. There is no radar echo signal in the shadow area, which seriously affects the quality of DEM acquisition by interferometric radar [32], thus the RMSE of the SRTM3 are directional.

Assessment of DEM vertical accuracy in a glacial area
Both DEMs of Yangbajing were used to evaluate the vertical accuracy of the DEMs in glacial area by comparing to DGPS measurements of a glacier surface (Fig. 5). The range of GDEM errors is -16.34m -22.65m, with a mean error of -1.43m, and a RMSE of 8.55m. The range of SRTM3 errors is -16.42m -6.74m, with a mean error of -1.39m, and a RMSE of 5.87m. Both DEMs are within the officially-stated accuracy range, and vertical accuracy is better for SRTM3 than for GDEM. Elevations in SRTM3 tend to be lower than those in DGPS. This is probably due to the C-band of SRTM's penetration capability into the glacier [33]. Height differences between SRTM3 and DGPS tend to increase with elevation (Fig. 6), indicating that SRTM3 errors increase with elevation in glacial area.

Assessment of DEM vertical accuracy for Laolongwan
In order to evaluate the vertical accuracy for different types of landform, the data from Laolongwan Valley were selected for analysis (Fig. 7). Table 4 shows that the accuracy of SRTM3 is higher than GDEM when presenting the actual elevations of mountain ridges and valleys, differing from the results obtained by Hayakawa (2008) [6]. Considering all the samples from Laolongwan, GDEM displays better elevation accuracy on mountain ridges but poorer elevation accuracy in mountain valleys, while SRTM3 presents elevation poorly for both mountain ridges and valleys. These contrasts probably result from mountains blocking the satellite signals in valleys, and, as SRTM3 has a larger raster size, the elevation changes on mountain ridges are obscured. The vertical errors produced by GDEM and SRTM3 increase with slope, which is consistent with the results we obtained in all the study regions. It is common to use DGPS measurements to assess the accuracy of remote sensing-derived DEMS (RS-DEMs). However, an elevation value for RS-DEMs is only an average value of true elevations that fall within each pixel [34,35], whereas a DGPS elevation value is simply a value in a point. This means we compare a very local elevation value with a value that represents the average elevation within a larger area. Any assessment of the vertical accuracy of RS-DEMs thus presents only an average vertical accuracy at the DGPS point, reflecting the macroscopic accuracy of RS-DEMs. Theoretically, the uncertainty of an elevation value of a point obtained from SRTM3 is higher than that from GDEM, considering the coarser resolution of SRTM3 than GDEM. However, the statistical results in this study suggest that SRTM3 has a higher accuracy than GDEM in the region of TP, assuming that the DGPS measurements represent the true elevations. Fig. 8 illustrates the visual effect produced by the three DEMs. GDEM reflects the rough surface of the terrain, there are many small fluctuations which reflect the detailed information of the terrain. Compared with GDEM, DEM10 also clearly shows the main terrain characteristics, but it is not smooth and with some contour texture.DEM10 is derived from topographic map and some information on the ground is ignored in the process of generating DEM. SRTM3 looks vague and does not reflect detailed terrain information. SRTM3 loses some terrain details because it is resampled to 90m from SRTM1 (30m).Comparing the three DEMs, GDEM shows visual effects than other two DEMs, and DEM10 performances better than SRTM3.

Slope analysis
In accordance with the result obtained by other studies [28,36], Table 5 shows that GDEM has the steepest mean slope (15.85°) and dem10 and SRTM3 have similar mean slopes, which are 13.93°and 13.46°, respectively, indicating that DEMs with a higher resolution generally present greater apparent gradients. Steep slopes (>15°) are more common in GDEM, whereas gentle slopes (0°~9°) are more frequent in DEM10 and SRTM3, implying that, compared to DEM10 and SRTM3, GDEM could better represent the topographic characteristics of the mountainous region with high relief. SRTM3 has the highest proportion in the slope range of 3°~9°. DEM10 has higher proportions than SRTM3 in the other two slope ranges (<3° and ≥ 25°), thus the slope of SRTM3 tend to be more median than that in DEM10. A possible reason is that SRTM3 loses some terrain details (too steep or too flat terrain information) when resampled to 90m from SRTM1 (30m). In conclusion, form visual comparisons and slope analysis, GDEM could better reflect the details information topographic characteristics than DEM10 and SRTM3 and DEM10 perform better that SRTM3, which is in accordance with the result obtained by other studies [28,36] that DEMs with a higher resolution generally reflect more detailed local terrain changes than those with a lower resolution.
The detailed topographic information reflected by three DEMs is related to the way of data generation. GDEM is an automatic generation of satellite images with a resolution of 15 meters, in which more original satellite image information is retained [26]; DEM10 is generated from digitized 1:100,000 topographic maps produced from aerial photographs. This process generating topographic map from aerial photo called cartographic generalization could lose a lot of detailed information and obscure presentation of local topographic fluctuations. SRTM data is derived from Radar image with data points posted every 1 arc-second, approximately 30 meters. When SRTM3 was resampled from SRTM1 [24], detailed topographic information could have been largely ignored.

Conclusion
Our assessment demonstrates that both ASTER GDEM (mean RMSE 18.6m) and SRTM3 (mean RMSE 10.4m) lie within their vertical nominal accuracies of 20m and 16m, respectively. The vertical accuracy of DEMs decreases with terrain slope and varies in aspect, but does not respond to elevation change. In general, SRTM3 has higher accuracies than ASTER GDEM in the TP, while the latter is more accurate for mountain ridges than valleys and the accuracy of SRTM3 is poor for both mountain ridges and valleys. In glacial areas, the RMSE of ASTER GDEM and SRTM3 are 8.55m and 5.87m, respectively. Elevation values tend to be higher for ASTER GDEM and lower for SRTM3 compared to those derived from DGPS. In the aspect of topographic information, the performance of ASTER GDEM is the best, followed by DEM generated from 1:100,000 topographic map and SRTM3 is the last.