Module 1.2: Data Quality-Standards

 Module 1.2: Data Quality-Standards

This weeks module is looking at positional accuracy of road networks. The data for the roads came from Albuquerque and TeleAtlas for Street Map USA. The Albuquerque dataset is considered pretty accurate while the TeleAtlas is distributed by ESRI in ArcGIS. We were supposed to look the different street views to see where they were similar and where they diverted from each other. It was interesting to see the ESRI Street Map USA seemed further away from the "true" data points. One would think the street views that ESRI produces would be very accurate. The way we put the "true" data points in were but doing a close up of the orthoimages to see where the streets seemed to be located. The image below shows data points and street views of both Street Map USA and Albuquerque data sets.

I did 40 data points because it was hard to get the sampling rules of 20% in each quadrant and >10% of diameter apart at the 20 data points. The way I did the accuracy assessment was using the table on Page 5 in the Positional Accuracy Handbook within excel. In order to make sense of the data I switched it to UTMs to get the Easting and Northing in meters. In the handbook we are supposed to see the difference between the test points and the "true" points of both the x and y coordinates. Also, we had to square both of the x and y coordinate differences. Then we would add together the squared differences and then add it all up to get the sum. After that we would get the average which is the sum/number of points. We would get the RMSE which is the average^1/2. Lastly, we would get the NSSDA which is 1.7308 * RMSE.

Once we got all that information we needed to write our final accuracy statement. This statement does change between a tested and complied to meet. The tested statement is used when the accuracy was determined by comparison with an independent data set of greater accuracy (Positional Accuracy Handbook, pp. 5). The compiled to meet statement is used when the data has been thoroughly tested and that method produces a consistent accuracy statistic (Positional Accuracy Handbook, pp. 5).  Based on the way these points were collected we would use the test statement. For the Streets Map USA the statement is Tested 306.9 meters horizontal accuracy at 95% confidence level and the Albuquerque Streets is Tested 16.7 meters horizontal accuracy at 95% confidence level.

By creating the different data points it helps know the positional accuracy of your shapefiles. It is important to try to get a higher accuracy data set to compare. After doing all the data points and getting the math correct it seems that the Albuquerque data set is more accurate than the Street Map USA data set. This is interesting because the Street Map USA dataset is used within ESRI ArcGIS and one would think that is more accurate then other datasets. 

Blog Post #1

 Blog Post #1

The internship program that I am doing is for the U.S. Army Corps of Engineers (USACE) as a GIS Analyst and Archaeologist. The way I plan to earn this credit is by creating an archaeological geodatabase for the Planning and Environmental Division. Create maps for different divisions and/or projects within the USACE, Mobile District. Create maps for reports and letters for the division. Do GIS analysis of LiDAR data and coastal flooding for the division. Lastly, conduct remote sensing for projects requiring quantifiable impacts analysis.

The GIS user group I chose was GIS Association of Alabama (GISAA). This group is a non-profit group that covers the whole state of Alabama. The focus is on geospatial professionals within the private and public sectors. The reason I have chosen this group is because the region I mostly focus on for my work is the state of Alabama. It seems they are able to connect a lot of different professionals together and look at different ways to use GIS data. Membership is Individual - Professional Membership - $50.00 (USD) and Individual - Student Membership - $10.00 (USD) for one year. 

The website is: https://gisaa.org/

Module 1.1: Calculating Metrics for Spatial Data Quality

 Module 1.1: Calculating Metrics for Spatial Data Quality

In this weeks lab we looked at data that was collected by GPS coordinates to see the precision and accuracy of that data. This is important to know because when using data you need to make sure you are paying attention to what is being given to you and not just accepting it. It is important to note that when looking at the data one needs to aggregate data for both precision and accuracy. 

In the first portion of this module we took the data points and made an average data point. This is necessary because it helps show where the data should be compared to the rest of the area. Then we needed to project the waypoints and the average waypoint to the correct projecting so the data could be used in feet or meters better than decimal degrees. Having a lot of data it is important to do the Batch geoprocessing tool. This makes sure to get all your data in the correct projection without doing it one at a time. Also, you can do multiple feature classes to make sure all the data is projected correctly. Next we need to use the Spatial Join tool to help with the distance that falls within the different percentages (50th, 68th, and 95th). This makes sure your data is correct without having to guess the different buffer zones and is much more accurate than a visual inspection. The map below is a visual representation of the horizontal accuracy and precision of the waypoints that were given from the GPS unit.

After getting all the data to make the map the next step was getting the horizontal accuracy. There was a reference point that was put in the data to help measure the distance between the average location of the waypoints and the reference point. Then we measured the distance in meters to see how far apart they were from each other to see the accuracy of the point. The following numbers were the numerical results of the horizontal accuracy and precision. The horizontal accuracy is low accuracy because everything is slightly offset from the reference point. This compares to the horizontal precision which is high because it is only 1.6 meters off from the reference point. Lastly, the way that horizontal accuracy is measured by using the average point and comparing it to the reference point that was taken in the field. On the other hand, the way the horizontal precision is measured by looking for the standard deviation of the points such as the different percentiles that were shown in the map above.

Module 6: Corridor Analysis

 Module 6: Corridor Analysis

In this weeks module we learned how to do corridor analysis, which considers a range of possible paths instead of a single optimal path. This is useful because the least-cost path is not always the most optimal path for what a person needs for the area. In scenario 4 we once again needed to do the conceptual steps before starting out analysis.

Step 1: State the Problem

Problem: Potential movement of black bears between the two protected national forest areas.

Goal: Create a suitability and corridor analysis of the black bear movement between the national forests.

Step 2: Break Down the Problem

Objective 1: Reclassify the roads into suitability classes.

Input dataset: Roads shapefile

Objective 2: Reclassify the slopes into suitability classes.

Input dataset: Elevation raster

Objective 3: Reclassify the Land cover into suitability classes.

Input dataset: Land cover raster

Objective 4: Invert suitability model

Input dataset: The reclassification models of the roads, elevation, and land cover.

Objective 5: Create a corridor analysis

Input: The cost path inputs for the two national forests.

Step 3: Explore Input Datasets

Once again this is from Model Builder within ArcGIS Pro which is helpful because it can do all the data for you without doing each step separately. Also, it lets you work through the problem before beginning the work. The image below is my flowchart for this scenario.

When I did the analysis, I did the corridor analysis. I made sure to get the least-cost path for both national forest polygons. First, I did the cost distance and cost path for both of the polygons. After changing the least-cost path to a polyline I then did the corridor analysis from Scenario 3. I used the corridor tool where I put in the two cost distance surfaces and inputs. There is no graduated color ramp, so I went to classify and switched it to three breaks and added in the threshold for each one. Similar to scenario 3 I times the minimum value by 1.05, 1.1, and 1.15. This showed the different areas and how they were less optimal as you went out. The image below shows the corridor analysis of the best path between the two national forests based on the criteria that was given to us in the beginning of the distance to the roads, elevation, and land cover.

When thinking about the data after the initial analysis there are more factors that I would see if we could get more information. First, I would possibly see if there were any criteria for the rivers in the area and what the black bears might want to be close to those areas as well. Another factor would be if there were any developments in the area that could disturb the bears and where they might want to go with the corridor analysis.

This scenario did have a couple of hiccups on trying to get the data to represent the best paths. Even though I did all the analysis correct I was only able to get one corridor analysis path. I tried different ways but it was only giving me one route each time. I am not sure the best way to fix this, but when doing the exercise it did say the Cost Distance, Cost Path, and Corridor Tools were being taken out and replaced with a newer one. This might help with getting the correct analysis in the future. At the same time, it is important to note that when doing the analysis it is important to take your time and try to understand what you are doing for each step. Lastly, make sure to name your outputs to where you can see which features are being represented and not to use the same name within the same geodatabase.

Module 6: Suitability Analysis

 Module 6: Suitability Analysis

In this weeks module we first focused on suitability analysis maps for a potential developer in Jackson County, Oregon. We needed to state the problem, break down the problem, and create a flowchart/Model Builder for this project. 

Step 1: State the Problem

Problem: How much land is suitable to build on.

Goal: Create a suitability map in raster form to find the areas that are suitable to build on.

Step 2: Break Down the Problem

Objective 1: Reclassify the land cover into suitability classes.

            Input dataset: Landcover raster

Objective 2: Reclassify the soils into suitability classes.

            Input dataset: Soils shapefile

Objective 3: Reclassify the slopes into suitability classes.

            Input dataset: Elevation raster

Objective 4: Reclassify the rivers into suitability classes.

            Input dataset: Rivers shapefile

Objective 5: Reclassify the roads into suitability classes.

            Input dataset: Roads shapefile

Step 3: Explore Input Datasets

The image below is the model builder for this scenario. These are the steps I took to finish this scenario.

Doing the three steps are helpful because you can work through the issues of the project before starting the analysis. Also, when using the model builder you can use it to create all your data at once instead of doing each step separately. We created a map with a side-by-side comparison of how the data would look if you had equal weight with all the information and if you changed the weight depending on the feature. This is helpful because it give you different scenarios of what it least to most suitable in your dataset. In the image below it shows that least suitable areas are on mountain tops and most suitable are by a river with equal weight. When you change to an unequal weight it is still the same but the kilometers square changes to being higher in group two. 

Looking at the information is able to give a better idea to the developer on the areas that seem the most suitable and the kilometers square of what they might be able to use in the areas. 

Module 5: Damage Assessment

 Module 5: Damage Assessment

This weeks module was about how to assess damage to structures after Hurricane Sandy. We were supposed to use FEMA "standards" to get the information they might need to help with relief.

The first part was to make a map that showed the tract of Hurricane Sandy from the Atlantic Ocean to the eastern coast. We learned how to create a line from points that were create from a XY Table. The geoprocessing tool is Points to Line (Data Management Tools) and it showed the path of Hurricane Sandy. The next part of the module we needed to change the symbology of the points so they would represent the different phases of a hurricane. The way we did this was to use Creating Marker Symbols because it makes it possible to make the points into a good symbology. The way to do this is to click on the symbology tab to change the different values into the correct symbols. We were given the exact ones to select for the points but under the properties tab then go to the layers menu. Under the Appearance tab choose Font. This will give you more options to change your symbology. The image below shows the final outcome of the path of Hurricane Sandy.

The next portion of this module we needed to create a survey on the Survey123 in ArcGIS Pro Online. This is useful to learn because you are able to send this out to the public and they can help you get data after a tragic event such as a hurricane. This survey was for the damage assessment for Hurricane Sandy. You are able to make so many different questions that have to be required in order to go on with the rest of the survey. Also, another important aspect is that you can add images and the exact date/time it was taken. Lastly, the survey can have a map put in and there is a point added when they open this portion of the survey. The link is where you can look at the survey for instructional view only: https://arcg.is/1zTePH1.

In the final portion of the module was getting the images for pre-and post-Sandy, adding domains, and analyzing the data for the "FEMA" client. We were giving the images that were going to be used for this exercise but when getting your own data being able to get good imagery data is important to assess any damage. First we needed to create a Mosaic Dataset and then add the raster data to the dataset. Having both of the images on top of each other is helpful because you can keep going back and forth to see where the damage is located. Next we needed to add domains so we can have it for attribute values for the points that we are going to use to assess the damage from Hurricane Sandy. You want to right click you geodatabase and choose Domains. It will open up a table similar to an attribute table. We put in the domains we wanted for this assessment, which were inundation, structure damage, wind damage, and structure type. After putting in the domain you want to add codes/descriptions that are going to be used when trying to interpret the data. The image below is a look to the different domains with the view of the code/descriptions of the structure damage.

After creating the domains and the codes/descriptions we created a new feature class that has the different fields with the domains. Even though the directions say "No" on the Allow Null portion you do need to say yes in order to actually create the features. In this portion we needed to classify the structure damage. When trying to identify the properties I was looking at the pre-storm image to see if the structure looked like people would be vacationing or living there versus a commercial place. The scale that worked the best for me was between 1:500 and 1:250. I was trying to see the surrounding area to see if the structure was affected or there was no damage. The decisions that were the most difficult were trying to decide the line of when the structure is inundated or not. There is a lot of sand around but in these areas many of the structures are higher from the ground so they could be perfectly fine. Another difficult decision was trying to decide wind damage because this is an image it makes it hard to know what was wind and what was water. I would say I would like another image that was at the same time as the previous image because it made it hard to know what was wrong with the structure and what was a shadow. Also, I think we should add another structure type of commercial because I was having a hard problem with wanting to put unknown because I knew it was commercial and not industrial. A last piece of information would be the structure height to know if the structure was inundated or not from the water and sand. The image below is a view of the different points I put on each structure.

Lastly, we needed to take the data we created an analysis the data to understand what it happening post-hurricane sandy. To get the information I needed to know the distance was by using the Near geoprocessing tool where I could get the distance of the structures from the coastline. It did show that there were four structures that were past the 300-meter radius. Also, to get more structures I did have to change my coastline a little because it was not showing the structures. Lastly, I looked at the near distance portion of the attribute table and saw that most of the structures were 92-meters and above. This is interesting because if you focus on just the distance you might not get the information for all the structures. It looks like the closer to the coastline the more likely the structure is either destroyed or has some major damage. At the same time, further away from the coastline it does look like the structures were more affected by the hurricane. I would say it is reliable because I kept the same thought process while I was analyzing the data. The only problem I had was only being able to see the top of structure I was having hard time telling how much was damaged. I based the decision on the sand a debris surrounding the structure.

Module 4: Coastal Flooding

 Module 4: Coastal Flooding

In this weeks module we were focusing on how to use DEM and LiDAR data to see storm surges within 1-meter. This module was more difficult than other ones because you had to do a lot of geoprocessing before being able to do analysis. I did have to rely on past modules to help me get the information I needed for the module.

We were given two data types which were dem_lidar and dem_usgs. The dem_lidar is a high-resolution elevation model for a portion of Collier County, FL created using Lidar data. The dem_usgs is a regular elevation model from the USGS created using traditional methods (i.e. photogrammetry) before Lidar data were available. The first thing we needed to do was reclassify the values to where it showed 1-meter or less within the values. After that we needed to use the raster to polygon geoprocessing tool so that we could use the spatial tool to get the information of how many buildings were disturbed by the storm surge. Being able to get this information is important because it shows how each of the data sets interpreted their storm surge. After creating the polygons I got the select by attribute table and got the join_count to get the selected buildings within each data set. This was more helpful for me to do so that I could focus on just the buildings I needed for the table. Also, when making the map having them as their own layer it made it easier to change the colors. The last aspect we needed for the map was to figure out the error of omission and error of commission. Errors of omission are those buildings impacted by the storm surge based on the Lidar DEM (the “true” scenario) but not identified as such based on the USGS DEM. Errors of commission are the buildings not impacted by the storm surge based on the Lidar DEM but incorrectly identified as such based on the USGS DEM. The image below is how the different buildings looked based off if they were within both the data sets, in one data set, or not in a data set at all. 

As stated earlier this module was one of the most difficult because you needed to not only change from feet to meters but use a lot of geoprocessing tools in order to do any analysis.