UAS and Lidar Data: Comparison, Fusion, and Analysis

Center for Geospatial Analytics at North Carolina State University

Corey White adapted from Helena Mitasova, Anna Petrasova, and Justyna Jeziorska

Objectives

• Explain why lidar and UAS SfM surfaces of the same site differ
• Evaluate the differences with difference maps and with profiles along stable features
• Separate systematic error, processing distortion, and real change in the landscape
• Patch and fuse DEMs with a fixed and a spatially variable overlap
• Evaluate a fused surface with an overland flow simulation

• 4B built surfaces from the points; today we combine surfaces from two sensors into one
• Same site again: Mid Pines at Lake Wheeler, the 2013 lidar and the June 2015 UAS flights
• The lab (5A) runs this whole sequence: difference maps, a road profile, patching, smooth fusion, and a flow simulation on each result
• Every concept has a GRASS tool next to it; the lab uses them in the same order

Lidar and UAS SfM data

• Section 1: properties of lidar and SfM data, and the motivation for fusion

Point clouds from lidar

• The measured variable is the time of flight of each pulse
• Georeferencing from the platform position (GNSS) and orientation (IMU), fused in the INS
• Each (x, y, z) comes from range, position, and attitude; ground control serves only as a check

• Recap from 4B: an active sensor, a few returns per pulse, about one point per square meter in this 2013 cloud
• Points fall in scan lines; the ground under the trees is sampled
• Accuracy comes from the navigation system, not from targets on the ground

Point clouds from SfM

• The measured variable is reflected light captured as overlapping images
• Georeferencing from GCPs, or from the camera positions when the UAS carries RTK or PPK GNSS
• Each (x, y, z) is reconstructed from matched image features

• Recap from 2A and 3B: a passive sensor, tens to hundreds of points per square meter, color on every point
• The cloud is a reconstruction of what the camera saw: the top of the crop, not the soil under it
• The GCP layout and the camera model decide the accuracy; 4A showed what happens without GCPs

Properties of lidar and SfM point clouds

Airborne lidar

• Penetrates vegetation, several returns per pulse
• Shifts between swaths, the corduroy effect
• Regional coverage, repeated every few years
• High cost per flight, low cost per hectare

UAS SfM

• Very high point density, color on every point
• Cannot see the ground under vegetation
• Distorted by cast shadows and by weak GCP layouts
• Low cost per flight, limited area, flown on demand

The errors and distortions of the two methods are different and independent, so each surface can be used to check the other

• Lidar gives the ground and the wide area, SfM gives the detail and the date of your choice
• Because the error sources are independent, a difference map between them shows the error of whichever surface is worse at that spot
• The lab uses this: lidar ground under the UAS crop

Motivation for fusion

• Low cost DEM updates where the landscape changes fast: construction sites, mines, eroding coasts
• Replace vegetation in a UAS DSM with lidar bare ground
• Watershed analysis when the UAS mapped only part of the watershed
• More accuracy and detail over stable features: buildings, bridges, road surfaces

• The lidar serves as the base map and the UAS as the update
• The second bullet is the lab’s Part 3: the crop in the UAS DSM is replaced by lidar ground so water can flow through it
• The third is why the lab fuses first and simulates flow after: the UAS clip alone does not cover the drainage

Fusion techniques

Approach	Cost
Merge the point clouds, bin or decimate, interpolate once	Heavy computation, one consistent surface
Interpolate each, sample randomly, patch, reinterpolate	Loss of detail, slow
Interpolate each and patch the rasters	Fast, but a sharp edge at the seam
Smooth fusion of the rasters	Fast, no edge; both DEMs must share a resolution

• The first row gives the most consistent surface but is rarely practical with tens of points per square meter or more
• The last row is what this lecture is about: a weighted blend across an overlap zone
• Shared resolution is the precondition; in the lab both surfaces are at 0.3 m

Workflow

1. Evaluate both point clouds and choose a common resolution
2. Interpolate the lidar and the UAS DEMs at that resolution
3. Map and measure the differences between them
4. Identify systematic errors and distortions; correct or exclude them
5. Fuse with a smooth overlap and check the result

• Steps 1 and 2 were 4B; today is steps 3 to 5
• Step 4 is often skipped; fusing a distorted DSM into a good DEM makes both worse
• The check in step 5 is a shaded relief first, and a flow simulation when the surface is for hydrology

Difference maps and profiles

• Section 2: difference maps and profiles; distinguishing error from change

Identifying distortions from processing

Difference map: one surface subtracted from the other cell by cell, r.mapcalc; the pattern tells you whether you are looking at error, distortion, or real change

Lidar DSM minus Agisoft DSM

Lidar DSM minus Pix4D DSM

• Same images, same GCPs, two software packages: the crop fields look alike, the open fields do not
• The cyan in the Pix4D map is a processing distortion, a gentle warp of the whole surface
• Lab Part 1 computes both of these maps and their statistics with the same color rules

Identifying changes in topography

• Vegetation growth or removal: the crop strips, two years apart
• Erosion and deposition: the red gully line
• Custom color rules center the scale on zero so that near-zero differences stay grey

• Not every difference is an error: here the green is sorghum that was not there in 2013
• The red line is a gully, a real change of a few decimeters, visible only because the rest of the field is near zero
• The lab provides a color rules file for this: grey within 10 cm, color beyond

Profiles: fields

• A vertical offset of the whole line is a systematic error
• A local bump is vegetation, or erosion and deposition

• Profiles show what a map does not: a constant offset versus a local change
• March and September lines sit on the lidar; June floats above it where the crop stands
• In the lab the GUI Profile Tool draws this; r.profile gives the numbers

Profiles: a building

• High accuracy in all surfaces, less detail in the lidar at the roof edges
• Negligible systematic error between lidar and UAS

• A stable, hard feature: if the surfaces disagree here, one of them has a problem
• The rounded corners are the one point per square meter lidar sampling, not an error
• Same test the lab does on the road

Profiles: a road

• Roads and parking lots are easy to find and do not grow
• Lidar and UAS SfM agree within 4 cm here, with a mean offset under 1 cm

• This is lab Part 1: r.profile of the difference map along the road, v.in.ascii to a vector, v.univar for the mean
• A mean near zero means no vertical shift; the 4 cm spread is noise in both surfaces
• The lab’s longer transect along the same road gives a mean of about 8 cm for the Agisoft DSM and 16 cm for the Pix4D DSM
• If the mean were 20 cm, you would subtract it before fusing

Correcting systematic errors

Systematic error: a difference that follows a rule over the whole surface, a constant shift or a tilt, as opposed to random noise or local change

• A constant vertical shift (GNSS datum or antenna height): subtract the median difference over stable features
• A tilt (weak GCP geometry): fit a plane or a low-order polynomial to the differences and subtract it
• Complex SfM distortions: interpolate the GCP residuals or the lidar differences and subtract that surface
• Well distributed, accurate GCPs prevent most of this (4A)

• Diagnose first with the profile, then pick the correction that matches the pattern
• Median, not mean: vegetation and outliers would pull the mean
• The lab’s June Agisoft DSM is shifted about 8 cm along the road, small against the differences in the fields, so the lab does not correct it

Patching

• Section 3: patching and the edge it produces

Updating a lidar DEM by patching

Patching: filling the cells of one raster with another where the first has data, r.patch; the first input wins wherever it is not null

A UAS DEM patched into a lidar DEM, elevation and shaded relief (images by Brendan Harmon)

• The elevation map looks fine; the shaded relief shows the seam at once
• Always check a patch in relief: r.relief with an exaggerated zscale finds steps of a few centimeters
• Lab Part 2 does this with the clipped UAS DSM and the lidar DSM

Edge between the patched DEMs

• A step of a few centimeters at the seam, and a change of texture
• Water flow follows the step: an artificial channel along the edge

• Two surfaces that each have a few centimeters of noise will never meet exactly
• The texture change is also real: the UAS sees crop rows the lidar never resolved
• The step matters most for hydrology, as Section 5 shows

Smooth fusion

• Section 4: weighted blending across an overlap

Smooth fusion

Smooth fusion: a weighted average of the two surfaces across an overlap zone along the edge, the weight going from one surface to the other with the distance from the edge; r.patch.smooth in GRASS

Patched

Fused

• Two steps: derive the overlap zone from the distance to the edge, then blend with distance-based weights
• The lab does it by hand first with r.grow.distance and r.mapcalc, then with the addon
• Doing it by hand once shows what the addon’s smooth_dist parameter does

Fusion method

A linear combination of the two elevation surfaces \(z_A\) and \(z_B\), weighted by the distance \(d\) to the edge of \(A\) over an overlap of width \(s\):

\[ z_{AB} = w\, z_A + (1 - w)\, z_B \] \[ w = \begin{cases} d / s & 0 \le d < s \\ 1 & d \ge s \end{cases} \]

• At the edge of A the weight is 0, so the fused surface equals B; s meters inside A it equals A
• In the lab: r.grow.distance -n gives d, then one r.mapcalc expression does the blend: if(distance > s, uas, if(distance == 0, lidar, (1 - distance/s) * lidar + distance/s * uas))
• r.grow.distance -n measures the distance inward from the null boundary, which is what d is here
• The lab runs this with s of 5 m and 20 m and compares the relief

Influence of the overlap width s

Overlap width: how far into the updated surface the blend reaches; too narrow leaves a visible step, too wide erases detail of the better surface

• The difference between the two surfaces at the edge decides the width you need
• With a 3 cm step a few meters is enough; with a 30 cm step you need tens of meters
• The next step lets the elevation difference set s

Fusion with spatially variable overlap

• The width \(s(x, y)\) follows the elevation difference \(\Delta z\) between the two surfaces along the edge
• Wide transition where they disagree, narrow where they agree
• Fine features of both surfaces survive where the difference is small

In GRASS: r.patch.smooth -s input_a=uas_ground input_b=lidar_dem output=fused overlap=overlap transition_angle=1.3 parallel_smoothing=13 (Petrasova et al. 2017)

• The -s flag turns on the variable overlap; transition_angle is the angle of the transition in degrees (1 to 5 recommended), so the width grows with the elevation difference at the edge
• parallel_smoothing is the size of the window that smooths the zone along the edge, an odd number; larger values make the width less variable
• Lab Part 3 runs both variants on the bare ground and looks at the overlap raster

Overlap width from the elevation difference

• Left: where the surfaces disagree by 20 to 30 cm; right: the weight raster the blend uses
• The zone is a few meters wide where the difference is near zero and tens of meters where it is large
• This is the overlap output the lab asks you to display

Patched and fused lidar and UAS DSMs

Patched

Fused, \(s = 20\) m

• Lab Part 2 output: the clip boundary is a rectangle, so the seam is four straight lines
• After fusion the rectangle disappears; the UAS detail inside is untouched beyond 20 m from the edge
• Compare the two in relief, and later in flow

Fusing UAS bare ground with the lidar DEM

UAS cells within 0.3 m of the lidar ground, in yellow

Fused bare ground

• The UAS cannot see the soil under the crop, so the crop cells are set to null and the lidar ground fills them
• The test is the height above the lidar DEM: under 30 cm counts as ground
• The result is a current bare ground surface with UAS detail where the ground was visible

Spatially variable overlap at Mid Pines

• Every crop strip edge gets its own transition zone
• Wide bands where the UAS ground sits a few decimeters off the lidar, narrow bands where they agree
• This is what overlap=overlap writes in Part 3

Overland flow simulation

• Section 5: overland flow simulation as a test of a fused surface

Modeling overland flow at high resolution

• Real depressions are features, not errors to fill
• Water ponds, overtops barriers, and splits: standard flow routing cannot represent this
• A noisy surface needs a robust algorithm that does not chase every cell

Path sampling: a stochastic solution of the flow continuity equations; many walkers are released as rainfall and move downslope, and their density gives the water depth; r.sim.water in GRASS (Mitasova et al. 2004)

• r.watershed routes every cell to an outlet, through depressions; at 0.3 m that removes the ponding the surface really has
• Path sampling keeps the depressions and lets water accumulate in them
• The lab uses r.sim.water with 100,000 walkers over 20 simulated minutes of rain at 30 mm per hour, writing a depth map every minute, on three surfaces

Path sampling

Walkers released over the terrain

Water depth from the walker density

• Each walker carries a bit of water downslope following the gradient, with some randomness
• Where walkers crowd together the depth is high: a stream, a pond
• The method needs the first derivatives, hence r.slope.aspect with dx and dy before the simulation

Flow over a UAS mapped field

The crop surface acts as a barrier: water ponds against it instead of flowing through

• This is a DSM, so the simulation runs water over the top of the sorghum
• Real water runs through the stems along the soil; the DSM is the wrong surface for hydrology
• That is the motivation for fusing lidar ground into the UAS surface

Flow on a patched versus a fused DEM

Patched: an artificial channel along the seam

Fused: the flow follows the terrain

• The seam step of a few centimeters is enough to capture the flow at 0.3 m resolution
• Smooth fusion removes the step and the artificial channel with it
• Same experiment, same site, in the lab’s Part 4

Assignment example: simple and smooth patching

r.patch

Weighted average, \(s = 20\) m

• Left: the blue lines along the clip rectangle are pure artifact
• Right: the artifact is gone, but water still ponds against the crop strips because this is still a DSM
• Compare the depth maps at the last minute of the simulation, at the edge of the UAS area

Assignment example: DSM and bare ground

Smooth patched DSM

Fused bare ground

• With lidar ground under the crop the water follows the terrain to the stream
• This is the surface you would use for erosion or runoff questions on this farm
• The lab’s last part: simulate on all three and explain the differences

Key terms

Difference map: one surface minus the other, cell by cell; the pattern separates error, distortion and change

Systematic error: a shift or tilt that follows a rule over the whole surface, found with profiles over stable features

Patching: filling one raster with another where the first has data; fast, but leaves a seam

Smooth fusion: a distance-weighted blend of two surfaces across an overlap zone along the edge

Overlap width: how far the blend reaches; fixed, or variable with the elevation difference at the edge

Path sampling: a stochastic overland flow simulation that keeps depressions and ponding; the test for a fused surface

• Midterm vocabulary, and the words the lab report has to use correctly
• The grading asks for conclusions in these terms: difference pattern, systematic error, seam, overlap, flow artifact

Wrap-up

Assignment 5A: Fusion of UAS and lidar data: map and measure the differences between the 2013 lidar DSM and two June 2015 UAS DSMs, patch and fuse the UAS DSM into the lidar, build a fused bare ground, and compare the overland flow simulated on each surface

• Recap: lidar and SfM have different error sources, so one checks the other; a difference map and a profile over a road show which
• Patching leaves a seam that water follows; smooth fusion across an overlap removes it
• Let the overlap width follow the difference, and test the result with r.sim.water
• 5B takes the same UAS flights as a time series

References

Mitasova, Helena, Chris Thaxton, Jaroslav Hofierka, Richard McLaughlin, Amber Moore, and Lubos Mitas. 2004. “Path Sampling Method for Modeling Overland Water Flow, Sediment Transport, and Short Term Terrain Evolution in Open Source GIS.” In Computational Methods in Water Resources: Proceedings of the XVth International Conference, vol. 55. Developments in Water Science. Elsevier. https://doi.org/10.1016/S0167-5648(04)80159-X.

Petrasova, Anna, Helena Mitasova, Vaclav Petras, and Justyna Jeziorska. 2017. “Fusion of High-Resolution DEMs for Water Flow Modeling.” Open Geospatial Data, Software and Standards 2 (1): 6. https://doi.org/10.1186/s40965-017-0019-2.