Assignment 3B
Validate the Lake Wheeler flight
Task
Measure the accuracy of the Lake Wheeler flight and decide what it can measure. You will establish the GNSS base coordinate three independent ways, evaluate the flight’s orthomosaic and DSM (processed in both Agisoft Metashape and WebODM) against the checkpoints collected at the field trip, report the accuracy in ASPRS Edition 2 terms (American Society for Photogrammetry and Remote Sensing 2024), and answer four measurement questions with arithmetic.
The checkpoints were withheld from processing, so this is an independent accuracy assessment: the honest kind (Sanz-Ablanedo et al. 2018). On-campus students collected this data with the Emlid Reach RS4 at the flight; distance education students work from the identical files and run the same analysis, including the GNSS post-processing.
Data
All files are from the Fall 2026 flight (September 16) at Lake Wheeler:
- Orthomosaic, Metashape and DSM, Metashape (EPSG:3358)
- Orthomosaic, WebODM and DSM, WebODM (also EPSG:3358; if a download reports a different CRS,
r.importreprojects it on the way in) - Checkpoints: the withheld points, exactly four columns in this order:
name, easting, northing, height(EPSG:3358, orthometric). This is the file the GRASS commands below expect - Checkpoints, full: the complete field export with ellipsoidal heights, occupation metadata, and the repeatability re-occupations; used only in the “Report the accuracy” step
- Base station bundle: base RINEX log, the NCRTN CORS RINEX for the same window, and the Method A and B coordinates recorded in the field
- Processing reports: Metashape and WebODM (PDF); Parts 2 and 3 read numbers out of both
The orthomosaics are large. They are cloud-optimized GeoTIFFs, so for the horizontal digitizing you can stream them without downloading: in QGIS add the URL prefixed with /vsicurl/, or in GRASS pass the same /vsicurl/ URL to r.import. The DSMs and reports are small enough to download directly.
Software
- Emlid Studio (free, Windows and macOS) or RTKLIB (free; the RTKPOST GUI is Windows, the command-line tools run everywhere) for Part 1. In either tool, if the solution stays FLOAT, restrict processing to the middle hour of the log
- GRASS 8.5 (or QGIS) for Parts 2 and 3
Part 1: The base coordinate, three ways
Every checkpoint inherits the base station coordinate, so the survey is only as good as that one position. The field protocol recorded it three ways; you compare them.
- Method A (autonomous average): the five-minute Single-mode average recorded in the field. No correction source. Read it from the field records.
- Method B (NCRTN VRS): the real-time network solution collected before the base started broadcasting. Read it from the field records.
- Method C (static post-processing): compute it yourself. In Emlid Studio, use the Static workflow with the base RINEX as the rover input and the NCRTN CORS file as the base; use the CORS station’s published coordinate as the known base position with an antenna height of 0 m (the CORS values are already at the antenna reference point), enter your base receiver’s antenna height from the field sheet, then process and record the solved coordinate in NAD83(2011). In RTKLIB, the same files go through RTKPOST with positioning mode set to Static; the remaining defaults are fine.
Then:
- Put the three coordinates in one table (easting, northing, ellipsoidal height, and the differences from Method C).
- Method A should sit meters away; Methods B and C should agree within a few centimeters. State which axis disagrees most and why vertical is the weak axis.
- Answer: if the base had broadcast the Method A coordinate all day, what error would every checkpoint carry, and could any later processing remove it?
Part 2: Validate the products
Vertical
Import a DSM and the checkpoints into GRASS and compute the residuals at each checkpoint. The v.* tools below are the vector siblings of the raster tools you used in Topics 2 and 3 and will be new; run the commands one at a time and read what each prints:
# in a GRASS project in EPSG:3358
r.import input=dsm_metashape.tif output=dsm_metashape
v.in.ascii input=checkpoints.csv output=checkpoints separator=comma skip=1 \
x=2 y=3 z=4 columns="name text, e double precision, n double precision, h double precision"
g.region raster=dsm_metashape
v.what.rast map=checkpoints raster=dsm_metashape column=dsm_z
v.db.addcolumn map=checkpoints columns="dz double precision, dz2 double precision"
v.db.update map=checkpoints column=dz query_column="dsm_z - h"
v.db.update map=checkpoints column=dz2 query_column="dz * dz"
v.db.univar map=checkpoints column=dz # mean error: is the surface biased?
v.db.univar map=checkpoints column=dz2 # RMSE_V1 = sqrt of the mean of dz2Repeat for the WebODM DSM. For each product:
- RMSEV1: the square root of the mean of
dz2. - Mean error: from the
dzstatistics. The blunder rule: the standard screens at 3 x the project’s target RMSE; this lab sets no target class, so flag any checkpoint with |dz| beyond 3 x the RMSEV1 you just computed, investigate it, and recompute after excluding confirmed blunders; a mean error above 25 percent of the RMSE points at a systematic bias, not noise (American Society for Photogrammetry and Remote Sensing 2024). - Map the residuals (
v.colorsondz, over the orthomosaic): is the error spatially structured (edges, one corner, a dome) or salt and pepper?
Horizontal
Pick at least 8 photo-identifiable checkpoints, digitize their positions on each orthomosaic (in QGIS: a new point layer with the ortho as basemap; in GRASS: right-click the map in the Layer Manager and choose Digitize), and compute for each the offsets to the surveyed coordinates. RMSEH is the square root of the mean of (de2 + dn2).
Report the accuracy
- RMSEV2 and RMSEH2 (the ruler): estimate the checkpoint survey accuracy from the repeatability re-occupations in the full checkpoint file (the RMS of the differences between independent occupations, divided by sqrt 2).
- Combine in quadrature: reported RMSEV = sqrt(RMSEV12 + RMSEV22), and the same for horizontal.
- Write the accuracy statement in Edition 2 language, including the checkpoint count and the Section 7.16.1 wording for a sample below 30 (American Society for Photogrammetry and Remote Sensing 2024).
Part 3: What can this flight measure?
- Compute the level of detection for a repeat of this flight at the same quality: LoD = 2.8 x your sigma-z (use RMSEV).
- Rule, with numbers, on each question:
- Counting the trees along the field edge
- Measuring canopy height to 0.5 m (Grybas and Congalton 2022)
- Detecting a 30 cm deep gully after a storm
- Quantifying 2 cm of sheet erosion across the field
- Fill in a comparison table for both tools, adapted from the lecture’s “two dialects” table by dropping the overlap and camera model rows and adding processing time: GSD, aligned images and coverage, camera position error, control fit, checkpoint RMSE (yours, from Part 2), and processing time.
- One paragraph: which product would you hand to a client, and what would you change about the flight or the control to halve the LoD (James and Robson 2014; Stott et al. 2020)?
Report
Prepare a report (4 to 6 pages including figures): the base coordinate table and discussion (Part 1); the validation methods, residual maps, and the Edition 2 accuracy statement for both products (Part 2); the LoD arithmetic, the four verdicts, the filled comparison table, and the closing paragraph (Part 3).
Submission
Upload to Moodle, Assignment 3B:
- Your report as a PDF.
- Your checkpoint residual table (CSV export) for both products.
Grading
- Base coordinate: all three methods reported with a correct reading of what the comparison shows.
- Validation: residuals computed correctly, blunder and bias checks applied, quadrature carried, accuracy stated in Edition 2 terms with the small-sample wording.
- Residual maps and table: the residual maps show spatial structure or its absence and are interpreted; the checkpoint residual CSV for both products is included.
- Measurability: the LoD is computed from the student’s own RMSEV and the four verdicts follow from it, not from the lecture’s example.
- Interpretation: the comparison table is filled and the closing paragraph ties changes to mechanisms from the lectures.
- On-time submission.