UAS Imagery Processing

Center for Geospatial Analytics at North Carolina State University

Justyna Jeziorska, Helena Mitasova & Corey White

Objectives

• Understand the photogrammetric data processing as a multistep process
• Indicate data needed for orthophoto/DTM generation from aerial imagery
• Understand the difference between interior and exterior orientation of the photo
• Describe the workflow of geoprocessing of aerial imagery

Today we’ll walk the full photogrammetric pipeline as a multistep process. By the end, you’ll know exactly what data are needed to produce an orthophoto and DTM/DSM from aerial imagery, the difference between interior and exterior orientation, and how to run the complete workflow in Agisoft Metashape Professional.

Photogrammetric process

This diagram is the big picture of the photogrammety process. We plan our flight, perform collect our data, process our data, and finally we are able to analyze our results.

And today we are going to be focusing on data processing.

So of you might be wondering why we are jumping ahead to data processing if we haven’t done flight planning or our flight yet…that is a good question, and the answer is that in order to better understand flght planning we first need to know how the data will be processed.

Data processing

%% Photogrammetric big-picture workflow
flowchart LR
    A["Pre-Processing"] --> B["Data Processing"]
    B["Data Processing"] --> C["Data Analysis & Export"]

Photogrammetry with SfM include 3 main steps, pre-processing, data processing, and analysis and export. Each step of course concains substeps which together form our full photogrammetric processing pipeline.

Photogrammetric process

%% Photogrammetric big-picture workflow
flowchart LR
  subgraph A["Pre-Processing"]
    IMAGES["(UAS Images)"]
    LOG["(Flight Log / EO)"]
    GCP["(GCPs)"]
  end

  subgraph B["Data Processing"]
    MATCH[Feature Matching]
    SOLVE["Camera Solving
        (SfM / Bundle Adjustment)"]
    DENSE[Dense Point Cloud]
    MESH[Mesh / Surface]
  end

  subgraph C["Data Analysis & Export"]
    ORTHO[Orthophoto]
    DSM[DSM / DTM]
    PCLOUD[Classified Point Cloud]
    TEXMODEL[Textured 3D Model]
    REPORT[Processing Report]
  end

  IMAGES --> MATCH --> SOLVE --> DENSE --> MESH
  SOLVE -. Feedback loop .-> IMAGES
  LOG --> SOLVE
  GCP --> SOLVE
  DENSE --> ORTHO
  DENSE --> DSM
  DENSE --> PCLOUD
  MESH --> TEXMODEL
  SOLVE --> REPORT
  DENSE --> REPORT
  GCP --> REPORT

This diagram is the big picture: images come in, features are matched, cameras are solved, dense geometry is built, cartographic products are exported. Keep this in mind; every button we click in software maps to one of these boxes.

Note the feedback loops: if alignment is weak, we go back, clean images or add GCPs, then re-optimize. This is normal—not a failure—photogrammetry is iterative

UAS data

What do we get after the flight mission?

So what do we get after the flight mission? You come home with three things: images, a flight log (often IMU/GNSS), and ideally GCP measurements. Everything else we’ll compute. The better these inputs, the less cleanup later.

Digital imagery

• usually on the camera SD card
• can be geotagged (depends on camera / UAS setup)
  • Camera lens location is “written into” each photo’s EXIF file
    • this not always the case..

Images usually live on the SD card, if you don’t forget to put it in the UAS before the flight, which is definately not something I’ve done before. Some cameras geotag photos; which is great for initial alignment, but treat tags as rough. We’ll still refine positions during bundle adjustment.

Flight log

• Onboard Inertial Measurement Unit (IMU) accurately measures the orientation of airborne sensors,
• Information is logged into a text file (flight log),
• Contains elements of exterior orientation (EO)

Flight attitude - describes an aircrafts orientation in space (yaw, pitch, roll)

The UAV’s IMU/GNSS writes a text log: attitude, altitude, and time. This provides exterior orientation hints. We’ll import these to seed alignment—then let bundle adjustment tighten them.

GCP coordinates

• Measured by GPS coordinates of the panels set before the flight
• Photo ID points (distinguishable ground features) can be surveyed later on
• It is important to know the GCPs coordinate system (spatial reference system)

GCPs are known points surveyed in the field. They anchor the model to a coordinate system and let us quantify accuracy. Always record the CRS you surveyed in and keep metadata tidy.

Spatial reference system

• Defines how the two-dimensional, projected map in your GIS is related to real places on the earth
• It is crucial to know what is your data reference system!

• There are global map projections, but most map projections are created and optimized to project smaller areas of the earth’s surface
• There are two different types of coordinate reference systems: Geographic Coordinate Systems and Projected Coordinate Systems
• Spatial reference list (EPSG codes for coordinate reference systems)

CRS defines how map coordinates relate to the earth. We’ll choose a projected CRS appropriate to our area, not a global one unless we need it. Remember: GCS = lat/long, PCS = meters/feet.

UAS data processing outputs

What do we get after processing the data?

From processing we get: orthophoto, DSM/DTM, point cloud, mesh/texture, and a processing report. Think of the orthophoto as your basemap and the DSM/DTM as your analysis surface.

Orthophoto

• Aerial imagery geometrically corrected (“orthorectified”) such that the scale is uniform
• Raster: consists of red, green, and blue bands

An orthophoto is imagery corrected for perspective and terrain, so scale is uniform. It’s just a raster (RGB bands), but every pixel is in the right place for measurement.

Digital Surface Model

• DEM/DTM - Digital Elevation Model / Digital Terrain Model
  • Representation of a terrain’s elevation
  • Bare-earth raster grid
• DSM - Digital Surface Model
  • Representation of a visible surface
  • Captures the natural and built features on the Earth’s surface

DTM/DEM: bare ground surface. DSM: the surface you see—trees, buildings, cars. SfM produces a DSM by default; getting a DTM requires classification and filtering.

Pointcloud

• Representation of the external surface of an object
• Set of vertices in a three-dimensional coordinate system
• Vector or raster?
• Dale Lutz once said, “point cloud is a badly behaved raster”

A point cloud is a collection 3D of points (x, y, z, plus color/intensity). It’s not quite raster or vector—think of it as raw geometric evidence. We can classify, thin, and grid it to surfaces.

Multiple-view geometry

• Scene geometry (structure):
  Given 2D point matches in two or more images, where are the corresponding points in 3D?
• Correspondence (stereo matching):
  Given a point in just one image, how does it constrain the position of the corresponding point in another image?
• Camera geometry (motion):
  Given a set of corresponding points in two or more images, what are the camera matrices for these views?

Multiple view geomoetry comes from the field of computer vision and gives us the tools to describe the spatial relationship between multiple images in a scence allowing us to construct 3D structure from 2D views.

• The scenece geometry gives us structure – where are 3D points?
• Stereo matching gives us Correspondence – which pixels match across images?
• And the Camera geometry allows us to infer Motion – where were the cameras?

SfM solves these together by minimizing reprojection error.

What do we need?

1. Digital imagery
2. (Digital elevation model or topographic dataset)
3. Exterior orientation parameters from aerial triangulation or IMU
4. Optional(Camera calibration report)
5. Optional(Ground Control Points parameters)
6. Photogrammetric processing software that utilizes collinearity equations

Required: images, EO hints (IMU/triangulation), and software that implements collinearity equations. Optional but recommended: DEM, camera calibration, and GCPs for accuracy.

Digital imagery

When we capture images, we’re always dealing with perspective projection. With a short focal length and close distance, we have full perspective — objects closer to the camera appear much larger, so scale varies dramatically across the image. This increases distortions, especially around the edges.

As we increase the focal length and move the camera farther away, we approach a weak perspective model. Here, perspective rays are nearly parallel, so objects maintain a more uniform scale across the image. This reduces distortion and simplifies photogrammetric calculations, but of course requires higher altitude or zoom lenses.

For UAS mapping, we’re often somewhere between these extremes. Wide-angle action cameras emphasize full perspective, while higher-altitude mapping with a longer focal length lens gets us closer to weak perspective — which is why flight height and lens choice both matter for geometry and accuracy.

And remember: overlap is still king — ~75–85% forward and 60–80% side — to give Structure from Motion enough redundancy to solve geometry robustly.

Digital Elevation Model

In the past: Shape of the ground surface must be known in order to remove the effects of relief displacement

Now: Computed automatically by Structure from Motion

Historically we needed a DEM to remove relief displacement. Modern SfM estimates both terrain and cameras together—still, a good a priori DEM can stabilize tricky projects.

Structure from Motion (SfM)

• Range imaging technique
• Process of estimating 3D structures from 2D image sequences
• May be coupled with local motion signals

SfM extracts 3D structure from multiple overlapping 2D images. It’s paired with MVS (multi-view stereo) to densify the model. Quality inputs = better tie points = better geometry.

Exterior orientation (EO)

Position and orientation in the object space

6 elements necessary for any photogrammetric processing:

The cameras:

• X (Latitude)
• Y (Longitude)
• Z (Altitude)

and Angular orientation:

Photogrammetry	Aviation analogy	Axis	Motion
ω (omega)	Roll	x	Tilt forward/back
φ (phi)	Pitch	y	Tilt side-to-side
κ (kappa)	Yaw	z	Rotation around vertical

Exterior orientation camera position (X, Y, Z) and yaw, pitch, roll. Logs/EXIF provide first guesses; bundle adjustment refines them to millimeter-to-centimeter pixel fits.

Flight log (Again)

• Log file contains elements of exterior orientation that are measured by onboard Inertial Measurement Unit (IMU) and written into a text file

• Sometimes (most DJI products) exterior orientation parameters are saved in photos’ EXIF file
• Log contains information about the location of the camera, not the location of the depicted object - more info in this section of lecture 3

Remember: logs tell you where the camera was, not the object. We’ll still need tie points and, ideally, GCPs to anchor the scene in the real world.

Interior orientation

• In the past: camera calibration report
• Now: Self-calibration (auto-calibration) is the process of determining intrinsic camera parameters directly from uncalibrated images

• Can be automatically derived using Structure from Motion (SfM) methods

IO = camera intrinsics: focal length, principal point, lens distortion. Older workflows used lab calibration; modern SfM does self-calibration as part of alignment.

Ground Control Points

• GCP - Target in the project area with known 3 coordinates (X, Y, Z or lat, long, alt)
• For more information about placing targets and importance of GCPs see this section of lecture 3
• For more information about processing the data with GCPs see intro to the assignment

GCPs with known X/Y/Z reduce bias (like doming) and let us report RMSE against independent check points. Place them across the full extents and elevation range.

Processing options

Everything boils down to… money (and time)

• What is my starting budget and equipment?
• How frequently will I fly?
• Do I have the experience/training necessary for processing (or am I able to hire people who do)?
• Do I have time to process the data by myself?

Your constraints: budget, flight frequency, team expertise, and compute time. Cloud services can offload compute; local runs give more control and transparency.

Processing options - software

• Agisoft Metashape

• OpenDroneMap

• Pix4D

• DroneDeploy

• Trimble Inpho UASMaster

• Drone2Map (ESRI)

• DroneMapper

• many many more…

There are many commerial and open source processing software avaliable. We will be learning with both Agisoft Metashape (proprietry) and WebODM (Open Source options).

Agisoft Metashape

• Image-based solution aimed at creating 3D content from still images
• Operates with arbitrary images and is efficient in both controlled and uncontrolled conditions
• Both image alignment and 3D model reconstruction are fully automated

• Installer
• Manual
• Tutorials for:
  • Orthophoto and DSM generation with GCPs
  • Dense Cloud Classification & DTM Generation
  • DEM based Measurements
• YouTube channel

Metashape is robust for uncontrolled, real-world imagery. It automates alignment and reconstruction but still benefits from good inputs and sensible settings. Keep the manual handy.

Processing workflow

Preprocessing stage:

• Loading photos into Metashape
• Inspecting loaded images, removing unnecessary images

Load photos, inspect for blur/over-exposure, and remove bad frames. If you have camera positions, import them now—they’ll seed alignment and save time.

Processing workflow

Processing stage:

1. Aligning photos
2. Building dense point cloud
   • (optional: editing dense point cloud)
3. Building mesh (3D polygonal model)
   • (optional: editing mesh)
4. Generating texture
5. Building DSM and orthomosaic

• Align photos (tie points + bundle adjustment).
• Dense point cloud (depth maps).
• Optional mesh + texture.
• Build DSM and orthomosaic.

At each step: check diagnostics before continuing.

Exporting results

We’ll export GeoTIFFs for DSM/ortho, LAS/LAZ for point clouds, and optionally OBJ/PLY for meshes. Always keep a Metashape report with parameters and errors for reproducibility.

We also have the option to export cloud optimized formats such as COGS and COPCs

Preprocessing

• Loading photos
• Loading camera positions (flight log)
• If the EO is in the photos EXIF file, the parameters will load automatically

If EXIF contains EO, Metashape loads it automatically. If not, import the flight log with the correct time offset and CRS. Verify positions overlay your area.

Aligning photos

At this stage, Agisoft Metashape implements SfM algorithms to monitor the movement of features through a sequence of multiple images:

• Obtains the relative location of the acquisition positions
• Refines camera calibration parameters
• Sparse point cloud and a set of camera positions are formed

Alignment finds features, matches them, and solves for cameras. The output is a sparse point cloud and camera poses. If coverage is weak, revisit image quality or overlap.

Bundle Block Adjustment

• Non-linear method for refining structure and motion
• Minimizing reprojection error
• Detecting image feature points (i.e., various geometrical similarities such as object edges or other specific details)

BBA is the math heart: a nonlinear least-squares that minimizes reprojection error of tie points by adjusting IO and EO together. GCPs strengthen the solution.

Bundle Block Adjustment

• Subsequently monitoring the movement of those points throughout the sequence of multiple images
• Using this information as input, the locations of those feature points can be estimated and rendered as a sparse 3D point cloud

We monitor point tracks across images to estimate 3D feature locations. The result is a coherent sparse cloud—our sanity check before investing time in densification.

Aligning cameras in Metashape

Accuracy

• Higher accuracy > more accurate camera position estimates (time-consuming)
• Lower accuracy > rough camera positions (less-time consuming)

Works by upscaling or downscaling images when identifying tie points.

Accuracy settings: High gives tighter camera estimates but takes longer; Low is useful for triage. Start High, downsample only if the project is truly huge.

Building dense point cloud (Depth Maps)

At the stage of dense point cloud generation, Agisoft calculates depth maps for every image

• Quality: Highest, High, Medium, Low, Lower > the higher quality, the more accurate camera position estimates, but the process is more time-consuming

Metashape computes a depth map per image, then fuses them. Quality levels trade time for detail. For class labs, High is a good balance; Highest only when needed.

Building dense point cloud (Depth Filtering Modes)

Depth Filtering modes

Algorithms sorting outliers (due to some factors, like poor texture of some elements of the scene, noisy or badly focused images)

• Mild depth filtering mode > for complex geometry (numerous small details on the foreground), for important features not to be sorted out
• Moderate depth filtering mode > results in between the Mild and Aggressive
• Aggressive depth filtering mode > sorting out most of the outliers (Recommended for aerial data processing)

Filtering prunes outliers. Mild preserves fine structure, Aggressive removes noise but may erase detail, Moderate is a safe default. Choose based on scene complexity.

Optional: Editing dense point cloud

• Manual points removal
• Automatic filtering based on applied masks
• Sparse cloud only:
  • Reducing the number of points in the cloud by setting tie point per photo limit
  • Automatic filtering based on:
    • Reprojection error
    • Reconstruction uncertainty
    • Image count

Clean with masks and selection tools. In sparse cloud, use reprojection error, reconstruction uncertainty, and image count filters to remove weak tie points, then re-optimize.

Building mesh

• Arbitrary > for modeling of any kind of object
  • Should be selected for closed objects (statues, buildings, etc.)
  • Memory consumption: high
• High field > for modeling of planar surfaces
  • Should be selected for aerial photography
  • Memory consumption: low
  • Allows for larger data sets processing
• Source data > the source for the mesh generation
  • Sparse cloud > fast 3D model generation (low quality)
  • Dense cloud > high-quality output based on the previously reconstructed dense point cloud

Use Arbitrary for closed objects—high memory but best geometry for statues/buildings. For aerial scenes we’ll prefer the next option. Use Height Field for terrain—lower memory and scalable. Choose Dense cloud as the source for quality results; Sparse is only for quick previews.

Building mesh (Face Count)

Face count > the maximum face count in the final mesh

low face count

high face count

Control face count to manage memory and export size. Generate a high-res master, then decimate copies for web or VR as needed.

Optional: Editing mesh

• Close Holes tool > repairs your model if the reconstruction procedure resulted in a mesh with several holes, due to insufficient image overlap
  • Necessary step for volumes calculation
• Decimation tool > decreases the geometric resolution of the model by replacing a high-resolution mesh with a lower resolution one

If overlap was thin, expect holes. Use Close Holes before computing volumes—open meshes give nonsense volume numbers.

Optional: Editing mesh

• Automatic filtering based on specified criteria:
  • Connected component size
  • Polygon size
• Manual polygon removal
• Fixing mesh topology
• Editing mesh in the external program
  • Export mesh for editing in the external program
  • Import edited mesh

Filter by component size or polygon size to remove floaters. For heavy edits, export to a DCC tool (e.g., Blender), fix topology, then import back.

Generating texture

• Determines how the object texture will be packed in the texture atlas
• Affects the quality of the final model

• Texture mapping modes:
  • Generic
  • Adaptive orthophoto
  • Orthophoto
  • Spherical
  • Single photo
  • Keep uv

Texture quality affects realism. The mapping mode controls how imagery is packed into the texture atlas. For terrain, Adaptive orthophoto or Orthophoto are typically best.

Texture mapping modes

Generic

• Creates as uniform texture as possible

Adaptive orthophoto

• The object surface split into the flat part and vertical regions
• The flat part of the surface textured using the orthographic projection, while vertical regions textured separately to maintain accurate texture representation in such regions
• More compact texture representation for nearly planar scenes + good texture quality for vertical surfaces

Generic seeks uniform texture; good all-purpose. Adaptive orthophoto splits flat vs. vertical faces—great for near-planar scenes with facades.

Texture mapping modes

Orthophoto

• The whole object surface textured in the orthographic projection
• Even more compact texture representation than the Adaptive orthophoto at the expense of texture quality in vertical regions

Spherical

• Only for objects that have a ball-like form

Single photo

• Texture from a single photo (photo can be selected from ‘Texture from’ list)

Keep uv

• Generates texture atlas using current texture parametrization; Rebuilding current texture with different resolution or generating the atlas parametrized in the external software

Orthophoto is compact but weaker on verticals. Spherical is niche. Single photo is for controlled cases. Keep UV preserves external UVs if you authored them elsewhere.

Generating DSM

Parameters

• Source data: Dense point cloud
• Interpolation
  • Disabled: leads to accurate reconstruction results since only areas corresponding to dense point cloud points are reconstructed
  • Enabled (recommended): Interpolation mode - Agisoft will calculate DEM for all areas of the scene that are visible on at least one image.

Build DSM from the dense cloud. With Interpolation Off, you’ll get only observed areas—good for accuracy checks. With On (recommended), gaps are filled where at least one image saw the ground.

Generating orthophoto

Parameters

• Surface: DEM
• Blending mode
  • Mosaic (default): Implements an approach with data division into several frequency domains, which are blended independently
  • Average: Uses the weighted average value of all pixels from individual photos
  • Disabled: The color value for the pixel is taken from the photo with the camera view being almost along the normal to the reconstructed surface in that point.

Set Surface = DEM so drape follows terrain. Blending: Mosaic is the usual choice; Average can reduce noise; Disabled chooses the most nadir, often increasing seam visibility.

Exporting results & saving intermediate results

Orthophoto export

Export as GeoTIFF or COG, check the CRS, and include overviews for fast display. Keep a world file if your GIS expects one.

Exporting results & saving intermediate results

DEM export

Export the DSM/DTM as GeoTIFF or COG. Verify units and vertical datum. Document cell size and interpolation choices in your report.

Exporting results & saving intermediate results

3D model export

Export OBJ/PLY/GLB depending on downstream tools. For web sharing, decimate to manageable size and consider GLB for portability.

Exporting results & saving intermediate results

Point cloud export

Export LAS/LAZ or COPC with CRS tags. If you plan to classify later, preserve RGB and intensity attributes—they’re useful for ground/non-ground separation or if you already classified points export all or only selected classes.

Exporting results & saving intermediate results

Processing report generation

Always generate the Metashape report. It captures camera calibration, tie point statistics, overlap maps, and GCP/check-point errors—critical for QA/QC and reproducibility.

Processing report

Includes:

• Orthophoto and digital elevation model sketch
• Camera parameters and survey scheme
• Tie points data export (matching points and panoramas)
• Image overlap statistics
• Camera positioning error estimates
• Ground control point error estimates

Look for image overlap, camera error bars, and GCP vs. check-point RMSE. If check-point error is much higher than GCP error, you’re over-fitting—revisit weighting and distribution.

Batch processing

For larger jobs, set up batches so stages run unattended. Save after each stage so you can rollback without recomputing everything.

Quality processing with GCPs

• Marker positions are defined by their projections on the source photos
• After optimizing alignment based on markers, Point cloud generation and other steps need to be performed
  • Used for:
    • Setting up a coordinate system
    • Photo alignment optimization
    • Measuring distances and volumes
    • Marker-based chunk alignment
• More on GCP placing and processing in Agisoft see intro to the assignment

Place markers, reproject on multiple photos, then optimize alignment with GCPs. Rebuild dense cloud/DSM/ortho afterward—optimization changes the geometry.

What did we learn?

• What is a general workflow for UAS imagery processing
• How do we transform UAS data into orthophoto, DSM, 3D model, and point cloud
• How to process the data in Agisoft Metashape Professional and how to set proper parameters in the program

You saw the end-to-end workflow: inputs → alignment → densification → surfaces → exports. You can now produce orthophotos, DSM/DTM, point clouds, and meshes in Metashape, and you know where IO/EO, GCPs, and CRS fit in.