GIS595/MEA792: UAV/lidar Data Analytics

3D modeling and rendering with UAV data

GIS595-004/603; MEA592-006/601:

UAS Mapping for 3D Modeling

Helena Mitasova, Payam Tabrizian, Justyna Jeziorska

Center for Geospatial Analytics,
North Carolina State University

Outline

  • Motivation for 3D modeling with UAV data

  • Basic concepts in 3D modeling and rendering

  • 3D GIS and 3DWeb GIS

  • Overview of Blender software and main functionalities

  • General workflows for modeling with UAV and GIS data in Blender

Motivation for 3D modeling with UAV data

  • Easier and more intuitive interaction with data
  • Data cleaning (artifact removal, adding details)
  • Adding additional GIS data and simulations (e.g., draping maps)
  • Animating flights and visualizing various data facets and analytics
  • Developing interactive online visualizations
  • Improving science communication and publication impact

3D modeling

  • Definition

  • Typologies

    • Polygonal modeling

    • Curve modeling

    • Digital sculpting (displacement, voxels, dynamic tesselation)

3D Mesh

    Vertex

    Edge

    Face

3D Rendering

  • Prerendering, viewport rendering, rendering, render engines

  • Materials

    • Shading

    • Texturemapping

  • Photorealism

Photorealism

Image credit: Mushrooms by John Wrbanek

Shaders

    Surface

    Volumetric

    Displacement

Surface Shaders

    Diffuse

    Glossy

    Transparent

Surface Shaders

    Emission

    Glass

    Mixed
    Diffuse + Glass

Texture mapping

    Satellite image used to create surface color

    render

Bump mapping

    Elevation image used to generate surface roughness

    render

Alpha mapping

    Alpha map of oceans used to extract land features

    render

Emission mapping

    Satellite imagery of lights used to illuminate the surface

    Render

Volumetric shading

    Satellite imagery of the clouds

    The sphere object and the light source

Volumetric shaders

    Rendering of the cloud sphere

    Rendering of the cloud & earth spheres

Lighting

    Spotlight

    render

Lighting

    point

    render

Lighting

    sun

    render

3D GIS

  • Georeferencing

  • 3D in GIS (e.g., ArcGIS pro, CityEngine)

  • GIS in 3D (e.g., Mapbox in Unity, BlenderGIS)

3D Web GIS

    Platform Interaction design capability Realism Web AR/VR Realtime GIS data integration API License Domain hosting
    ArcGIS full medium -- yes JS, HTML Proprietary No
    CesiumJS full low -- yes JS Apache 2.0 No
    Sketchfab limited high VR -- HTML Proprietary Yes
    Blend4web full high VR no HTML CSS JS Python GNU V3 No
    Mapbox full high full Yes Python JS C# Proprietary Yes

Blender

  • Free and open-source

  • Powerfull mesh editing tools

  • GIS connection

  • Easy Scripting python API

  • Direct export for Sketchfab and Blend4Web

Components of Blender interface

    Environments attributed to five editors

Editors: 3D viewport


    3D viewport

Editors: Outliner


    Outliner showing Camera, Cube and Lamp objects

Editors: Python console

  • Quick scripting/prototyping
  • Python libraries already loaded


    Python console

Editor: Python text editor

  • Scripting and executing workflows
  • Libraries should be loaded

Editor: Properties editor

  • Materials, objects, textures, modifiers, lights, cameras

    Render options

    Object attributes

    Material attributes

    Texture attributes

Node editor

Designing complex photorealistic material

Node tree with shader and output nodes, parameters and links

General workflows for processing UAV data

  • Importing data
  • Model optimization
  • Shading
  • Output

Importing surface models

  • Geotiff (.tif) through BlenderGIS
  • Wavefront (.obj) directly from Agisoft, etc.
  • Stanford (.ply) pointclouds

Model optimization

  • Decimation
  • Smoothing
  • Mesh correction
  • Texture correction

Decimation with Decimate modifier

    Initial mesh
    5mil faces, 5GB memory

    Decimate to %50
    2.5mil faces, 3.6GB memory

    Decimate to %10
    500K faces, 2.2GB memory

Smoothing with Smooth modifier

    Initial mesh

    After medium smoothing

    After high smoothing

Mesh correction with sculpting tool

    Blender Sculpt tool Brushes

    Initial mesh

    Refined mesh using Inflate, Flatten and Smooth brush

Texture correction with Texture Paint tool

    Blender Texture Paint tool


Initial mesh


Refined texture using clone brush

Shading

  • Draping orthophoto and GIS analyses using image textures
  • Blending multiple textures using mix shaders
  • Extracting features (e.g., buildings) using alpha maps
Draping orthoPhoto and GIS analyses using image textures

Viewsheds by ptabriz on Sketchfab

Blending multiple textures using mix shaders

    Wire shader blended with UAV RGB texture

General workflows for processing UAV data

  • Output
  • Single frame rendering
  • Animations (multiple frame)
  • Web

Animation

Scene Camera is following a circular curve to create the animation (100 frames 10fps)

Web export: Sketchfab

  • Imports, shaders, lights, cameras, and animations
    • Option A) Directly exporting from Blender using Sketchfab addon
    • Option B) Importing (.blend) in Sketchfab

Web export: Blend4Web

  • Blender addon for simple export to (.html)
  • Primitive interactions and simple annotations

Web export: Blend4Web

  • Advanced interactions, UI
  • Combine with other media (video, sounds, photos, text)
  • Need to host the model yourself
  • Requires Javascript
  • Example project