NCSU GIS 714:
Geospatial Computing and Simulations

Running basic analysis with GRASS on Hazel

The goal of this section is to become familiar with running basic GRASS commands on HPC, including creating a project, importing data, running a simple analysis and exporting data.

The class' scratch directory is /share/gis714s26. After you login for the first time, create your own directory and use that as your scratch space: 

mkdir /share/gis714s26/$USER

Using the GRASS Conda Environment

We will be using GRASS in conda environment. Refer to Hazel Conda instructions for more details. One-time setup: 
module load conda
conda init bash
This will create .bashrc file in your home folder. Logout and login again. Verify the setup: 
conda activate /usr/local/usrapps/gis714s26/grass_env
grass --version
conda deactivate

Creating a new project

In your scratch space (/share/gis714s26/$USER) create a new project intro with CRS matching NLCD data located /share/gis714s26/: 
mkdir /share/gis714s26/$USER/
conda activate /usr/local/usrapps/gis714s26/grass_env
grass -c /share/gis714s26/nlcd_2019_land_cover_l48_20210604.img /share/gis714s26/$USER/intro --text
This will automatically start in a new mapset PERMANENT and set the default computational region to match the NLCD raster. Now, review projection and computational region info and exit: 
g.proj -p
g.region -p
exit
conda deactivate
Which projection is NLCD data in? How many cells?

Importing NLCD

We will import 2019 NLCD land cover into our newly created project. We will import data in an interactive session on a compute node
We request a single core and 2GB of memory for 30 minutes: 
bsub -Is -n 1 -R "rusage[mem=2GB]" -W 30 bash

Once you are on a compute node, activate the environment:

source ~/.bashrc
conda activate /usr/local/usrapps/gis714s26/grass_env
Start GRASS in the newly created project: 
grass /share/gis714s26/$USER/intro/PERMANENT
Set region to smaller area so that we don't have to wait too much: 
g.region n=1690185 s=252165 e=1838925 w=514635 res=30 -p
Import NLCD land cover data (will take couple minutes): 
r.in.gdal input=/share/gis714s26/nlcd_2019_land_cover_l48_20210604.img output=nlcd_2019 memory=2000 -r
Once you are done, you can launch GUI with g.gui add the new raster. Alternatively, launch a GRASS monitor and add the raster like this: 
d.mon wx0
Once the display opens, run: 
d.rast nlcd_2019
Take a screenshot for the report. Then close the window and exit GRASS and the compute node. 
exit
exit

Running an analysis and export

Next we will compute a simple analysis with r.neighbors (number of different values within a 3x3 neighborhood) and export the result into tif. We use compression option when exporting to minimize file size (consult GDAL documentation for further explanation). For that we submit a job, we request 1 core, 2GB of memory for 5 hours. GRASS tool is executed using the --exec interface which starts a GRASS session in specified project, executed the command and ends the session. We will create a text file in your home folder using nano editor: 
nano /home/$USER/bsub_neighbors.sh
Copy and paste the following content and exit (Ctrl+X) and save (Shift+Y): 
#!/bin/bash
#BSUB -n 1
#BSUB -W 3:00
#BSUB -R "rusage[mem=2GB]"
#BSUB -oo neighbors_out
#BSUB -eo neighbors_err
#BSUB -J neighbors

source ~/.bashrc
conda activate /usr/local/usrapps/gis714s26/grass_env

grass /share/gis714s26/$USER/intro/PERMANENT --exec r.neighbors input=nlcd_2019 output=diversity_2019 size=3 method=diversity
grass /share/gis714s26/$USER/intro/PERMANENT --exec r.out.gdal input=diversity_2019 output=/share/gis714s26/$USER/diversity_2019.tif type=Byte createopt="COMPRESS=LZW,PREDICTOR=2,BIGTIFF=YES"
Submit the job from scratch space: 
cd /share/gis714s26/$USER/
bsub < /home/$USER/bsub_neighbors.sh
Now check that job is submitted. 
bjobs
Once it runs, check the error output for progress messages from GRASS. 
less neighbors_err
Once the job is finished, inspect the summary output and see how long it took: 
less neighbors_out
Check the tif file is created and copy it on your local machine and open it in a GIS. Take a screenshot for the report.

Creating your own GRASS conda environment

If you need your own environment (e.g., to add extra packages), you can create one from a YAML file. This is not part of the assignment.

Step 1: Create a YAML file

Save the following as grass_env.yml:

name: grass_env
channels:
  - conda-forge
dependencies:
  - grass=8.4.2

  # Compilers for building addons
  - c-compiler
  - cxx-compiler
  - make

  # Essential libraries that addons might need
  - gdal
  - proj
  - geos

  # Python geospatial stack
  - numpy
  - scipy
  - pandas
  - geopandas
  - matplotlib

Step 2: Create a .condarc file

When installing packages, conda downloads temporary files that can fill your home directory quota. Create ~/.condarc to redirect them:

cat > ~/.condarc << 'EOF'
pkgs_dirs:
  - /share/$GROUP/$USER/conda/pkgs
channels:
  - conda-forge
EOF

Step 3: Create the environment

All installations must be done from a login node (not a compute node).

conda env create --prefix /usr/local/usrapps/$GROUP/$USER/grass_env -f grass_env.yml

This will take several minutes.

Step 4: Use it

conda activate /usr/local/usrapps/$GROUP/$USER/grass_env
grass --version