Deep Learning for Tree Counting

and Building Extraction

Yoga Abdilah
June 11th, 2020
 GeoAI Pattern

Object Detection Prediction Pattern Detection

Buildings, Road Segments, Water Pipe Breaks, Diseases, Top Risky Segments, Emerging
Swimming Pools, Blight, Crimes, Crashes, Incidents, Hotspots of 911 Calls, Disease
Graffiti, Overgrowth, Road Fires, Congestion, 911 Calls, Clusters, and more
Signs, Vehicles from CCTVs,
and more
 Examples for Imagery AI Workflows
Object Detection, Instance Segmentation, Land Cover, Change Detection..

Tree Counting Building Footprints Land Cover Pipeline Encroachment

Oil Pads Cars

Road Cracks
Swimming Pools Roads Damaged Structures
ArcGIS Pro Professional GIS
New

Improvements • Reports
Coming
• Dimensions
• Utility Management
• Spell Check •
• Faster Startup
• Publish to Server •
• Editing Tools
• Deep Learning •
• Catalog Preview
• Parcel Management
• Definition Expressions
• Contingent Attributes

ArcGIS Pro Apps

ArcMap
Integrating Open Science, AI and Machine Learning
Revolutionizing Spatial Analysis and Data Science

Analytic Services
Python
Notebooks Python API

Integration

AI & Machine Learning

Open Science Tools

Geospatial
Infrastructure
GeoAI User Experience
Pro Web Python API
Tree Counting
With Deep Learning in ArcGIS Pro
Tree Counting
with Deep Learning in ArcGIS Pro

• Only available in ArcGIS Pro.

• Require Image Analyst extension.
• Object detection workflow first introduced in ArcGIS Pro 2.3
• Begin with installing deep learning frameworks for ArcGIS.
 End to End Tree Counting Workflow

• Identify how we access imagery data.

- Raster file?
- Mosaic Dataset?
- Mosaic Dataset in a shared directory/datacenter?

Imagery Imagery Data Train & Consume Deploy Models to Run Inference at Feedback Take
Access Prep Labelling Models Production SCALE Loop Action
 End to End Tree Counting Workflow

• Identify image format, spatial reference, number of bands.

• Specify processing extent.

Imagery Imagery Data Train & Consume Deploy Models to Run Inference at Feedback Take
Access Prep Labelling Models Production SCALE Loop Action
 End to End Tree Counting Workflow

• Create training samples.

• Number of samples might vary based on:
- Method used
- Image size

Imagery Imagery Data Train & Consume Deploy Models to Run Inference at Feedback Take
Access Prep Labelling Models Production SCALE Loop Action
 End to End Tree Counting Workflow

• Start by Exporting data

- GP Toolbox: Export Training Data for Deep Learning

Imagery Imagery Data Train & Consume Deploy Models to Run Inference at Feedback Take
Access Prep Labelling Models Production SCALE Loop Action
 End to End Tree Counting Workflow

• Next: Train a deep learning model!

- Using ArcGIS Pro (since 2.5)

Imagery Imagery Data Train & Consume Deploy Models to Run Inference at Feedback Take
Access Prep Labelling Models Production SCALE Loop Action
 End to End Tree Counting Workflow

• Next: Train a deep learning model!

- Using ArcGIS Pro (since 2.5)
- Single Shot Detector
- U-Net
- PSPNET
- RetinaNet
- MaskRCNN

Imagery Imagery Data Train & Consume Deploy Models to Run Inference at Feedback Take
Access Prep Labelling Models Production SCALE Loop Action
Tree Counting
Image Classification vs Object Detection

Image Classification Object Detection

Predict object in an image

Predict object in an image
+
Location of the image (bounding boxes)
 Tree Counting
Method – Single Shot Detector (SSD)

• SSD divides the image using a grid

• Each grid cell be responsible for detecting objects
- Probability that there is an object?
- Height of the bounding box?
- Width of the bounding box?
- Horizontal coordinate of the center point of the bounding box?
- Vertical coordinate of the center point of the bounding box?

Anchor box Zoom level

Aspect ratio

https://developers.arcgis.com/python/guide/how-ssd-works/
 End to End Tree Counting Workflow

• Next: Train a deep learning model!

- Using ArcGIS Pro (since 2.5)
- Using arcgis.learn
(ArcGIS API for Python)
- Using open libraries

Imagery Imagery Data Train & Consume Deploy Models to Run Inference at Feedback Take
Access Prep Labelling Models Production SCALE Loop Action
 End to End Tree Counting Workflow

• Deploy Deep Learning model

- Accessible also as emd or
Deep Learning Package

Imagery Imagery Data Train & Consume Deploy Models to Run Inference at Feedback Take
Access Prep Labelling Models Production SCALE Loop Action
 End to End Tree Counting Workflow

• Deploy model & start object detection

- GP Toolbox: Detect Objects Using Deep Learning

Imagery Imagery Data Train & Consume Deploy Models to Run Inference at Feedback Take
Access Prep Labelling Models Production SCALE Loop Action
 End to End Tree Counting Workflow

• QC
• Utilize Result:
- Tree Health Monitoring

Imagery Imagery Data Train & Consume Deploy Models to Run Inference at Feedback Take
Access Prep Labelling Models Production SCALE Loop Action
Tree Health Monitoring
• Assessing vegetation health using Visible Atmospherically Resistant Index (VARI)
- Indirect measure of leaf area index (LAI) and vegetation fraction (VF)
- Using reflectance values from the visible wavelength
(Rg - Rr) / (Rg + Rr - R(Rg - Rb))
- Preferred to use both visible and near infrared (NIR) wavelength
Demo
Tree Counting
with Deep Learning in ArcGIS Pro

- Tree Health Monitoring

- Field Survey for Verification
Tree Counting
with Deep Learning in ArcGIS Pro

- Tree Health Monitoring

- Field Survey for Verification
Tree Counting
with Deep Learning in ArcGIS Pro

- Tree Health Monitoring

- Field Survey for Verification
Building Extraction
With Deep Learning in ArcGIS Pro
Building Extraction
with Deep Learning in ArcGIS Pro

• Only available in ArcGIS Pro.

• Require Image Analyst extension.
• Object detection workflow first introduced in ArcGIS Pro 2.3
• Begin with installing deep learning frameworks for ArcGIS.
 End to End Building Extraction Workflow

Imagery Imagery Data Train & Deploy Models Run Inference at Feedback Take
Access Prep Labelling Consume to Production SCALE Loop Action
Models
Demo
Conclusion

• Deep Learning workflow helps simplify & accelerate object detection process
• Deep Learning is supported in ArcGIS Pro + Image Analyst Extension
• ArcGIS Pro support end-to-end workflow
• GeoAI capabilities could be integrated with other open science libraries