Configurable

Computing &
Embedded Systems
VIP Class
Fall 2016
Week 1

Dr. Lee W. Lerner

[email protected]
Mike Ruiz
[email protected]
Jay Danner
[email protected]

VIP PROJECT THEME - TECH CITIES

Georgia Tech has been researching
methods for the collection, analysis, and
display of extremely dense and varying
streams of geographically meaningful data

GTs smart-city testbed technology

Reconfigurable sensor nodes that measure
environmental conditions and human influence
Collects real-time data about traffic (vehicle,
foot, skateboard, bicycle, pets, etc.) and
environmental conditions (weather, air quality,
noise, etc.)

Potential deployments
GT Campus Array
City of Atlanta
City of Decatur
City of Marietta
Cities all over the world via integration with AoT
2

GT CAMPUS ARRAY

Deploy some number along Ferst

Other locations?

DECATUR
City of Decatur
Deployed one, early
version (ugly), sensor
node in early 2015 in
Decatur Sq.
Plan to deploy 5-10
nodes in 2016
around entire city

Developed public
facing web
dashboard for
Atlanta BeltLine
version of sensor
deployment
Integrate Decatur
testbed with other
city testbeds

ATLANTA

Have 1 node deployed on North

Ave/Spring
Need to deploy 5 more this semester on
North Ave

ATLANTA
Ponce City Market - Within the next week
Mid October
Northside Dr. - Within a week or 3
October
5th Location - TBD

Freedom Parkway Peachtree St. - Mid

THE ARRAY OF THINGS (AOT)

PROJECT
Lead by Argonne National Laboratory and U.
Chicago
A network of interactive, modular sensor units around
Chicago and other major cities collecting real-time data
on the urban environment, infrastructure, and activity
for research and public use
AoT intends to operate as a national instrument for
urban science and technology research

Partnering with Georgia Tech to research:

Security and trust of sensors nodes and data

Sensor/human personal device networking
New sensors/applications
In-situ accelerated computation and real-time analytics
Deployment in the metro-Atlanta area

Sensors already planned for:

Traffic (human and other)

Volatile Organic Compounds (VOCs)
Sound
Temperature
Ozone (O3)
Humidity
Carbon Dioxide (CO2)
Light
Sulfur Oxides
Carbon monoxide (CO)
Particulate Matter
Nitrogen dioxide (NO2)
Precipitation
Vibration

Wind

https://arrayofthings.github.io/

CONFIGURABLE COMPUTING AND

EMBEDDED SYSTEMS

Designed for a dedicated

function
Complexity varies from
single microcontroller to
complex networked systems
Embedded devices often
impact the physical world

PROGRAMMABLE LOGIC
DEVICES (PLDS)
Programmable logic devices
Function is determined by a configuration bitstream that
is downloaded to the device
Examples include:
Field Programmable Gate Array (FPGA)
Complex Programmable Logic Device (CPLD)
Electronic Programmable Logic Device (EPLD)

Most devices have volatile configuration memory and

require the bitstream to configure the device every time
power is supplied to the system
This is the case for most high-performance or low-cost devices,
e.g., static random access memory (SRAM) based devices

Few devices are non-volatile through the use of antifuses or flash-based memory

PROGRAMMABLE LOGIC DEVICES

(PLDS)

Traditional
Configurabl
e

HW is

HW can be
changed by SW or

PLD TAXONOMY

Numerous device types exist to support

different design size,Device
power, cost, and
Manufacturer
security requirements
E.g. Xilinx, Altera,
Microsemi

Xilinx Devices

Altera Devices

E.g. CPLDs, FPGAs

E.g. FPGAs, CPLDs

FPGA Series

FPGA Series
E.g. 6-Series

E.g. 7-Series

Device Family

Device Family

E.g. Virtex-6

E.g. Spartan-6

Device Member
By array size and tile
types, e.g. Virtex-6
SX315T

Device Member
By array size and tile
types, e.g. Virtex-6 LX75T

PLD ARCHITECTURE

Example:
generic FPGA
device
architecture
Tens to
hundreds of
configurable
logic blocks
Majority of
configurable
design
resources are
for routing
Mohammed M. Farag, Architectural Enhancements to Increase Trust in Cyber-Physical Systems Containing Untrusted Software and Hardware, Virginia Tech
Dissertation, September 17, 2012

System-level Design
Specification
Device-level Design
Specification

Example:
FPGA design
flow

Design Entry
Hardware Description Language (HDL)
Software models
Design Constraints

Synthesis and Implementation

Device-specific, gate-level netlists
Intermediate, testable formats

Programming File Generation

Vendor-proprietary device programming formats
Formatting for various in-system memory
devices

Programmed Device

Increasing difficulty of design verification

PLD DESIGN

PROJECT FOCUS AREAS/SUBTEAMS

Topic Area
Skill Area

General Design

Embedded
Vision

Security &
Trust

Algorithms &
Analytics

E.g. Data
reduction and
sanitization

E.g. object
identification and
counting

E.g. Sensor-tosensor trusted

communications

Embedded
Hardware

E.g. interfacing
with sensors

E.g. hardware
acceleration

E.g. root-of-trust
hardware

Embedded
Software

E.g. data
reporting and
system health
monitoring

E.g. OpenCV

E.g. system
firmware
management
and data privacy

E.g. traffic count

results pattern
analysis software

E.g.
management
software for
updates and
status
monitoring

Backend Software

E.g. data
dashboards and
research query
frameworks

14

GENERAL DESIGN

Advisors: Mike Ruiz

Technologies
Embedded hardware (FPGAs) and software (Linux)
Embedded Platforms (e.g. Zedboard, ODROID,
Arduinos, Rpi, etc).
Sensors (Environmental, Physical, Cameras, etc)

XILINX ZYNQ: DESIGN NOW ADAPT

LATER

ZYNQs FPGA architecture is fairly

consistent within its family of devices.
Designs made for the lowest-end chip
can run on any other ZYNQ-based
platform with minimal tweaking.

Z-TURN
MICROZED

ZEDBOARD
16

SOFTWARE ARCHITECTURE

PetaLinux kernel
Based on kernel.org release

Ubuntu core rootfs

Uniform automated build
and development process

Proprietary Software
Ubuntu Core rootfs

All proprietary software

encrypted on SD

PetaLinux Kernel

Only decrypted in RAM

Python, Bash, and C++

FPGA Hardware

GENERAL DESIGN
Focus areas
Defining and implementing
Hardware and Software
architectures
Interfacing with sensors and
other system components
Processing and transferring
data to backend servers
Casing, Construction, and
Deployment

GENERAL DESIGN GOALS

Update current v1.0 design for final

deployments
Begin designing components and
features for the v2.0 release.
Examples include:

Power-over-Ethernet
Wireless sensor node
Integration with S&T black magic
Moving to new Zynq platform
20

EMBEDDED VISION

Advisors: Jay Danner and Dr. Linda

Wills
Topics

Traffic Analysis
Dense Crowd Counting
Event Detection
Pattern Recognition optical
Pattern Recognition
flow. Taken from opencv.org
Object Detection
Hardware Acceleration

EMBEDDED VISION ALGORITHM

We developed Precedent-Aware Classification (PAC), which uses

low level heuristics in combination with the observed common
pathways to classify pedestrians/cars, with minimal algorithmic
complexity.

PAC is introduced in Rapid Precedent-Aware Pedestrian and Car

Classification on Constrained IoT Platforms, accepted at IEEEs
ESTIMedia conference, as part of Embedded Systems Week (October 27, 2016).

EMBEDDED VISION ALGORITHM

PAC uses traditional
detectors, such as
Histogram of Oriented
Gradients (HOG), to gather
additional low-level
heuristics about
pedestrians during a
calibration period.
Heuristics used for
classifying objects:
Aspect ratio
Pixel count
Pixel density
Size
Probably pathway (is the
object located on a
previously recorded
path?)

Probable pathway Pedestrian

EXAMPLE RESEARCH OPPORTUNITIES

Dense Crowd Counting

OPENCV

Computer Vision libraries

suitable for Linux, Windows,
and embedded system
development
C/C++, Python

Provides tools so that we

can focus on higher level
algorithm implementation

SECURITY & TRUST

Advisors: Dr. Lee W. Lerner

Topics
Secure firmware updates
Secure (private) data exchange
Trustworthy operations
Hardware and software
Not only operates as intended, but has no additional
functionality

Trustworthy data exchange

Explore software- and hardware-based
mechanisms for instilling trustworthy operations
Enforce security rules and system specifications at
run-time
26

26

SECURITY AND TRUST - PHILOSOPHY

Traditional
Software
Applications:

FPGA Security
Improvements:

SECURITY IMPROVEMENTS

Traditional Software
Applications:
System Identification

FPGA Security
Improvements:

Unique System Identifie

Xilinx Device DNA
DNA Port Primitive
Simple FSM

SECURITY IMPROVEMENTS

Traditional Software
Applications:

FPGA Security
Improvements:

System Identification
Boot Operation

Secure Key Storage

EFUSE
Battery Backed RAM
Authentication / Decryption
BOOTROM
Immutable
Root of Trust
FSBL -> U-Boot -> OS

SECURITY IMPROVEMENTS

Traditional Software
Applications:

FPGA Security
Improvements:

System Identification
Boot Operation
Software Update

Authentication &
Decryption keys
stored in BRAM
Only encrypted
data on SD
Extracted to RAM
disk

SOFTWARE UPDATE MECHANISM

SECURITY IMPROVEMENTS

Traditional Software
Applications:
System Identification
Boot Operation
Software Update
Firmware Update

FPGA Security
Improvements:
Hardware based
decryption engine
U-Boot sends new
images for
verification
EFUSE, BBRAM

FIRMWARE UPDATE MECHANISM

FUTURE SECURITY WORK

Anti-tamper Mechanisms
Secondary FPGA
Immutable system to monitor
and validate base system
operations
SIMON Decryption core
System Attestation

Anti-cloning Functions
Etc.

End Goals:
Interactive FPGA
based Security
Measures
Expanding defense
in depth
methodology

DASHBOARD

Advisors: Caroline Foster

Web-based dashboard
Not a student sub-team this semester

CLASS LOGISTICS
Review general VIP syllabus and example design notebook
http://www.vip.gatech.edu/
http://www.vip.gatech.edu/design-notebooks

GitHub
https://github.gatech.edu/llerner3/vip
Wiki pages for general project and each sub-team
Repository for all code
Revision controlled
Bug tracking/ticket system
Everyone must join the project

Sub-teams should meet weekly outside of class

Bi-weekly in-class presentations by sub-teams
~ 1 PowerPoint slide (or 2 minutes) per person, presented by each
person
~1-2 slides for overall team presented by Team Lead

Document your progress on the team wikis and your notebook

A notebook is not optional
You will get midterm notebook grade feedback

36

CLASS LOGISTICS
Tentative schedule
Week 1 Class/project overview
Week 2 Sub-teams formed; initial tasks set; returning students
discuss previous work
Week 3 Sub-teams present on task plan, schedule, and midterm/final
milestones
Week 4 Sub-teams present on progress
Week 5 Round table discussion
Week 6 Sub-teams present on progress
Week 7 Round table discussion; self-grades due
Week 8 Midterm sub-team presentations/demos/wiki overviews
Week 9 Round table discussions
Week 10 Sub-teams present on progress
Week 11 Round table discussion
Week 12 Sub-teams present on progress
Week 13 Round table discussion
Week 14 No class (Thanksgiving Break)
Week 15 Final sub-team presentations/demos/wiki overviews
37

CLASS LOGISTICS
Resources
Configlab2 machine in VIP student room
Use your personal GT account to sign in
Share data in /share folder (set file permissions for
sharing)

Configlab.vip.gatech.edu
ssh XY configlab.vip.gatech.edu
Need to be on ECE subnet (VPN if remote;
anyc.vpn.gatech.edu)

Virtual Box VM
Will be provided by us
Will have tools, code repo, etc. ready to go

Materials
E.g., ZedBoards
Use in or check out from VIP room (note your checkout on Wiki)
38

GIT REPOSITORY

Get familiar with GIT

Directory structure demo
Easy-to-use build script
Read guidelines about what to put in the repo
Download the repos

Log into github so I can add you to the project

Upload your SSH key to github
git clone [email protected]:llerner3/vip.git
git clone [email protected]:llerner3/vip.wiki.git

39

CLASS LOGISTICS

Grades
This is not a lecture- or test-based class
Grades are based on your level of effort, technical

contributions, and communications/management

Items considered in grading
Notebook quantity and quality
Wiki quantity and quality
Code repo quantity and quality
Communications efforts (includes class participation;
final semester presentation/demo)
Technical growth throughout semester

40

WEEK 1 ASSIGNMENTS
Review
Atlanta smart cities developments

http://www.bizjournals.com/atlanta/news/2016/01/05/at-t-aims-to-turn-atlanta-into-a-smart- city.html

AoT project information

https://arrayofthings.github.io/

https://github.com/waggle-sensor/waggle

Create GT GitHub account with your GT credentials

https://github.gatech.edu/
Familiarize yourself with the concepts of GIT repositories and revision control

Email the following no later than 2 days before next class:

Rank sub-team topic areas in terms of interest, and state any reasons why you

want to be on a team
For all class emails (besides between sub-teams):

To: [email protected] ; [email protected] ; [email protected]

Subject: [VIP] <etc.>

But always try to use Issue tracket, not email!

Returning students: prepare to discuss your previous progress in Week 2

41