Kolban's Book On C.H.I.P.
Kolban's Book On C.H.I.P.
Kolban's Book On C.H.I.P.
Table of Contents
Introduction The CHIP Book..................................................................................................................9
The CHIP story.........................................................................................................................................10
Introducing the CHIP...............................................................................................................................10
Physical architecture............................................................................................................................10
CPU.....................................................................................................................................................12
The storage subsystem.........................................................................................................................12
The power system................................................................................................................................12
Video output........................................................................................................................................15
Audio output........................................................................................................................................15
Keyboard input....................................................................................................................................15
Comparison against Raspberry Pi Zero...............................................................................................16
Getting Started.........................................................................................................................................17
Flashing new software.........................................................................................................................17
Installing packages...................................................................................................................................23
Installation tools..................................................................................................................................23
apt-cache.........................................................................................................................................24
apt-file.............................................................................................................................................25
apt-get.............................................................................................................................................25
apt...................................................................................................................................................26
dpkg................................................................................................................................................26
synaptic...........................................................................................................................................26
Available applications.........................................................................................................................26
Office suites word processors, spreadsheets, presentation packages...........................................26
PDF viewers...................................................................................................................................29
Graphics..........................................................................................................................................30
Email clients...................................................................................................................................32
Web Browsers.................................................................................................................................33
The CHIP Linux environment..................................................................................................................34
The shell..............................................................................................................................................34
The file system....................................................................................................................................35
Mounted file systems......................................................................................................................35
File permissions..............................................................................................................................36
Visualizing the File system.............................................................................................................36
Processes.............................................................................................................................................40
Kernel mode vs User mode.................................................................................................................42
Root privileges and running applications............................................................................................43
Memory management..........................................................................................................................44
The sysfs file system...........................................................................................................................45
Open file descriptors...........................................................................................................................45
System applications.............................................................................................................................48
Writing your own units...................................................................................................................50
The systemd-ui................................................................................................................................52
Storage.....................................................................................................................................................53
Page 2
Networking...............................................................................................................................................54
NetworkManager.................................................................................................................................56
Networking tools.................................................................................................................................57
ifconfig network command.............................................................................................................57
ip network command......................................................................................................................57
iwconfig network command...........................................................................................................58
iwlist network command.................................................................................................................58
iwconfig network command...........................................................................................................58
nmcli...............................................................................................................................................58
wpa_cli............................................................................................................................................59
wavemon network command..........................................................................................................59
Remote login.......................................................................................................................................60
Domain Name Service.........................................................................................................................60
DNS Service Discovery and Multicast DNS.......................................................................................62
Accessing your CHIP from the Internet.........................................................................................64
Sharing Windows files.........................................................................................................................64
Creating an access point......................................................................................................................67
Network File System (NFS)................................................................................................................69
Sockets API programming...................................................................................................................71
Finding an IP address from a hostname..........................................................................................71
Creating a socket client...................................................................................................................72
Creating a socket server..................................................................................................................72
Sending data through a socket........................................................................................................73
Receiving data through a socket.....................................................................................................73
Closing a socket..............................................................................................................................74
UDP programming..........................................................................................................................74
JavaScript socket.io.............................................................................................................................75
Sending an email.................................................................................................................................75
MQTT..................................................................................................................................................76
MQTT Protocol..............................................................................................................................78
Mosquitto MQTT...........................................................................................................................79
Building mosquitto from source................................................................................................80
Installing on Windows...............................................................................................................81
Writing MQTT clients....................................................................................................................84
Eclipse paho...............................................................................................................................84
C Mosquitto client library.......................................................................................................84
Node.js JavaScript MQTT......................................................................................................86
Browser JavaScript MQTT.....................................................................................................87
Ethernet...............................................................................................................................................90
X-Windows..............................................................................................................................................90
Using xming on a Windows PC..........................................................................................................91
Using mobaXterm on a Windows PC..................................................................................................94
Building X applications.......................................................................................................................97
Customizing the X desktop.................................................................................................................97
Running Headless..................................................................................................................................103
Using your PC as a graphical CHIP desktop.....................................................................................103
Connecting to CHIP using a serial connection..................................................................................104
Page 3
Connecting to CHIP using the USB cable.........................................................................................106
VNC Virtual Network Computing..................................................................................................107
Programming..........................................................................................................................................110
Programming in general....................................................................................................................113
ELF structure................................................................................................................................113
Libraries of code...........................................................................................................................114
Static vs Dynamic libraries......................................................................................................116
Building shared libraries..........................................................................................................116
General development tools...........................................................................................................117
ar...............................................................................................................................................117
ldd.............................................................................................................................................117
nm.............................................................................................................................................117
objdump...................................................................................................................................118
ranlib........................................................................................................................................118
readelf.......................................................................................................................................118
C Programming.................................................................................................................................118
pkg-config.....................................................................................................................................119
JavaScript Programming...................................................................................................................119
Node.js..........................................................................................................................................120
NPM.........................................................................................................................................120
Writing an NPM module..........................................................................................................122
Johnny-Five..................................................................................................................................123
The Pin object..........................................................................................................................125
I2C with Johnny Five...............................................................................................................126
CHIP specific Johnny Five objects..........................................................................................126
I2C Component Backpacks......................................................................................................126
Node-RED.........................................................................................................................................126
Node-RED CHIP local installation...............................................................................................128
Node-RED Windows installation.................................................................................................128
Node-RED writing flows...........................................................................................................133
Advanced Node-RED configuration.............................................................................................134
Including additional npm packages..............................................................................................135
Writing Custom Node-RED nodes...............................................................................................135
Node definition properties............................................................................................................138
Installing Custom Node-RED modules during development.......................................................139
Sharing Node-RED flows.............................................................................................................140
GPIO with Node-RED..................................................................................................................140
Security..................................................................................................................................................141
Security and userids...........................................................................................................................141
Audio......................................................................................................................................................141
USB attachable audio........................................................................................................................142
Linux ALSA Advanced Linux Sound Architecture........................................................................143
Pulse Code Modulation.................................................................................................................143
ALSA programming.....................................................................................................................144
ALSA device enumeration.......................................................................................................146
ALSA audio input / capture.....................................................................................................147
ALSA from JavaScript.............................................................................................................147
Page 4
Frequency Spectrum Analyzer..........................................................................................................147
The libsndfile library.........................................................................................................................149
Speech output....................................................................................................................................151
Logging..................................................................................................................................................151
Performance...........................................................................................................................................154
Resource Utilization..........................................................................................................................154
Memory Utilization......................................................................................................................154
Performance Commands...................................................................................................................155
Performance characteristics of different programming languages....................................................159
Hardware Interfacing.............................................................................................................................160
Terminal connections.........................................................................................................................160
GPIO interfacing...............................................................................................................................162
GPIO theory..................................................................................................................................163
GPIO on CHIP..............................................................................................................................163
GPIO using libsoc....................................................................................................................164
GPIO using Node.js.................................................................................................................165
I2C.....................................................................................................................................................166
I2C theory.....................................................................................................................................166
I2C on CHIP.................................................................................................................................169
I2C with libsoc.........................................................................................................................171
I2C with Node.js......................................................................................................................172
PWM.................................................................................................................................................172
PWM theory.................................................................................................................................172
PWM on CHIP..............................................................................................................................173
UART (Serial) Interface....................................................................................................................173
UART theory................................................................................................................................173
UART on CHIP.............................................................................................................................173
USB...................................................................................................................................................174
Bluetooth...........................................................................................................................................175
Bluetooth specification.................................................................................................................175
Bluetooth GATT.......................................................................................................................178
Service Discovery Protocol......................................................................................................179
Bluetooth C Programming............................................................................................................179
hci_get_route............................................................................................................................180
hci_open_dev...........................................................................................................................180
hci_inquiry...............................................................................................................................180
hci_read_remote_name............................................................................................................181
str2ba........................................................................................................................................181
ba2str........................................................................................................................................181
Bluetooth Audio............................................................................................................................182
Bluetooth RFCOMM....................................................................................................................182
Bluetooth tools..............................................................................................................................184
l2ping.......................................................................................................................................184
rfcomm.....................................................................................................................................184
bluetoothctl..............................................................................................................................185
hciconfig..................................................................................................................................186
hcitool......................................................................................................................................186
Page 5
gatttool.....................................................................................................................................186
Micro SD...........................................................................................................................................188
Hardware interfacing examples..............................................................................................................188
Simple GPIO Output change.............................................................................................................189
Simple GPIO Input change................................................................................................................192
GPIO extenders.................................................................................................................................194
PCF8574.......................................................................................................................................194
MCP23017....................................................................................................................................197
NeoPixels...........................................................................................................................................200
NeoPixel theory............................................................................................................................200
NeoPixels on CHIP.......................................................................................................................203
NeoPixels through the Arduino....................................................................................................203
NeoPixels through Node.js/Johnny-Five/Arduino I2C backpack................................................209
LCD dot matrix display HD44780.................................................................................................212
PCF8574 module..........................................................................................................................218
GPS....................................................................................................................................................220
The gpsd package.........................................................................................................................222
GPS from your cell phone............................................................................................................227
Video / Webcams...............................................................................................................................228
Streaming a webcam video...........................................................................................................231
Recording a webcam image for a period of time..........................................................................231
Recording a video.........................................................................................................................232
Playing video................................................................................................................................232
Barcode and QR Code scanning...................................................................................................233
OpenCV Webcam programming................................................................................................233
Analog to Digital conversion ADC0832........................................................................................234
Analog to Digital conversion ADS1015.........................................................................................238
Using JavaScript......................................................................................................................241
Accelerometer and Gyroscope MPU-6050 (aka GY-521).............................................................242
The math of Accelerometers.........................................................................................................249
Visualizing orientation..................................................................................................................250
Compass HMC5883L (aka GY-271) (aka CJ-M49).......................................................................250
Using WiringCHIP........................................................................................................................255
Using Johnny-Five........................................................................................................................256
Tilt compensation of the compass................................................................................................257
Using the shell..............................................................................................................................258
Temperature and pressure BMP180...............................................................................................258
Motion detectors Passive Infrared Sensor (PIR)............................................................................259
Ultrasonics HC-SR04.....................................................................................................................260
nRF24L01 networking......................................................................................................................262
Using the Arduino APIs................................................................................................................263
Graphic Equalizer..............................................................................................................................266
Microphone inputs.............................................................................................................................267
FM Radio...........................................................................................................................................267
LED 7-Segment displays...................................................................................................................269
Opto isolators / Opto couplers...........................................................................................................270
CD4051 Digital switches................................................................................................................272
Page 6
Software Defined Radio DVB-T Receiver.....................................................................................273
ADS-B Decoding..........................................................................................................................274
Bluetooth tracker iTag....................................................................................................................277
Real time clocks................................................................................................................................279
Arduino IDE on CHIP.......................................................................................................................282
Electronics..............................................................................................................................................283
Attaching your Chip to circuit boards...............................................................................................284
Drawing circuits and breadboards Fritzing....................................................................................284
Drawing Schematics Digi-Key Scheme-it.....................................................................................285
Drawing Schematics and PCBs CircuitMaker Software................................................................286
Using a logic analyzer.......................................................................................................................286
Overcoming trepidation of using Integrated Circuits........................................................................287
Logic Level Shifting..........................................................................................................................288
Transistors as switches......................................................................................................................290
Power inputs......................................................................................................................................291
Current and Voltage meters...............................................................................................................293
Web programming..................................................................................................................................294
Browser security................................................................................................................................294
REST requests...................................................................................................................................295
JSON processing in C...................................................................................................................295
Using postman for testing.............................................................................................................296
Writing a REST request using C...................................................................................................296
Writing a REST request using Node.js.........................................................................................296
Writing an App that listens for REST requests.............................................................................297
Listening for REST requests using Node.js.............................................................................297
WebSocket.........................................................................................................................................298
A WebSocket browser hosted application....................................................................................299
CHIP as a WebSocket server........................................................................................................300
WebSocket Server for C using libwebsocket...........................................................................301
WebSocket Server for C using noPoll......................................................................................304
WebSocket Server in JavaScript..............................................................................................305
Node.js as a Web Server....................................................................................................................306
Node.js and Express......................................................................................................................306
Express middleware.................................................................................................................306
Express routing........................................................................................................................306
Express WebSocket support.....................................................................................................307
Express as an https server........................................................................................................308
jQuery JavaScript framework.........................................................................................................308
Using jQuery.................................................................................................................................308
Making REST requests using jQuery...........................................................................................308
Using Bower......................................................................................................................................309
Robotics.................................................................................................................................................309
Heading in a direction.......................................................................................................................310
Projects...................................................................................................................................................311
Wiring-CHIP......................................................................................................................................311
Visualizing MPU6050 orientation.....................................................................................................311
Building libsoc...................................................................................................................................311
Page 7
Recommended packages........................................................................................................................312
How CHIP boots....................................................................................................................................312
Knowledge Sources................................................................................................................................316
Internet Relay Chat............................................................................................................................316
Research Areas.......................................................................................................................................316
Page 8
Introduction The CHIP Book
Howdy Folks,
I've been working in the software business for over 30 years but until recently, hadn't been playing
directly with Micro Processors. When I bought a Raspberry Pi and then an Arduino, I'm afraid I got
hooked. In my house I am surrounded by computers of all shapes, sizes and capacities any one of
them with orders of magnitude more power than any of these small devices however, I still found
myself fascinated.
As the number of available single board computers continues to grow, CHIP caught my attention. As
one of the cheapest on the marketplace yet remaining full featured ... I had to take a look ... and I liked
what I found.
As I studied the devices, I started to make notes and my pages of notes continued to grow and grow.
This book is my collated and polished version of those notes. Rather than keep them to myself, I offer
them to all of us in the CHIP community in the hope that they will be of some value. My plan is to
continue to update this work as we all learn more and share what we find in the community forums.
As such, I will re-release the work at regular intervals so please check back at the book's home page
for the latest. I do not plan on ever calling this book "finished" as there will always be more to add.
Because of the breadth and depth of material that we want to cover, it is unlikely that you will want to
read the book linearly from cover to cover. Instead, I recommend that you familiarize yourself with the
table of contents and come back to it as and when you need information.
As you read, make sure that you fully understand that there are undoubtedly inaccuracies, errors in
my understanding and errors in my writing. Only by feedback and time will we be able to correct
those. Please forgive the grammatical errors and spelling mistakes that my spell checker hasn't
caught.
The home page for the book is:
https://leanpub.com/chip
Please don't email me directly with technical questions. Instead, let us use the forum and ask and
answer the questions as a great community of CHIP minded enthusiasts, hobbyists and professionals.
Neil Kolban
Texas, USA
Page 9
The CHIP story
In May 2015, a small company called Next Thing Co launched a Kickstarter project for a new
MCU module which they called C.H.I.P. (which in this book I just call CHIP). The goal was to
produce a credit-card sized Linux computer for $9. Their initial ask was $50,000 to get the
project going. They received $2,071,927!!! The Kickstarter was successfully funded in June
2015 with 39,560 backers and from there the challenge was on. Kickstarter backers had to
have their early orders fulfilled and pre-orders beyond the Kickstarter campaign were taken. I
submitted an order for 2 instances of CHIP during the first week of January 2016 and
received them mid-July. It is taking a little time, but with patience and perseverance and with
the volume of CHIPs being made, it is hoped that their availability should increase.
The details of CHIP, and how to get the most from it is the subject of this book.
The operating system running on CHIP is based on the Linux Debian distribution.
Physical architecture
The following images are reproduced from the CHIP documentation pages. They illustrate
the major component of the physical structure of the device.
Page 10
The CHIP architecture is open source. Schematics for CHIP can be found on GitHub:
https://github.com/NextThingCo/CHIP-Hardware
Third party vendors have build cases for CHIP. The following is from C4Labs:
NextThingCo also make a $2 cover for the rear of the CHIP. I recommend some case over
the circuitry. Sooner or later you will put your CHIP down on something either wet or metallic
and be disappointed that you shorted something out (perhaps even damage the device).
See also:
Zebra CHIP Case
Page 11
CPU
The heart of any computer is its CPU. CHIP uses an Allwinner R8 processor running at
1GHz. The architecture of the chip is Arm Cortex-A8.
See also:
AllWinner R8
Allwinner R8 Datasheet
Allwinner R8 User Manual
This sounds troubling. It seems to indicate that the storage has errors and this is indeed true.
Flash memory checks the integrity of data re-flashed and validates that what was written was
what was expected. When errors are found, the blocks of storage used to hold data are
flagged as invalid and become no longer available for use, thus reducing the overall storage
size. In the current technology available to us, blocks can and do become (or initially some
are) invalid. This is the nature of the high density storage mechanisms in play. To determine
how many blocks are available and how many are bad, one can examine the output of dmesg
looking for "bad PEBs". For example:
$ dmesg | grep -i "bad PEBs"
ubi0: good PEBs: 2015, bad PEBs: 29, corrupted PEBs: 0
In the above, we see there are 29 blocks flagged as "bad" which is about 1.4%.
See also:
Single Board Revolution: Preventing flash memory corruption
Page 12
is off, then holding the button down for a second will power it on. If the CHIP is on, then
holding the button down for 10 seconds will power it down however, try and avoid that
method. The correct way to shutdown CHIP is to run the "shutdown" command. If we
remove power from the CHIP either by literally removing the power or holding the power off
button down, then we run the risk of corrupting our device as any files that may be in the
process of being updates may not complete and we will end up with corrupt files.
If we need to supply power to CHIP from an external source (as opposed to through the USB)
then we should apply the 5V voltage to the header terminals marked CGH-IN (U13-2) and
GND (U13-1 or U13-4). This input arrives at a power regulator. If we power the CHIP through
CHG-IN, the LEDs may flash briefly once when power is applied but the CHIP not appear to
boot. This is as it should be. Hold down the power button for 10 seconds and then the LEDs
will light and CHIP boot.
The AXP209 is an I2C device found on I2C bus 0 at device address 0x34.
CHIP has two on-board LEDs. One is called PWR and is connected directly to a 3.3V source.
This LED is magenta in color. This means that when power is active through the AXP209 (i.e.
CHIP is on) then the LED will be lit. There is no control over this LED. The second LED is
called STAT and is connected to one of the four GPIO lines (GPIO2) of the AXP209. That
means that it can be controlled from software. The AXP209 register that controls the signal
on GPIO2 is 0x93. This LED is white in color.
To switch off the LED, the following I2C command can be used:
$ sudo i2cset -f -y 0 0x34 0x93 0x00
The battery connector is a JST-PH 2 connector. Be sure and connect power the right way
around. +ve is to the right with the U14 header to the left. I failed to get this right and
immediately fried a precious CHIP. It made me very sad please don't make the same
mistake.
Page 13
LiPo batteries with the correct plugs can also be found.
The AXP209 also has an internal temperature sensor. The data is available via I2C by
reading registers 0x5E and 0x5F from the AXP209 device. The first register is the low order
4 bits of the result while the second byte is the next 8 bits giving us 12 bits in total. To
interpret the result, the data is in steps of 0.1C with a base of -144.7C. As such, our
calculation will be:
(((value(0x5e) & 0x0f ) | ((value(0x5f) & 0xff) << 4))/10) 144.7
A script is supplied with Linux image called "battery.sh". When run as root, it returns a wealth
of information about the battery subsystem:
$ sudo battery.sh
BAT_STATUS=0
CHARG_IND=1
BAT_EXIST=1
CHARGE_CTL=0xc9
CHARGE_CTL2=0x45
Battery voltage = 0mV
Battery discharge current = 0mA
Page 14
Battery charge current = 111.5mA
Internal temperature = 55.2c
CHIP might be corrupted if we power it down without first shutting down Linux. An elegant
program is available on GitHub called "blink" that runs as a service at boot time. This
program can (among other things) detect the button press and cause a clean shutdown to
occur. See:
https://github.com/fordsfords/blink
See also:
AXP209 Datasheet
Video output
A jack socket is found on CHIP into which a supplied cable can be plugged. The cable
terminates in composite video output of audio left, audio right and video.
Audio output
The jack socket used for composite video output can also be used for stereo output. A
standard 3.5mm audio plug such as those for headphones or amplifier input can be inserted.
Keyboard input
A USB keyboard can be plugged into the USB socket. These can be picked up very cheaply.
Thinking of the small form factor of CHIP, one of my favorites is the hand-held wireless
keyboards that are available for about $10-$15.
These devices use a 2.4GHz dongle that plugs into the USB port. Not only does one get a
hand held keyboard, but the mouse pad at the top works surprisingly well. Running the 4.4
Kernel image, I was delighted to find that it worked first time out of the box with no tweaks or
other configurations needed. Note that this is not a bluetooth keyboard and hence the dongle
needs to be inserted into the USB socket. Connection to the dongle can be validated by
running "lsusb" and looking for the USB ID of the dongle:
$ lsusb
Bus 002 Device 002: ID 25a7:0701
Page 15
Comparison against Raspberry Pi Zero
When CHIP was first mentioned, the closest equivalent on the market was the $35 Raspberry
Pi 2. At over 3 times the price of the proposed CHIP and without networking, it seemed that
they would be addressing different markets however, in October 2015, the Raspberry
foundation made a fantastic announcement the Raspberry Pi Zero. This is also a Linux
computer which is physically smaller than a credit card with the astonishingly low price of only
$5. I'm a great fan of choice and competition while at the same time not a fan of redundant
duplication ... I saw the arrival of two very similar devices in a mixed light. I don't hold a bias
on CHIP vs Raspberry Pi Zero (I've written books for both). Let us try and do as factual a
comparison as possible between the two:
Concept CHIP Raspberry Pi Zero
Price $9 $5
Availability Poor (July 2016) Constrained (July 2016)
Processor 1GHz 1GHz
RAM 512MB 512MB
Storage 4GByte flash Micro SD
WiFi yes no
Bluetooth yes no
USB yes (Type A) yes (Micro)
Video Composite HDMI
GPU support No Yes
I2C yes yes
SPI no yes
Serial yes yes
ADC no no
PWM no yes
Camera bus no yes
Battery charging yes no
OS Linux Linux
Community Fledgling Mature
If we compare exclusively on price, obviously $9 is more than $5 but let us not forget that in
order to get a Pi Zero running, you will need a Micro SD card. If you want to network, you will
also need a WiFi dongle and since the USB port on the Pi Zero is micro USB and the WiFi
dongles are USB Type A plugs, you will need an adapter. While none of these are particularly
expensive, the additions start to bring the prices in line with each other.
See also:
C.H.I.P. vs Pi Zero: Which Sub-$10 Computer Is Better?
Page 16
Getting Started
To boot up the CHIP, we merely need to supply power to the micro USB power socket. CHIP
will boot and be running. CHIP comes supplied with a cable that outputs composite video that
can be plugged into a TV or monitors with compatible inputs. During booting we will see the
Linux boot output scrolling by. At the end, we will be presented with a graphical environment
with which we can interact. And now things start to get interesting. We need a mouse and a
keyboard to be connected to allow us to interact
Now this is important before you start customizing your CHIP you need to ask yourself
the question about upgrading the software that is currently on it as shipped from the factory.
It is likely that what is on the CHIP is not the latest and greatest low-level kernels and
applications. Unfortunately, we can't yet do an in-place upgrade and must re-install the whole
image in order to get to the latest. When we do this re-install, ALL configuration changes,
applications and data that you place on the CHIP is lost.
If you think you might want to go to the latest level, see the section called Flashing new
software before going too much further.
When you have started CHIP, try hard not to simply power it off by removing power. That can
corrupt the storage. Instead issue the command "sudo shutdown now". This will cleanly
end the run of the device and after the command has shutdown the device, then you can
safely remove power from it. If you need to restart the device, you can run the command
"sudo reboot". The device will shutdown and then restart.
Your CHIP doesn't know where in the world it is. It will be able to get the universal time from a
time server over the internet, but without knowing where in the world you are, it doesn't know
your local time zone. To set this up, run the command:
$ sudo dpkg-reconfigure tzdata
From there you will be presented with a series of menus to help you setup your local
timezone.
Page 17
To start understanding how to flash the CHIP, visit the website called http://flash.getchip.com
using a copy of the Google Chrome Browser. The reason for using Chrome is that the flasher
application used to install the new image runs within the Chrome environment. Once there,
you will see a page similar to the following:
Once added, the page will now show the images that can be flashed onto the CHIP:
Page 18
The recommendation is now to use the "cloud" icon in the upper right of each image. This will
download the corresponding image file. These files end in ".chp". Once an image file has
been downloaded, we can find the following image to flash this file to the CHIP:
The first time we want to flash, we will see the following dialog:
Page 19
Now we install the driver. Clicking the "INSTALL THE DRIVER" button, downloads an
executable called "InstallDriver2.exe". When we run this, we get am installation
wizard:
Now we are ready to flash. To do this, we add a wire from the "FEL" terminal to a ground
Page 20
terminal.
Page 21
Unfortunately, in my experience, it wasn't that simple. When I plugged it in directly and then
followed the later steps, I ended up with failure. After using a powered USB hub, it
immediately started working.
Page 22
You can determine your new currently installed version by running the command:
$ uname -a"
Linux chip 4.4.11-ntc #1 SMP Sat May 28 00:27:07 UTC 2016 armv7l GNU/Linux
Installing packages
Only a subset of the possible available applications are pre-installed on CHIP. If you want to
install more, you can. The tools that are available for that are "apt-get". Before adding new
programs run apt-get with the "update" flag to refresh the catalog of available applications:
$ sudo apt-get update
This will determine what you have installed and perform the upgrades to get you to the latest
levels of these installed packages. Performing an upgrade can take some time so be sure
that you are available to leave the CHIP running un-attended for a while.
Installation tools
When we think of a Linux operating system, we will find that the state of a system is governed
by the existence and versions of files on the file system. If we consider a "package" to be a
collection of files necessary to perform some discrete operation, then we can also think of the
files on the Linux file system as being members of some package or another (with the obvious
Page 23
exception of application files that you create).
If we wish to install a package, we must obtain a copy of all the files associated with that
package and place them in the correct directories where they are expected to be found. If we
were attempt to do this by hand, we would end up in all sorts of troubles. Not least of which
would be the potentially of making mistakes with manual copies. In addition, some packages
are not self contained but instead have prerequisites which are themselves packages. What
we need is some registry mechanism where we can learn what packages are installed and a
repository where packages can be found and downloaded. This is exactly what the apt
technology does for us.
The apt technology is a suite of related applications which coordinate together to perform
package management functions.
To begin with, the apt system needs a list of sources from which packages can be found.
This is specified in a file called /etc/apt/sources.list. It is from this list of sources that
new packages and updates to existing packages can be found. A typical entry in the
sources.list file looks like:
deb http://ftp.us.debian.org/debian/ jessie main contrib non-free
deb-src http://ftp.us.debian.org/debian/ jessie main contrib non-free
It is important to note that a package that is downloaded and installed from the repository may
not be the very latest version of that package available. In fact, experience is showing that
some packages downloaded from the repository are very old indeed.
Next we will look at some of the primary commands for working with packages. The most
important of these is apt-get. We will be using apt-get extensively to install new
packages that aren't present in a default distribution.
See also:
apt-rpm.org
apt-cache
List available packages.
$ apt-cache pkgnames
Show statistics on the cache. Primarily what we see here is disk space consumed by the
cache as well as counts of the known packages.
$ apt-cache stats
Total package names: 55911 (1118 k)
Total package structures: 55916 (2684 k)
Page 24
Normal packages: 43172
Pure virtual packages: 655
Single virtual packages: 4455
Mixed virtual packages: 529
Missing: 7105
Total distinct versions: 46909 (3377 k)
Total distinct descriptions: 88535 (2125 k)
Total dependencies: 293481 (8217 k)
Total ver/file relations: 48617 (778 k)
Total Desc/File relations: 88535 (1417 k)
Total Provides mappings: 8201 (164 k)
Total globbed strings: 140 (1534 )
Total dependency version space: 1200 k
Total slack space: 72.4 k
Total space accounted for: 15.3 M
See also:
man(8) apt-cache
apt-file
The apt-file tool allows one to find which package might contain a specific file. Note that
apt-file needs to be installed with the following command:
$ sudo apt-get install apt-file
apt-get
The apt-get command is a command line interface to the Advanced Packaging Tool (APT)
library. It is by far the most commonly used of the package management tools. To install a
package we can run:
$ sudo apt-get install <pkgname>
The downloaded package will end in ".deb". We can then install that package using"
Page 25
$ sudo dpkg --install <fileName.deb>
If we just wish to see the effect of performing a command without actually commit to run it, we
can add the --simulate option.
To refresh our list of packages, we can run:
$ apt-get update
See also:
man(8) apt-get
25 Useful Basic Commands of APT-GET and APT-CACHE for Package Management
Linux and Unix apt-get command
apt
To see which packages are installed, run
$ apt --installed list
dpkg
The dpkg command is the base command for package installation. If we have a local ".deb"
file which contains a package, we can install it using:
$ sudo dpkg -i <filename.deb>
synaptic
There is also a GUI interface to package management called "synaptic". This can be installed
through:
$ sudo apt-get install synaptic
See also:
Wikipedia Synaptic
Available applications
There are hundreds (thousands?) of installable applications available for CHIP.
Page 26
be installed through:
$ sudo apt-get libreoffice
It's a big package and takes a bit of time to install, so be patient and grab a coffee.
After installation you will find the tools in the "Office" menu:
Page 27
Very intuitive and easy to learn.
The spreadsheet is called Calc and looks like:
Page 28
Again, if you are familiar with other spreadsheets, you will find this one quick to learn too.
See also:
LibreOffice
PDF viewers
I often find myself wanting to read PDF documents on my CHIP. A good tool for PDF reading
is Okular. It can be installed with:
$ sudo apt-get install oklular
There is quite a bit to it so it may take some time to install. Once installed, we run:
$ okular
and open any PDF files we may have. As we can see in the following screen shot, a pretty
comprehensive PDF viewer:
Page 29
See also:
Okular home page
Graphics
The GNU Image Manipulation Program or as it is more commonly called "GIMP". This can be
installed with:
$ sudo apt-get install gimp
Page 30
Once launched, it provides a rich assortment of tools for drawing and image manipulation:
Page 31
See also:
GIMP
Email clients
IceDove is a version of the popular Mozilla Thunderbird email client. It can be installed with:
$ sudo apt-get install icedove
IceDove has all the usual features including calendaring and newsfeed support. Once
installed it can be found under the Internet category:
Page 32
Web Browsers
The popular FireFox browser is available.
It provides the majority of functions one expects from a browser but you will be dissapointed
with the performance compared to a PC desktop. However, that doesn't stop us using it for a
variety of purposes including as a kiosk or as a host for browser based applications.
Page 33
The CHIP Linux environment
The operating system of CHIP is Linux based on the Debian distribution. There are many
excellent books, web pages and forums covering Linux so it is not my attempt to try and
recreate any of those. Instead, here is a very short summary to give you some of the core
commands and concepts you will find useful and use these as a springboard into further study
as needed.
See also:
Wikipedia Linux
The shell
Without exception, an environment on CHIP that you will be working with is the shell. The
shell is a command interpreter that takes the commands you enter at the keyboard and
executes them. Whether you use the "ssh" program to remotely login or start a terminal
emulator on the graphical desktop, both present you with an instance of the shell.
When you enter a command, the command is looked for in each of the directories specified
Page 34
by the PATH environment variable. The first directory which contains a file with the name of
your command is the one that is executed.
You can see a history of your previous commands by entering "history". A list of
commands prefixed by a number is displayed. Entering "!<num>" will cause that command
to be re-executed.
See also:
man(1) ln
Page 35
File permissions
On a Linux OS, files and directories have permissions associated with them. By and large
there are three permissions a file can have which are "read", "write" and "execute". These
declare whether a file can be read from, written to or be executed. A file also has the concept
of an owner and a group. When we talk about permissions, we further categorize the
permissions into those that are allowed for the owner of the file, those that are allowed for
members of the group associated with the file and those that are allowed for everyone else
(i.e. neither the owner nor a member of the group).
We thus end up with nine specific permissions:
owner read / write / execute
group member read / write / execute
others read / write / execute
An encoding scheme has been used for visualizing and describing permissions for files using
the octal numbering system (base 8). If we imagine that the three permissions were
described in 3 bit binary with an order of read / write / execute then we would have
permissions of "rwx" or [1/0][1/0][1/0]. So in binary, permissions of 101 would say we have
permission to read and execute but not write. If we translate this binary into octal, it would be
written as 4+0+1 = 5. Thus we can represent a set of read / write / execute permissions as a
single octal digit (0-7). Combining this with the notion that we have three sets of permissions
(user, group and other), we can completely describe all the permissions on a file with three
octal digits. For example 755 would be a value of 7 for user (read / write / execute) and 5 for
both group and other (read / execute).
The Linux command called chmod can be used to set the permissions. For example:
chmod 755 myFile
Does this sound complicated? Yes it does however it is how Unix has been handled since
its inception. There are other mechanisms should you not wish to remember these encodings
and those can be read about in the man pages for chmod.
Page 36
This can be started from the desktop context menu by running "File Manager". The feel of
this is similar to the Microsoft Windows Explorer environment. I'll assume that most of us are
familiar with navigating a file manager like this but here are some of the more interesting
features.
Another popular file manager is called "Nautilus". This is also provided on CHIP. It looks as
follows:
A third file manager is called "PCManFM" however this is not installed by default. We can
install it using:
$ sudo apt-get-install pcmanfm
Page 37
This is the file manager used by default on the Raspberry Pi Raspbian operating system so if
you are moving from a Pi environment to CHIP, this may be an attractive choice.
This can be started from the command line tool "pcmanfm" or from the icon in the task bar.
The feel of this is similar to the Windows Explorer environment. I'll assume that most of us
are familiar with navigating a file manager like this but here are some of the more interesting
features.
First there is a dual pane mode (View Dual Pane Mode / F3). The screen splits into two
parallel views where you can look at two distinct folders. From here you can drag and drop
files between them.
Sometimes the large icon view is distracting and you can switch to a detail view (View
Folder View Mode Detailed List View):
Page 38
Or you can go somewhere in between with the compact view:
You can open multiple tabs where each tab is its own folder view (File New Tab / CTRL+T).
Possibly one of the most important functions is the ability to open a shell window at the
current folder. We can do this from Tools Open Current Folder in Terminal / F4.
An especially elegant feature in this package is the ability to "bookmark" frequently used
folders. Once bookmarked, it appears on the list on the length and provides a quick
Page 39
navigation to that folder when needed.
The choice of which one you use is purely personal preference. To choose the default in use,
open up the "Settings Manager" and select "Preferred Applications":
In the Utilities section, we will find an entry called "File Manager" where we can specify
which of the available File Manager packages we want to use:
An alternative to the graphic environment is the tool called "tree" which visualizes a directory
structure as an ASCII tree layout. This program is not installed by default and one must install
using:
$ sudo apt-get install tree
See also:
Wikipedia Thunar
Wikipedia Nautulius (GNOME Files)
PCManFM File manager
Processes
A process is the name we give to a running instance of a program. When we start a program
it runs within its own context separate from the context of other programs. The process
Page 40
manages that context and includes the files that program opened, the executable running, the
environment variables, the user running the program and more. Each process is unique
identified with an integer identifier call the Process ID or PID. To see which processes are
running on our CHIP we can issue the "ps" command. This has a ferocious number of
options and deep reading of the manual page would be required to gain full appreciation. In
its basic form, when we run it it will show the PID and command associated with that pid. For
example:
$ ps
PID TTY TIME CMD
7232 pts/1 00:00:00 bash
7757 pts/1 00:00:00 ps
Notice that we get to see the PID column. Another command available to us is called
"pidof". This takes the name of a command and returns us the PIDs for all instances of that
command. For example:
$ pidof geany
4823
Once we know the PID of a running program, we can terminate it by sending it a "kill"
signal. These are asynchronously transmitted signals that indicate to the process that some
external event has happened. To see the list of possible signals run:
$ /bin/kill -L
1 HUP 2 INT 3 QUIT 4 ILL 5 TRAP 6 ABRT 7 BUS
8 FPE 9 KILL 10 USR1 11 SEGV 12 USR2 13 PIPE 14 ALRM
15 TERM 16 STKFLT 17 CHLD 18 CONT 19 STOP 20 TSTP 21 TTIN
22 TTOU 23 URG 24 XCPU 25 XFSZ 26 VTALRM 27 PROF 28 WINCH
29 POLL 30 PWR 31 SYS
To send a signal we can use one of the following two syntax formats:
kill -<number> <PID>
or
kill -<name> PID
For example:
kill -1 4823
and
kill -HUP 4823
are identical. Each signal can be trapped by the process that receives it allowing the process
to determine how it wishes to handle its arrival. The one exception is (9) KILL which is
un-trapable and causes the immediate termination of the process.
The way that a new process is created is through a technique called "forking". There is a low
level system call named fork() that, when executed, causes a brand new instance of the
original process to be created. The new instance shares everything from the first one
including open files, variable values, environment variables everything. The difference
between them is that the return code from fork() call for the original process is the process
id for the new process while the new process receives a value of 0. In the new forked
Page 41
process, it is then very common for a second system call to be made called exec(). What
this does is load a new executable into the address space of the existing process (the forked
copy) and begin its execution. If this feels a trifle convoluted, I can't argue with that
however it does provide a mechanism for a newly started process to inherit the environment
of the starter which can include aspects such as current userid, group membership, open files
and more. And that is a desirable situation.
When we run a new shell command, that also causes a new process to run. The shell starts
the process and then waits for it to complete. If we don't wish to wait, we can run it in the
background by adding an "&" to the end of the command.
For example, running:
$ geany myfile.c &
causes the geany editor to open/create a file and immediately detaches it from the starting
terminal allowing us to run additional commands in that terminal.
From within a C program, we can also start another process by using the system() call.
This takes the file system path of an executable and starts it while waiting for it to complete.
This is a quick way of performing work by letting sub-programs do complex jobs for us. For
example, should we wish to play a sound, we can call aplay() to play the sound through the
system() call.
See also:
man(1) ps
Page 42
Root privileges and running applications
Linux is a multi-user operating system. This means that multiple distinct users can
simultaneously login to a single system and run applications concurrently with each other. It
is not desirable for one user to be able to read the files from another and hence there are
permission capabilities associated with each individual user. However, there is a special user
in a Linux system that is called "root". This user is also know as the "superuser". Root is not
constrained by normal security constraints. Root can terminate any process and read/write to
any file. As such, root is normally given only to the most trusted system administrators and
ordinary users run with their ordinary userids. However, on the CHIP, there are times when
we wish to access hardware or run powerful commands (anything that affects the system as a
whole is usually considered to be under this category). A special command is supplied called
"sudo" that takes as a parameter another command to run. That second command is then
executed with root permissions. Effectively, it escalates the privileges of the command to root
just for the duration of that process.
A question that arises from time to time is that we know that the default userid is "chip" with
a password of "chip", what then is the default password for "root"? The answer is that
there is none and by that we mean that login as root is disabled. If you want to become root,
then run:
$ sudo -i
On occasion there is discussion in the forums on the correctness of having to run some user
written CHIP applications as root. On a production, multi-user operating system, I wouldn't
question that notion but I maintain that the CHIP is not that. Since I can't imagine a CHIP
user not knowing the root password or being allowed to run sudo, the CHIP is already
compromised from a multi-user perspective. From a system stability point of view, any
program that runs as root can potentially destroy the system. As such, take back-ups of your
Page 43
system as needed so that you have a restore point. In my experience, I have yet to make
such a serious accidental error that my system has been destroyed.
The sudo command is governed by a "/etc/sudoers" file which lists the users that are
authorized to elevate their permissions via sudo. To add a user to the set of sudo authorized
users, we can run:
adduser <username> sudo
This will add the user to the "sudo" group and members of that group are authorized to run
sudo. After the change, the user must logout and log back in again for the change to take
effect.
See also:
Security and userids
Memory management
In a Linux environment, each application sees its own address as a virtual address space.
They believe that they are alone in memory and the operating system takes care of mapping
the application addresses to the real memory addresses. Since CHIP has a 512MBytes of
RAM, this would seem to imply that the maximum number of programs that CHIP can run is
constrained. The operating system and related components use some RAM and then there is
the cost for running each of your own applications in parallel. In most Linux systems, there is
the concept of swap space on disk. What that means is that should we start to run low on
available RAM, applications that haven't touched memory in a while can have some or all of
their allocationw written to disk. This is called paging. The data that was in RAM isn't lost ...
its safely stored on disk and hence can be recovered ... however the space used by what is
now on disk is available for re-use. Should the original application need the memory again,
the operating system will re-load the memory, swapping it out with some other programs
allocation if needed.
Now we need to remember that there isn't actually a disk on CHIP, instead we have flash
memory. While in principle, we could swap memory to flash, that is a really bad practice.
When the operating system is performing swap operations, it is potentially reading and writing
lots and lots data. While flash is great, it does "wear" and the amount of data being moved for
OS swapping would just cause it to wear that much faster.
To list the processes sorted by their memory usage, we can use:
$ ps auxw sort=-vsz
or
$ ps auxw sort=-%mem
See also:
Memory Utilization
Page 44
The sysfs file system
In the Linux world, a guiding principle is that everything can be considered a file on the file
system. If we look in the /dev directory we will see entries for terminals, serial ports, printers
and video input devices. Most of these have physical counterparts and access is quite simple
... we read and write and are done. But what of more subtle interactions such as I2C, PWM,
or GPIO. Linux has added a concept that is a virtual file system called "sysfs". This file
system is implemented in the OS kernel and provides a bridge from user applications
(applications that run in "user space") to the drivers in the kernel for the components in
question.
On CHIP, the file system is automatically mounted at /sys.
At the highest level we have:
sys
|-- block
|-- bus
|-- class
|-- dev
|-- devices
|-- firmware
|-- fs
|-- kernel
|-- module
`-- power
The structure is not arbitrary and the locations and names of the files and directories have
meanings.
block Entries for block oriented devices.
bus Entries for data buses. This includes I2C and SPI. For each entry in bus, that
directory itself contains two directories called "devices" and "drivers":
devices An entry for each device connected to that bus.
drivers Drivers associated with the bus
class Entries for devices by class.
devices Entries for each known device.
See also:
Wikipedia sysfs
The sysfs Filesystem
GPIO on CHIP
Page 45
on the file system or a simply a file descriptor of an opened physical device, from the
Raspbian process perspective, it is a file descriptor. Since these are owned and managed by
the kernel itself, there is the possibility of using a tool to show us what file descriptors a
process has open or, conversely, look at a device or file and ask what processes have it open.
The tool in question is called "lsof". It is not installed by default so it must be manually
installed from the "lsof" package:
$ sudo apt-get install lsof
This command has a deep richness of options and great study of the man page would be
needed to understand all the possibilities. However, there are some common recipes that
one can use without knowing too much additional details.
Here is a recipe that shows which processes have a particular file open.
$ sudo lsof /dev/tty1
COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME
agetty 634 root 0u CHR 4,1 0t0 1042 tty1
agetty 634 root 1u CHR 4,1 0t0 1042 tty1
agetty 634 root 2u CHR 4,1 0t0 1042 tty1
When it comes to networking, lsof is again extremely valuable. The primary TCP/IP
networking API is called sockets and uses file descriptors as the means of maintaining
context for a connection. Running "sudo lsof -i" lists the socket domain protocols.
$ sudo lsof -i
COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME
rpcbind 282 root 6u IPv4 5269 0t0 UDP *:sunrpc
rpcbind 282 root 7u IPv4 5272 0t0 UDP *:881
rpcbind 282 root 8u IPv4 5273 0t0 TCP *:sunrpc (LISTEN)
rpcbind 282 root 9u IPv6 5274 0t0 UDP *:sunrpc
rpcbind 282 root 10u IPv6 5275 0t0 UDP *:881
rpcbind 282 root 11u IPv6 5276 0t0 TCP *:sunrpc (LISTEN)
rpc.statd 291 statd 5u IPv4 5364 0t0 UDP chip1:891
rpc.statd 291 statd 8u IPv4 5371 0t0 UDP *:57196
rpc.statd 291 statd 9u IPv4 5374 0t0 TCP *:46133 (LISTEN)
rpc.statd 291 statd 10u IPv6 5377 0t0 UDP *:39540
rpc.statd 291 statd 11u IPv6 5380 0t0 TCP *:49870 (LISTEN)
sshd 308 root 3u IPv4 8381 0t0 TCP *:ssh (LISTEN)
sshd 308 root 4u IPv6 8383 0t0 TCP *:ssh (LISTEN)
avahi-dae 322 avahi 12u IPv4 6229 0t0 UDP *:mdns
avahi-dae 322 avahi 13u IPv6 6329 0t0 UDP *:mdns
avahi-dae 322 avahi 14u IPv4 6331 0t0 UDP *:46349
avahi-dae 322 avahi 15u IPv6 6332 0t0 UDP *:52287
ntpd 390 ntp 16u IPv4 6590 0t0 UDP *:ntp
ntpd 390 ntp 17u IPv6 6591 0t0 UDP *:ntp
ntpd 390 ntp 18u IPv4 6689 0t0 UDP chip1:ntp
ntpd 390 ntp 19u IPv4 6690 0t0 UDP 172.20.0.1:ntp
ntpd 390 ntp 20u IPv6 6691 0t0 UDP localhost:ntp
ntpd 390 ntp 21u IPv6 6692 0t0 UDP [fe80::cc79:cfff:fe21:db95]:ntp
ntpd 390 ntp 25u IPv4 8001 0t0 UDP 192.168.1.109:ntp
ntpd 390 ntp 26u IPv6 8002 0t0 UDP [fe80::ce79:cfff:fe21:db95]:ntp
dnsmasq 404 dnsmasq 4u IPv4 6698 0t0 UDP *:bootps
dnsmasq 404 dnsmasq 6u IPv4 6701 0t0 UDP *:domain
dnsmasq 404 dnsmasq 7u IPv4 6702 0t0 TCP *:domain (LISTEN)
dnsmasq 404 dnsmasq 8u IPv6 6703 0t0 UDP *:domain
dnsmasq 404 dnsmasq 9u IPv6 6704 0t0 TCP *:domain (LISTEN)
Page 46
sshd 778 root 3u IPv4 9000 0t0 TCP 192.168.1.109:ssh-
>192.168.1.200:49712 (ESTABLISHED)
sshd 783 root 3u IPv4 9038 0t0 TCP 192.168.1.109:ssh-
>192.168.1.200:49713 (ESTABLISHED)
sshd 791 chip 3u IPv4 9000 0t0 TCP 192.168.1.109:ssh-
>192.168.1.200:49712 (ESTABLISHED)
sshd 791 chip 10u IPv6 9219 0t0 TCP localhost:6010 (LISTEN)
sshd 791 chip 11u IPv4 9220 0t0 TCP chip1:6010 (LISTEN)
sshd 791 chip 13u IPv6 9251 0t0 TCP localhost:6010->localhost:36970
(ESTABLISHED)
sshd 791 chip 16u IPv6 9269 0t0 TCP localhost:6010->localhost:36972
(ESTABLISHED)
sshd 791 chip 17u IPv6 9276 0t0 TCP localhost:6010->localhost:36974
(ESTABLISHED)
sshd 791 chip 18u IPv6 9341 0t0 TCP localhost:6010->localhost:36976
(ESTABLISHED)
sshd 791 chip 19u IPv6 9348 0t0 TCP localhost:6010->localhost:36978
(ESTABLISHED)
sshd 791 chip 20u IPv6 9391 0t0 TCP localhost:6010->localhost:36980
(ESTABLISHED)
sshd 791 chip 21u IPv6 9464 0t0 TCP localhost:6010->localhost:36982
(ESTABLISHED)
sshd 791 chip 22u IPv6 9472 0t0 TCP localhost:6010->localhost:36984
(ESTABLISHED)
sshd 791 chip 23u IPv6 9622 0t0 TCP localhost:6010->localhost:37002
(ESTABLISHED)
sshd 791 chip 24u IPv6 9508 0t0 TCP localhost:6010->localhost:36988
(ESTABLISHED)
sshd 791 chip 25u IPv6 9513 0t0 TCP localhost:6010->localhost:36990
(ESTABLISHED)
sshd 791 chip 26u IPv6 11156 0t0 TCP localhost:6010->localhost:37046
(ESTABLISHED)
sshd 791 chip 27u IPv6 9644 0t0 TCP localhost:6010->localhost:37008
(ESTABLISHED)
sshd 791 chip 28u IPv6 9609 0t0 TCP localhost:6010->localhost:37000
(ESTABLISHED)
sshd 791 chip 30u IPv6 10280 0t0 TCP localhost:6010->localhost:37020
(ESTABLISHED)
sshd 791 chip 31u IPv6 9671 0t0 TCP localhost:6010->localhost:37012
(ESTABLISHED)
sshd 791 chip 32u IPv6 9744 0t0 TCP localhost:6010->localhost:37014
(ESTABLISHED)
sshd 791 chip 33u IPv6 11455 0t0 TCP localhost:6010->localhost:37108
(ESTABLISHED)
sshd 797 chip 3u IPv4 9038 0t0 TCP 192.168.1.109:ssh-
>192.168.1.200:49713 (ESTABLISHED)
xfce4-ses 808 chip 3u IPv6 9275 0t0 TCP localhost:36974->localhost:6010
(ESTABLISHED)
dbus-laun 811 chip 3u IPv6 9250 0t0 TCP localhost:36970->localhost:6010
(ESTABLISHED)
dbus-laun 811 chip 4u IPv6 9268 0t0 TCP localhost:36972->localhost:6010
(ESTABLISHED)
xfwm4 820 chip 3u IPv6 9340 0t0 TCP localhost:36976->localhost:6010
(ESTABLISHED)
xfce4-pan 824 chip 3u IPv6 9390 0t0 TCP localhost:36980->localhost:6010
(ESTABLISHED)
Thunar 826 chip 3u IPv6 9471 0t0 TCP localhost:36984->localhost:6010
(ESTABLISHED)
xfdesktop 828 chip 3u IPv6 9463 0t0 TCP localhost:36982->localhost:6010
(ESTABLISHED)
Page 47
volti 830 chip 5u IPv6 9643 0t0 TCP localhost:37008->localhost:6010
(ESTABLISHED)
blueman-a 834 chip 4u IPv6 9740 0t0 TCP localhost:37014->localhost:6010
(ESTABLISHED)
pk-update 838 chip 3u IPv6 9512 0t0 TCP localhost:36990->localhost:6010
(ESTABLISHED)
nm-applet 840 chip 4u IPv6 9621 0t0 TCP localhost:37002->localhost:6010
(ESTABLISHED)
xfce4-pow 841 chip 3u IPv6 9507 0t0 TCP localhost:36988->localhost:6010
(ESTABLISHED)
polkit-gn 843 chip 3u IPv6 9608 0t0 TCP localhost:37000->localhost:6010
(ESTABLISHED)
xfsetting 844 chip 3u IPv6 9347 0t0 TCP localhost:36978->localhost:6010
(ESTABLISHED)
panel-6-s 850 chip 3u IPv6 9670 0t0 TCP localhost:37012->localhost:6010
(ESTABLISHED)
xfce4-ter 936 chip 4u IPv6 10279 0t0 TCP localhost:37020->localhost:6010
(ESTABLISHED)
geany 1091 chip 3u IPv6 11155 0t0 TCP localhost:37046->localhost:6010
(ESTABLISHED)
node 1141 chip 11u IPv6 11392 0t0 TCP *:3000 (LISTEN)
nautilus 1152 chip 8u IPv6 11454 0t0 TCP localhost:37108->localhost:6010
(ESTABLISHED)
This command is particularly useful is you are writing server applications and some other
process already has the port you want to listen upon open. You can use "lsof" to determine
which process that might be.
To filter the results we can use the "-i<filter spec>" where the filter specification is:
[46][protocol][@hostname|@IP Address][:service|:port]
See also:
man(8) lsof
System applications
When Linux starts, it needs to know which services it needs to also start. Examples of
Page 48
services would be demons such as web servers, NFS servers and more. The package called
systemd provides management of such requests.
To see the list of units defined we can run:
$ systemctl list-units
This not only shows the defined units but their status. The columns shown include:
UNIT The name of the unit
LOAD
ACTIVE
SUB
DESCRIPTION A description of the unit
We can qualify the list-units command with a --type=<type> flag to filter on just
specific types of units. For example:
$ systemctl list-units --type=service
To see the status of a specific service we can add the service name:
$ systemctl status <service name>
This last command will also show the script used to start the service.
To start a service we can run:
$ sudo systemctl start <service name>
The details of what a particular unit is going to do when started or stopped can be found by
running:
$ systemctl cat <service name>
Page 49
Understanding and Using Systemd
YouTube: Tutorial systemd Basics
[Service]
EnvironmentFile=-/etc/default/ssh
ExecStart=/usr/sbin/sshd -D $SSHD_OPTS
ExecReload=/bin/kill -HUP $MAINPID
KillMode=process
Restart=on-failure
[Install]
WantedBy=multi-user.target
Alias=sshd.service
At a high level, a configuration file is composed of a series of sections with the section name
in square brackets. Following those are a series of name=value properties for the
corresponding section.
Amongst the sections we have:
[Unit]
[Service]
[Install]
For the [Unit] section we will invariably wish to include a Description line that provides an
English description of what the unit means. For example:
[Unit]
Description=My service that does XYZ
In the [Service] section, here we describe how to start a particular service. There are
many possible options here and we will touch on just a few. The first is the Type line. This
describes how the service will be started. The value of simple means that systemd will
simply fork() and exec() the service. This is commonly just what we want. A value of
forking means that again systemd will fork() and exec() the service but that service
will itself fork() and exec() other instances. Next we have the ExecStart line. This is
Page 50
vital. This specifies the path and parameters that will be used to start the service. The
WorkingDirectory line (if present) specifies the directory that will be the working directory
for the newly started service. The User line is used to name a local Linux user that the
service will run as.
After having edited a unit file, we must run systemctl daemon-reload to cause systemd
to re-scan its configuration files and unit files.
$ sudo systemctl daemon-reload
Which will cause the service to be started. Hopefully now you may get an aha moment.
What this means is that for each service defined, we as administrators of our CHIP Linux OS
no longer have to remember mechanics of how to start daemons. All we need to know is their
logical service name and issue a start command.
But what if we want to stop a service? Simply issuing:
$ sudo systemctl stop testService
will stop it. By default, the way that systemd performs that task is to execute a kill against
the previously remembered process id of the start request. If we have an alternate shutdown
procedure, we can define another statement in our [Service] section called ExecStop.
This command should stop the previously started service in whatever clean manner is
possible. A special variable called $MAINPID is made available to the command which is the
process Id of the previously started service.
We have now touched upon what it takes to start and stop a service manually, but what if we
want the service to start automatically? We can again do this with systemd but it takes a
little more consideration. The first thing to realize is that services typically depend on each
other. For example, there is no point in starting the e-mail processing service until and unless
the network services have already been started. We can define relationships between units
with additional statements in the [Unit] section. The first one we will touch upon is the
Requires statement. This names additional units which should also be started when the
current unit is started. We also have the statements Before and After. These define the
relative order in which multiple Units should be started. For example if we have:
[Unit]
Requires=xyz.service
After=xyz.service
then we have declared that when this current unit is started we had better also start the
xyz.service but it must be started after xyz.service has in fact started.
Page 51
For example, if we wish our service to start but only after we have networking available, we
might code:
[Unit]
After=network-online.target
There are other statements that are similar to Requires but with slightly altered semantics.
Requires states that if a dependent unit fails to start then so do we. The Wants statement
again lists units to be started but should they fail, then our unit will still be allowed to start.
If we wish to automate the start-up of a service, in the Unit file for that service we should add
an [Install] section. This will include a "WantedBy" entry which will list the unit that, if it is
started will also start our new unit. A good example would be "multi-user.target". For
example,
[Install]
WantedBy=multi-user.target
See also:
man systemd.service
The systemd-ui
A graphical user interface is available for systemd operations. It can be installed from the
systemd-ui package using:
$ sudo apt-get install systemd-ui
Page 52
Storage
Your CHIP uses flash memory for storing data. The CHIP has a 4GByte flash chip built in.
This storage holds the operating system, your installed applications and data that your
applications need. Now 4GBytes seems like a lot but compared to a PC which might have
hundreds of GBytes (or even a terabyte or more), it isn't huge. However it is more than
sufficient for the majority of tasks we will need to perform. Before you start working on your
CHIP and creating new files, some words of caution and some suggestions. Since the CHIP
storage is physically wired flash memory as opposed to a removable storage medium such as
micro SD memory cards, if your CHIP device becomes damaged or broken, you may never
be able to retrieve your data from the device. Think about that for a moment. Imagine you
have spent weeks or months writing an application to run on CHIP and the source code to
your app resides on the file system contained in the flash memory of the CHIP. Now you find
that your CHIP no longer boots and a sinking feeling comes over you. Obviously, good
practice is to take backups regularly however, few of us do make those backups as often as
we should. If we weren't using CHIP but using a single board computer with removable
storage (eg. a Raspberry PI), we could also get ourselves a new device and remove the
storage from the now defunct system and place it in the new system and we are back up and
running. We can't do that with CHIP.
I do a lot of programming and if I were to lose my data I'd be very unhappy. What I do instead
is have external network attached storage that can be network "mounted" onto my CHIP. This
could be an NFS export from a Linux based PC or an export from a dedicated network back-
up system such as the Western Digital "My Cloud" devices. These can be plugged into your
LAN without a dedicated PC and expose a few gigabytes of shared storage.
To determine how much space is available, run the command "df -k":
$ df -k
Filesystem 1K-blocks Used Available Use% Mounted on
ubi0:rootfs 3777124 733344 3043780 20% /
devtmpfs 253292 0 253292 0% /dev
tmpfs 253552 0 253552 0% /dev/shm
tmpfs 253552 6756 246796 3% /run
tmpfs 5120 4 5116 1% /run/lock
tmpfs 253552 0 253552 0% /sys/fs/cgroup
tmpfs 50712 4 50708 1% /run/user/1000
The result shows the different file systems using the storage and how much is used and
remains. The file system of most interest to us is the one mounted at "/" which is called
"ubi0:rootfs". If we look at this example output, we see that its size is 3777124 1K
blocks. Thinking in terms of 1K blocks isn't that useful and fortunately there are other flags.
For example:
$ df -kBM
Filesystem 1M-blocks Used Available Use% Mounted on
ubi0:rootfs 3689M 717M 2973M 20% /
Page 53
devtmpfs 248M 0M 248M 0% /dev
tmpfs 248M 0M 248M 0% /dev/shm
tmpfs 248M 7M 242M 3% /run
tmpfs 5M 1M 5M 1% /run/lock
tmpfs 248M 0M 248M 0% /sys/fs/cgroup
tmpfs 50M 1M 50M 1% /run/user/1000
shows us the same output but this time in 1Mbyte blocks. From this we quickly see that the
file system hosts 3.7GBytes of which the operating system and installed apps uses
717MBytes leaving us 2.9GBytes free.
A graphical tool for showing disk usage is called "baobab". It is not supplied by default with
CHIP but can be installed with:
$ sudo apt-get install baobab
While resulting in a very rich and powerful interface, my experience is that it takes too long to
build the data for rendering.
See also:
man(1) df
Baobob
Networking
The CHIP hardware provides two distinct networking layers. Those are WiFi and Bluetooth.
WiFi is used to create or join a wireless TCP/IP network while Bluetooth is the communication
style used for personal device connections with a more limited range.
Page 54
If we are in a graphical environment, we can configure networking by running the command
"nm-connection-editor" as root. For example:
$ sudo -E nm-connection-editor
Page 55
Notice the DNS servers are 8.8.8.8 and 8.8.4.4. These are public DNS servers offered and
managed by Google (thank you Google!).
You are also likely going to want to change the host-name of your CHIP. The default is
"chip" but if you happen to have multiple instances of CHIP, they will step on each other.
Running "nmtui" allows us to change the value:
Experience seems to show that although this does change the host-name of CHIP, we also
need to edit the /etc/hosts file and change the name there.
NetworkManager
The core of the networking subsystem in CHIP is provided by the NetworkManager daemon.
There are many ways to interact with this component including:
nmcli Command line tool
Page 56
nmtui Text based user interface
nm-applet GUI task-bar tool
nm-connection-manager GUI user interface
See also:
WikiPedia NetworkManager
NetworkManager home page
man nmcli
Networking tools
ifconfig network command
This is a classic Unix command that is now claimed to be deprecated in favor of "ip".
One of the primary commands is to list configured ip addresses. For example:
$ ip addr
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: wlan0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP group default qlen
1000
link/ether 00:13:ef:80:3d:bd brd ff:ff:ff:ff:ff:ff
inet 192.168.1.103/24 brd 192.168.1.255 scope global wlan0
valid_lft forever preferred_lft forever
inet6 fe80::7846:21a0:ecc2:95e7/64 scope link
valid_lft forever preferred_lft forever
3: wlan1: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc mq state DOWN group default
qlen 1000
link/ether 00:36:76:21:97:a3 brd ff:ff:ff:ff:ff:ff
inet6 fe80::70e:70ab:40bc:f69a/64 scope link tentative
valid_lft forever preferred_lft forever
See also:
man(8) ifconfig
ip network command
To say that the "ip" command does a lot is a bit of an understatement. It has a wealth of
options.
See also:
man(8) ip
ip command cheat sheet
Page 57
iwconfig network command
See also:
man(8) iwconfig
See also:
man(8) iwlist
See also:
man(8) iwconfig
nmcli
The nmcli is the Network Manager Command Line Interface. It has a wealth of options.
Some of the more interesting ones to us include:
nmcli device wifi list Produce a listing of a scan of access points
See also:
man nmcli
Page 58
wpa_cli
wpa_cli -i <if> status List the status of an interface. For example:
bssid=a4:2b:8c:81:47:95
freq=2437
ssid=sweetie
id=0
mode=station
pairwise_cipher=CCMP
group_cipher=CCMP
key_mgmt=WPA2-PSK
wpa_state=COMPLETED
address=00:36:76:21:97:a3
uuid=00b41cd5-94ef-5ab0-804a-f8bd3adf72af
See also:
man(8) wpa_cli
Page 59
Remote login
To login from one machine to another we can use a variety of different techniques. It is likely
the case you want to login from your PC into your CHIP, however the reverse is also possible
depending on what is running on your PC. We will assume for just now that the goal is to get
from the PC to CHIP.
One of the most common remote login tools is called ssh. We can use the command:
$ ssh <userid>@<host>
to open a new session to the remote host and login with the supplied userid. At this point, we
will be prompted for the password of the userid. It is common in our development
environment that we will be doing lots of remote ssh commands and we want our local PC
userid to be a trusted partner to the remote. If our PC is running a Linux flavor, the following
recipe can be followed to set up trust between the two. In our story we will assume that the
PC has a hostname of "host-pc" and that CHIP has the hostname of "host-chip". We will
also assume that the userid on the PC is "pcuser" while on CHIP it will be "chip".
On the PC, in the user's home directory, run:
$ ssh-keygen -t rsa
When prompted for a pass phrase, simply hit enter and not provide one.
On CHIP, login as "chip" and make a directory in the home directory called ".ssh".
Back on the PC, in the user's home directory, we want to copy the public key into the newly
created directory on CHIP appending to a new file called "authorized_keys". We can do
this with:
$ cat .ssh/id_rsa.pub | ssh chip@host-chip 'cat >> .ssh/authorized_keys'
on the PC, we will immediately be logged into CHIP without a password challenge.
There are alternative commands for remote login such as telnet and rlogin but ssh is
considered the current best practice.
Page 60
communicate with the target directly. As you can imagine the database of IP addresses to
names could be incredibly large and it is. In fact it is what is known as a distributed
database meaning that the mapping is itself partitioned down into lower and lower levels. So
instead of one massive database, there are smaller and smaller databases that have
administrative control over subsections. If we were to think of it in a hierarchical fashion, we
have databases for each of the "top level domains". These are:
com
org
net
gov
etc etc etc
For each entry in those databases, they then point to machines which may then host further
databases for example "google.com" would look up the database entry at "com" for the
entry for "google". If we further ask for "play.google.com", this would then look up the
entry for "google.com" which would then ask a DNS server there for the address of the
machine called "play". Thankfully, we don't have concern ourselves with these levels of
details when actually resolving host names to IP addresses when using CHIP. Instead, we tell
CHIP about a single DNS server that is part of the hierarchy and we send it requests. It in
turn knows the locations of top level domain servers and the requests get forwarded up the
chain until we achieve resolution. There are other optimizations at play including caching
which make this more than acceptably fast. The question though is how does CHIP find
the address of the original DNS server that it should use to begin with? The answer to this
depends on how CHIP connects to your network. If you are using DHCP, then when the
network starts your CHIP will contact the DHCP server on your own network and will be boot-
strapped with all the information it needs from the DHCP server. This obviously moves the
problem further to the right but the chances are high that your DHCP server is built into
your modem supplied by your internet service provider (ISP) such as your cable or DSL
provider. When they supplied the modem, they configured the up-stream services and your
modem "by magic" gets the information it needs.
However there is also the concept of your LAN. Addresses in the range 192.168.1.x are
defined to be local addresses. This means that the network traffic on that LAN lives only
locally and doesn't propagate into the Internet. For example, in my house, my CHIP might
have the IP address 192.168.1.2 while in your house, your CHIP might have exactly the same
address. This means that you can connect to other devices in your own house that have IP
addresses in the range 192.168.1.1 to 192.168.1.254 with impunity while I can do the same
and none of us will tread on each other. This too is great but what now if I don't want to
remember that 192.168.1.2 is my CHIP while 192.168.1.3 is Windows PC while 192.168.1.4
is a Linux box. What we want is the ability to resolve host names just as we do with DNS.
For example, I might want to ping "mychip" or "mywindows" or "mylinux" by name rather
than remember the addresses.
This is where the system file called /etc/hosts comes into play. This is a text file which we
Page 61
can edit. Within the file we can add lines of the form:
<IP Address> <hostname>
for example:
192.168.1.2 mychip
192.168.1.3 mywindows
192.168.1.4 mylinux
Once added to the file, any attempt to resolve these host names would succeed. What
happens is that the name resolution subsystem of CHIP OS first looks for a match of host
name in /etc/hosts and, if not resolved there, then makes contact with a DNS server for
Internet resolution.
If we know the host name of a machine and want to see its IP address, we use the command:
$ host <hostname>
for example:
$ host www.google.com
www.google.com has address 216.58.217.4
www.google.com has IPv6 address 2607:f8b0:400f:802::2004
Note that the order of resolution for a host name is governed by the content of the file called
/etc/nsswitch.conf.
Page 62
The avahi-daemon utilizes a configuration file found at /etc/avahi/avahi-daemon.conf.
The default name that avahi advertizes itself as is the local hostname.
When host-name resolution is performed, the system file called /etc/nsswitch.conf is
used to determine the order of resolution. Specifically the hosts entry contains the name
resolution. An example would be:
hosts: files mdns4_minimal [NOTFOUND=return] dns
Which says "first look in /etc/hosts, then consult mDNS and then use full DNS". What this
means is that a device which advertizes itself with mDNS can be found via a lookup of
"<hostname>.local". For example, if I boot up CHIP which gets a dynamic IP address
through DHCP and the hostname of that machine is "chip1", then I can reach it with a
domain name address of "chip1.local". If the IP address of the device changes,
subsequent resolutions of the domain name will continue to correctly resolve.
Avahi tools are not installed by default but can be installed using the "avahi-utils"
package:
$ sudo apt-get install avahi-utils
To see the list of mDNS devices in your network, we can use the avahi-browse command.
For example:
avahi-browse -at
+ wlan1 IPv6 chip1 [ce:79:cf:21:db:95] Workstation local
+ wlan1 IPv4 chip1 [ce:79:cf:21:db:95] Workstation local
+ wlan0 IPv6 pizero [00:36:76:21:97:a3] Workstation local
+ wlan0 IPv6 raspi3 [b8:27:eb:9d:fc:60] Workstation local
+ wlan0 IPv6 chip1 [cc:79:cf:21:db:95] Workstation local
+ wlan0 IPv4 pizero [00:36:76:21:97:a3] Workstation local
+ wlan0 IPv4 raspi3 [b8:27:eb:9d:fc:60] Workstation local
+ wlan0 IPv4 chip1 [cc:79:cf:21:db:95] Workstation local
+ wlan0 IPv6 pizero Remote Disk Management local
+ wlan0 IPv6 raspi3 Remote Disk Management local
+ wlan0 IPv4 raspi3 Remote Disk Management local
+ wlan0 IPv4 pizero Remote Disk Management local
+ wlan0 IPv4 WDMyCloud Apple File Sharing local
+ wlan0 IPv4 WDMyCloud _wd-2go._tcp local
+ wlan0 IPv4 WDMyCloud Web Site local
+ wlan0 IPv4 Living Room _googlecast._tcp local
+ wlan0 IPv4 123456789 _teamviewer._tcp local
To access an mDNS advertized server from Microsoft Windows, you will need a service
similar to Apple's Bonjour installed. Bonjour is distributed as part of Apple's iTunes product.
Once installed, we should be able to access the published servers at their <name>.local
address. A partner windows tool called "Bonjour Browser" is available which displays an
mDNS listing of servers on windows.
See also:
avahi home page
man(1) avahi-browse
man(5) avahi-daemon.conf
Page 63
Bonjour Browser
Zero Configuration Networking (Zeroconf)
Within CHIP OS, we will see a number of demon programs that collaborate together to
provide the Samba services. These include:
smbd The demon that makes the data available from the CIFS server and provides
security control.
Page 64
nmbd Implements the NetBIOS Name Server or NBNS which is also called WINS
from a Windows perspective.
On the windows side, we have some useful networking commands including:
ipconfig
net
nbtstat List NetBIOS over TCP information.
Let us assume that we wish to share files from CHIP so that they are accessible from a
Windows machine.
To start we need to add a package to our CHIP environment. This is called samba.
$ sudo apt-get install samba
After the samba package is installed, we want to set a samba password for our sharing user
using the command "smbpasswd":
$ sudo smbpasswd -a chip
Having made the configuration changes, we stop and restart the Samba demons using:
$ sudo systemctl stop smbd
$ sudo systemctl start smbd
If we now open a windows explorer and enter the IP address of our CHIP (in the following
example my CHIP is at 192.168.1.103):
Page 65
We see the exposed shares. If we try and open it, we will be prompted for a userid and
password:
Page 66
/var/log/samba
An obvious next question after having seen how to expose a CHIP file system to Windows is
can CHIP access a Windows file system? On most Linux variants (including Raspberry Pi)
the answer is a solid yes. Sadly, at the time of writing, the answer is no for CHIP. In order to
work with Windows file systems, a kernel module called "cifs" needs to be present and in the
current builds of CHIP, it simply is not there.
See also:
How to Create a Network Share Via Samba Via CLI (Command-line interface/Linux Terminal) -
Uncomplicated, Simple and Brief Way!
Next we want to define wlan1 as an interface and give it a static IP address. We edit the file
/etc/network/interfaces:
$ sudo nano /etc/network/interfaces
Page 67
iface wlan1 inet static
address 172.20.0.1
netmask 255.255.255.0
What remains is to configure the hostapd program. This is the daemon which handles
connection requests from WiFi stations. We need to configure it with a number of
parameters. We create a file called /etc/hostapd.conf using:
$ sudo nano /etc/hostapd.conf
Of the above lines, you are likely going to be want to be customized for your own
environment. The line which starts "ssid=". This is where you supply the SSID (network)
name of your WiFi environment. This is what WiFi stations (end users) will see as an
available WiFi access point.
This configuration file has to be associated with hostapd. We edit the file called
/etc/default/hostapd:
$ sudo nano /etc/default/hostapd
Within the file we find the line that references DAEMON_CONF and change its value to point to
our configuration file. We will change the line to read:
DAEMON_CONF="/etc/hostapd.conf"
Now if we browse the WiFi network, we will find a new access point called "chip_ap".
Page 68
You will notice that we can connect to it without providing any credentials. That isn't a great
situation as it allows anyone to connect to your CHIP. What we will do next is add a passkey
that the WiFi station needs to present for authentication. To do this we edit the
/etc/hostapd.conf file and add the following lines:
wpa=2
wpa_key_mgmt=WPA-PSK
wpa_passphrase=password
The "wpa_passphrase=" line contains the password you want users to enter to be able to
connect.
It is important to realize that the current CHIP kernels do not support network address
translation (nat). What this unfortunately means is that you can not route traffic receive
through the wlan1 interface out through the wlan0 interface. The implication of this is that you
can't use CHIP as either a WiFi router, bridge or extender.
See also:
How to create a very basic WiFi access point on CHIP
Wikipedia hostapd
Hostapd home page
hostapd configuration file
dnsmasq
man(8) dnsmasq
Page 69
branches and the story continues. In Linux, we have a directory called "/" also known as the
root of the file system. Beneath this we have more and more directories. In early Unix, each
physical disk could be formatted with the Linux file system and "grafted" onto the hierarchical
structure. For example, if I created a physical disk that I wanted to use as the storage area
for all my user's data, I could graft this disk onto the "/home" directory of the file system.
When a user transition as follows:
$ cd /
$ cd home
Then the transition into "home" resulted in access to the storage of the disk drive. The act of
grafting disk storage onto the Linux file system is called "mounting" and we can now say that
we have "mounted the /home file system" and that "/home" represents a mount point where
we transition from one file system to another. This mechanism allows us to extend the Linux
file system with more storage as needed.
With the arrival of fast networking, an additional concept was brought to light. What if we
could graft onto our local file system a "reference" to a file system owned by a remote
machine? A transition through the mount point where the remote file system is attached
would then cause network requests to read and write data from the remote machine. This
idea has been implemented in Unix systems through the Network File System (NFS).
The Network File System (NFS) allows us to mount directories hosted by one Linux machine
on the file system of another. For example, imagine the CHIP exposed a local file system and
our development Linux system mounted that exposed file system. We could then edit and
compile files on our PC desktop and the files would immediately be present on the CHIP.
To setup NFS on a Linux machine, we need the following packages installed:
nfs-common
nfs-server
portmap
To see what has been exported from a remote NFS server, run:
$ sudo showmount -e <host>
to see what has been exported from our local NFS server run:
$ sudo showmount -e
Page 70
3. Reload the NFS configuration by running
$ sudo exportfs -r
To un-mount an NFS mounted file system, we can use the unmount command. If we try and
access an NFS mounted file system and the network is not present or down, we will block the
caller. If we want to force an un-mount an NFS file system while the network is down, we
must use the unmount -l command which performs a lazy un-mount.
Page 71
mechanism to convert a hostname into its corresponding IP address. Fortunately there is a
function that will do just that for us. It is called "gethostbyname". It and its related data
structures are defined in the header called "netdb.h". To use, we invoke
gethostbyname() passing in a string representation of the hostname or a string
representation of the IPv4 dotted decimal address. What we get back is a pointer to a
struct hostent. This structure has been populated by the gethostbyname() call and
we can examine its content to obtain the results. Specifically, to find the IP address we can
use:
hostent.h_addr
The return type of this is a "char *" which will point to the IP address. For example:
struct hostent *myHost = gethostbyname("www.kolban.com");
char *ipAddress = myHost->h_addr;
If we can't resolve the hostname or some other error has occurred, the return from
gethostbyname() is NULL.
The addresses returned are already in network byte order so no conversion from host byte
order is needed nor should be performed.
Having just explained gethostbyname() we are now going to say that it has been mostly
superseded by the getaddrinfo() call.
See also:
man(3) gethostbyname
Page 72
for incoming connections requests from clients. Since the server doesn't know when a client
will connect, it executes an accept() call which normally blocks waiting for a client
connection to arrive. When a client does connect, accept() returns with a new socket
connection that is ready to be used and constitutes the connection to the client.
int s = socket(PF_INT, SOCK_STREAM, IPPROTO_TCP);
struct sockaddr_in serverAddress;
serverAddress.sin_family = AF_INET;
serverAddress.sin_addr.s_addr = htonl(INADDR_ANY);
serverAddress.sin_port = htons(port);
If we don't care about the address information of the client then the address structure and
length parameters can be supplied as NULL.
See also:
man(2) accept
man(2) bind
man(2) listen
See also:
man(2) send
Page 73
man(2) recv
Closing a socket
When either the server side or the client side of the socket connection wishes to indicate that
it has finished, it should close the socket with a call to close().
See also:
man(2) close
UDP programming
Where TCP socket programming is stream based and data pushed in one side appears on
the partner side, UDP socket programming works on the concept of datagrams. A datagram
is a packet of data that is transmitted from one end to another as a unit. The maximum size
of a datagram is 64K. Unlike TCP, there is no assured delivery. What this means is that if a
sender sends a datagram it is possible for the datagram to never reach the intended recipient
and neither end will know it was lost. UDP is a fire and forget protocol. With these apparent
limitations, what then is the value of UDP? The first is that because it is datagram oriented, it
has a smaller overhead in terms of transmission. There is no receipt of delivery and hence no
extra chatter. Since a datagram is delivered as a unit, it also means that there is a logical
record boundary which is not present in TCP. However the most compelling reason for UDP
is that there needs to be no pre-acquired connection between a sender and a receiver. This
means that a sender can send a datagram to a recipient without first having to form a
connection. A significant value for this is that a sender can broadcast a datagram to a set of
endpoints and all the recipients will receive a copy. Contrast this with TCP where if I want to
send data to a set of machines, I have to send a distinct copy of that data to each partner.
We use UDP by creating a socket but flag it as a datagram:
int s = socket(AF_INET, SOCK_DGRAM, 0);
Next, on the server, we bind the socket to a local port to act as the endpoint using the
bind() API.
Finally, on the server, we can block waiting for an incoming datagram using the recvfrom()
API.
On the client, we perform a socket() call to create a socket and then execute a sendto()
API call to send a datagram to the target server.
See also:
man(2) sendto
man(2) recvfrom
Page 74
JavaScript socket.io
A package for JavaScript called "socket.io" provides a wealth of network programming
capabilities. To install, we can use npm:
$ npm install socket.io
See also:
npm socket.io
Sending an email
There may be times where you wish to send an email from your CHIP. Hopefully it isn't
because a sensor had detected that your house is on fire (although you may actually want
that). Maybe you want to receive an email when Johnny comes home from school. In this
case you could build a motion sensor outside the door that, when triggered, takes a photo of
the scene and sends it to your camera.
The way mail works on a Linux system is that a shell program called mail communicates with
a local program called sendmail. The mail is handed off to sendmail and then sendmail
relays it onwards out onto the Internet. The sendmail application also handles incoming
emails and it can become quite complex. If all we want to do is send emails from CHIP, there
is an easier solution through the package called ssmtp.
First we want to install ssmtp using:
$ sudo apt-get install ssmtp
# The place where the mail goes. The actual machine name is required no
# MX records are consulted. Commonly mailhosts are named mail.domain.com
mailhub=smtpout.secureserver.net
[email protected]
AuthPass=*********
UseTLS=NO
UseSTARTTLS=NO
# Where will the mail seem to come from?
rewriteDomain=kolban.com
Page 75
FromLineOverride=YES
After installation we are ready to send emails, however the mail sending tools are also not yet
installed. To install those, we need to install the package called mailutils:
$ sudo apt-get install mailutils
Once installed we have the classic mail command. To send an email from the command line
we can use:
$ echo "Here is your mail" | mail -s "Your Mail" -A /etc/passwd [email protected]
With this in place, we can now send an email from almost any app. For example, when a
sensor attached to the Pi detects something un-toward, we can send an email to alert
someone.
One of the things to consider about emails is that they are "people interrupters". When an
email arrives we usually want to attend to it. Receiving irrelevant emails or emails that don't
tell us anything we didn't already know can be a strong negative. As such, be very cautious
when writing applications that send emails. At a minimum, think through the frequency with
which emails are sent. If you have a sensor which, when it triggers can trigger again
repeatedly, design your solution so that only one email is sent and not hundreds of emails that
will merely clutter (and annoy) the recipient.
MQTT
The MQ Telemetry Transport (MQTT) is a protocol for publish and subscribe style messaging.
It was originally invented by IBM as part of the MQSeries family of products but since has
become an industry standard governed by the Oasis standards group. The latest
specification version is 3.1.1.
Being Pub/Sub, this means that there is a broker (an MQTT Broker) to which subscribers can
register their subscriptions and publishers can submit their publications. Publications and
subscriptions agree on the topics to be used to link the messages together. A client can be a
publisher, a subscriber or both.
The value of MQTT is that it can be used to deliver data from an application running on one
machine to an application running on another. Immediately we seem to see an overlap
between MQTT and REST calls but there are some major differences. In a REST
environment, when you form a connection from a client to a server, the server must be
available in order for the client to deliver the data. With MQTT that is not necessarily the
case. The client can publish a message which can then be held by the broker until such time
as the receiving application comes on-line to retrieve it. This is a store and forward
mechanism.
Every message must have a topic associated with it.
Topics are broken into topic levels. There are wild-cards.
+ Single topic level wild-card
Page 76
eg. a/+/c
would subscribe to a/b/c and a/x/c.
# Multi topic level wild-card
eg. a/#
would subscribe to a/<anything>.
MQTT is built on top of TCP/IP. Clients connect to the broker (not to each other) over a TCP
connection.
There is a quality of service requested by a client. This is encoded in the QoS field:
QoS=0 Send at most once. This can lose messages. At most once means perhaps
never.
QoS=1 Send at least once. This means that the message will be delivered. Saying
this another way, a message will not be discarded or lost. However, duplicates can
arrive i.e. the message can be delivered twice or more.
QoS=2 Send exactly once. This means that the message will not be lost and will be
delivered once and once only.
MQTT also has the capability to buffer messages for subsequent delivery. For example, if a
client subscriber is not currently connected, a message can be queued or stored for delivery
to the client when it eventually re-connects. We call a client that is not connected an off-line
client. For a subscription, we have the choice to deliver all the queued messages for a client
or just the last message. To understand the difference, we can imagine a published message
that says "I will be attending the party" we want all such messages sent to the client
because they are all of interest. However if we think of a published message of "Today's
forecast is sunny and warm" then there may be no need for old messages and only the
current weather forecast is of interest to us. The last message published on a topic is called
the "retained message". When a client subscribes, it can ask to receive the last message
immediately so even if a subscription takes place after a previous publication, it can still
receive data immediately.
Clients make their status known to the broker so the broker can tell if a client is connected.
This is achieved via a keep-alive/heartbeat.
If a client connection is lost because of a network disconnection, the broker can detect that
occurrence. This is where we get morbid. We define this as the client having "died". In the
real world, when someone dies, there may be a last "will and testament" which are the
desired of what the person wanted to happen when they die. MQTT has a similar concept. A
client can register a message to be published in the event of the clients death. This is
remembered by the broker and in the event of the client dieing, the broker will perform the
role of the attorney and publish the last registered "will and testament" message on behalf of
the deceased client.
See also:
Page 77
MQTT Hive
Mosquitto
MQTT.org
Oasis MQTT spec 3.1.1
Mosquitto,org
YouTube: Internet of Things Why You Need MQTT
OASIS Message Queuing Telemetry Transport (MQTT)
OASIS MQTT V3.1.1 Specification
MQTT Protocol
Protocol
Client sends Connect
Broker replies with Connack
Connect MQTT packet
clientId The identity that the client claims to be ...
cleanSession
username
password
lastWillTopic
lastWillQos
lastWillMessage
keepAlive
Connack
sessionPresent
returnCode
Publish
packetId
topicName
qos
retainFlag
payload
dupFlag
Page 78
Puback
??
Pubrec
packetId
Pubrel
packetId
Pubcomp
packetId
Subscribe
packetId
qos
topic1
qos2
topic2
...
Suback
Disconnect
Mosquitto MQTT
One of the most prevalent implementations of MQTT is called Mosquitto and is available as
an open source implementation. On a Linux system we would install with:
$ sudo apt-get install mosquitto
Unfortunately, the version of Mosquitto supplied by the repository is 1.3.4 which is quite old
and pre-dates many important functions such as websocket support. To resolve this, you may
have to download the latest version and build yourself.
Where systemd is installed, mosquitto is controlled by it. To see if it is running execute
$ systemctl status mosquitto
mosquitto.service - LSB: mosquitto MQTT v3.1 message broker
Loaded: loaded (/etc/init.d/mosquitto)
Active: active (running) since Fri 2016-08-05 23:44:17 CDT; 1min 1s ago
CGroup: /system.slice/mosquitto.service
3046 /usr/sbin/mosquitto -c /etc/mosquitto/mosquitto.conf
Page 79
mosquitto_sub and mosquitto_pub. These are distributed as part of the mosquitto-
clients package.
As a simple test, open two terminal sessions. In one run:
$ mosquitto_sub -t greeting
You will see the published message appear in the subscriber window.
Let us now dig a little further into a mosquitto configuration file. Within there, we can
specify multiple "listener" entries. These can bind to a port or a port and interface. Within
a listener section, we can configure the protocol that the listener should use. By default, it will
be plain TCP for normal client connections but we can specify that the protocol should be
"websockets". For example, adding the following into the mosquitto.conf file:
listener 8081
protocol websockets
will cause mosquitto to listen for websocket clients on port 8081. If you want to listen on
both standard mqtt protocol and websocket, then you will need multiple listener definitions.
listener 1883
protocol mqtt
listener 8081
protocol websockets
See also:
Moqsuitto.org
man(1) mosquitto_sub
man(1) mosquitto_pub
man(5) mosquitto.conf
MQTT Community Wiki
Page 80
to
WITH_UUID:=no
and
WITH_SRV:=yes
to
WITH_SRV:=no
and
WITH_WEBSOCKETS:=no
to
WITH_WEBSOCKETS:=yes
If we are enabling websockets, we must also install the websockets libraries. See the
following for instructions:
https://libwebsockets.org/index.html
In summary, we clone a git project, within the project directory create a directory called
"build", in the build directory execute "cmake .." and when that completes, execute a
"make" against the Makefile that was just built for us.
This install needs cmake which can be installed via:
$ sudo apt-get install cmake
Installing on Windows
We can download and install a version of Mosquitto for Windows. Binary downloads are
available here:
http://mosquitto.org/download/
The version illustrated here is 1.4.8.
Page 81
Page 82
Page 83
Writing MQTT clients
Having an MQTT environment available is nice but without clients doesn't add much value.
Your clients will be the applications that will register subscriptions and publish messages. The
subscriber will name the topic on which messages published will be received.
While the MQTT protocol may be standardized, the client APIs appear to not be. As such
there are multiple APIs for any given language.
Eclipse paho
See also:
Paho home page
Practical MQTT with Paho
Page 84
mosquitto_reinitialize Destroy a client and then create a new one
mosquitto_username_pw_set Set the userid and password for authentication
mosquitto_will_set Set the last will of the client for estate planning purposes
mosquitto_will_clear Revoke the last will
mosquitto_connect Connect a client to a broker
mosquitto_connect_bind Same as mosquitto_connect but constrains
interface to bind with
mosquitto_connect_async Asynchronous connection to the broker
mosquitto_reconnect Reconnect to a broker after a lost connection
mosquitto_reconnect_async Same as mosquitto_reconnect but
asynchronous
mosquitto_disconnect Disconnect a client from a broker
mosquitto_publish Publish a message on a given topic
mosquitto_subscribe Subscribe to messages on a topic
mosquitto_unsubscribe Unsubscribe to messages on a topic
mosquitto_loop Perform processing for the mosquitto client. It is here that
incoming message from the broker are received or previously unsent publications
transmitted.
mosquitto_loop_read
mosquitto_loop_write
mosquitto_loop_misc
mosquitto_loop_forever Same as mosquitto_loop but does not return and
keeps processing until the client is disconnected
mosquitto_socket Retrieve the low level TCP/IP socket that the mosquitto
client is using
mosquitto_want_write Return true if there is pending data to be sent to the
broker
mosquitto_loop_start Start a thread to process a mosquitto_loop in the
background
mosquitto_loop_end Stop a previously started loop that was created using
mosquitto_loop_start
Page 85
#include <string.h>
#include <mosquitto.h>
mosquitto_lib_init();
struct mosquitto *mosq = mosquitto_new(
NULL, // Generate an id
true, // Create a clean session
NULL); // No callback param
int rc = mosquitto_connect(
mosq, // Client handle
host, // Host of the broker
port, // Port of the broker
false); // No keepalive
if (rc != MOSQ_ERR_SUCCESS) {
printf("Error with connect: %d\n", rc);
return(rc);
}
mosquitto_publish(
mosq, // Client handle
NULL, // Message id
topic, // Topic
strlen(message)+1, // Length of message
(const void *)message, // Message to be sent
0, // QoS = 0
false); // Not retained
mosquitto_destroy(mosq);
mosquitto_lib_cleanup();
}
See also:
man(3) libmosquitto
mosquitto.h
Page 86
var mqtt = require("mqtt");
var client = mqtt.connect("mqtt://pc9100");
client.on('connect', function() {
client.subscribe("greeting");
});
client.on('message', function(topic, message) {
console.log("Received message for topic=" + topic + ", message=" + message);
});
Error handling can be accommodated by registering a client handler for an error event:
client.on("error", function(error) {
console.log("We detected an error: " + error);
});
The full details of the API can be found on the NPM home page for the MQTT package. At a
high level, the functions are:
connect Connect to a broker.
Client.publish Publish a message. The message content may be a Buffer or
a string. The identity of the topic on which the message is being published must
also be supplied.
Client.subscribe Subscribe to a topic.
Client.unsubscribe Unsubscribe from a topic. The signature is:
client.unsubscribe(topic, [options], [callback])
Page 87
The MQTT broker chosen must support MQTT protocol through WebSocket. Popular brokers
such as Mosquitto are able to perform that role. From a security standpoint, browsers are
only allowed to make WebSocket requests back to the HTTP server from which the page
running in the browser was originally loaded. As such, the browser can only make
WebSocket requests to the MQTT broker if the MQTT broker served up the page. We can
alleviate that issue by introducing full function Web Servers and proxies into the story but for
simple purposes, brokers such as Mosquitto can serve up static web pages directly. This
means that we can point our browser to Mosquitto which will then look for an HTML file on the
local file system to Mosquitto and send that back to the browser. The browser will now see
the MQTT broker as the server of the web page and when the JavaScript in that web page
asks for an MQTT subscription, it will succeed. Within the Mosquitto configuration file, each
listener definition that wishes to also provide HTTP files must supply an "http_dir" option
which names the directory that will be searched for HTML file requests from the browser.
We are not limited running an MQTT broker on a separate server machine, we can run the
MQTT broker directly on the Pi should we desire.
Now let us look and see what is needed to run an MQTT client in the browser.
First we need to download the Paho JavaScript client. If we visit the Paho downloads page:
https://projects.eclipse.org/projects/technology.paho/downloads
we will find an entry for "JavaScript client 1.0.1". Follow through that link and download the
ZIP file. At the time of writing the result will be:
paho.javascript-1.0.1.zip
we can then unzip the file using unzip. Within we will find a file called mqttws31.js. This
is the JavaScript source of the client and needs to be included in the HTML file that will be
using MQTT. Here is an example HTML with embedded JavaScript illustrating us connecting
to a broker and registering a subscription. When a message is published to the matching
topic, it is logged to the browser console.
<html>
<head>
Page 88
<script src="mqttws31.js"></script>
<script>
var client = new Paho.MQTT.Client("192.168.1.101", 8081, "/", "CLIENT1");
client.onMessageArrived = function(message) {
console.log("Message arrived ...: " + message.payloadString);
};
client.onMessageDelivered = function(message) {
console.log("Message delivered ...");
};
client.connect({
onSuccess: function() {
console.log("On success ... we are now client connected to the broker!");
client.subscribe("greeting", {
onSuccess: function() {
console.log("Subscription success ...");
},
onFailure: function(context, errorCode) {
console.log("Failed to make a subscription ... code=" + errorCode);
}
});
},
onFailure: function(context, errorCode, errorMesage) {
console.log("On failure ... code=" + errorCode + ", message=" + errorMesage);
}
});
</script>
</head>
<body>
</body>
</html>
Within the mosquitto.conf file, we will have an entry that looks like:
listener 1883
protocol mqtt
listener 8081
protocol websockets
http_dir /mnt/pc/projects/robot
This defines that we will listen on port 1883 for standard MQTT protocol while we will also
listen on port 8081 for websocket HTTP requests and if a request to load a page arrives, we
will serve it from a given directory.
The API to publish a message is called "send" and takes as a parameter an MQTT message
object.
See also:
Eclipse Paho JavaScript client
Paho API JavaScript documentation
The Mosquitto MQTT broker gets Websockets support
Page 89
Ethernet
While the CHIP by itself does not support Ethernet, there is a plugin board available which
adds an Ethernet connection. The board is called "Hummus" and is available from Chip Dip
Shop:
See also:
Chip Dip Shop Hummus
X-Windows
Linux has a graphical user interface called X-Windows. When you start Linux and connect
your CHIP to a TV, it is X-Windows that provides the graphical environment into which you
can open windows and run other graphics oriented applications.
Unlike other graphics systems, X-Windows was designed to operate in a client/server mode.
Since Unix is a multi-user environment, it isn't reasonable to think that all users could sit
around the same graphics terminal so a mechanism to run one's applications on Unix while
having the graphical results displayed on a local terminal was needed. The X-Windows
environment provides a client/server architecture. In that model, the client is the application
that wishes to display graphics and the server is the software or hardware that the client
draws upon. It is the server that an end user sits in front off.
Page 90
It is likely you have a PC as your normal workstation environment that includes a monitor,
keyboard and mouse. I like to use this as my environment for working with CHIP. In order to
do such, I run an X-Windows server on my windows PC, open an SSH connection to my
CHIP, point the X-Windows output back to my PC and then start the CHIP X-Windows
desktop. The result of this is a window on my PC which is the desktop of CHIP. This allows
me to sit in one seat and switch between working on the CHIP and working on the PC.
Although the CHIP is fantastic, it will be unlikely to be anywhere near as powerful as your PC
which is likely loaded with your favorite browser and other goodies.
On my Windows PC, I use the free version of xming (http://sourceforge.net/projects/xming/) to
run a local X-Server.
To start the CHIP desktop on the X-Server output, use xfce4-session.
When the device running an application wishes to display its output on an X-Windows server,
the application has to know which X-Windows server to use. After all, an X-Windows server is
merely an application sitting at the end of TCP/IP connection. The way to tell the application
where to send the graphics request is through an environment variable called DISPLAY. The
format of the variable is <HOST or IP>:<Screen>. Since an X-Windows server can
display multiple screens, we specify which screen to use. 99 times out of 100 we want to use
0.
Here is an example of use:
$ export DISPLAY=192.168.1.2:0
$ xfce4-terminal
X-Windows provides a level of security configuration to prevent any arbitrary computer from
accessing your screen (note that this means reading your screen as well as writing to it). On
your X-Windows server environment, you can control this by running the "xhost" command.
I typically run "xhost +" which allows any machine on my local network to access the X-
Windows server. This should normally not pose an undue security risk as my X-Windows
servers are on my local LAN inside my house and there is no inbound route to my X-Windows
servers through my router / firewall from the Internet.
Page 91
Since X-Windows is itself a windowing environment and we are already running on a
windowing environment (Microsoft windows itself), we have the option of how to map the two
together. The choice here is a matter of taste. I prefer the one window flavor.
Page 92
The next screen asks us whether or not we want to start a remote application when xming
starts. I usually select no.
Page 93
The next screen is about server settings. I want clipboard copying enabled and I also disable
access control.
The final screen allows us to start the server and optionally save the configuration.
Once done you will see the X-Windows server background which is a dull checkerboard like
appearance:
What remains for you to do now is connect into your CHIP and run xfce4-session.
Page 94
plus file browsing and a host of other features.
When started, we can select to attach to a remote Linux system.
Once selected, we are presented with a very nice terminal emulator which is set up for X
clients and provides a file browser:
Page 95
specify that we create a single desktop window in which other X-Windows open. We do that
in the configuration settings:
Another tip when using mobaXterm is to create a macro for starting the desktop. A macro is a
recording of keystrokes/command that can be played back from a menu selection. I create a
directory called "bin" in my home directory and in there create a shell script that I call
"runx.sh". In that script I code:
#!/bin/bash
nohup xfce4-session > /dev/null 2> /dev/null &
This causes the desktop window manager to startup in the background. I then create a
macro that runs:
~/bin/runx.sh
and then when I start an SSH session into CHIP, I find a menu entry for the macro:
which starts the CHIP X-Windows desktop. This saves keystrokes, typing errors and having
to remember commands.
See also:
YouTube: CHIP Tutorials: Using mobaXterm with CHIP
Page 96
Building X applications
If we wish to build UI applications (such as X-applications) then the GTK toolkit is what we
want to use. There is a screen designer called "glade" that makes the job easier for us. We
can install this with:
$ sudo apt-get glade
If we change the style, for example to "Xfce-dawn", then the whole set of windows look and
feel change:
Page 97
The desktop is the background against which we perform work. We can change its
configuration by launching the "Desktop Settings" application:
Here we can change the background image, colors, menu content and icons.
After making such changes, you will end up with a look and feel of your own preferences:
Page 98
Don't be afraid to experiment and tinker with the adjustments until you end up with the desired
result.
At the top of your display is a bar called a "panel". This is actually the application called
"xfce4-panel". It is a container used to show icons of currently open applications, launch
menus and a variety of widgets items. If we wish to add new items to the panel, we bring up
its context menu and select "Add New Items":
Upon selection, a list of the available item types are shown. We can select these and add
them to the panel:
Page 99
We can see the current widget items already on the panel by selecting "Panel Preferences":
Page 100
The order of the items can be changed as well as adding or removing items. If a widget has
customization properties, they too can be edited in this page.
We are not limited to just one panel. We can add additional panel instances and position
them as we like. For example, we can have a bar at the top and a second bar at the bottom.
When the desktop starts, we are presented with the CHIP splash screen that looks as follows:
We can replace this splash screen with any image we desire (or even none at all). If we
launch the Settings > Session and Startup tool we see the following:
Page 101
From here we can customize the image by clicking the "Configure" button:
The image file used for the splash screen can be selected. If you change it to any image on
the file system, that image will be used for the splash.
We can add our own applications and directories to the desktop. We do this by creating a
new file that is read by the desktop manager. The content of the file is described in a
"desktop-entry-spec" specification document (see references). To make your own entries,
create a new file in the "Desktop" folder in your home directory. The extension of the file
should be ".desktop". Within the file we have "sections" and "properties". A section is a name
surrounded in square brackets. For example:
[Desktop Entry]
Page 102
Property Name Description
Name The name of the entry on the desktop.
Type The type of the desktop entry. Allowable values are:
Application An application to launch.
Link Link to a web page.
Directory A directory to navigate to. The file type for this entry should be ".directory" as
opposed to ".desktop".
Running Headless
The term "headless" simply means that your CHIP doesn't have a display, keyboard and
mouse connected to it. This then begs the question of "How do I interact with the CHIP if it is
headless?". There are a couple of answers to that puzzle, the most common ones being:
Connect the CHIP to a TCP/IP network (eg. WiFi)
Connect the CHIP via a serial cable
Page 103
that you commonly sit at. This may be a Windows PC, a Mac or a Linux machine. This PC
will likely have a mouse, keyboard and monitor attached. Wouldn't it be great if you could
interact with your CHIP through that PC?
There are a number of choices available to us to achieve that task. The first one we are
going to look at is running an X-Windows server on the PC and opening windows from your
CHIP on the X-Server. There are a wide variety of free X-Servers that run on PCs. My
favorite is "mobaXterm". When you install and run this package on Windows, you can open
an SSH session from the PC to CHIP and get a terminal command prompt. From there, you
can run the command "xfce4-session" which will start a desktop on your PC.
By default, when you boot CHIP, it starts a local X-Server. If you are remotely connecting into
CHIP, there is no need to run this as it consumes system resources. To prevent CHIP from
loading an X-Server at boot, we can can run the following command:
$ sudo systemctl set-default multi-user.target
The next time CHIP is restarted, we will be left at the login prompt.
To re-enable the graphical environment, run the following command:
$ sudo systemctl set-default graphical.target
See also:
Using mobaXterm on a Windows PC
Using xming on a Windows PC
Page 104
For example, if we take a CP2102 USB to UART:
And connect the TX pin of the USBUART to the RX pin of the CHIP and the RX pin of the
USBUART to the TX pin of the CHIP, then if we run Putty against the COM port that shows
up in Windows, we get a Linux login prompt.
There are a number of good serial terminal applications available, one of the most popular is
PuTTY which is available for both Linux and Windows. We can install this on CHIP by
installing the package called "putty". PuTTY is an X-Windows application so will need an X-
Server on which to display the output.
My favorite connection tool, mobaXterm also provides a serial connection capability:
See also:
UART on CHIP
Using mobaXterm on a Windows PC
Page 105
Connecting to CHIP using the USB cable
The USB cable that one uses to power CHIP can simultaneously be used to communicate
with it. We plug the micro USB end into the micro USB socket on CHIP and plug the other
end into a Linux based PC. If all has gone well, running "lsusb" on Linux will show an entry
that contains:
Bus 001 Device 021: ID 0525:a4aa Netchip Technology, Inc. Linux-USB CDC Composite Gadge
(Ethernet and ACM)
What this means is that the cable connection will show up as both a network connection and
a serial connection. We will focus on the serial connection for just now. Look for a directory
entry called /dev/ttyACM0 on the Linux PC. If found, then that is an 115200 baud connection
to CHIP. If we install and run a terminal emulator such as "putty", we will connect to CHIP.
Page 106
VNC Virtual Network Computing
VNC is a technology that allows one to see the desktop of one machine from another. For
example, if CHIP is showing graphics output on the screen, then VNC can be used to mirror
that output to another computer such as your PC. There are a number of implementations of
VNC available, the two most prevalent being:
Real VNC
TightVNC
To get started, we must install a version. For example:
$ sudo apt-get install tightvncserver
Once installed, we need to run tightvncserver to create a login password and an optional view
only password.
To start the VNC server we can run "vncserver". To see if the VNC server is already
running, we can use "ps -ef | grep -i vnc".
Look for processes that are called "Xtightvnc". Each one of those is an instance of a VNC
srever. Notice also the display number that follows it.
Here is an example:
$ ps -ef | grep -i vnc
chip 2242 1 2 21:17 pts/1 00:00:00 Xtightvnc :1 -desktop X -auth
/home/chip/.Xauthority -geometry 1024x768 -depth 24 -rfbwait 120000 -rfbauth
/home/chip/.vnc/passwd -rfbport 5901 -fp
Page 107
/usr/share/fonts/X11/misc/,/usr/share/fonts/X11/Type1/,/usr/share/fonts/X11/75dpi/,/usr/shar
e/fonts/X11/100dpi/ -co /etc/X11/rgb
chip 2292 1908 0 21:17 pts/1 00:00:00 grep -i vnc
for example:
$ vncserver -kill :1
Page 108
When the "Connect" button is clicked, we are prompted for the login password we created
earlier:
Page 109
Of the clients available and the ones I have played with, RealVNC is my personal choice.
See also:
VNC (Virtual Network Computing)
man(1) - vncserver
Programming
What is a computer if not for programming? Before we get into programming proper, let us
first talk about editing source files. A source file is the source code of your application that will
then be run. Some languages require that the source be compiled into executables (eg. C)
while others interpret the source code (JavaScript, shell, Python). In both cases, we need to
use an editor to edit the source. The choice of editor is as much personal preference as
anything in our IT world. If I said use editor XYZ, someone else would immediately turn
around and say that we should use editor ABC. The best I can offer is a short list of some of
the more popular editors and let you take your pick.
The vi editor is the historic text based editor that has been around for many decades. Some
users love it, and some hate it. It is definitely not for the Unix beginner. Its commands are
cryptic. If you know vi today, awesome if you don't know vi, then I would not recommend
it as the editor to learn. Its strength today is predominantly that it is ubiquitous across every
known Unix variant.
The nano editor is another text editor and this is significantly easier to use than vi. The
manual page and/or the editor documentation should be consulted for details however the
basic model is to use the cursor keys and enter/change/delete text.
Page 110
The geany editor is perfect for editing code in an X-Windows environment. It includes
language knowledge as well as build commands. I use geany most of the time for my own
editing when editing locally.
Page 111
So far we have assumed that we are editing files directly on CHIP and also that our source
lives on the CHIP device. While that is certainly a solution, if you are working on a large
project, I might caution against that for a number of reasons. Firstly, realize that the source of
your applications represent hours and hours worth of work. How would you feel if you
accidentally deleted your files and couldn't recover them? What if your files simply got "lost".
While the CHIP is a fantastic device, it doesn't have the maturity of a desktop PC. There is no
recycle bin to recover your files if you hit delete. Also, since you are likely building a project
that may involve some hardware interfacing, consider that all too common "oopsie" where you
plug a wire into the wrong socket. On the CHIP, that could fry your board. Thankfully, since
the CHIP only cost $9, you can simply get another one ... but what of all your work locked up
in the flash memory of the device. Oh Oh!!! No removable storage to take out and place in a
new device. Ouch!!!
Instead, I recommend editing and saving your source on a desktop operating system.
Personally I use either Windows 10 or Ubuntu. My development environment is Eclipse (for
C, Java, JavaScript and Web). Eclipse has built in support for GIT and I regularly commit my
changes back to Github for secure storage and duplication as a backup and to return to a
previous state (if needed). With my source code on the desktop, I then export that directory
for mounting access on CHIP. CHIP then treats that file system with the illusion that it is local
Page 112
and from there I can do my compiles and tests. The source really lives on the desktop. A
side benefit of this technique is that should I get low on CHIP file system storage, the source
code, compiled objects and executables doesn't count against CHIP storage since all data is
remotely held on the desktop.
Programming in general
One of the exciting aspects of the CHIP environment is that one is not forced to work in any
particular programming language when building CHIP applications. One has a wealth of
choices available. Languages available include C, C++, Java, JavaScript, Python and more.
Some are much more commonly used and prevalent than others and for some languages we
even have multiple choices of compiler and environment to use. With choice comes decision.
When you wish to undertake a project, which language do you use? Despite what many folks
might argue, my vote is to use the one you "want" to use (where possible). The choice of one
language over another can quickly break down into a religious battle where personal bias
holds as much argument as technical merits.
An early decision that you will make is whether you develop on CHIP itself or on a PC. I am
assuming that you have a Windows or Linux PC available to you. It is a safe bet that your PC
is going to be much more powerful than your CHIP. Your PC will be faster, have more RAM
and have disk capacity to hold all the files you need. My recommendation is to develop your
applications on a PC and either cross-compile or copy across for compilation the files and
artifacts necessary. Cross compilation is the better option of the two. With cross-compilation,
you edit and compile your source code on the PC and then test and run it on CHIP. This
gives you the power of a PC for development and insulates you from CHIP crashes which are
all too possible when working with low-level parts and electronics in general. Accidents can
happen and if you short your CHIP, then you will be restarting your environment from a fresh
boot. The reduction in time that it takes to perform a build on a PC is also very dramatic.
ELF structure
When we compile an application on a Linux environment for execution, the result is a file
containing executable instructions. The format of the file is known as the Executable and
Linkable Format (ELF) and contains a program that "thinks" it has an address space of
0x0000 to 2^n where n is either 32 or 64. The ELF file contains logical sections which
contain data for distinct purposes. Among the sections are:
.text Instructions for execution
.data Initialized data. This could be data that can be read-and written to that has
an initial value. For example:
int myVar = 123;
char *myString = "Hello World";
.bss Uninitialized data. This is data that can be read and written to that has a zero
Page 113
initial value.
static int myVar;
.rdata Read-only data. This is data that can be read from but not written to. For
example:
const int myVar = 123;
printf("Hello World!\n");
It would not be sensible to believe that large applications can be written in just one source file.
Instead, applications typically are broken up into multiple source files where the code in one
file typically performs some specific function. In order for this to be useful to us, we must also
allow code that exists in one source file to invoke code that may exist in a different source file.
This brings us to the concept of exposed symbols. When we compile a source file to its
object file representation, the resulting object file can expose the symbols of functions and
variables. By exposing these symbols, we can then find their memory locations during link
time. The flip side of this is the concept that we can declare variables and functions in our
source code as being defined elsewhere. This means that during compilation, these variables
and functions will not be found but, at linking time, the linker will ensure that somewhere in the
aggregate of files being brought together, all undefined symbols are resolved to some defined
symbol. We can use the nm command to list the defined and undefined symbols within an
object file or archive.
Libraries of code
When you compile a C program, an object file is produced which is the compiled
representation of your C source. It consists of instructions ready to be executed by the CPU.
Only when the object file is linked with the C run-time does it become an executable. Now
imagine that you have authored some code that you want to use over and over again.
Perhaps it is a general logging function or some specialized algorithm for drawing graphics on
the display. When ever you want to re-use that code, you could include the source in your
project, recompile it and link it with the rest of your application. However this is not an optimal
idea for many reasons. First, you have broken your desire for encapsulation. You can go into
the source file of the function and tinker with it. This can make maintenance a challenge as
you will now have multiple distinct copies of the source floating around. You will also be
wasting compilation time re-building the object file each time you compile.
A better solution would be to compile the original source once and re-use the object file
Page 114
directly when needed. Now the object file becomes an encapsulation of the function and can't
be contaminated with inadvertent tweaks. It is also likely that you won't just have one re-
usable piece of function but instead have many. This means that you may end up with
dozens of object files each of which need to be linked with your final application. There may
also be dependencies between these object files and trying to remember which object files
you need to pass to the linker can become a challenge.
Fortunately, Linux provides an easy and elegant solution in the form of the supplied archiver
tool called ar. The ar tool allows us to take a set of object files and aggregate them together
into a single file that we call an archive file. By Linux convention, the name of an archive file
begins with "lib" (for library) and ends with the file suffix of ".a" for archive.
If you are familiar with taking multiple files and producing a ZIP file, this is a similar concept
(but without compression).
The linker has knowledge of the meaning of an archive file and when you add that to the
compilation steps, references to symbols (functions and variables) will be sought within the
object files contained within the archive. The end result is that you can group together useful
libraries of function into a single archive file and merely link with that "black box" to
incorporate the function contained within into your own applications.
Having spoken about building your own libraries, you should also realize that this story also
applies to libraries that you may leverage from others. If a new whiz-bang Open Source
library becomes available, you can use the functions it exposes in your own applications
without having to know how they work.
With C programming, we have the need to define functions and variables which are not locally
present within the compiled app. For example, imagine that you want to call a function called
whizBang() that is supplied in a library. If you simply call whizBang() from your code, you
should end up with a compile time warning or error that the compiler doesn't know anything
about whizBang(). The solution is to include a C language header file that contains the
definition of the function. For example:
void whizBang(char *name, int age);
Page 115
This gives the compiler all the knowledge it needs to validate that a call to the function is as
expected. As such, when a library is provided there should be one or more header files that
are accompanied with it that provide the relevant definitions to allow you to correctly use it.
Page 116
the additional compiler flag called "-fPIC". To aggregate the object files together into a
shared library, we don't use the ar tool, instead we use the compiler again adding the "-
shared" flag.
Let us look at the "-fPIC" flag. PIC is an acronym for "Position Independent Code". By
specifying this flag, we are indicating to the compiler that is should produce code that is not
dependent on where it will be loaded in the address space of a process. This allows different
applications to load the same code at different memory locations since each application may
have its own needs as to where code is found.
See also:
man(1) ar
UNIX ar Examples: How To Create, View, Extract, Modify C Archive Files (*.a)
ldd
The ldd tool shows the shared library dependencies upon an executable.
See also:
man(1) - ldd
nm
The nm command can be executed against an object file or an ELF executable to display the
symbol tables contained within. Contrast this command with objdump. The objdump
command displays memory layouts while the nm command displays symbol table layouts.
Some of the more important flags are:
--undefined-only Display undefined symbols only.
--defined-only Display defined symbols only.
--print-file-name Prefix each symbol with the file it comes from.
Page 117
See also:
man(1) nm
objdump
The objdump command can be executed against an object file or an ELF executable. It
dumps and disassembles these files with a wife variety of options.
Some of the more important flags are:
--section-headers Display a summary of the sections contained in the file.
See also:
man(1) objdump
ranlib
See also:
man(1) ranlib
readelf
See also:
man(1) readelf
C Programming
Since CHIP runs the Linux operating system and Linux is a version of a Unix and Unix is
heavily based on the C language, we must immediately start thinking about C. The C
compiler is supplied in the package called "build-essential".
After installation, we can now compile C programs. For example, create the following in a file
called "test.c":
#include <stdio.h>
Page 118
You have just edited, compiled and run a C program. That wasn't hard.
pkg-config
When we compile C programs, we commonly have to supply definitions, include directories,
flags and libraries with which to link. Since we can't absolutely guarantee where these files
and definitions will be found, this results in our Makefiles being complex. Fortunately, there is
an excellent tool called "pkg-config". This command can be run in the steps of the
compiler to provide environment specific details. Each package that you may want to use in
your development has a ".pc" file associated with it.
A well behaved library will provide its own ".pc" file. To see a list of such files, run:
$ pkg-config list-all
We can use pkg-config within makefiles. For example, here is some usage that inserts the
correct libraries and headers:
libsoc-input: libsoc-input.c
$(CC) -o $@ $< $(shell pkg-config --libs --cflags libsoc)
Note that pkg-config is not installed by default, you must install the package called "pkg-
config":
$ sudo apt-get install pkg-config
See also:
Guide to pkg-config
man(1) pkg-config
JavaScript Programming
JavaScript is a language that was originally invented as a scripting language running in
browsers. Even today that is still predominantly the case. It is an interpreted language that
has leanings towards Java but is sufficiently different that it is very much in a class of its own.
There are many schools of thought and opinions on JavaScript. It is a loosely typed language
which means that one doesn't have to declare variables before using them. This can result in
tricky puzzles to track down at run-time that would have been caught with other languages at
development time. For example:
var sale = {
name: "widget",
price: 12.34
};
would not catch the mistake that we originally named the property in sale as price but
referred to it as cost. Only at run-time when the statement was reached would we catch it.
Page 119
However, discussions along those lines tend to result in religious views and we will leave
those alone. From the perspective of CHIP, there are also non-browser variants of the
language including Node.js and Nashorn that are at our disposal.
Node.js
Node.js is not pre-supplied with CHIP and must be manually installed. An armv7 distribution
is available. The following recipe illustrates an example of installing Node.js on CHIP:
$ cd /tmp
$ wget https://nodejs.org/dist/v4.4.7/node-v4.4.7-linux-armv7l.tar.xz
$ tar -xvf node-<version>.tar.xz
$ cd node-<version>
$ sudo cp -R * /usr/local/
$ node -v
$ npm -v
Node.js is evolving as the underlying JavaScript evolves. A web site called node.green
contains a list of JavaScript features by availability within versions of Node.js
There is also a tool that will manage our Node.js environments for us. The tool is called
"nvm". To install this, one would run:
$ wget -qO- https://raw.githubusercontent.com/creationix/nvm/v0.31.7/install.sh | bash
Always check the project home page to find the latest version of "nvm". Once installed, we
can install multiple versions of node. The latest can be installed with:
$ nvm install node
To switch to using the system version (the one that is system defined) run:
$ nvm use system
See also:
Adafruit Why node.js?
Node.js home page
YouTube: Installing Node.js
Github: creationix/nvm
NPM
NPM is the Node Package Manager which is a tool and environment for packaging JavaScript
libraries for distribution. In simple terms, an NPM package is a directory which contains
Page 120
JavaScript plus a control/configuration file called package.json. Contained within
package.json is the meta-data about the package itself. Even if we are creating a local
application, it is still very useful to create a package.json. The content of the file is a
JSON object:
{
"name": "mypackage",
"version": "1.0.0",
"dependencies": {
"<SomePackage>": "<SomeVersion>"
}
}
When we are building a module that will be included be require(), we should realize that
require() will return an object. In our JavaScript that implements the module, the object
called "exports" is the one that will be returned. So if we create a JavaScript source file
called "index.js" and in that source file code:
exports.myFunction = function() {
console.log("Hello world!");
}
Then the object returned from a require() of our package will contain a function called
"myFunction()" which, when called, will print to the console.
Semantic versioning is used to indicate the nature of changes. New projects should start with
1.0.0.
<major>.<minor>.<patch>
Page 121
See also
NPM home page
Page 122
main
bin
man
directories
repository The repository where the source can be found. Form example:
"repository": {
"type": "git",
"url": "https://github.com/npm/npm.git"
}
and from within the directory that contains the module source, run:
$ npm publish
We can have the npm tool create a package.json file for us by running "npm init"
Within your implementation, you will find an object called "exports". Properties added to
this will be available to the user of the package.
See also:
Creating Node.js modules
package.json
Johnny-Five
We can't talk about IoT devices like CHIP and JavaScript without mentioning the Johnny-Five
project.
Page 123
To build an environment, install both chip-io and johnny-five:
$ npm install chip-io johnny-five
board.on('ready', function() {
// do Johnny-Five stuff
});
Within the Johnny-Five framework, we have now "normalized" our complete interaction with
hardware. We can now think of higher level modules that work against specific technologies.
Some are specialized, some are generic. We will look at a few of these in the coming
sections.
The CHIP headers exposed by Johnny-Five are:
Page 124
If we need to switch on debugging, we can set the environment variable "DEBUG" to "chip-
io".
See also:
Johnny-Five home page
npm chip-io
Page 125
pin.write(value);
Node-RED
When we think about sensors, we quickly find that we want something to happen when a new
reading is taken or something changes. Generically, we can look at this as commonly being:
1. Accept incoming event
2. Parse the content of the event
3. Examine the content
Page 126
4. Test if the content meets some criteria
5. Optionally do something as a result of having examined and tested the content
If we agree that, loosely speaking, this is a common pattern then maybe we can find a
framework that makes processing this pattern easier. We are used to writing application logic
that is driven by the arrival of an event. This logic is typically written in a programming
language such as C, JavaScript, Python or others. Although not an inherent issue with the
languages, when we look further at our puzzle, we realize that these are generic
programming languages and not any form of domain specific language while what we have
here is indeed a very specific domain namely the processing of events. It is not
uncommon at this point for yet another new programming language to be created. Whether it
is compiled or interpreted, procedural, object oriented or functional are just details of the
language. Node-RED has taken a different approach. Instead of providing just another
language it has provided the ability to describe the logic we want executed through a drag,
drop and wire paradigm.
Node-RED provides a drawing canvas onto which building-blocks called "nodes" may be
placed. These nodes can then be wired together such that the output of one node becomes
the input to the next node down stream. By combing these nodes in a variety of different
combinations, we can achieve the goal we desire. Node-RED comes with a rich set of pre-
supplied nodes and a growing library of community supplied nodes is also available. If you
don't find a node that suits or you have a very specific need, you can even augment Node-
RED with additional nodes of your own creation.
The development environment for Node-RED is the browser and hence no special software
need be installed.
When one node passes control to another node, data is passed between them. On node
entry, the incoming data can be found in the JavaScript variable called "msg" which contains a
property called "payload" that contains the body of the message. A node can then modify
(or create a new) output object and that then becomes the message passed on to the next
downstream node.
In addition to the message that is passed from node to node, there are additional objects that
serve as contextual information at different states. Specifically we have:
context
flow
global
This command can take some time to run and will download a lot of prereqs.
Once installed, we can start it from the command line with:
$ node-red
Page 127
The server now listens on port 1880 for incoming traffic.
In addition to running Node-RED standalone, Node-RED can be embedded into your own
Node.js application.
See also:
Node-RED home page
Node-RED Programming Guide
YouTube: C.H.I.P. Tutorials: Installing Node-RED
This will install a copy of Node-RED relative to your current directory. On my CHIP, this tool
about 6 minutes to install. From there, we can navigate to
./node_modules/node-red
In there we will find an executable called "red.js". If we run that program with "node
red.js" we will start Node-RED and we should see messages similar to the following:
Welcome to Node-RED
===================
We can now start a browser and point it to port 1880 on the CHIP. Personally, I like to use a
browser on my desktop PC just for speed and convenience.
Page 128
Next, the obligatory license agreement:
Now we get to supply the directory into which Node.js will be installed:
Page 129
Now we get to select the options we wish to include:
Page 130
And now Node.js is installed:
After installation we can bring up a command windows and validate that the installation
succeeded:
Page 131
With Node.js installed, we can now execute the command to install Node-RED:
C:> npm install -g node-red
Once installed, we can run it with the command "node-red". The result will be the console
log as follows:
See also:
Node-RED installation
Page 132
Node-RED writing flows
When we author a flow we do so in the browser. On the left of the screen are a series of
categories within which node types may be found. These may be dragged and dropped onto
the drawing canvas. From there, the output of one node may be linked to the input of
another. When a flow diagram has been changed but not yet deployed, the nodes that are
different from the deployed versions have a blue circle to show that they are as yet un-
deployed. For example the blue circle here indicates it is un-deployed.
After we have written a flow we can export that flow for distribution to others. From the menu
at the top right, we can select "Export > Clipboard" which will open up a dialog from
which the JSON describing the flow can be copied.
We can rename a flow by double clicking on its header in the editor panel.
When we wire one node to another, we are explicitly passing data in the form of a message
between the nodes. Since the flow executes in one direction, we can use the phrase
"upstream" to mean nodes executed earlier in the flow that others which are "downstream".
Page 133
The message passed between the nodes is a JavaScript object. By convention, we
commonly use a property of that object called "payload" to identify the content.
When a flow is started, it is always a new message that flows downstream. If we need to
manage state across flow instances we can do that too. There is a variable called "context"
that maintains data specifically for that instance of the node. We can invoke it with
"context.get(<name>)" and "context.set(<name>, <value>)".
A second level of state data is the "flow" object. This is a data object that is available to all
instance of nodes within a given flow. Again we can access it with "flow.get(<name>)"
and "flow.set(<name>, <value>)".
Finally there is state data that is available to all nodes in all flows across the environment.
This is accessed through the "global" object with "global.get(<name>)" and
"global.set(<name>, <value>)".
We can log debug messages with the functions node.log(<text>), node.warn(<text>)
and node.error(<text>).
To throw an error, we should call node.error(<text>, <message>).
See also:
Error: Reference source not found
Node-RED documentation Writing Functions
Page 134
functionGlobalContext An object that contains other objects that can be
retrieved through global.get(<name>).
logging Controls for logging.
level The debug output level.
metrics -
audit -
See also:
Node-RED Configuration
then at run-time, you will be able to code global.get("xyz") in your function nodes.
It is also important to install the underlying packages that you want to use. These should be
added to the node_modules directory in $HOME/.node-red. For example, if you want to
use package XYZ, then you should change to $HOME/.node-red and then use npm to install
XYZ relative to there. We will then find the XYZ package in the relative node_modules
directory.
Page 135
// do something
node.send(msg);
}
}
RED.nodes.registerType("<nodeName>", <nodeName>Node);
}
To understand this, we find that there is a function called <nodeName>Node that is registered
to be invoked when ever a new instance of the node is created. This function is passed an
object that contains the node specific properties that were set by the user in the editor.
When a node is reached in a flow, it receives an incoming event called "input".
The "config" parameter is passed in for each instance of the node. It contains both the
properties defined specifically for that node as well as some other system properties. The
system properties include:
id A unique ID for this node in the flow.
type The type of node that this represents (a string)
z Unknown
x X location in the canvas
y Y location in the canvas
wires An array of wires
The HTML for a new custom node defines a number of items. First there is a script that is run
that initializes the node. This has a fixed form and looks like:
<script type="text/javascript">
RED.nodes.registerType("<nodeName>", {
category: ,
color: ,
defaults: {
},
inputs: 1,
outputs: 1,
icon: "<image file name>",
label: function() {
return "<label string>";
}
});
</script>
In addition to the script, we can provide components for the configuration and for the help.
The configuration is formatted as:
<script type="text/x-red" data-template-name="<nodeName">
<div class="form-row">
// Other HTML
</div>
</script>
The HTML that we inject on a per configuration option basis is of the form:
Page 136
<div class="form-row">
// HTML here
</div>
We want the row to start with a label that describes what the option. This is added with:
<label for="<id>"> </label>
These are Font Awesome font names. Following the label will be the <input> tag that is
used to prompt the user for the value of the option. It must have an id value of "node-
input-<optionName>". Here are some examples:
For a simple text input:
<input type="text" id="node-input-database" placeholder="database">
For a selection:
<select type="text" id="node-input-operation">
<option value="insert">insert</option>
<option value="delete">remove</option>
</select>
Within the Node-RED browser environment, we should assume the existence of jQuery. This
means that we can also include jQuery JavaScript. We can use this to update the page as a
function of what has been changed.
For example, the following shows a selection that shows or hides options as a function of
what is selected:
<div class="form-row">
<label for="node-input-retrievalType"><i class="icon-tasks"></i> Type</label>
<select type="text" id="node-input-retrievalType">
<option value="byId">... by Id</option>
<option value="byView">... by View</option>
</select>
</div>
<div class="form-row node-input-designDoc">
<label for="node-input-designDoc"><i class="icon-tasks"></i> Design doc</label>
<input type="text" id="node-input-designDoc" placeholder="Design doc">
</div>
<div class="form-row node-input-viewName">
<label for="node-input-viewName"><i class="icon-tasks"></i> View name</label>
<input type="text" id="node-input-viewName" placeholder="View name">
</div>
<script>
$("#node-input-retrievalType").change(function() {
var type = $("#node-input-retrievalType").val();
if (type == "byId") {
$(".node-input-designDoc").hide();
$(".node-input-viewName").hide();
} else {
$(".node-input-designDoc").show();
$(".node-input-viewName").show();
}
});
Page 137
For the help, the format is:
<script type="text/x-red" data-help-name="<nodeName">
<p>Help text</p>
</script>
See also:
Node-RED Creating Nodes
defaults Definitions for each of the editable/customizable properties for the node.
We create an object called defaults and for each new property to be exposed from the
node, we create a property as an immediate child of defaults. The value of that
property will itself be an object with a property called value that will contain the default
value. For example, if we wanted our node to expose an editable property called
"street" we would define:
defaults: {
Page 138
street: {
value: "<default street>"
}
}
In there will be a directory for each module that is in addition to the default modules supplied
with the Node-RED package. For example, if we wish to add a node called "MyNode", we
would create a directory called:
C:\Users\<userName>\.node-red\node-modules\MyNode
Page 139
Sharing Node-RED flows
The flows created within Node-RED can be exported as JSON documents and shared/given
to others as this textual representation.
Initialize some Johnny-Five library and make that function available to the rest of the flow.
The gpio input and output nodes are pretty straightforward, however, the "johnny5" node
needs a little more explanation. This node hosts some JavaScript that is executed ONCE
when the flow is deployed or Node-RED is started and the Johnny-Five onReady() callback
is fired. At this time, we can register any Johnny-Five setup functions with callbacks. If a
Johnny-Five callback fires, it can then fire a message out of the "johnny5" node output
terminal causing a new flow instance to run.
From here, we can run:
node.send({topic: "something", payload: "somethingelse"});
See also:
npm node-red-contrib-gpio
npm chip-io
Johnny-Five
Page 140
Security
Security and userids
When you login to Linux, you login with a userid. The default is the user "chip" with
password "chip". This is an "ordinary" userid and not the same as root or superuser. One
can always switch to that user but then all commands executed are performed as root and if a
typing mistake is made, you can literally ruin your environment. A solution to the problem is to
use the "sudo" command. Sudo takes as a parameter the name and parameters of another
command. Sudo then switches to root, and then runs that command as root. However, when
the command completes, the switch to root is not maintained. Thus, in effect, we run as root
for just the duration of the command.
One of the primary configuration files for sudo is /etc/sudoers. This contains the rules that
govern which users may execute sudo and for what commands. On the CHIP, the userid
called "chip" is fully authorized to run any command and does not have to provide any
passwords.
Within that file we will find a rich set of options. The one of interest to us reads:
%sudo ALL=(ALL:ALL) ALL
which says that any user that is in the group "sudo" (which chip is) can run any command as
root. This is great ... but we are prompted for a validation password. If we want to avoid
being asked for a password, we can change the line to read:
%sudo ALL=(ALL:ALL) NOPASSWD:ALL
See also:
man(8) sudo
sudo Home Page
Audio
CHIP is capable of generating an audio output signal. There are a number of possible output
destinations. The first is to carry the audio signal as part of the composite output. A second
solution is to plug in a set of audio speakers using the same jack.
There are a number of tools available to play audio.
Tools:
aplay Play a PCM sound from a file. Some of the more important flags are:
-l list devices
-L list PCMs
-D=NAME select device
Page 141
arecord Record an audio input stream. Note that aplay and arecord are basically
the same application.
alsactl
speaker-test Play tones and other sounds to test the speaker. Some of the more
important flags are:
-D NAME select device
-t {pink|sine|wav} The type of output to produce.
-f {freq} frequency of sine wave in HZ
alsamixer a full screen (text) audio mixer
amixer a command line audio mixer
mpg123 MP3 player
One of the easiest ways to play audio from within your application is to spawn a sub-process
such as aplay or mpg123 using a system call.
To play audio through this device requires us to specify it as the hardware output. For
example:
$ mpg123 -o alsa -a plughw:1,0 example.mp3
Another popular device is based on the CM108 chip set. Devices based on this architecture
can be found on eBay for about $1.50. These cards show up as:
Bus 001 Device 004: ID 0d8c:013c C-Media Electronics, Inc. CM108 Audio Controller
If you are attaching a microphone for audio input, examine carefully the plug. The plug needs
to match the input specs of the adapter. Specifically, there are two primary styles of plug.
Page 142
These are Tip Ring Sleeve (TRS) and Tip Ring Ring Sleeve (TRRS). For these cheap
devices, the microphone input is almost invariably Tip Ring Sleeve style.
To program against ALSA, we need to install the development environment found in the
libasound2-dev package.
$ sudo apt-get install libasound2-dev
See also:
Introduction to Sound Programming with ALSA
Programming Tutorial: ALSA Tutorial 1 Initialization
A close look at ALSA
Page 143
The next thing we need to consider is what does an individual sample consist of? In audio, it
is the amplitude of the signal. This is an analog value. There could be complete silence (0)
or maximum volume that we could sense (1) or a value ANYWHERE in between. To encode
an analog value in a computer can take a lot of memory. A typical floating point number might
be 8 or 16 bytes. Our reality is that we don't usually need this level of granularity. Instead
what we can do is perform a step known as "quantization". Simply put, this is the mapping
from an analog data point to one of a much smaller set of discrete possibilities. Imagine a
computer byte. This is an 8 bit value that can range from 0 to 255. Now imagine we map the
analog values in the range of 01 to the 8 bit values of 0255. So 00 and 1255 and the other values
would map proportionally. For example, an analog value of 0.5 would map to 128. The result of this quantization is that we
lose information in exchange for compactness. For example, 0.999, 0.998, 0.997 would all map to 255 and if all we have
is the number 255, we no longer know which of the possible analog values was the real value that we encoded into.
However, if we divide 1/255 we get 0.004 so our encoded value will be pretty darn close. Of course, we are not limited to
encoding our analog value in only 8 bits, we could certainly use more bits (for example 2 bytes = 16 bits) to achieve higher
fidelity.
Putting these concepts together, our encoding scheme is to sample a value many times per second and for each value,
quantize it by mapping it to the appropriate binary value. This stream of bytes then becomes the encoding of our original
audio signal.
See also:
Music and Computers
ALSA programming
The ALSA programming model is a C language library that can be linked in with your own
applications. The device drivers for driving the hardware are provided by the ALSA
implementation and our concerns become primarily those focused on driving the higher level
logic. The high level library is called libasound. ALSA itself is composed of a number of
major interfaces. These include:
Control interface
PCM interface
MIDI interface
Mixer interface
Logically, think of ALSA as the driver for the sound system. Since audio signal input and
output is best performed by hardware devices as opposed to the CPU, we are primarily
concerned with driving these devices. Since a Pi can have multiple audio devices attached to
it, we need a mechanism of specifying which device we want to interact with. ALSA allows us
to identify devices by name. If we run the "aplay -L" command, we will be rewarded with a
list of ALSA addressable devices. For example:
null
Discard all samples (playback) or generate zero samples (capture)
default
dsp0
dmixer
sysdefault:CARD=sun4icodec
sun4i-codec,
Page 144
Default Audio Device
dmix:CARD=sun4icodec,DEV=0
sun4i-codec,
Direct sample mixing device
dsnoop:CARD=sun4icodec,DEV=0
sun4i-codec,
Direct sample snooping device
hw:CARD=sun4icodec,DEV=0
sun4i-codec,
Direct hardware device without any conversions
plughw:CARD=sun4icodec,DEV=0
sun4i-codec,
Hardware device with all software conversions
If we plug in a USB based sound card, even more options become available:
default:CARD=Device
Generic USB Audio Device, USB Audio
Default Audio Device
sysdefault:CARD=Device
Generic USB Audio Device, USB Audio
Default Audio Device
front:CARD=Device,DEV=0
Generic USB Audio Device, USB Audio
Front speakers
surround21:CARD=Device,DEV=0
Generic USB Audio Device, USB Audio
2.1 Surround output to Front and Subwoofer speakers
surround40:CARD=Device,DEV=0
Generic USB Audio Device, USB Audio
4.0 Surround output to Front and Rear speakers
surround41:CARD=Device,DEV=0
Generic USB Audio Device, USB Audio
4.1 Surround output to Front, Rear and Subwoofer speakers
surround50:CARD=Device,DEV=0
Generic USB Audio Device, USB Audio
5.0 Surround output to Front, Center and Rear speakers
surround51:CARD=Device,DEV=0
Generic USB Audio Device, USB Audio
5.1 Surround output to Front, Center, Rear and Subwoofer speakers
surround71:CARD=Device,DEV=0
Generic USB Audio Device, USB Audio
7.1 Surround output to Front, Center, Side, Rear and Woofer speakers
iec958:CARD=Device,DEV=0
Generic USB Audio Device, USB Audio
IEC958 (S/PDIF) Digital Audio Output
dmix:CARD=Device,DEV=0
Generic USB Audio Device, USB Audio
Direct sample mixing device
dsnoop:CARD=Device,DEV=0
Generic USB Audio Device, USB Audio
Direct sample snooping device
hw:CARD=Device,DEV=0
Generic USB Audio Device, USB Audio
Direct hardware device without any conversions
plughw:CARD=Device,DEV=0
Generic USB Audio Device, USB Audio
Hardware device with all software conversions
Page 145
Prepare the device for output;
While there is data to be played {
write PCM data to the output device;
}
To write a C program that works with ALSA, we start by including the header file called
"<alsa/asoundlib.h>". This provides the signatures and constants for our code.
Our program will begin by opening the device. We use the snd_pcm_open() call for this
task. For example:
snd_pcm_t *handle;
rc = snd_pcm_open(&handle, "<deviceName>", SND_PCM_STREAM_PLAYBACK, 0);
A return code of less than zero indicates an error. Next we setup some hardware parameters:
snd_pcm_hw_params_t *params;
snd_pcm_hw_params_alloca(¶ms);
snd_pcm_hw_params_any(handle, params);
The hardware parameters inform the drivers of some of the core data items we are going to
use including:
data format (eg. SND_PCM_FORMAT_S16_LE)
number of channels (1 or 2)
sampling rate (eg. 44100)
Once we have populated our hardware parameters structure, we can then write those new
parameters to the driver with snd_pcm_hw_params().
See also:
ALSA project the C library reference
Introduction to Sound Programming with ALSA
Page 146
}
if (name) {
free(name);
}
if (desc) {
free(desc);
}
if (IOID) {
free(IOID);
}
i++;
}
snd_device_name_free_hint(hints);
} // End of listDevices
Page 147
What these graphs are showing is the frequency content in an audio signal. Think of an audio
signal like a beam of white light. Just like light can be decomposed into the sum of light
waves at different frequencies, so too can an audio signal. What we hear is a combination of
partial sounds at different frequencies that when combined are the totality of what we
perceive. Given an input audio signal, we can decompose it into its constituent parts. From
there, we can choose to graph it out. If the input source is changing (such as playing music)
then we can translate the amounts of different frequencies into other perceptions such as light
intensity or color changes.
The key to breaking down the audio into its component parts is the mathematics of the
Fourier Transform. That is much more math than we will need to comprehend (nor do I
understand or could possibly explain nor would want to try and explain to you). However
that doesn't stop us from using the technique. The algorithm we will use is called a Fast
Fourier Transform (FFT). It takes as input a sample of data (ideally a power of 2 in size) and
returns us an array of Fourier coefficients. Its okay, there is not any real need to know what
that means. Each member of the array is a mathematical complex number. This means it
has a real part and in imaginary part. Generically, a complex number is represented as:
a+bi
where "a" is the real part and "b" is the complex part ("i" is the imaginary number). Let us imagine we
have N samples then our array from the FFT will be:
The value we want instead of complex numbers is the magnitude of the complex number which is given by:
(a +b )
2 2
Each element in the array will now represent the amount of a specific frequency. The N
values will be an equally spaced distribution of the input frequency. For example, if we have
captured audio at a max rate of 44100 samples/second (44.1KHz), then the each element in
the array will span 44.1KHz/N. For example, if we are working on a sample size of 1024
elements, each element in the array will contain the amount of frequency in (approximately)
43Hz chunks. There is a little more to consider. In an FFT result, only the resulting data is
of use to us. If we have an output of N results, we only care about the first N/2 values.
Page 148
One of the primary implementations of FFT is called FFTW. This can be installed on a Pi from
the package "libfftw3-dev".
$ sudo apt-get instal libfftw3-dev
See also:
FFTW
How to extract frequency information from samples from PortAudio using FFTW in C
How to generate the audio spectrum using fft in C++?
Analyze audio using Fast Fourier Transform
Sine tone generator
To build an application one should include "<sndfile.h>" and link with "-lsndfile".
Notes:
The API is well document and pretty straightforward
Take care with sf_count_t. It is an int64 and when used with printf() formats
needs to be cast to an int or used with the format specifier of "%lld".
Here is a sample application that lists the headers and reads some data:
#include <stdio.h>
#include <sndfile.h>
#include <string.h>
#include <stdlib.h>
Page 149
printf(" - channels: %d\n", info.channels);
printf(" - format: %x\n", info.format);
switch(info.format & SF_FORMAT_TYPEMASK) {
case SF_FORMAT_WAV:
printf(" - WAV\n");
break;
default:
printf(" - Unknown format\n");
break;
}
switch(info.format & SF_FORMAT_SUBMASK) {
case SF_FORMAT_PCM_S8:
printf(" - Signed 8 bit\n");
break;
case SF_FORMAT_PCM_16:
printf(" - Signed 16 bit\n");
break;
default: {
printf(" - Unknown sub-format\n");
break;
}
}
switch(info.format & SF_FORMAT_ENDMASK) {
case SF_ENDIAN_FILE:
printf(" - Default file endian\n");
break;
case SF_ENDIAN_LITTLE:
printf(" - Little endian\n");
break;
case SF_ENDIAN_BIG:
printf(" - Big endian\n");
break;
default:
printf(" - Unknown endian\n");
break;
}
sf_close(file);
free(buffer);
}
For writing an output WAV file, here is some more sample code:
#include <stdio.h>
#include <sndfile.h>
#include "soundUtils.h"
static SNDFILE *soundFile;
Page 150
void soundFile_writeData(short *data, int items) {
sf_write_short(soundFile, data, items);
}
void soundFile_endSave() {
sf_write_sync(soundFile);
sf_close(soundFile);
}
See also:
libsndfile
Github: erikd/libsndfile
libsndfile api
Speech output
There is a software package called "eSpeak" that can be installed by:
$ sudo apt-get install espeak
To simply speak phrases, run espeak by itself. Each line entered followed by return speaks
the result. The software can include languages. Run:
$ espeak --voices
to see a list of the available voices. To supply a different voice use the "-v <voice>"
command. For example:
$ espeak -v en-sc "I can speak with a Scottish accent." --stdout | aplay
Another text to speech tool is called "festival" that can be installed by:
$ sudo apt-get install festival
See also:
eSpeak text to speech
Festival/festvox downloads
Festival manual
Logging
Messages from various systems and subsystems are logged into files in the /var/log
directory. There are many log files from all different types of sources here. Some of the more
important are:
Page 151
kern.log The kernel messages log
user.log User application messages
auth.log Authorization messages
daemon.log Daemon messages
kern.log Kernel messages
The dmesg command displays the kernel logs from a memory ring buffer (a memory buffer
that loops around on itself).
Now let us look and see how logging is actually implemented on CHIP. It is based on the
rsyslogd demon. It is started by default and we can validate that it is running with:
$ systemctl status rsyslog
rsyslog.service - System Logging Service
Loaded: loaded (/lib/systemd/system/rsyslog.service; enabled)
Active: active (running) since Thu 2015-12-31 15:37:16 CST; 1 day 3h ago
Docs: man:rsyslogd(8)
http://www.rsyslog.com/doc/
Main PID: 489 (rsyslogd)
CGroup: /system.slice/rsyslog.service
The configuration file for rsyslogd can be found at /etc/rsyslog.conf. This is a rich file
and I don't recommend editing it without first studying in detail the documentation on
rsyslogd.
The high level notion is that when a message is generated, it is generated by a particular
"type" of application. The rsyslogd calls this a "facility". In addition, when a message is
issued it is issued for a reason. Obviously the reason is to log something for future
examination but here we take the reason to mean why the message was issued. For
example, is it a warning? Is it debug? The rsyslogd calls this the "severity" of the
message. Now we have a richer context for considering a message. When a message is
issued it contains within both the facility and severity. The configuration of rsyslogd can
then use this additional meta information to route the message to the appropriate destination.
To log a message, we use the syslog function. We must also include syslog.h. The
signature of syslog is:
void syslog(int priority, const char *format, )
The priority is the encoding of both the facility and severity. The format parameter is a
printf() style format string with optional parameters following.
There are definitions provided for the facility:
LOG_AUTH Authorization messages
LOG_AUTHPRIV Private authorization messages
LOG_CRON Messages related to the cron daemon
LOG_DAEMON Messages related to daemon processes
Page 152
LOG_FTP Messages related to file transfer through FTP
LOG_KERN Messages from the kernel
LOG_LOCAL0 LOG_LOCAL7 Messages sources for local use
LOG_LPR Messages related to interaction with printers
LOG_MAIL Messages related to the receipt or delivery of mail
LOG_NEWS Messages related to the Usenet news subsystem
LOG_USER Generic messages
LOG_UUCP Messages related to the Unix to Unix copying program
And there are corresponding definitions for the severity:
LOG_EMERG Messages that indicate a disaster scenario
LOG_ALERT Messages that indicate immediate attention
LOG_CRIT Messages that indicate a critical condition that needs addressed
LOG_ERR Messages that indicate an error
LOG_WARNING Messages that indicate a warning
LOG_NOTICE Normal messages but considered important
LOG_INFO Informational messages
LOG_DEBUG Debugging or diagnostic messages
Now lets us see what happens when we log a message.
#include <stdio.h>
#include <syslog.h>
Since we are logging to LOG_USER, once run, we can look at the end of the
/var/log/user.log file and we will see:
Jan 1 19:23:31 chip2 a.out: Hello World!
Of course, the destination of the message is governed by the configuration of rsyslogd and
this log file is only the default.
See also:
rsyslog home page
man(8) dmesg
Page 153
Performance
When running your applications, if they are not running fast enough, or stutter or run out of
memory, you need to perform some performance analysis. Fortunately, Linux has many tools
and utilities at your disposal to do just that.
Resource Utilization
If we have performance issues, the chances are that we may be constrained on resources.
As such, we want to determine how to measure the resources we are using and have
available.
Memory Utilization
Running vmstat produces a quick summary of virtual memory availability:
$ vmstat
procs -----------memory---------- ---swap-- -----io---- -system-- ------cpu-----
r b swpd free buff cache si so bi bo in cs us sy id wa st
0 0 0 170052 0 151772 0 0 0 0 2030 4097 6 5 89 0 0
The column called free shows us the number of bytes unused RAM for our applications. In
this example, we see 170052 bytes. Since we have a large amount of free ram, we would not
expect to be using any swap space and this is backed up by the swpd column.
$ top
A similar command is the first few lines of the top command.
top - 00:00:46 up 1:07, 4 users, load average: 0.12, 0.16, 0.14
Tasks: 162 total, 2 running, 160 sleeping, 0 stopped, 0 zombie
%Cpu(s): 16.4 us, 12.1 sy, 0.0 ni, 67.4 id, 0.0 wa, 0.0 hi, 4.0 si, 0.0 st
KiB Mem: 503632 total, 333828 used, 169804 free, 0 buffers
KiB Swap: 0 total, 0 used, 0 free. 151836 cached Mem
We see that we have a total amount of RAM of 50362 bytes of which 333828 is used and
168904 is free. The output here contains more details than vmstat as it shows us the total
amount of RAM and used RAM as well as unused RAM.
Again, looking at the following line, we see that we have no swap space being used.
KiB Swap: 0 total, 0 used, 0 free. 151836 cached Mem
Since Linux supplies virtual memory, this means that each process sees its own address
space as though it were the only application in the machine. Since the operating system itself
is managing memory, if the accumulation of memory usage across all the programs running
on your CHIP starts to get close to the total amount of RAM available, the CHIP can start
paging out RAM to storage. By writing pages of RAM to storage, the operating system can
free up those pages for use by other applications. The thinking here is that there will be some
applications that are holding on to RAM but are themselves not actively using it. Perhaps
these applications sleep for long periods of time or are demon programs. Either way, the
chances of them accessing the page of RAM that has been swapped out to disk is believed to
Page 154
be lower than that of other applications. Should those applications later need to access the
pages of RAM that have been swapped out, some other application will have some of its
pages swapped out and the RAM released by that swapped used to hold the original
program's swapped out pages. The algorithms in play here that decide which pages get
swapped out is beyond our discussion.
From a performance perspective, it is not necessarily a bad thing to see swap space being
used. It simply means that our combined applications are needing more physical RAM than
we actually have and the swap space is being used to provide an illusion that we have
enough. However, we can enter a situation where we get so low on available RAM relative to
the needs of the applications that we spend more and more time swapping in and out pages
of RAM than doing productive work. Such a circumstance is called thrashing. Your Pi as a
whole will start to feel sluggish at this point.
Should your applications consume more and more RAM, you can imagine that more and
more of their pages will be swapped out. Since the amount of disk space allocated to holding
swapped out pages (called the swap space) is of a finite and fixed size, we can reach a state
where we have filled all our real RAM and also filled the disk space allocated for swap space.
At that point your CHIP is in trouble. It can no longer allocate extra space for memory
requests and applications will start to fail. Raspbian will start to terminate applications to free
up their resources.
Performance Commands
top
The top command is a real-time text/full-screen application that shows the current system
state as well as the processes currently running and their resource consumption:
Here is an example screen capture:
Page 155
One should spend quite some time studying the manual page to become skilled in the
meaning of each of the options. Without explaining where you can see them, the top output
includes %CPU utilization, how the CPU is being used, how much memory is free, the list of
top running processes, how many processes are active and much more.
See also:
top(1) - Linux man page
vmstat
The vmstat command logs statistics about number of processes, virtual memory utilization,
swap space, IO traffic, and CPU usage.
$ vmstat
procs -----------memory---------- ---swap-- -----io---- -system-- ------cpu-----
r b swpd free buff cache si so bi bo in cs us sy id wa st
2 0 0 169548 0 151984 0 0 0 0 1873 3790 6 7 87 0 0
See also:
man(8) vmstat
Task Manager
An X application called "Task manager" is pre-installed to provide an X based task manager
tool:
Page 156
When ever I have attempted to use it, its output appears corrupted but ... worse ... it seems to
consume the majority of the available CPU cycles ... and since our goal is to monitor resource
utilization, this seems a bit pointless.
gnome-system-manager
An attractive system manager tool is available called "gnome-system-manager". This is not
installed by default but can be installed through:
$ sudo apt-get install gnome-system-manager
Page 157
The down side of this app is that it consumes a lot of CPU just by itself.
time
When we run a command from the shell prompt, we might want to know how long it took and
also how much of that time was spent in different areas. The command "time" takes as a
parameter a command to run. The command is executed and, at the end, we get a report of
what CPU utilization and elapsed time the execution of the command took. For example,
when I ran an install of a an npm module that involved compilations, here is the result:
real 6m2.749s
user 5m2.060s
sys 0m48.645s
Page 158
Performance characteristics of different programming languages
There is usually a trade-off between programming abstraction levels and performance. We
know that a Pi runs a CPU and that the CPU executes CPU instructions. If we were to write
our applications directly in assembly language then we would likely achieve optimal
performance. However, in reality, we find that is rarely done. The reason for that is that we
are willing to sacrifice that level of performance for the time, ease and maintainability of the
source code itself. This is a story that repeats for higher and higher levels of abstraction. For
example, simple C programming translates pretty efficiently to assembly language but yet C is
both portable across CPUs and gives us higher levels at which to work. Up from C is C++
adding an Object Oriented flavor. Again, C++ translates to assembly language and hence
native executable code. Further up the scale we move away from natively executable code to
interpreted code such as JavaScript. Here we have new paradigms that don't have
equivalents in the instruction sets of processors such as lambda functions and closures
features which our human minds can use to simplify our algorithmic tasks when writing
programs but which take more execution resources to achieve. Typically these interpreted
programs are parsed at run-time. With some advanced interpreters, compilation is done on
the fly or just in time and with other interpreters, no compilation is performed and the
instructions are simply re-interpreted each time they are executed. Between interpreted and
native executable code there are languages such as Java which compile their source code to
a fictitious assembly language for a CPU architecture which does not exist in hardware.
Instead a software application called a Virtual Machine "executes" these compiled instructions
in software. The advantage here is that a program compiled once to produce these
"bytecode" applications is portable to all platforms which have implemented the virtual
machine irrespective of the underlying hardware CPU that is actually executing the virtual
machine.
The choice of programming language you use will most commonly be made based upon your
own skills and experience with one or the other. In addition, if you are using specialized and
pre-existing libraries, they may only be available for a subset of languages (or even only one).
However, when one writes a program no matter how elegant that program may be, if it
doesn't "work" then it is useless. If we write an application and it takes too long that users
aren't prepared to use it or worse, it is driving hardware components so slowly that they
themselves simply don't operate then we may be forced to change from our higher level
language of choice down to lower level but more efficient languages.
There are also times where we can design around performance problems while keeping high
level concepts. The trick here is to determine why an application is not performing as we
desire and see if we can't alleviate those problems with better coding design. Examples
might include the use of multi-threading to perform multiple tasks in parallel or we might
employ mixed languages. For example a program written in Java can invoke subroutines
written in C. These subroutines could encapsulate the tasks that are expensive in Java.
Page 159
Hardware Interfacing
Terminal connections
Before we say anything else ... I want to say very clearly ... the GPIO pins on the CHIP are
3.3V. Repeat after me ... the GPIO pins on the CHIP are 3.3V. They are NOT 5V tolerant.
The reason I mention this ... and I will mention it again ... is that one of the quickest ways to
damage or destroy your CHIP is to connect a 5V source to a GPIO pin. Don't do it. Don't
even think of doing it. Check, check and double check when you wire up a project that you
haven't done it (by accident).
The CHIP has two sets of sockets. One on the left of the board and one on the right. Each
set of sockets is composed of 40 terminals giving us 80 terminals in total. To be able to
discuss these terminals we have to define a naming and numbering scheme. The one
chosen is that the terminals on the left (with the USB at top) are called the U13 terminals
while the terminal on the right are called the U14 terminals. We choose to number and name
these as follows:
U13
GND 1 2 CHG-IN
VCC-5V 3 4 GND
VCC-3V3 5 6 TS
VCC-1V8 7 8 BAT
TWI1-SDA I2C bus 1 data 9 10 PWON
TWI1-SCK I2C bus 1 clock 11 12 GND
X1 13 14 X2
Y1 15 16 Y2
LCD-D2 17 18 PWM0 Pulse Width Modulation
LCD-D4 19 20 LCD-D3
LCD-D6 21 22 LCD-D5
LCD-D10 23 24 LCD-D7
LCD-D12 25 26 LCD-D11
LCD-D14 27 28 LCD-D13
LCD-D18 29 30 LCD-D15
LCD-D20 31 32 LCD-D19
LCD-D22 33 34 LCD-D21
LCD-CLK 35 36 LCD-D23
LCD-VSYNC 37 38 LCD-HSYNC
GND 39 40 LCD-DE
Page 160
U14
GND 1 2 VCC-5V
UART1-TX 3 4 HPL Headphones left
UART1-RX 5 6 HPCOM Headphones common
FEL 7 8 HPR Headphones right
VCC-3V3 9 10 MICM
LRADC ADC input 11 12 MICIN1
XIO-P0 13 14 XIO-P1
XIO-P2 15 16 XIO-P3
XIO-P4 17 18 XIO-P5
XIO-P6 19 20 XIO-P7
GND 21 22 GND
AP-EINT1 23 24 AP-EINT3
TWI2-SDA I2C bus 2 data 25 26 TWI2-SCK I2C bus 2 clock
CSIPCK 27 28 CSICK
CSIHSYNC 29 30 CSIVSYNC
CSID0 31 32 CSID1
CSID2 33 34 CSID3
CSID4 35 36 CSID5
CSID6 37 38 CSID7
GND 39 40 GND
These terminals are exposed in two banks of sockets. I have a concern (which may be
irrational) that plugging in and out breadboard wires into these sockets will weaken them. To
Page 161
solve that, I bought sets of extension sockets that look as follows:
There are 40 pin female sockets with long legs. These can be pushed into the CHIP headers
and then the breadboard wires pushed into these extensions. Should these upper sockets
wear out, I can pull these and plug in new instances. An extender header can be found on
eBay for about $0.90 each.
When plugged into CHIP, they look as follows:
Because the pins are so long, you can also solder these headers onto a strip board or PCB
and have the components "face-up" while docked to the headers underneath. An advantage
of that is that during assembly, you don't have to invert your thinking with mirror images of
what pins go where.
GPIO interfacing
One of the great joys of CHIP is that you can build projects that involve your own electronics.
You can attach sensors, LEDs, and much more to the CHIP. GPIOs are the General Purpose
Page 162
Input/Outputs. They provide the primary mechanism to read and write electrical signals to
and from the CHIP.
GPIO theory
The theory of General Purpose Input/Output (GPIO) is the simplest of all the hardware
interfaces to understand. With GPIO a device exposes a set of physical pins. These pins can
either be used to output an electrical signal for consumption by a second device connected to
the pin or can be used to read the electrical signal present on a pin as set by a second
device. At any one time, each pin can either be defined as an output pin or an input pin.
Obviously it can't be both at the same time though may switch roles as needed. When a pin
is flagged as output, it can source a current that can be consumed by a connected peripheral
however care must be taken. GPIOs on MCUs are only able to source a small amount of
current and trying to draw too much can irreparably damage the device. Take care to consult
the appropriate data sheets for any devices connected to determine their current
requirements and ensure that you are not asking for too much current from your MCU.
GPIO on CHIP
Exposed from the U14 header on CHIP are 8 terminals labeled XIO-P0 to XIO-P7. These are
available for GPIO usage. Access to these terminals is through TWI bus 2 and the device
address 0x38.
For programs that wish to access the GPIO, CHIP uses the "GPIO Sysfs" interface. In
English, what that means is that access to the GPIOs is performed through file system
operations in the directory tree starting at /sys/class/gpio. If we look in this directory we
will find lots of "things" and we will gloss over most of them. However, here is the general
idea
We have 8 GPIOs with which we may wish to work. By default, NONE of them are exposed
for our usage but we can access which ones we want by writing the text of the GPIO we
want to work with into the file called /sys/class/gpio/export. It would be ideal that if we
wanted to work with XIO-P3 we might execute:
echo "3" > /sys/class/gpio/export
but life in programming is never that simple. Instead we have to add a "base value" to our
desired GPIO. To make matters worse, the base varies whether we are running on a 4.3 or a
4.4 kernel. If we are running on a 4.3, the base is 408. If we are running on 4.4, the base is
1016. Making this explicit, if we are running on a 4.3 kernel and want to interact with XIO-P3,
we would run:
echo "411" > /sys/class/gpio/export
Page 163
GPIO Identity 4.3 Value 4.4 Value
XIO-P0 408 1016
XIO-P1 409 1017
XIO-P2 410 1018
XIO-P3 411 1019
XIO-P4 412 1020
XIO-P5 413 1021
XIO-P6 414 1022
XIO-P7 415 1023
The act of writing the value into the export file causes a new directory structure to appear in
/sys/class/gpio. The name of the new directory is "gpio<value>". For example, if we
write "411" into export, then a new directory called /sys/class/gpio/gpio411 will be
found.
With a gpio terminal directory, we find the meat of the file we need. First let us look at the file
called "direction". If we read from this file we will find a value of either 'in" or "out". The
value "in" means that the GPIO is in input mode. If the value is "out", then the GPIO is in
output mode. When a GPIO is in input mode, if read from the file called "value" we will
either get "0" or "1" reflecting the signal level found on the terminal. If the direction mode is
"out" then the signal found on the terminal will either be "0" or "1" depending on what we
write into "value".
The file called "edge" is the value of the pin that will cause a "poll()" call to return. Values
can be:
none
rising
falling
both
See also:
The sysfs file system
GPIO, SPI and I2C from Userspace, the True Linux Way
Page 164
When we request a GPIO, we haven't yet said whether or not we re using it for input or
output. We must now set the direction with a call to libsoc_gpio_set_direction().
The direction will be either INPUT or OUTPUT. To determine the current direction of a GPIO
we can call libsoc_gpio_get_direction().
The value of an output GPIO can be set with libsoc_gpio_set_level() and the value of
an input GPIO can be read with libsoc_gpio_get_level(). The values can be LOW or
HIGH.
We might not want to continually poll a GPIO but instead wait for some event to happen such
as a change in value. We have a function called libsoc_gpio_wait_interrupt() which
will block waiting for an interrupt to occur or a timeout to pass. If we don't want to block, we
can call libsoc_gpio_callback_interrupt() to register a callback function that will be
invoked when an event happens. We can cancel a previously registered callback handler
with a call to libsoc_gpio_callback_interrupt_cancel(). What we consider to be
an event that will register an interrupt is also settable by us via the
libsoc_gpio_set_edge() function. Triggers including:
RISING Signal goes low to high
FALLING Signal goes high to low
BOTH Signal transitions in either direction
NONE No triggering
See also:
libsoc gpio
Building libsoc
board.on('ready', function() {
var pin = new five.Pin("XIO-P0");
var b = true;
setInterval(function() {
if (b) {
pin.high();
Page 165
} else {
pin.low();
}
b = !b;
}, 1000);
});
See also:
npm chip-gpio
npm onoff
I2C
The Inter-IC (I2C) protocol is is used for connecting multiple devices together that provides a
way to transmit 8 bit bytes over a serial bus link. Originally developed by Phillips
Semiconductors it is now part of NXP Semiconductors. An alternate name for I2C is the Two
Wire Interface (TWI). This is so-called because there are only two lines needed for
communication. I2C is important for us as there are many interesting components that use
this protocol to which we may wish to attach.
I2C theory
The I2C interface is a serial interface technology for accessing devices. The protocol is also
called the Two Wire Interface as it uses only two signal bus lines. These signal bus lines are
Page 166
called SDA (Data) and SCL (Clock). I2C allows for bidirectional communication. Attached to
these lines are devices that wish to communicate. One device holds a special role and is
known as the bus master. It is the master that is the coordinator of all activities and is
responsible for generating the clock. All other devices attached to the bus are known as slave
devices.
The data transmission rates are up to 100kbits/second in standard mode and 400kbits/second
in fast mode.
Each slave device has a distinct and unique address upon the bus. It is the master that
always initiates communication which may be either a send request to send data to a specific
slave or a read request which asks the slave to respond with specific data. Unless the master
explicitly addresses a slave, it must keep silent. Only the slave that has been instructed to do
so may transmit data on the data line. The device addresses are commonly 7 bits in size.
The master sends the address of the device with which it wishes to communicate.
Immediately following the address is a bit that indicates whether this is a read request (1) or a
write request (0).
A A A A A A A R / W
Because of this encoding, take care when reading data sheets and looking at the address of a
device. Sometimes addresses are written as values from 0x00 0x7f describing the device
address while sometimes addresses are described as values from 0x00 0xff which
include the read / write bit. For example the following are all correct:
The device has address 0x1e.
The device can be read from 0x3d.
The device can written to 0x3c.
The buses are "open collector" with an active low. This means that pull-up resistors should
be attached to the lines to ensure that they are logic high by default.
There are hundreds of devices which are I2C capable. Here is a selection broken out by
category:
Page 167
Accelerometers, Gyroscopes, Compass
HMC5883L (GY-271) Compass
MPU-6050 (GY-521) Accelerometer and Gyroscope
MMA7455 Accelerometer
Data sensors
BMP180 Temp and pressure
PWM controllers
PCA9685 16 port PWM/Servo controller
GPIO Extenders
PCF8574 8 bit GPIO extender
MCP23017 16 bit GPIO extender
Analog to Digital
ADS1015 Analog to digital
Audio
TEA6767 FM Radio
Real time clock
DS1307 Real time clock
From a signal perspective the SDA line will be read when the clock goes high. The signal on
the SDA line must not change while the clock is high (with the exception of the stop bit
marker). When the clock goes low, the SDA line can be changed to represent the next bit of
data to be transmitted.
At any one time, the master will be in communication with either no slave or exactly one
slave. This sounds very academic but is important to understand. If we think of an initially
idle environment, then the master is not communicating with any slaves. When the master
now wishes to send data to a slave, it will begin a "transaction". For the life of the transaction,
the master may only now communicate with that one slave. The start of a transaction is
indicated by a transition from high to low on the SDA line while the clock is high. This is one
of only times that the SDA line can change its state while the clock is high and is reserved for
a start indication.
A second indicator signal called the stop signal is used to indicate the end of a transaction.
This is flagged by an SDA signal line change from low to high while the clock line is high.
After every 8 bits of data are transmitted, the receiver of the data must generated an
acknowledgment. The acknowledgment is sent by the receiver and consists of a low value
during the high period of the clock. If the receiver detects a situation that it considers and
error, it can send a negative acknowledgment by bringing the SDA high low during the high
period of the clock.
Data transmission is MSB to LSB.
A summary of the transmission from master to slave and slave back to master for both read
and write requests is shown next:
Page 168
See also:
Wikipedia I2C
I2C-bus specification and user manual
YouTube: How I2C Communication Works and How To Use It with Arduino
I2C on CHIP
CHIP is 3.3V device. As such be very careful when connecting to a 5V based I2C device. If
CHIP is a master, then you can connect the CHIP I2C clock to the input CLK on the 5V slave
but do not directly connect the SDA lines between a 5V slave and CHIP. Instead, place a
level shifter between them.
There are two distinct buses for I2C exposed on CHIP. These are three buses CHIP shown
as Bus 0 and Bus 2 (as listed by i2cdetect -l). However, that is a moment in time story.
Bus 0 should not be exposed as it is used for internal operations. Buses 1 and 2 are for user
applications but, due to a believed error, the current 4.4 Kernel exposes buses 0 and 2.
The tools for I2C are in the apt-get package called i2c-tools. If these tools are not
installed, further commands such as i2cdetect will not be found.
To install these tools, run:
$ sudo apt-get install i2c-tools
When using I2C we have the notion of a bus with devices attached to this bus. Ideally, we
know the identity of the device on the bus but if we aren't sure and/or want to verify that the
device is present, we can use the i2cdetect command. This command can send the start
of an I2C transmission to each of the possible addresses on the bus. By I2C protocol, when
we send an address, the device should respond with an acknowledgment that it is ready to
proceed. By sending the start of a protocol request to each device, the principle is that those
that respond are obviously present and we have thus determined which devices are present
on the bus. We can't tell what the device actually is only that it is there. Here is an
example of i2cdetect output:
$ i2cdetect -y 0
0 1 2 3 4 5 6 7 8 9 a b c d e f
Page 169
00: -- -- -- -- -- -- -- -- -- -- -- -- --
10: -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
20: -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
30: -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
40: -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
50: -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
60: -- -- -- -- -- -- -- -- 68 -- -- -- -- -- -- --
70: -- -- -- -- -- -- -- --
Running this command is potentially dangerous as we are trusting that each of the devices on
the bus won't be confused if it merely receives the start of an I2C request that immediately
ends without further interaction. In practice, we know of no real-world circumstances that
cause any problems.
Another useful command is i2cdump. This takes the bus number and the slave address and
returns the values of all registers addressable from the device. Here is an example of output:
$ i2cdump -y 0 0x68 b
0 1 2 3 4 5 6 7 8 9 a b c d e f 0123456789abcdef
00: 83 01 01 0d 21 0e ec 24 05 f9 03 f6 28 4c 6e 92 ????!??$????(Ln?
10: 3a 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 :...............
20: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
30: 00 00 00 00 00 00 00 00 00 00 01 fc 00 ff 9c 3b ..........??..?;
40: fc ec 20 fe c7 00 4f 00 94 00 00 00 00 00 00 00 ?? ??.O.?.......
50: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
60: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 f0 90 ..............??
70: 00 00 00 00 00 68 00 00 00 00 00 00 00 00 00 00 .....h..........
80: 83 01 01 0d 21 0e ec 24 05 f9 03 f6 28 4c 6e 92 ????!??$????(Ln?
90: 3a 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 :...............
a0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
b0: 00 00 00 00 00 00 00 00 00 00 01 fc 5c ff 34 3b ..........??\.4;
c0: 90 ec 00 fe c4 00 53 00 9b 00 00 00 00 00 00 00 ??.??.S.?.......
d0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
e0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 f3 50 ..............?P
f0: 00 00 00 00 00 68 00 00 00 00 00 00 00 00 00 00 .....h..........
In addition, there are two further commands called i2cget and i2cset that read and write
from the I2C bus.
From the shell, we have a couple of commands that can be used to read and write to an I2C
slave device. The first is i2cset. There are options on it but the simplest is:
$ i2cset -y 0 <address> <dataAddress> [value]* I
Page 170
/dev/i2c-2
Access to the bus can be achieved through access to these files. Note the permissions on
the files are read/write for root and the group called i2c.
First we must include <linux/i2c-dev> to be able to access the I2C related constants.
With that done, we can open the appropriate /dev/i2c-x file. To specify which device on
the bus we are going to work with, we execute an ioctl command that looks as follows:
ioctl(fd, I2C_SLAVE, deviceAddress)
Note that if we specify an invalid device address, we don't get an error indication.
Once the address has been supplied, we can perfom reads and writes to read and send data.
If we need to mix activities in a single transaction, we can use:
ioctl(fd, I2C_RDWR, struct i2c_rdwr_ioctl_data *msgSet)
See also:
i2c-dev interface
man(2) ioctl
Level shifting techniques for I2C bus design
Page 171
libsoc_i2c_read().
We can set a timeout for an I2C request using libsoc_i2c_set_timeout().
PWM
PWM theory
The idea behind pulse width modulation is that we can think of regular pulses of output
signals as encoding information by how long the signal is kept high vs low. Let us imagine
that we have a period of 1Hz (one thing per second). Now let us assume that we raise the
output voltage to a level of high for of a second at the start of the period. This would give
us a square wave which starts high, lasts for 500 milliseconds and then drops low for the next
500 milliseconds. This repeats on into the future. The duration that the pulse is high relative
to the period allows us to encode an analog value onto digital signals. If the pulse is 100%
high for the period then the encoded value would be 1.0. If the pulse is 100% low for the
period, then the encoded value would be 0.0. If the pulse is on for "n" milliseconds (where n
is less than 1000), then the encoded value would be n/1000.
Typically, the length of a period is not a whole second but much, much smaller allowing us to
output many differing values very quickly. The amount of time that the signal is high relative
to the period is called the "duty cycle". This encoding technique is called "Pulse Width
Modulation" or "PWM".
There are a variety of purposes for PWM. Some are output data encoders. One commonly
seen purpose is to control the brightness of an LED. If we apply maximum voltage to an LED,
it is maximally bright. If we apply the voltage, it is about the brightness. By applying a
fast period PWM signal to the input of an LED, the duty cycle becomes the brightness of the
LED. The way this works is that either full voltage or no voltage is applied to the LED but
because the period is so short, the "average" voltage over time follows the duty cycle and
even though the LED is flickering on or off, it is so fast that our eyes can't detect it and all we
see is the apparent brightness change.
See also:
Wikipedia Pulse-width modulation
Page 172
PWM on CHIP
Sadly, it is not currently present in the kernels pre-built for CHIP. To consume easily, we are
going to have to be patient.
UART on CHIP
CHIP has a UART that it exposes on terminals 3 and 5 of the U14 connector. There are three
terminals next to each other which are GND (1), UART1-TX (3) and UART-RX(5) which are
just what we need. The serial port is exposed in the file system at /dev/ttyS0.
By default, the UART is used to provide a terminal for access. This means that there is
already an application reading and writing to it. That application is an instance of the Linux
getty program which is handling terminal access. There are going to be times when we
want to use the UART for our own I/O purposes and hence it will be our application that needs
access to it. Since we can't have two programs thinking they have control over the UART, we
must disable the getty program. This is registered as a service which can be stopped with:
sudo systemctl stop [email protected]
Page 173
During boot, CHIP writes the boot messages to serial. This is great if we are doing debugging
but if we are building a project, the text of the console log sent to serial will likely confuse our
partner device listening to CHIP. A recipe to resolve this is still being sought.
From a Node.js environment we can use the serialport npm package.
We can look at the settings for the serial port using:
$ sudo stty -F /dev/ttyS0
See also:
Connecting to CHIP using a serial connection
npm serialport
USB
The nature of USB devices can be found with the lsusb command. From there, you can add
the -v (verbose) option and filter with the -s devNum option.
For example, here is an output from lssub:
Bus 002 Device 001: ID 1d6b:0001 Linux Foundation 1.1 root hub
Bus 001 Device 002: ID 1e4e:0110 Cubeternet
Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
Bus 003 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
When we look at the output of a USB listing, we sometimes see pairs of numbers. In the
above, we can see 1e4e:0110. The first of these numbers is called a vendor id and each
product vendor has been allocated a unique code. The second number is a product id and
corresponds to a product registered by that vendor. When a USB device connects to a
computer, the computer can query these codes and learn what kind of device it is that has
attached.
We can use this vendor/product pair as keys in a web look-up to determine more details
about that device. A website such as http://thesz.diecru.eu/content/usbid.php will allow us to
enter the vendor/product pair and retrieve its details.
Should one need more USB sockets than the single one provided and don't want to use an
external USB adapter, a product called "Queso III" is available that adds four additional USB
ports.
Page 174
See also:
man(8) lssub
Queso III 4 port USB extender
Bluetooth
CHIP has native Bluetooth support. This means that CHIP can interact with Bluetooth
devices such as keyboards, mice and cell phones. Let us review what Bluetooth means for
us. Bluetooth is a wireless communication protocol/technology that provides data transfer
over a radio signal. Let us assume that CHIP is one end of the connection and any other
Bluetooth device can be the other. For security purposes, arbitrary Bluetooth devices can't
simply be "used" without some explicit authorization. For example, it would be very wrong if I could bring my Bluetooth
headset near your phone and start listening to your calls. To achieve security, a process called "pairing" needs to be
performed. The achieves a level of trust between the two Bluetooth devices such that they remember allowing subsequent
connection without having to be re-paired.
Packages of interest:
bluetooth
bluez
blueman A gui package that may be un-stable
We can check the status of our bluetooth demon with:
$ sudo systemctl status bluetooth
bluetooth.service - Bluetooth service
Loaded: loaded (/lib/systemd/system/bluetooth.service; enabled)
Active: active (running) since Thu 2016-08-04 21:37:33 CDT; 2h 10min ago
Docs: man:bluetoothd(8)
Main PID: 389 (bluetoothd)
Status: "Running"
CGroup: /system.slice/bluetooth.service
389 /usr/lib/bluetooth/bluetoothd
Bluetooth specification
Bluetooth is a specification for wireless communication between multiple electronic devices.
At present, there are two primary standards these are Bluetooth (Classic) and Bluetooth
LE. The "LE" stands for Low Energy and is the specification for devices that wish to be
powered by simple batteries and yet have sufficient life span.
In the Bluetooth story, we have devices which are "masters" and devices which are "slaves".
A slave can only form a connection and communicate with a master while a master can form
Page 175
concurrent connections with multiple slaves. One slave can not directly communicate with
another slave. Instead it must communicate with the master and the master relay the
request.
The simplest communication is one master and one slave but if we have multiple slaves
connected to the same master, the resulting "network" is termed a "piconet".
Each device that participates in the conversation has a unique address that is a 48 bit value
commonly written as 12 hex values. This address is known as the "Bluetooth Device
Address" and may be seen in other documentation abbreviated to "BD_ADDR".
The encoding of a Bluetooth address is that the first 24 bits encode the organization
responsible for allocating the remaining address and the remaining 24 bits are the address
itself. However the full 48 bits are the complete identity of the device.
As well as having a unique address, each device can have a symbolic name to help us
meaningfully identify it. This is termed the display name. The display name is only a mapping
to a Bluetooth address and it is really the address that is used to distinguish one device from
another. It is also important to note that there are no uniqueness constraints on device
names. Multiple devices may select the same device name.
Let us assume that initially, we have two devices and neither of them know about the other.
What must now happen is a discovery process. One of them will broadcast an "inquiry"
request. Devices receiving the inquiry can respond with their own existence by transmitting
their own address and possibly additional information. It is common that the response to an
inquiry does not contain the display name of the responding device. If the display name is
needed, the inquirer can transmit a directed request now that it knows the address of the
responding device which will solicit the name as a further response.
A device does not have to respond to an inquiry request. It has a property setting called an
"inquiry scan" that controls whether or not it responds to such. If the attribute is on, then it will
respond to an inquiry request and if off, then it will not respond to such. Think of the phrase
"inquiry scan" as the devices choice as to whether it performs the action of "scanning for
inquires".
Once the two devices know each other's addresses, a connection can be formed between
them through a process known as "paging". Again, a device does not have to service a
received connection request. It has a property setting called a "paging scan" which controls
this. If on, then a paging request causes a new connection to be formed. If off, it will not
accept a new connection request.
Once a connection is formed between the devices, that connection can be maintained in a
variety of states, the most common being active. However other states are available and are
used to save power when there is no active communication of data anticipated.
In order to permit devices to communicate with each other, there has to be an element of
security involved. We usually don't want arbitrary devices to be able to connect with each
other and share arbitrary information. To achieve this, a process called "bonding" is enacted.
Bonding is achieved through the notion of "pairing". In pairing, the devices exchange their
addresses, names and other data and generate keys that they share with each other. Pairing
Page 176
typically requires an explicit interaction from the user to permit the pairing to succeed. The
user interaction can be as simple as "I approve this device" with a button click or it can be
richer with the entry of a pass-code to authenticate and prove that one is who one claims to
be.
The Bluetooth protocol provides support for different classes of power. This translates directly
into the signal strength of the radio. Remember that the more power used by the radio, the
heavier the drain on the power source. If the power source is batteries, the more power used
to transmit data, the shorter the life of the batteries.
At the lower levels, Bluetooth takes care of exchanging data between partners. However,
there is much more to Bluetooth than just simple data exchange. In order to provide
interoperability between devices built by multiple manufacturers, higher level protocols called
"profiles" have been defined. These profiles define "what" is transmitted over Bluetooth for a
given device function.
Some of the profiles we will come across include:
HSP Head Set Profile (eg. a Bluetooth ear piece).
HFP Hands Free Profile (eg. Bluetooth communication in a car).
HID Human Interface Device (eg. a keyboard or mouse).
SPP Serial Port Profile.
A2DP Advanced Audio Distribution profile (eg. connection to Bluetooth speakers).
AVRCP Audio Visual Remote Control Profile.
Knowing that these protocols exist, it is not sufficient that two Bluetooth devices are in range
of each other, they must also both support the same profile that is desired to be used.
At a higher level that the connection are the transport protocols available to us. These
include:
RFCOMM Radio Frequency Communications Protocol. This protocol provides a
reliable stream oriented transmission. Loosely, you can compare it to TCP.
L2CAP Logical Link Control and Adaption Protocol. This is a packet oriented
protocol. RFCOMM builds on L2CAP.
ACL Asynchronous Connection oriented Logical protocol. L2CAP builds on ACL.
SCO Voice quality audio protocol.
Page 177
Certain port numbers used by L2CAP are designated as reserved for well defined purposes.
Bluetooth GATT
The Generic Attribute Protocol (strangely called GATT) provides a mechanism for passing
data in a standard form. GATT is always present in BLE. Think of GATT as a way to send
and receive data that is "remembered" at the GATT server. In addition, a client can request
the values of data items as well as receiver asynchronous notifications as events when
something interesting happens.
At the high level of the protocol there is the concept of a service. The service is the grouping
of functionally related attributes. Within a service are characteristics where a characteristic is
a property of that service. Each characteristic type is identified by its own UUID value. In
addition a characteristic contains a value, properties, security and may also include
descriptors.
See also:
Generic Attribute Profile (GATT)
generic attributes (gatt) and the generic attribute profile
GATT-Based specifications
GATT services Assigned numbers
GATT delarations Assigned number
GATT characteristics Assigned numbers
gatttool
Page 178
Service Discovery Protocol
When a client application wishes to request that a connection be established, the client needs
to know the port number on which the server is listening. In TCP/IP land, this is achieved by
having the client and the server share the implicit knowledge of the port number to use. In
Bluetooth, extra functions have been made available. Specifically, there is a service available
called the Service Discovery Protocol or "SDP". At the server, when a service is offered, the
port number of that service is registered to the local SDP. When a client now wishes to use
the target service, it first requests endpoint information from the SDP running on the server.
The SDP returns the endpoint information and the client now has all the information it needs
to create a direct connection to the desired target service. The unit of information managed
by the SDP server is called a "service record" or "SDP record".
A command line interface tool called sdptool is available to examine a Bluetooth device's
SDP data. A simple command is:
$ sdptool browse <Bluetooth Address>
Bluetooth C Programming
Within the C / Linux environment we have an implementation of the API stack called "BlueZ".
The BlueZ implementation supports RFCOMM, L2CAP, SCO and HCI.
In order to perform Bluetooth programming we must install the package called "libbluetooth-
dev" using:
$ sudo apt-get install libbluetooth-dev
Page 179
hci_get_route
Retrieve a handle to the specified Bluetooth device or the first available if NULL is supplied.
int hci_get_route(bdaddr_t *bdaddr)
hci_open_dev
Open the specified device and get a handle to that device. The returned value is a socket.
int hci_open_dev(int dev_id)
If the return is -1 then an error was encountered and the details of the error can be found in
errno.
hci_inquiry
int hci_inquiry(
int dev_id,
int len,
int max_rsp,
const uint8_t *lap,
inquiry_info **ii,
long flags)
where:
dev_id the device id of the adapter as retruned by hci_get_route.
len The duration of the scan * 1.28 seconds.
max_rsp The maximum number of responses we are willing to accept.
lap may be NULL
ii Pointer to an array of inquiry_info structures to be populated. The storage must
exist and be at least of size max_rsp * sizeof(inquiry_info).
flags
0 Cached results allowed
IREQ_CACHE_FLUSH Any cached values are discarded and only new responses
will be used.
Page 180
uint8_t dev_class[3] The device class is encoded in the assigned numbers
baseband.
uint16_t clock_offset
See also:
Assigned numbers baseband
hci_read_remote_name
Retrieve the display name of a specified device.
int hci_read_remote_name(
int hci_sock,
const baddr_t *addr,
int len,
char *name,
int timeout
)
len the size of the name buffer to hold the display name.
name a buffer to hold the display name.
timeout Maximum number of milliseconds to wait before giving up.
On return, a value of 0 indicates success.
str2ba
Convert a string representation of a Bluetooth address into an address.
str2ba(const char *str, bdaddr_t *ba)
Where str is the string representation of the Bluetooth address and ba is a pointer to a
bdaddr_t structure to hold the resulting address.
ba2str
Convert a Bluetooth address to a string. The string buffer must be at least 18 bytes long.
ba2str(const bdaddr_t *ba, char *str)
The ba is a pointer to a bdaddr_t structure while str is the buffer to be populated with the
string representation.
sdp_connect()
sdp_service_search_attr_req()
Page 181
sdp_record_register()
Bluetooth Audio
Bluetooth speakers and headphones are common so a natural question would be how to get
sound out of them from the Pi. The answer is to install an application called PulseAudio.
$ sudo apt-get install pulseaudio pulseaudio-module-bluetooth
The Bluetooth profile we want to work with is called A2DP (Advanced Audio Distribution
Profile).
Bluetooth RFCOMM
A serial protocol is available via Bluetooth and is called "RFCOMM" for "Radio Frequency
Communication".
When programming with C, we want to create a socket using:
int s = socket(AF_BLUETOOTH, SOCK_STREAM, BTPROTO_RFCOMM);
A socket address structure for Bluetooth RFCOMM is a struct called sockaddr_rc which
contains:
sa_family rc_family This will always be AF_BLUETOOTH.
bdaddr_t rc_bdaddr The address of the device to which we wish to connect or
listen upon. If any local Bluetooth adapter will suffice when we are a server, we can
supply BDADDR_ANY.
uint8_t rc_channel The channel to which we wish to connect.
To be a client of an RFCOMM server, we would use:
#include <stdio.h>
#include <unistd.h>
#include <sys/socket.h>
#include <bluetooth/bluetooth.h>
#include <bluetooth/rfcomm.h>
addr.rc_family = AF_BLUETOOTH;
Page 182
addr.rc_channel = 1;
str2ba(dest, &addr.rc_bdaddr);
if(status < 0) {
perror("connect");
}
close(s);
return 0;
}
// allocate socket
s = socket(AF_BLUETOOTH, SOCK_STREAM, BTPROTO_RFCOMM);
ba2str(&rem_addr.rc_bdaddr, buf);
fprintf(stderr, "accepted connection from %s\n", buf);
memset(buf, 0, sizeof(buf));
// close connection
close(client);
close(s);
Page 183
return 0;
}
See also:
RFCOMM with TS 07.10
man(1) rfcomm
An Introduction to Bluetooth Programming
Bluetooth tools
l2ping
In TCP/IP networking, we have a tool called ping which sends an ICMP packet over the
network to which the partner responds. We can use this command to determine the
"liveness" of the partner as well as get the round trip response times. For Bluetooth, we have
a similar tool called "l2ping". This command sends a L2CAP echo request to the partner
and waits for the response.
At the highest level, we use:
$ sudo l2ping <bd_addr>
where bd_addr is the address of the target device. Here is an example output:
$ sudo l2ping 00:1A:7D:DA:71:13
Ping: 00:1A:7D:DA:71:13 from B8:27:EB:62:03:9F (data size 44) ...
44 bytes from 00:1A:7D:DA:71:13 id 0 time 11.32ms
44 bytes from 00:1A:7D:DA:71:13 id 1 time 66.06ms
44 bytes from 00:1A:7D:DA:71:13 id 2 time 19.84ms
44 bytes from 00:1A:7D:DA:71:13 id 3 time 52.38ms
As of 05/2016, running the l2ping command on current Raspbian ends after about 5
seconds of pinging with a message:
Send failed: Connection reset by peer
Current thinking is that this is caused by a deliberate kernel timeout of L2CAP requests that
don't result in a connection. The theory believed to be that to save battery life in real
Bluetooth devices, if a connection isn't established in the timeout period, don't bother to keep
trying and, presumably, waste power resources.
The l2ping command is delivered as part of the "bluez" package.
See also:
man(1) l2ping
rfcomm
See also:
man(1) rfcomm
Page 184
bluetoothctl
One of the primary tools for working with Bluetooth is called bluetoothctl. Oddly, this
command seems to have only the bare bones of a man page (shame).
One should run bluetoothctl as root using:
$ sudo bluetoothctl
If you fail to run it as root, it will just silently sit there until you interrupt it. This command line
tool has the following commands:
list List available controllers
show [ctrl] Controller information
select <ctrl> Select default controller
devices List available devices
paired-devices List paired devices
power <on/off> Set controller power
pairable <on/off> Set controller pairable mode
discoverable <on/off> Set controller discoverable mode
agent <on/off/capability> Enable/disable agent with given capability
default-agent Set agent as the default one
scan <on/off> Scan for devices
info <dev> Device information
pair <dev> Pair with device
trust <dev> Trust device
untrust <dev> Untrust device
block <dev> Block device
unblock <dev> Unblock device
remove <dev> Remove device
connect <dev> Connect device
disconnect <dev> Disconnect device
version Display version
quit Quit program
See also:
Page 185
hciconfig
As mentioned, HCI is the "Host-Controller Interface" which is the layer of communication
between the higher level protocols of Bluetooth and the Bluetooth lower level controller. The
"hciconfig" command allows us to execute commands through this logical interface.
Running hciconfig by itself will list all the Bluetooth devices found on the computer:
$ hciconfig
hci0: Type: BR/EDR Bus: UART
BD Address: B8:27:EB:62:03:9F ACL MTU: 1021:8 SCO MTU: 64:1
UP RUNNING PSCAN
RX bytes:19100 acl:150 sco:0 events:457 errors:0
TX bytes:7952 acl:150 sco:0 commands:184 errors:0
Each Bluetooth device has a logical identifier of the form "hciX" where the devices are
numbered 0, 1, 2 etc.
To refer to a specific device, most of the commands that we issue through hciconfig will take
the hciX device identifier to target the correct instance.
Some of the more interesting commands we can perform with hciconfig include:
Getting and setting the devices display name property
Enabling/disabling page support
Enabling/disabling scan inquiry support
The hciconfig command is supplied as part of the "bluez" package.
See also:
man(8) hciconfig
hcitool
The hcitool is delivered as part of the "bluez" package.
To scan for Bluetooth LE devices, use:
$ sudo hcitool lescan
See also:
man(1) hcitool
gatttool
Interact with a BLE device at the GATT level. In order to interact with a BLE device at the gatt
level, we need its device address. Using "hictool lescan" is a good way to get the
address. Typically the program is run with:
$ sudo gatttool --device=<Address> --interactive
This will return a command prompt that starts with the partner device address:
Page 186
[FF:FF:45:19:14:80][LE]>
Once connected, we can interrogate the device about its primary function by running the
"primary" command:
[FF:FF:45:19:14:80][LE]> primary
attr handle: 0x0001, end grp handle: 0x0005 uuid: 00001800-0000-1000-8000-00805f9b34fb
attr handle: 0x0006, end grp handle: 0x0008 uuid: 0000180f-0000-1000-8000-00805f9b34fb
attr handle: 0x0009, end grp handle: 0x000b uuid: 00001802-0000-1000-8000-00805f9b34fb
attr handle: 0x000c, end grp handle: 0x000e uuid: 0000ffe0-0000-1000-8000-00805f9b34fb
Notice specifically the UUIDs. These correspond to the assigned numbers of the GATT
specifications. Contrast this with the output of "bluetoothctl info" which shows the following:
[bluetooth]# info FF:FF:45:19:14:80
Device FF:FF:45:19:14:80
Alias: FF-FF-45-19-14-80
Appearance: 0x03c1
Icon: input-keyboard
Paired: no
Trusted: no
Blocked: no
Connected: yes
LegacyPairing: no
UUID: Generic Access Profile (00001800-0000-1000-8000-00805f9b34fb)
UUID: Immediate Alert (00001802-0000-1000-8000-00805f9b34fb)
UUID: Battery Service (0000180f-0000-1000-8000-00805f9b34fb)
UUID: Unknown (0000ffe0-0000-1000-8000-00805f9b34fb)
The UUIDs of the declarations and characteristics can then be examine in the assigned-
numbers lists.
For example
Page 187
0x...2800 is
0x...2803
0x...2A06... - Alert Level
See also:
Bluetooth GATT
man(1) gatttool
Micro SD
While CHIP doesn't come supplied with a Micro SD card reader, a board is available to add
one. The board is called "Tzatziki" and is available from Chip Dip Shop:
See also:
Tzatziki Micro SD reader
Page 188
these language comparisons as a Rosetta stone to provide insight into how different language
features work.
When interacting with new components, stop and Google them. It is almost a certainty that
someone, somewhere has attempted this task before. In fact, it is as likely that many folks
have achieved your task in the past. As such, learn from them. Don't re-invent the wheel.
When a manufacturer releases a new component, they commonly make available a
document called the "data sheet". Think of this as the specification, operations guide and
instructions for using that component. If it is more than a trivial device, then it will expect
protocols and bit sequences with associated timings. If you have a working knowledge of
electronics including protocols such as SPI, I2C and UART and are able to write applications
in C, then the chances are high that you could start from the data sheet and build your own
drivers to make the component work. The question I would pose to you though is why do
that? If you could just lift and re-use a high level library that performs the task why would you
not just do that? Fortunately, such libraries are prolific. Some may be for the Pi and some
may be for other MCUs such as the popular Arduinos. For non-Pi devices, you may have to
port the code to the Pi before you can use it.
This brings us to the topic of Open Source projects. These are source code projects where
the source is made publicly available and any associated licenses are very permissive. You
can usually find an Open Source project that comes close to meeting your needs and you can
start from there. Open Source benefits from the notion of others contributing back. This may
be in the form of improvements or enhancements to the code base or it may be in the form of
improvements to the documentation (if any). If you start to use an Open Source project and
find problems and fix them, submit them back so that the next person who walks your path
may benefit from the time and investment you yourself made.
When working on a project and it starts to come together, it is natural to feel proud of one's
work. However, I caution against the idea of thinking that your Open Source library is better
than an existing Open Source library. Instead, think about how you can improve what is
already there as opposed to duplicating an existing effort because (perhaps) you disagree
with one small aspect of such an existing project. Don't publish a new wheel just because
you can, it will confuse future Open Source users when they look for a functional area and
find dozens of them as opposed to just one good one.
The Github repository is an excellent place to find and work with Open Source projects. It is
based upon the Git source code control paradigms and Git is now available on all operating
systems and built into the majority of Integrated Development Environments including Eclipse.
Page 189
other pins, one can even create complex signals across multiple pins that can recreate
complex protocols such as I2C or SPI.
After we write our applications to test signal changes on output, we can determine if
everything is working in a number of fashions. We can attach a simple meter or logic probe
or get sophisticated and attach a signal analyzer. However the simplest mechanism is still the
attachment of an LED with a resistor. When the signal is high, the LED lights up. When the
signal is low, the LED remains dark.
We can create a bit of a more sophisticated circuit which is a simple logic probe as follows:
In this circuit, if the input is high, the Red LED (LED1) will switch on and the Green go dark,
while if the input is low, the Green LED (LED2) will switch on and the Red go dark. If the input
is open, then both LEDs will be on.
As a breadboard layout, here is the same circuit:
With this test circuit available to us, we are now at a place where we have an environment in
which to test simple signal changes. We will pick a GPIO pin, attach our logic probe and
Page 190
toggle the value periodically to watch it change.
The last two statements control the signal appearing upon the output. A "1" written to the file
called "value" causes the corresponding terminal to go high while a "0" written to the file
causes the terminal to go low.
See also:
GPIO interfacing
Using libsoc
Here is an example of using libsoc to toggle on and off XIO-P0 on a 4.4 Kernel using libsoc:
#include <stdio.h>
#include <stdlib.h>
#include <libsoc_gpio.h>
int main(int argc, char *argv[]) {
gpio *mygpio = libsoc_gpio_request(1016, LS_SHARED);
if (mygpio == NULL) {
printf("Failed to open gpio\n");
return 0;
}
libsoc_gpio_set_direction(mygpio, OUTPUT);
int i;
for (i=0; i<10; i++) {
libsoc_gpio_set_level(mygpio, LOW);
sleep(1);
libsoc_gpio_set_level(mygpio, HIGH);
sleep(1);
}
printf("All done!\n");
return 0;
}
Page 191
Using Node-RED
Node-RED provides a powerful component for controlling output signals and looks like this
when added to the canvas:
See also:
GPIO with Node-RED
YouTube: C.H.I.P. Tutorials: GPIO Output with Node-RED
Using libsoc
Here is an example of using libsoc to read the value from XIO-P2 on a 4.4 Kernel using
libsoc:
#include <stdio.h>
#include <stdlib.h>
#include <libsoc_gpio.h>
int main(int argc, char *argv[]) {
gpio *mygpio = libsoc_gpio_request(1018 /* XIO-P2 */, LS_SHARED);
if (mygpio == NULL) {
printf("Failed to open gpio\n");
return 0;
Page 192
}
libsoc_gpio_set_direction(mygpio, INPUT);
while(1) {
gpio_level value = libsoc_gpio_get_level(mygpio);
printf("Level = %d\n", value);
sleep(1);
}
return 0;
}
See also:
GPIO using libsoc
Using Node-RED
With the same physical setup as described, we can write a Node-RED flow. Here is what it
looks like:
We monitor the input terminal called XIO-P2. This is polled and passed continuously down
the line. The "Changed" node looks at a flow variable to see if the current value is the same
as the last seen value. If it is NOT (i.e. it has changed) then we propagate onwards to "Save
latest". Here we save the new value read from the input into the flow variable. Finally we
write a debug message to show that an input change was detected.
An even better solution is to use the Johnny-Five "Button" capability. The Node-RED flow will
Page 193
look as follows:
See also:
YouTube: C.H.I.P. Tutorials: GPIO Input with Node-RED
GPIO extenders
Although CHIP already has exposed GPIO headers, there are always times when you find
yourself wanting more. We have the capability to expand the number of GPIOs available to us
through some relatively inexpensive integrated circuits.
PCF8574
An example of an extender is called the PCF8574. The PCF8574 can be found on eBay for
about 40 cents each.
This is an I2C device and hence works using only two wires. Using this IC we configure it
with a 3 bit address (000-111) that is used to select the slave address of the device. There
are 8 permutations of 3 bits. Since each address has 8 IOs and we can have up to 8 devices,
this means a potential total of 64 additional pins. The PFC8574A is the same IC but has a
different set of I2C addresses. If we mix PCF8574 and PCF8574A devices, we can reach a
total of 128 pins.
It appears that the device will use a pull-up resistor for a high and bring the pin to ground for
low. This means that if we want to use any of the pins for input, we should set their write
mode to high first. This will allow either a high or low input signal to be detected. It would
appear that if we set the output signal to be low and then fed a raw high signal into the device,
we would have a short.
Note that the output currents vary on signal. For a low output, the typical current that can be
drawn is 25mA while for a high level output, the maximum current that can be drawn is
300uA. Note that is micro amps and not milliamps. This means that you can't drive a load
high.
Page 194
Here is the pin diagram for the device:
The address that the slave device can be found upon is configurable via the A0-A2 pins. It
appears at the following address:
PCF8574
6 5 4 3 2 1 0
0 1 0 0 A2 A1 A0
PCF8574A
6 5 4 3 2 1 0
0 1 1 1 A2 A1 A0
Page 195
A2 A1 A0 Address Address
PCF8574 PCF8574A
0 0 0 0x20 0x38
0 0 1 0x21 0x39
0 1 0 0x22 0x3a
0 1 1 0x23 0x3b
1 0 0 0x24 0x3c
1 0 1 0x25 0x3d
1 1 0 0x26 0x3e
1 1 1 0x27 0x3f
And on a breadboard:
Page 196
See also:
Datasheet
MCP23017
The MCP23017 from Microchip is a 16 bit input/output port expander that uses the I2C
interface. The going price for an instance of one of these on eBay is about $1. The device
has 16 GPIO pins that can be set as input or output controlled in two banks (ports). We can
read or write the values of one bank at a time meaning that if we want to write all 16 bits, this
would be two I2C operations and the same for reading. The device is also capable of
generating interrupts for input signal detection. If we imagine a 100Khz clock rate, then to
switch a bit on or off (and we can do these in groups of 8) then that would be 3 bytes of data
plus acknowledgments ~30 bits which would imply a maximum switching rate of about
0.3ms.
The device operates on a volate of 3.3V or 5V.
The pin interface is:
Page 197
Pin Label Description
1 GPB0 Bi-directional I/O
2 GBP1 Bi-directional I/O
3 GBP2 Bi-directional I/O
4 GBP3 Bi-directional I/O
5 GBP4 Bi-directional I/O
6 GBP5 Bi-directional I/O
7 GBP6 Bi-directional I/O
8 GBP7 Bi-directional I/O
9 VDD Power (3.3V - 5V)
10 VSS Ground
11 NC Not connected
12 SCL Serial clock input
13 SDA Serial data input/output
14 NC Not connected
15 A0 Address pin
16 A1 Address pin
17 A2 Address pin
18 RESET Hardware reset
19 INTB Interrupt output for port B
20 INTA Interrupt output for port A
21 GPA0 Bi-directional I/O
22 GPA1 Bi-directional I/O
23 GPA2 Bi-directional I/O
24 GPA3 Bi-directional I/O
25 GPA4 Bi-directional I/O
26 GPA5 Bi-directional I/O
27 GPA6 Bi-directional I/O
28 GPA7 Bi-directional I/O
Page 198
Register Name Description
address
0x00 IODIRA Direction control for port A:
1 input
0 output
0x01 IODIRB Direction control for port B:
1 input
0 output
0x02 IPOLA Polarity inversion for input for port A:
1 inverted
0 as-is
0x03 IPOLB Polarity inversion for input for port B:
1 inverted
0 as-is
0x04 GPINTENA Interrupt enable for a change for port A:
1 Enable input pin for interrupt on change event
0 Disable input pin for interrupt on change event
0x05 GPINTENB Interrupt enable for a change for port B:
1 Enable input pin for interrupt on change event
0 Disable input pin for interrupt on change event
0x06 DEFVALA Default values for interrupt change comparison for port A.
0x07 DEFVALB Default values for interrupt change comparison for port B.
0x08 INTCONA Interrupt control for port A:
1 Compare the value on the pin to the value in DEFVALA
0 Compare the value on the pin to its previous value
0x09 INTCONB Interrupt control for port B:
1 Compare the value on the pin to the value in DEFVALB
0 Compare the value on the pin to its previous value
0x0a IOCON Same register as 0x0b
Bit Label Description
7 BANK Register mapping:
Registers in banks
Registers sequential
6 MIRROR Interrupt pin mapping:
1 INT pins connected
0 INT pins separate. INTA is associated with
port A and INTB is associated with port B
5 SEQOP Sequential operation mode:
1 Sequential operation disabled
0 Sequential operation enabled
4 DISSLW Slew rate control
3 HAEN Hardware address enable
2 ODR Open drain output for INT
1 INTPOL Polarity of INT output:
1 Active high
0 Active low
0 N/A Not used. Set as 0.
Page 199
0 Pull-up disabled
0x0d GPPUB Pull up control for Port B:
1 Pull-up via a 100K resistor
0 Pull-up disabled
0x0e INTFA Interrupt flags for Port A:
1 Pin caused an interrupt
0 No interrupt detected
0x0f INTFB Interrupt flags for Port B:
1 Pin caused an interrupt
0 No interrupt detected
0x10 INTCAPA Values of GPIO when interrupt occurred on port A:
1 Pin was high
0 Pin was low
0x11 INTCAPB Values of GPIO when interrupt occurred on port B:
1 Pin was high
0 Pin was low
0x12 GPIOA Read Reads the values on port A
0x13 GPIOB Read Reads the values on port B
0x14 OLATA
0x15 OLATB
See also:
Data sheet
NeoPixels
NeoPixel theory
NeoPixels are LEDs that are driven by a single data line of high speed signaling. Most
NeoPixels have a +ve and ground voltage source as well as a data line for input and a data
line for output. The output of one NeoPixel can be fed into the input of the next one to
produce a string of such LEDs. The input data to the LED is a stream of 24 bits of encoded
data which should be interpreted as 8 bits for the red channel, 8 bits for the green channel
and 8 bits for the blue channel. Each channel can thus have a luminance value of between 0
and 255. By mixing the values for each of the channels together, you can color an LED to
any color you may choose. After sending in a stream of 24 bits, if we send in a second
stream of 24 bits quickly after the first stream, the second stream is "pushed" through to the
next LED in the chain. This can be repeated as far as desired. If we pause sending in data,
the current values are "latched" into place and each LED them remembers its own value.
The timings of the data signals for these LEDs can be quite tricky but fortunately great minds
have already built fantastic libraries for driving them correctly so we need not concern
ourselves with these low level timings and can instead concentrate on devising interesting
projects and purposes to which the LEDs can be placed. There are a number of different
types of these LEDs with the most common ones being known as WS2811, WS2812 or
PL9823.
Page 200
NeoPixels are usually 5V devices but check the specs carefully. Since NeoPixels draw a lot
of current (60mA each), do not try and power them from low current sources. You also need
to be careful when sending signals into them and getting their polarity correct. If you mess
up, they are unforgiving. I recommend buying more instances that you will need in case you
make a mistake.
The eight piece strip is labeled CJMCU-2812-8. It has 4 pins at each end of the strip labeled:
Label Description Color
GND Ground. Yellow
DIN/DOUT One side is DIN the other DOUT this means that the strip is polarized. These are the control data Green
input and output lines.
4-7VDC Source voltage. Red
GND Ground. Yellow
The data is sent as 24 bits in Green/Red/Blue order (GRB). Note that this is NOT RGB. The
format of the data uses NZR (Non-return-to-zero) encoding.
Specifically, the logic 0 and logic 1s are encoded as following.
Both bits correspond to a transition from high to low to high again but the duration at which
they are held indicates the encoding of a 1 or 0.
The data send to an individual device is 24 bits corresponding to 8 bits of color data for each
of the RGB components. The 8 bits identify the brightness of each of the channels with 1
being lowest and 255 being brightest.
Various diagrams show the timing as follows:
Page 201
Various data sheets show different timing values in micro seconds:
Device T0H T0L T0 T1H T1L T1
WS2811 0.5 2.0 2.5 1.2 1.3 2.5
WS2812 0.35 0.8 1.15 0.7 0.6 1.3
WS2812B 0.4 0.85 1.25 0.8 0.45 1.25
PL9823 0.35 1.36 1.71 1.36 0.35 1.71
As we see, for a WS2812B, the "width" of a bit is about 1.25 microseconds. For a full pixel,
this would be 24 bits or 30 microseconds. Thinking of it another way, that would be over
33,000 pixels that could be updated per second.
For testing circuits, I like to use PL9823. These can be had on eBay for between 25 cents
and 35 cents each. It is physically easy to work with and in the event of a design or assembly
error, they are cheap enough to shrug off damage and throw away ones you damage or
break.
See also:
WS2812 Data Sheet
PL9823 Data Sheet
Use a $1 ATTiny to drive addressable RGB LEDs
Adafruit NeoPixel Uberguide
Page 202
NeoPixels on CHIP
The timings required to operate NeoPixels natively on CHIP are either not precise enough or
because of the pre-emtive nature of Linux, not consistent enough. As such, the
recommended solution is to drive them through an intermediate such as an Atmel processor.
The Adafruit NeoPixel library is one of the libraries that is integrated into the Library Manager and can be installed quickly
from there:
Page 203
The sample I recommend to run comes from the File Examples Adafruit NeoPixel strandtest. Edit the file to
change the number of pixels in your strand (down to one if needed).
The high level operation of the library is that you create an instance of the class:
Adafruit_NeoPixel strip = Adafruit_NeoPixel(numPixels, dataPin, NEO_RGB + NEO_KHZ800);
With a variable now referencing an instance of the class, we can initialize the environment
with:
strip.begin();
Calling:
strip.show();
Transfers the internal model of what values each pixel should have to the correct pixel. Now
we can set the value of a pixel with a call to:
strip.setPixelColor(pixel, color);
The color value is a 32 bit integer with the values set for red, green and blue. If you know the
exact RGB code you want, you can invoke:
strip.Color(red, green, blue)
Page 204
Here is an example construction of this schematic:
When the circuits are wired together between the CHIP and Arduino, a quick check can be
made using the i2c detection tool which will list a device at address 0x10:
$ i2cdetect -y 1
0 1 2 3 4 5 6 7 8 9 a b c d e f
00: -- -- -- -- -- -- -- -- -- -- -- -- --
10: 10 -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
20: -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
30: -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
40: -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
50: -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
60: -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
70: -- -- -- -- -- -- -- --
The Arduino receives its instructions via an I2C connection. A suitable Arduino sketch would
be:
/**
* Receive data over the I2C data bus and use that data to drive an attached string of NeoPixels.
* The notion here is that we will be connected to a MCU that, for one reason or another, can't
* drive NeoPixels directly. An example of this would be the Raspberry Pi which doesn't have
* the (easy) non-preemtable control to drive at the timings required.
*
* Design Notes:
* The Arduino I2C interface doesn't handle buffers > 32 bytes. As such, we can't simply
* receive a "long string" of data and instead have to break it down into an array of
* communications.
*
* The data format expected is:
* [ 2 ] - 2 bytes of number of NeoPixels in the stream in network byte order.
* [ 3 ]* - 3 bytes of Red/Green/Blue data ... 1 byte per channel. This repeats the number of times
* described in the first field (number of pixels).
*
* The architecture/design of the solution is that we are receiving a stream of bytes that will be
Page 205
* translated into instructions to driver the NeoPixels. We are assuming that the unit of data
* is the byte and hence have to process one incoming byte at a time. Because data being processed
* is *NOT* in single byte chunks, we have to maintain a state machine in order to know "where we
* are" as the bytes arrive. We define the following states:
*
* * START - Waiting for the start data which is the length of the neopixel data to come.
* * PIXEL_RED - Waiting for the byte for red channel of the current pixel.
* * PIXEL_GREEN - Waiting for the btye for the blue channel of the current pixel.
* * PIXEL_BLUE - Waiting for the byte for the green channel of the current pixel.
*
* The transitions are simple:
*
* START -> PIXEL_RED -> PIXEL_GREEN -> PIXEL_BLUE
* || -> START - If we have received all the pixel data we expected
* || -> PIXEL_RED - If we have more pixels to process
*
* 300 pixels = 300 * 3 bytes = 900 bytes;
*
* @author: Neil Kolban ([email protected])
* @date: 2016-07-11
*/
#include <Adafruit_NeoPixel.h>
#include <Wire.h>
#define DEBUG 1
enum STATE {
START,
PIXEL_RED, PIXEL_GREEN, PIXEL_BLUE
} state;
void i2c_requestHandler() {
#ifdef DEBUG
Serial.print(F("i2c_requestHandler - We need to return some data\n"));
#endif
Wire.write(0x99);
} // end of i2c_requestHandler
/**
* Initialize the state of the environment for the START state.
*/
void setStart() {
#ifdef DEBUG
Serial.println(F("Entered state START"));
#endif
state = START;
lengthProcessed = 0;
numPixels = 0;
} // End of setStart
/**
* Process one byte of data received for processing.
*/
void processByte(unsigned char byte) {
Page 206
// If we have not seen a byte in over a second, then something has gone wrong
// and let us assume that we are restarting our state.
if ((millis() - lastReadTime) > 1000) {
setStart();
}
lastReadTime = millis();
/*
#ifdef DEBUG
Serial.print(F("We got a byte through I2C: "));
Serial.println(byte, HEX);
#endif
*/
if (state == START) {
numPixels = (numPixels <<8) + byte;
lengthProcessed++;
if (lengthProcessed == sizeof(numPixels)) {
#ifdef DEBUG
Serial.print(F("The number of pixels following is: "));
Serial.println(numPixels);
#endif
if (!configured) {
strip = Adafruit_NeoPixel(numPixels, 6, NEO_GRB + NEO_KHZ800);
if (strip.numPixels() == 0) {
Serial.println(F("Failed to create strip"));
return;
}
configured = 1;
strip.begin();
} // End of ! configured
state = PIXEL_RED;
currentPixel = 0;
} // End of got our length ...
} else if (state == PIXEL_RED) {
red = byte;
state = PIXEL_GREEN;
} else if (state == PIXEL_GREEN) {
green = byte;
state = PIXEL_BLUE;
} else if (state == PIXEL_BLUE) {
blue = byte;
state = PIXEL_RED;
strip.setPixelColor(currentPixel, strip.Color(red, green, blue));
currentPixel++;
if (currentPixel == numPixels) {
strip.show();
setStart();
}
}
} // End of processByte
void setup() {
lastReadTime = millis();
Serial.begin(115200);
setStart();
Wire.begin(DEVICE_ADDRESS);
//Wire.onReceive(i2c_receiveHandler);
//Wire.onRequest(i2c_requestHandler);
#ifdef DEBUG
Serial.println("Ready! We are now able to start receiving I2C data.");
#endif
} // End of setup
void serialEvent() {
if (started) {
while(Serial.available()) {
unsigned char inChar = Serial.read();
processByte(inChar);
}
} // End of started
} // End of serialEvent
void loop() {
if (!started) {
Page 207
int c = Serial.read();
if (c != -1) {
// we got a byte
headerBuffer += (char)c;
if (c == '\n') {
if (headerBuffer.startsWith("###NEOPIXELDRIVER###")) {
Serial.println("WE SAW THE START!!!!");
digitalWrite(LED_PIN, HIGH);
started = 1;
} else {
headerBuffer = "";
}
}
Serial.write(c);
Serial.println(headerBuffer);
} // End of we got a byte
} // End of !started
} // End of loop
While this circuit and program fragment work and sound great in theory, the practice actually
left a lot to be desired. I found that there were random errors between the CHIP and Arduino
over the I2C bus. These may have been able to be eliminated with shorter cables but all in
all, it wasn't satisfactory. Thinking further, it dawned on me that both CHIP and the Adruino
have serial buses (UART). Given that we want to send a max of 300 pixels which equals
300*3 bytes which equals 300*3*8 bits end up with ~7200 bits to color all the pixels. If we
choose a bit rate of 115200 that should give us a max of about 12 complete refreshes a
second which is far more than we desire so from a speed perspective, we are fine. The
circuit for serial processing is even simpler than that for I2C:
See also:
Adafruit: NeoPixel uberguide
Github: adafruit/Adafruit_NeoPixel
Page 208
NeoPixels through Node.js/Johnny-Five/Arduino I2C backpack
After writing the previous section, it was brought my attention that a significant amount of
work on this technique had already been done and done far better than any effort I may have
undertaken. A Github project called "ajfisher/node-pixel" had been implemented that uses an
Arduino as a Johnny-Five accessible "backpack".
To prepare the Arduino for use as a NeoPixel back-pack, we must install the Arduino sketch
provided as part of the solution. First we clone the node-pixel Github project. From there, we
will find a directory called firmware/build/backpack. In that directory we will find the sketch
called "backpack.ino". We should open this sketch in our Arduino IDE, compile it and
deploy it to our Arduino Nano instance. If you are using the schematic above, the way to load
the Arduino is to detach it from the circuit, program it using a USB connector and then
disconnect it from the USB and attach it back to the circuit. Do not try and power it from both
Page 209
the CHIP 5V output and the USB. I don't know what would happen if you tried but I doubt it
would be good at all.
If all has gone well, when we run "i2cdetect", we should see the existence of the Arduino at
address 0x42:
$ sudo i2cdetect -y 2
0 1 2 3 4 5 6 7 8 9 a b c d e f
00: -- -- -- -- -- -- -- -- -- -- -- -- --
10: -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
20: -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
30: -- -- -- -- -- -- -- -- UU -- -- -- -- -- -- --
40: -- -- 42 -- -- -- -- -- -- -- -- -- -- -- -- --
50: -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
60: -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
70: -- -- -- -- -- -- -- --
The node-pixel package is well documented at the Github home page. This guide should be
a cheat-sheet only and not a definitive.
The high level of usage is that we initialize a new Johnny-Five board and when get an
indication that the board is ready, we can start working with the NeoPixels. The notion of
node-pixel is that we have a "strip" of pixels. Whether they are physically arranged in a strip,
circle or other combination is immaterial we logically think of them as a strip. Each pixel
has an ordinal within the strip starting at 0, 1, 2 etc. To use node-pixel, we ask for the
construction of a strip object that models the NeoPixel strip. This has a number of constructor
properties but for our purposes (when using an Arduino Backpack) we care about:
board the Johnny-Five board object.
controller Indicate that we want to use an Arduino as a backpack. Use
"I2CBACKPACK".
bus The I2C bus on which the Arduino is attached. For CHIP use 2.
strips An array of integers where each integer represents the number of LEDs on
the corresponding pin off the Arduino. For example, an array of [2, 3, 0, 1] would say
that there are 2 LEDs attached to Pin 0, 3 attached to pin 1, none attached to pin 2 and
one attached to pin 3. A common usage might be [0,0,0,0,0,0,10] which would
say that we have 10 NeoPixels attached to Arduino pin 6.
For example:
var pixel = require("node-pixel");
var strip = new pixel.Strip({
board: johnnyFiveBoard,
controller: "I2CBACKPACK",
bus: 2,
strips: [0,0,0,0,0,0,10]
});
When we ask the strip to be created, we need to wait for its "ready" event to be published
before we can use it. For example:
strip.on("ready", function() {
// strip is now ready for use ...
Page 210
});
Now that we have a strip object, we can start to use it. Since a strip is a model of an array of
NeoPixels, we can work with any of those pixels. We obtain a pixel object that represents a
pixel by calling:
var pixel = strip.pixel(address);
when "address" is the numeric address of the pixels starting at 0. Once we have the
corresponding pixel object, we can invoke the color() method upon it. This sets the color
value which can be supplied in a number of formats including:
#rrggbb A hex code value for example #ff0000.
name A color name for example "orange".
[r,g,b] A color array.
Note that setting a pixel's color does not cause that pixel to change color immediately.
Instead we set the colors of all the pixels we want to change and then invoke the "show()"
method on the strip object. There are also some convenience functions available on strip
including:
off() - switches off all the pixels.
color(colorString) set the color of all the pixels to the given color string.
A simple application illustrating a test would be:
"use strict";
var five = require('johnny-five');
var board_io = require('chip-io');
var pixel = require("node-pixel");
Page 211
}, 1000);
});
strip.on("error", function() {
console.log("Strip error detected");
});
});
The device is a 5V device but since it is write only, we don't have to worry about level shifting.
The device has a slew of wiring connections though and if one consults the data sheet, one
will find a wealth of options.
Page 212
Pin Name Description
1 GND Ground
3 VE (Contrast)
5 R/W Read/Write
1 read
0 write
14 DB7 Used in 4bit or 8bit mode. Also contains the BUSY flag.
The device has two banks of RAM. One is for display data and is called DDRAM while the
other is for character generation and is called CGRAM. DDRAM is 80 bytes in size. The
selection of which bank of RAM to be used is made by setting the RS (register select) pin.
The device can be used in either a 4 bit transfer mode or an 8 bit transfer mode. In 8 bit
mode, all the pins labeled DB0-DB7 are used. In 4 bit mode, only the pins labeled in DB4-DB7
are used. To transfer 8 bits of data in 4 bit mode, two transfers are required. The order of
data is the 4 high order bits in the first transmission and the 4 low order bits in the second
transmission.
For two lines of display, take care to realize that the address of the 1st position on line two
does not immediately follow the last position on line 1.
Address 1 2 3 ... 39 40
Page 213
The combination of RS and R/W provide control information:
RS R/W Operation
1 0 Write data.
1 1 Read data.
Instructions:
Clear display
Clears the display and sets the DDRAM address counter to 0.
0 0 0 0 0 0 0 0 0 1
Return home
Sets the DDRAM address counter to 0. Also resets any display shifting that may be present.
0 0 0 0 0 0 0 0 1 -
0 0 0 0 0 0 0 1 I/D S
A constant called LCD_ENTRYMODESET with a value of 0x04 may be used as a base. Other
related constants are:
LCD_ENTRYRIGHT = 0x00
LCD_ENTRYLEFT = 0x02
Page 214
LCD_ENTRYSHIFTINCREMENT = 0x01
LCD_ENTRYSHIFTDECREMENT = 0x00
Display mode properties
Sets display controls.
0 0 0 0 0 0 1 D C B
0 0 0 0 0 1 S/C R/L x x
Page 215
RS R/W DB7 DB6 DB5 DB4 DB3 DB2 DB1 DB0
0 0 0 0 1 DL N F x x
Page 216
Read busy flag and address
Always check the busy flag (BF) to determine if the device is busy before performing an
operation.
Write data to RAM
Writes data to either CGRAM or DDRAM. The choice of whether CGRAM or DDRAM is
written is based on the last previous address change of the address counter. For example, if
the address counter for DDRAM was last set, then the next write will be to DDRAM.
1 0 D7 D6 D5 D4 D3 D2 D1 D0
After a write to RAM, the address is automatically incremented in preparation for the next
write operation.
Read data from RAM
Reads data from either CGRAM or DDRAM. The choice of whether CGRAM or DDRAM is
read is based on the previous address change of the address counter. For example, if the
address counter for DDRAM was last set, then the next read will be from DDRAM.
1 1 D7 D6 D5 D4 D3 D2 D1 D0
After a read from RAM, the address is automatically incremented in preparation for the next
write operation.
Character table
Page 217
0000 0001 0010 0011 0100 0101 0111 1000 1001 1010 1011 1101 1110 1111
0000 0 @ P ` p
0001 ! 1 A Q a q
0010 " 2 B R b r
0011 # 3 C S c s
0100 $ 4 D T d t
0101 % 5 E U e u
0110 & 6 F V f v
0111 ' 7 G W g e
1000 ( 8 H X h x
1001 ) 9 I Y i y
1010 * : J Z j z
1011 + ; K [ k {
1100 , < L l |
1101 - = M ] m }
1110 . > N ^ n
1111 / ? O _ o
PCF8574 module
A cheap PCF8574 based module is available for about $1 on eBay that provides an I2C
mapper for this device. Instead of driving each of the LCD pins directly, they can be mapped
onto a PCF8574 GPIO extender.
Page 218
Since the module is based on a PCF8574A and the PCF8574A has 8 extended GPIOs, the
mapping between these and the HD44780 are as follows:
PCF8574 GPIO HD44780 function
P0 RS
P1 RW
P2 Enable/Strobe
P3 BL
P4 D4
P5 D5
P6 D6
P7 D7
Here is a wiring schematic illustrating connection between CHIP and the LCD1602 with I2C:
To access the module from Node.js, an npm package called "lcd-pcf8574" is available.
When connected to the CHIP I2C bus number 2, it shows up address 0x37.
See also:
PCF8574 I2C driver module for 1602 LCD displays
npm lcdpcf8574
Page 219
Using JavaScript and Johnny-Five
See also:
Johnny-Five
Johnny-Five LCD
GPS
The world is a sphere (hopefully that isn't news to you). We can specify a point on the Earth
through a coordinate system known as latitude (lat) and longitude (lng). Given a lat/lng
coordinate, we know where we are. The Global Positioning System (GPS) is a technology
that allows a GPS receiver to determine its own lat/lng coordinates.
Given a pair of coordinates, for example one coordinate that says where a device is and a
second coordinate saying (for example) where we want to eventually be, we can calculate
some interesting data such as how far away are we from the target and what compass
bearing we need to head to get there.
There are many applications of GPS and it has entered the popular domain. Most cell
phones now have in-built GPS receivers such that they can display a map showing your
current location. Combine that with a database of driving directions and we have real-time
route planners where we mount either a dedicated GPS receiver or a cell phone on our car
dashboards and provide destination details and the devices tell us the turns to make.
The Global Positioning System (GPS) is a set of satellites in orbit. They are continually
transmitting a very precise time signal. If we have a suitable receiver, we can receive the time
signals from some number of those satellites that are over head at any given time. Since the
speed of radio transmission is constant and not instantaneous and each satellite is producing
an extremely accurate and synchronized time signal, then a signal clock pulse emitted by all
satellites at exactly the same time will be received at very slightly staggered times by the
receiver as a function of the distance that the signal had to travel through space. This is the
same as the distance that the satellite was from the receiver. By receiving enough signals
from a sufficient number of distinct satellites and by using complex mathematics, the receiver
can then triangulate its own position and thus know where it is on the surface of the Earth.
Putting that in perspective, an electronic module can determine physically where it is. Such
modules are now common within your cell phone and within many car dashboards. They are
often used in conjunction with mapping software to provide a real-time map showing your
location. The accuracy of GPS is commonly about 4 meters.
Devices can be picked up for about the $12 mark. The unit I worked with is called the GY-
GPS6MV2 which is based on the u-blox NEO-6m. The pins on this breakout board are vcc,
gnd, RX and TX. This is 5V device. As such, you must use a level shifter or voltage divider
between the TX pin of the GPS and the RX pin of CHIP as the CHIP can't accept a 5V signal
input.
Since it is a UART device, we can test this on a regular PC. If we connect a serial port
Page 220
terminal, we can watch the data be received. The data that comes across is in NMEA format
(National Marine Electronics Association). There are various NMEA reader applications freely
available which can format the data.
To test the data, you should really be out-doors or otherwise have an un-obstructed view of
the sky. Testing in the interior of a building is basically fruitless as the GPS signals will not
penetrate and you will have learned nothing.
The device has an LED on the circuit that illuminates (flashes) when a lock has been made.
On a CHIP, we can connect the device directly to the UART. Connect the devices TX pin to
the CHIP RX through a level shifter or voltage divider, the device's RX pin to the CHIP TX and
the device 5V to the CHIP 5V line and the devices GND to GND on the CHIP.
To see it in operation, stop any getty program listening on /dev/ttyS0 and run a terminal
emulator such as PuTTY to connect to the serial port. You will immediately start to see the
Page 221
protocol data being sent by the GPS device:
See also:
UART (Serial) Interface
You tube: Tutorial 15 for Arduino: GPS Tracking
Wikipedia: Global Positioning System
Wikipedia: Geographic Coordinate System
The result of this is a curses based text application that shows a wealth of data related to the
GPS information being decoded from the device:
Page 222
Core amongst this data are:
Time The time value received from the satellites
Lat The latitude of the device
Lon The longitude of the device
The core though of the gpsd package is the gpsd demon. This demon program runs on a
machine which knows how to interact with a GPS device, reads and parses the data and then
makes it available as a higher level protocol.
When we run the gpsd demon, we can then connect to the device using gpsmon over the
network.
Using the gpspipe command the protocol from gpsd is captured and written to stdout.
An alternative to gpsmon is cgps which displays character data as follows:
Page 223
One final tool of interest to us is xgps which requires an X-Windows server and shows more
details on the satellites:
Page 224
From a programming perspective, there are libraries available for both C and C++ (as well as
other languages). These libraries simplify the work we need to perform to get the information
we desire. The single most important data type available with this library is called
gps_data_t. This contains the state and data retrieved from the GPS device. Some of its
fields are:
Page 225
Field Description
gps_mask_t set
timestamp_t online
gps_fix_t fix
int satelites_used
int satelites_visible
int status
Among these, the gps_fix_t fix member contains the data we most commonly want:
Field Description
timestamp_t time Time stamp of this fix record
int mode Mode of the fix:
0 not acquired
1 2D acquired
2 3D acquired
double latitude Fix latitude value
double longitude Fix longitude value
double altitude Altitude above sea level
double track
double speed Speed
double climb
double epc
double epd
double eps
double ept
double epy
double epv
double epx
double epy
Page 226
if (gps_read(&gps_data) != -1) {
printf("fix: %f, %f\n", \
gps_data.fix.latitude,
gps_data.fix.longitude);
} else {
printf("Error\n");
}
}
}
gps_close(&gps_data);
}
See also:
gpsd
main(3) libgps
The architecture of Open Source Applications gpsd
Page 227
}
});
}, function(error) {
console.log("Error: %O", error);
});
console.log("Waiting for position!\n");
</script>
</head>
<body>
Hello world!!
</body>
</html>
It works by using the HTML geolocation object which has a series of functions to allow one
to obtain GPS sensor information from within the web page. When a sensor reading has
been obtained, the values of the latitude and longitude are sent back to the Pi via a REST
request. The JavaScript uses the services of jQuery to make the REST request.
See also:
MDN Using geolocation
Making REST requests using jQuery
Video / Webcams
How can we possibly resist attaching a video input source to our CHIP?
Many webcams are available very cheaply and are USB attachable. Once attached to your
CHIP, you should check that it has been recognized. A quick way to do this is to run lsusb
before plugging in the camera and then re-running it after the camera has been attached.
The newly added device will be your camera. For example, on my system:
Bus 002 Device 001: ID 1d6b:0001 Linux Foundation 1.1 root hub
Bus 001 Device 002: ID 1e4e:0110 Cubeternet
Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
Bus 003 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
Once you have located your USB device entry, you can ask for details that the device has
made known to Linux by running the command:
lsusb -v -s <device number>
This returns a lot of detailed information and we won't be going into the details of it. However,
one can just about see the list of different resolutions and other attributes supported by the
camera.
Since we are running on CHIP OS and CHIP OS is a version of Linux, we can leverage all the
work that has been on video processing for Linux. This is known as the "video4linux"
interface and manifests itself as devices of the for /dev/video0. An interesting thing to note
is that video is considered a securable resource. What we mean by that is that access to
your video camera is restricted to only certain users. This makes sense as if your CHIP were
to be somehow compromised or you gave others access to your CHIP then, in principle, they
could switch on your camera and watch you. If you were to watch me, it would be deathly
Page 228
boring but you can obviously see that it could be abused. When a video camera is attached,
the device has restrictive file permissions:
$ ls -l /dev/video0
crw-rw---- 1 root video 81, 0 Jan 9 16:37 /dev/video0
The most powerful tool available for working with Webcams is called fswebcam. The
fswebcam package is not part of the standard CHIP distribution and needs to be installed
using:
$ sudo apt-get install fswebcam
I suggest reading the manual page once installed before using it as it has a ton of options.
However, the most basic is:
$ fswebcam <fileName>
for example:
$ fswebcam image1.jpg
This will grab an image from the default webcam and write into the file. If this works, you
should find a file on your file system. To view it, I recommend installing the powerful package
called "imagemagick". This can be installed with:
$ sudo apt-get install imagemagick
There are a lot of tools associated with the package so read the manual pages as needed.
One of the useful tools is called "display" which displays an image in an X-Windows
environment. For example:
$ display image1.jpg
Note: When I tried this, I found that my image was pure black and it took me a long time to
figure out the problem. The Webcam I was using has auto-brightness enabled. The way this
works is to start sampling frames when the camera starts up and examining the light levels
from those. After a few frames have been received, the on-board electronics determines the
correct brightness level. However, immediately after start-up (i.e. each time I request an
image), the camera hadn't yet seen enough frames to determine the level and hence the
black image. The fswebcam tool has an option called "--skip <frame count>" which
ignores the first number of frames before taking the desired snap. For example, I have to
run:
$ fswebcam --skip 10 image1.jpg
Once you have validated that the you can grab an image, it is time to move on to some of the
other options of fswebcam. Specifying the --resolution <width_x_height> allows us
to declare what resolution we wish to capture. Since not all devices support arbitrary
resolutions, determine the allowable sizes. For example, by closely examining the output of
lsusb -v -s <device> I found that my camera supports:
Page 229
640x480
352x288
320x240
176x144
160x120
Some of these pairings seem "odd" to my eyes but I'm sure there are good reasons why they
exist.
The image generated by fswebcam has a banner by default. This is a text bar at the bottom
which includes the date/time the picture was captured. There are a variety of output options
we can supply to control the appearance and these can be read about in more detail in the
man page. The one that I do want to cover though is --no-banner which disables the
banner.
Another tool for our consideration is called motion. This is quite a catch-all tool for working
with video being captured from the device. Its primary ability is to continually watch the video
input source and when the image contained in the video changes, perform an action such as
capture the changed scene. In effect, this provides a motion capture capability. One can
imagine running this application against a webcam near a doorway and only folks arriving or
leaving are recorded and not the empty spaces in between. Alternatively, one could record
some wildlife prone area and only when an animal enters the scene, will the recording start.
In addition to detecting motion in a scene, the tool allows us to stream video over a network.
The frame rate is poor maybe only a few frames/second (if that) but it does work and is
simple to use.
The tool is configured by a rich and detailed configuration file called motion.conf. The
search order is to look for motion.conf in the current directory and then in a directory called
.motion in the home directory and then at /etc/motion/motion.conf. It is likely that
this file will have to be edited before you use motion. The tool can be started as a demon
which puts itself into the background. However it can also be forced to run in the foreground
when testing. To run in the foreground, use:
$ sudo motion -n
Again, this is not supplied with a default CHIP distribution so we must install it with:
$ sudo apt-get install motion
Motion can be started as either a demon or in non demon mode. When setting it up, I
recommend running in the non-demon mode. This can be controlled by passing in the "-n"
flag.
To say that there are a lot of options associated with motion is an understatement. We will
see some of these in the following sections.
See also:
Using a standard USB webcam
Page 230
man(1) fswebcam
ImageMagick home page
Motion home page
man(1) motion
cheese
avconv documentation
Page 231
and it will start recording. If we set it off, then it will no longer be emulating motion but real
motion will trigger the recording. If we only wish to record when we want to record, then we
can disable detection of motion this says "Don't watch the video output and look for
motion" however, if we switch on emulate_motion while detection is disabled, then video
will be recorded. Is this a hack? Possibly, but it works.
We can use REST requests to control this. Sending:
http://192.168.1.101:8080/0/detection/pause
disables emulation.
Since REST requests can be sent either based on the CRON scheduling demon or by
application logic when a sensor is triggered, we now have the ability to capture video under
application control.
Recording a video
We can record a video using ffmpeg. For example:
$ ffmpeg -f v4l2 -framerate 25 -video_size 640x480 -i /dev/video0 output.mkv
We can add the -t <seconds> option to force recording for some period of time in seconds.
See also:
ffmpeg documentation
avconv
Playing video
The "vlc" package provides video over X-Windows.
$ sudo apt-get install vlc
Be aware that this is a big package. When launched it provides the ability to view video. The
video files can be opened as desired.
Page 232
Other players for playing video include mplayer/mplayer2.
Page 233
Once we have a Mat that contains data, we can interrogate it to manipulate pixels. We can
retrieve a pixel value with the at(row, col) and can also set the value the same way.
A common value of a pixel is 3 bytes, one each for red, green and blue. This can be
represented in OpenCV as a Vec3b (a vector of 3 bytes). The order of pixels is BGR and not
RGB.
We can read a static image from the file system using the "imread" method.
If we wish to create a window to show graphics, we can create a named window using
"namedWindow". To display an image in a window use "imshow".
See also:
OpenCV Installation in Linux
OpenCV API Reference docs
Page 234
A conversion is requested when CS (pin 1) goes from high to low. It should stay low until the
reading of the analog signal and its conversion is complete.
A clock signal starts low, goes high and then back to low. The clock signal should be sent to
the CLK (pin 7).
The data in (DI) is used to provide instructions to the IC. Its value is clocked in on the rising
edge of the clock signal. Following the start of the conversion, the DI is used to send in three
bits.
First a Start Bit used as a flag. This should be a 1 value.
Second a SGL/DIF selection bit The IC can work in an absolute or relative mode in
terms of analyzing the incoming analog voltage value. The SGL (on) (Single Ended)
mode means that the absolute value on the channel will be used while the DIF (off)
(Differential) mode will compare the difference between the values on CH0 (pin 2) and
CH1 (pin 3).
Third an ODD/SIGN selection bit. If we are in SGL mode, the value of this selection
chooses between reading from channel 0 (CH0) or channel 1 (CH1). If we are in DIF
mode, this selection choose which of the channels should be subtracted from the other
(we cant have negative values).
Following this we get 8 bits of analog converted data twice. The first 8 bits are MSB first
while the second 8 bits are LSB first. The signal data can be found on D0 (pin 5).
The following is reproduce from the data sheet and pictorially illustrates the timing diagrams:
Page 235
The high level bit banging algorithm is as follows:
set CLK mode = Output
set CS mode = Output
loop for 8
set CLK = 1
get DIO /// Get a data bit
delay
set CLK = 0
delay
end of loop
Page 236
set CS = 1
Here is a JavaScript program that implements this algorithm using the JavaScript "onoff"
package.
/**
* adc0832 module
*/
"use strict";
var Gpio = require('onoff').Gpio;
var sleep = require("sleep");
const LOW = 0;
const HIGH = 1;
var clkPin;
var dioPin;
var csPin;
module.exports.init = function() {
Page 237
clkPin = Gpio(CLK, "out");
dioPin = Gpio(DIO, "out");
csPin = Gpio(CS, "out");
};
module.exports.getValue = function() {
dioPin.setDirection("out");
clkPin.writeSync(LOW);
csPin.writeSync(LOW);
dioPin.writeSync(HIGH);// SGL
clkPin.writeSync(HIGH);
sleep.usleep(DELAY);
clkPin.writeSync(LOW);
sleep.usleep(DELAY);
var value = 0;
for (var i=0; i<8; i++) {
clkPin.writeSync(HIGH);
var bit = dioPin.readSync();
value = (value << 1) | bit;
sleep.usleep(DELAY);
clkPin.writeSync(LOW);
sleep.usleep(DELAY);
}
csPin.writeSync(HIGH);
return value;
}; // End of getValue()
Page 238
The device is extremely power efficient and can operate in a number of power modes. In
single-shot mode, it samples the value and makes it available and then goes into a lower
power mode. In continuous operation mode, it starts calculating the next sample as soon as
the previous has been calculated.
The pin out on the device is:
Pin Label Description
1 VDD 3.3V, 5V
2 GND Ground
3 SCL I2C Clock
4 SDA I2C Data
5 ADDR Address selection
6 ALRT Digital comparator / Conversion ready
7 A0 Channel 0
8 A1 Channel 1
9 A2 Channel 2
10 A3 Channel 3
Page 239
101 AIN1 GND
110 AIN2 GND
111 AIN3 GND
2 Lo_threshold register
3 Hi_threshold register
The default address of the device is 0x48 0b100 1000. This can be altered by wiring the
ADDR pin to different values. What this allows us to do is to have multiple instances of the
devices attached to our I2C bus. If we needed more than 4 analog inputs, this would be
essential.
Addr Pin I2C Address
GND 100 1000 (0x48)
VDD 100 1001 (0x49)
SDA 100 1010 (0x4a)
SCL 100 1011 (0x4b)
Page 240
When plugged into chip, we can see that it is available by running:
$ sudo i2cdetect -y 2
0 1 2 3 4 5 6 7 8 9 a b c d e f
00: -- -- -- -- -- -- -- -- -- -- -- -- --
10: -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
20: -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
30: -- -- -- -- -- -- -- -- UU -- -- -- -- -- -- --
40: -- -- -- -- -- -- -- -- 48 -- -- -- -- -- -- --
50: -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
60: -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
70: -- -- -- -- -- -- -- --
Using JavaScript
An existing library for JavaScript is available. See npm node-ads1x15 (alphacharlie/node-
ads1x15 on Github). This has the following methods exposed:
readADCSingleEnded(channel, gain, samplesPerSec, callback)
channel the channel to read from ... values 0-3.
gain default us 6144
Page 241
samplesPerSec default is 250
callback(error, data)
readADCDifferential(chP, chN, gain, samplesPerSec, callback)
readADCDifferential01(gain, samplesPerSec, callback) Get the differential between
channels 0 and 1.
readADCDifferential03(gain, samplesPerSec, callback) Get the differential between
channels 0 and 3.
readADCDifferential13(gain, samplesPerSec, callback) Get the differential between
channels 1 and 3.
readADCDifferential23(gain, samplesPerSec, callback) Get the differential between
channels 2 and 3.
startContinuousConversion(channel, gain, samplesPerSec, callback)
stopContinuousConversion(callback)
getLastConversionResults(callback)
startSingleEndedComparator(channel, thresholdHigh, thresholdLow, gain,
samplesPerSec, activeLow, traditionalMode, latching, numReadings, callback)
startDifferentialComparator(chP, chN, thresholdHigh, thresholdLow, gain,
samplesPerSec, activeLow, traditionalMode, latching, numReadings, callback)
While experimenting with this code, it was found that the I2C bus was hard coded to
"/dev/i2c-1" which is no good for a CHIP of kernel 4.4 where our bus is "/dev/i2c-2".
An issue has been raised on the GitHub project. As a circumvention, the underlying code can
be edited to change the bus.
See also:
npm: node-ads1x15
Github: alphacharlie/node-ads1x15
Page 242
pushed back in your seat when the gas is pressed and you would feel yourself wanting to
move forward if the brake is pressed. Similarly, if you stand in an elevator and press a button,
you feel yourself being pushed into the floor when it starts to rise and you would find yourself
wanting to rise when it comes to a stop. Acceleration is a vector quantity meaning that it has
a directional quality associated with it. In your car (assuming you are driving forward and we
call the forward direction x) then stepping on the gas produces an acceleration in the x
direction while stepping on the brake produces an acceleration in the -x direction. Similarly
with the elevator, when you rise there will be an acceleration in the z direction and when you
stop, there will be an acceleration in the -z direction. There is one more twist to the story
gravity. Although you may not have thought about it, as you sit in your chair there is a force
acting on you that wishes to accelerate you. That force is called gravity and the direction of
the acceleration is -z. If it weren't for the chair, your body would be accelerated towards the
floor. Even though your body isn't moving, we can consider this an acceleration force being
applied. What this means for us is that when a device such as the MPU-6050 is sitting on
your desk, it will be reporting that it is accelerating downwards.
The practical implication of this is very interesting. If the MPU-6050 is flat on the desk, it will
report an acceleration in its -z direction. If we now were to tilt the device, the acceleration
due to gravity is still present but now the direction of that force will have changed. The
magnitude of the acceleration will remain the same but the measurements will now show it
spread over the x, y and z axises. Working backwards, by examining the values of the
acceleration on the x, y and z axises, we can calculate the orientation of the device relative to
the direction of gravity (i.e. relative to up and down).
Having just discussed the concept of measurement of acceleration through the notion of an
accelerometer, we can now look at how a gyroscope comes into play. A gyroscope measures
the change in velocity (if at all) of a device rotating around its own axis, A gyroscope
measures changes in angular velocity.
The MPU-6050 combines these to measure both linear acceleration and angular velocity.
The device measures both qualities in the x, y and z axis and hence is considered a
measurement in 6 degrees of freedom (linear acceleration in x, y and z and angular velocity
in x, y and z). The measurement values have a 16 bit resolution.
If our goal is to measure the orientation of the device, we may be able to see that the
orientation can be found by examining the accelerometer values however these values only
result in accurate answers if the device is at rest. If it is moved, then the acceleration due to
movement can produce jittery results. If we look at the gyroscope, if we start from a known
orientation (eg. flat on the desk), then in principle we should also be able to determine the
device's current orientation by adding together all the gyroscopic changes that have
happened to it since its original known (calibrated) position. Unfortunately, both of these
techniques introduce errors. Using the accelerometer, we can determine our orientation by
averaging values over time to remove jitter. As such, it gives good values over time but poor
for short term measurement. The gyroscope gives us good angular momentum values in the
short term but the errors accumulate over time when we try and calculate the orientation of
the device from its base state by successive addition of delta values. Fortunately, we can
Page 243
combine these two techniques through a variety of algorithms to give us a good value that is
built from the combination of the two measurements.
The MPU-6050 device connects via an I2C bus (default address is 0x68 but can be changed
to 0x69). This is 3.3V device and hence is safe to connect to CHIP pins directly.
The device has the following pins exposed from the breakout boards:
Pin Description
VCC 3.3V
GND Ground
ADO I2C address selection, either 0x68 or 0x69. Low = 0x68, high = 0x69.
INT
Page 244
After wiring, if we run i2cdetect we will see the device:
$ sudo i2cdetect -y 2
0 1 2 3 4 5 6 7 8 9 a b c d e f
00: -- -- -- -- -- -- -- -- -- -- -- -- --
10: -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
20: -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
30: -- -- -- -- -- -- -- -- UU -- -- -- -- -- -- --
40: -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
50: -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
60: -- -- -- -- -- -- -- -- 68 -- -- -- -- -- -- --
70: -- -- -- -- -- -- -- --
Page 245
Register Offset Name Description
You will sometimes see these devices referred to as Inertial Measurement Units (IMUs).
The value returned from the accelerometer is a raw value. The actual result we want is
measured in gravities (g) which is about 9.81 ms -2. The device has various sensitivities built
in. To get to the real value, we need to divide by a scaling factor as show below:
Acceleration Sensitivity
Limit Factor
2g 16384
4g 8192
8g 4096
16g 2048
Thus to get the required value from the raw value we would use the equation:
raw value
required value=
sensitivity factor
By default, the device is configured for a 2g acceleration limit
Similarly, there is an angular velocity scaling:
Page 246
Angular Sensitivity
Velocity Limit Factor
250/s 131
500/s 65.5
1000/s 32.8
2000/s 16.4
Johnny-Five also provides a module for gyroscopes which includes the MPU-6050. At a high
level, we use this is as follows:
var gyro = new five.Gyro({
controller: "MPU6050":
bus: 2
});
Page 247
gyro.on("change", function() {
// handle data changes ...
// this <dot> x, y, z, pitch, roll, yaw, rate, isCalibrated
});
The class exposes two events. The first is called "change" which is fired when ever a
detection of change more than the configured threshold. The second is called "data" which
will be fired as often as possible.
Here is an example Johnny-Five application:
"use strict";
board.on('ready', function() {
var gyro = new five.Gyro({
controller: "MPU6050",
bus: 2
});
gyro.on("change", function() {
debugger;
console.log("Gyro changed! pitch=", this.pitch,
", roll=", this.roll,
", yaw=", this.yaw);
});
console.log("gyro initialized!!");
});
See also:
Accelerometers
Tilt Sensing Using a Three-Axis Accelerometer
MPU-6050 Data Sheet
MPU6050 Register Map and Descriptions
MPU-6050 Accelerometer + Gyro
I2Cdevlib MPU-6050
Filters
InvenSense MPU-6050 Home Page
Gyroscopes and Accelerometers on a Chip
Using an accelerometer for inclination sensing
YouTube: MPU-6050 Data with a Complementary Filter
Johhny-Five: Gyro
Page 248
The math of Accelerometers
Our real world is composed of three spatial dimensions. In English we might think of these as
left/right, up/down and in/out (or forward/back). In our math, we will label these by the
common x, y and z. If we want to think about the orientation of something, we can think of it
as the rotation about these axis. We define the rotation around the x axis as "roll" designated
by the symbol "". We define the rotation around the y axis as "pitch" designated by the symbol ""
and finally, we define the rotation around the z axis as "yaw" designated by the symbol "". So, to
think about the orientation of an object, we can think about its roll, pitch and yaw.
Axis Name Symbol
x roll
y pitch
z yaw
Now, let us merge in the notion of how we are going to measure our roll, pitch and yaw. Assuming no
acceleration of our device (I.e it is stationary) then there will be a constant force on it of 1g in the z
axis. So the force measured in the z axis will be 1 and x and y will both be 0. As we tilt the device
from its horizontal orientation, that force will start to be distributed across multiple axis and not just the
z axis. So a tilt will produce a new measured force value in both x and y. And here comes the magic.
The measured force change is proportional to the roll and pitch angles. But wait, what about the yaw?
Well if we think about it, a rotation around the z axis does not change the tilt of what we want to
measure. Think of it like this, if we have a perfectly flat table top and place some marbles on its
surface, if we tilt the table left or right, the marbles will roll. If we tilt it forward or back, the marbles
will roll. However, if we are gentle and rotate the table around its central vertical axis, the marbles will
not roll. They will remain in their same position relative to the table.
Another way to think about this is to hold your cell phone in your hand flat while sitting on a chair. If
you now spin around on your chair, have you changed the "tilt" of your cell phone? I would say no.
What this means to us is that we can calculate the orientation of our device only by figuring out the roll
and pitch. The yaw becomes immaterial to our story (for just now).
By some math, we can find that:
G py
tan()=
G pz
G px
tan()=
G 2
py +G2pz
See also:
Using an Accelerometer for Inclination Sensing
Understanding Euler Angles
Page 249
Visualizing orientation
It is all very good getting raw numbers representing the amount and direction of force on our
device against different axis, but triplets of numbers isn't at all intuitive to our ways of thinking.
Instead, what we want is a mechanism to "see" the orientation of the device. Since
orientation is inherently a three dimensional physical consideration, what we want is to
visualize the data as a 3D scene. There are many 3D programming packages on the
Internet with varying degrees of sophistication and capabilities. Since my belief is that all user
interfaces should be browser based as opposed to thick client, I wanted one that could run in
a browser. I also wanted one that is open source. For this I chose "three.js".
See also:
three.js
The pin configuration for GY271 is (Left-Right as seen from chip side with connector at top):
Page 250
Pin Function
GND Ground
The pin configuration for the CJ-M49 is (Left-Right as seen from chip side with connector at
top):
Pin Function
GND Ground
The module makes its data available via the I2C bus. The I2C address of the device is 0x1E.
If we run
$ i2cdetect -y 2
Page 251
Address Description Access
00 Configuration register A Read / Write
01 Configuration register B Read / Write
02 Mode register Read / Write
03 Data output X MSB register Read
04 Data output X LSB register Read
05 Data Output Z MSB register Read
06 Data Output Z LSB register Read
07 Data output Y MSB register Read
08 Data output Y LSB register Read
09 Status register Read
10 Identification register A Read
11 Identification register B Read
12 Identification register C Read
Think of the device as having a register cursor. We can move the cursor simply by sending
the id of the register to move to. For example, sending the device 0x03 will move the cursor
to register 3. Subsequent reads will read from the cursor location and auto-increment the
cursor. So moving to register 3 and reading 6 bytes will read the next 6 register values
[0x03-0x08].
Configuration Register A (register 00) is as follows:
Name Bits Description
7 Reserved. Must be 0.
Page 252
Name Bits Description
The three identification registers return the ASCII values 'H', '4' and 'C'.
To create an angle in degrees, the following formula can be used:
angle = atan2((double)y,(double)x) * (180 / 3.14159265) + 180; // angle in degrees
Attachment of the device to CHIP is particularly easy. As you can see it only needs 4 wires to
connect.
Page 253
Here is a schematic of how an instance of the board may be wired to CHIP.
When we use a compass, the result we usually want is "which way is North". On a real
physical compass, we have a needle that points in that direction and we know our answer.
Through a digital device, what we should expect to get back is an angle that the device would
have to be oriented for it to point North. This assumes that there is a reference mark on the
device from which the angle has meaning.
When experimenting with the device, don't bring strong magnets too close to the device as it
may become magnetized or otherwise damaged.
See also:
Data sheet
Sparkfun Tutorial
Arduino Nano + GY-271 (Digital Compass module) + OLED
Electrodragon - HMC5883L - Three-Axis Compass
YouTube Arduino How To: HMC5883L Compass Magnetometer Tutorial
YouTube Use the HMC5883L 3-axis sensor with an Arduino Tutorial
YouTube Let's build an Arduino electronic Compass using the HMC5883L and a Ring of LEDs -
Tutorial
Wikipedia atan2
Github Arduino Library
Page 254
Using WiringCHIP
We can easily use the WiringCHIP I2C functions to interact with the device. Access is
relatively simple with no complex operations. Once we have retrieved the values, we can
perform the math to calculate the angle to magnetic North.
#include <stdint.h>
#include <stdlib.h>
#include <stdio.h>
#include <math.h>
#include <wiring-CHIP.h>
#define PI 3.14159265
#define HMC5883L_MODE_CONTINUOUS (0x00)
#define HMC5883L_ADDRESS (0x1e)
#define HMC5883L_REG_MODE (0x02)
#define HMC5883L_REG_MSB_X (0x03)
#define HMC5883L_REG_LSB_X (0x04)
#define HMC5883L_REG_MSB_Y (0x05)
#define HMC5883L_REG_LSB_Y (0x06)
#define HMC5883L_REG_MSB_Z (0x07)
#define HMC5883L_REG_LSB_Z (0x08)
while(1) {
uint8_t msb = wiringCHIPI2CReadReg8(fd, HMC5883L_REG_MSB_X);
if (msb == -1) {
perror("msb");
}
uint8_t lsb = wiringCHIPI2CReadReg8(fd, HMC5883L_REG_LSB_X);
if (lsb == -1) {
perror("lsb");
}
short x = msb << 8 | lsb;
Page 255
msb = wiringCHIPI2CReadReg8(fd, HMC5883L_REG_MSB_Z);
lsb = wiringCHIPI2CReadReg8(fd, HMC5883L_REG_LSB_Z);
short z = msb << 8 | lsb;
return 0;
}
See also:
Wiring-CHIP
I2C
Using Johnny-Five
Johnny-Five provides a model for compass interaction that includes the HMC5883L.
We construct a new instance of a compass with:
var compass = new five.Compass({
controller: "HMC5883L",
bus: 2
});
Once we have a compass, we can use two properties exposed from it which are "heading"
and "bearing". The heading is the current heading in degrees. The bearing is the compass
beating which contains the following properties:
point A cardinal direction such as "north", "south", "east", "west"
abbr An abbreviation of the point such as "N", "NE" and "NEbE"
low, mid, high
Two events are fired from the Compass object. These are "change" and "data". The
"change" event is fired when the value of the compass changes while "data" is fired as often
as possible.
Here is a sample that logs the current heading. If we turn the compass module, we will see
the changes take place in real-time:
"use strict"
var five = require('johnny-five');
var chipio = require('chip-io');
board.on('ready', function() {
let compass = new five.Compass({
controller: "HMC5883L",
Page 256
bus: 2
});
compass.on("change", function() {
console.log("Compass value changed: " + Math.round(compass.heading));
});
});
See also:
Johnny-Five Compass
Again by looking at the above image, consider an accelerometer. This will measure the force
of gravity and gives us a reference to the orientation of the compass sensor. When the
sensor is horizontal, the acceleration measured will be exclusively down the Z axis however
when the sensor is titled, the acceleration will be distributed across X, Y and Z and by using
math, we can determine the orientation of the board. As a side note, when the board is
horizontal, the compass bearing will be the yaw value.
We will define the following definitions:
Page 257
phi Roll (X)
theta Pitch (Y)
psi Yaw (Z)
See also:
Implementing a Tilt-Compensated eCompass using Accelerometer and Magnetometer Sensors
Wikipedia: Axes conventions
Once connected, we can run i2cdetect to list the devices on the I2C bus and we should
find the BPM180 at 0x77.
$ sudo i2cdetect -y 2
0 1 2 3 4 5 6 7 8 9 a b c d e f
00: -- -- -- -- -- -- -- -- -- -- -- -- --
Page 258
10: -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
20: -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
30: -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
40: -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
50: -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
60: -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
70: -- -- -- -- -- -- -- 77
See also:
Data sheet
Sparkfun BMP180 Barometric Pressure Sensor Hookup
BMP180 I2C Digital Barometric Pressure Sensor
Johnny-Five Barometer BMP180
Johnny-Five Thermometer BPM180
These devices have two small potentiometers on the boards. One is used to adjust the
sensitivity (Sx) and the other is used to adjust the delay time (Tx) before it resets. There is
also a jumper that controls whether or not the detection is "repeatable". By this we mean that
if there is a detection of presence, the signal goes low to high. Now, what happens if there is
STILL presence after the reset time? In non-repeatable mode, the signal goes low after the
reset time and will then go high again (possibly immediately) when presence detection is
made. With repeatable mode, the signal may NOT go low after the reset time IF there is still
presence detected. Only after a reset time AND the change to no further detection of
presence will the signal go low again. Some modules have jumper pins wired, others just
have pads which can be soldered.
Page 259
In addition to these slightly larger models, there is also the HC-SR505 mini PIR motion
sensor.
Their wiring could not be simpler. They have three pins labeled +5v, GND and Out. When
motion is detected within the sensor range of the device, the output pin transitions from low to
high and stays there until a configurable delay time says that is desirable to go low again.
There isn't much more to say about them than this. Because they have no input requirements
other than power, and the output can be voltage divided from 5V to 3.3V for CHIP, all we need
do is choose a GPIO pin for input and wire it up. We can then poll or trigger on a value
change on the pin and we will have learned that there is movement within the range of the
device.
See also:
Wikipedia Passive infrared sensor
HC-SR501 PIR Motion Detector datasheet
Ultrasonics HC-SR04
Imagine that we wish the CHIP to discover the distance to an object, how might we go about
achieving that? One way is through the use of an ultrasonic transmitter / receiver. These
devices are very cheap and also very easy to use. The principle of operation is very simple.
Even though when someone speaks to you, it feels like their voice reaches you instantly, this
isn't actually the case. Sound moves through the air at a relatively slow rate (relative to light
for example). The approximate speed of sound is 340 meters/second. Thus if you were 340
meters away, a loud sound would take 1 second to reach you. If you were 680 meters away,
the sound would take two seconds to reach you. You probably recognize that this is what
happens in a thunder storm. If you see a lightning flash, it can be many seconds before you
hear the thunder. This is because light travels very fast but sound travels much slower.
The HC-SR04 device emits a pulse of sound but at such a high frequency, our ears can't hear
it. When the sound is emitted it travels outwards away from the device. If there is an object
in its path, some of that sound is reflected back the way it came back towards the HC-
SR04. If the HC-SR04 receives an echo from a previous pulse, it indicates that on an output
pin.
The HC-SR04 has a high resolution timer within it that starts when the output is transmitted
and stops when an echo is received. The duration of the time between transmission and
echo response is output to us. We now have learned the duration of time between when a
sound was emitted and the echo received. Since we know the speed of sound and we know
Page 260
a time period, we now know how far the sound wave traveled. Since the distance it traveled
is the distance to a remote object and back again, if we halve the distance we have
learned the distance to that object.
Speed of sound is 340.29 m/s (340.29 * 39.3701 inches/sec). Call this V sound.
There are some important limitations in using an ultrasonic distance sensor. The distance is
calculated by reflecting a sound wave off a remote object and receiving the response. This
works great if the target object is perpendicular to the source of the sound. However, if the
target object is at an angle to the source then the results can be either off or the target may
simply not be detected. To see why this is, consider a wall at an angle greater than 45 to the
source. When the sound wave strikes the wall, rather than be reflected back to the source, it
will be reflected off somewhere else. If we remember back to a bouncing object from high-
school, the angle of incidence equals the angle of reflection. So a perfect response will be
found when the sound wave strikes the object at exactly 90. Anything else and we have
introduced either error or a missed response.
Page 261
The HC-SR04 is a 4 pin device
Vcc The input voltage is 5V.
Trig Pulse to trigger a transmission minimum of 10 microseconds.
Echo Pulses low to high to low when an echo is received. Warning, this is a 5V
output.
Gnd Ground.
Send a minimum of a 10 microseconds pulse to the Trig pin (low to high to low). Later, the
Echo will go low/high/low. The time that Echo is high is the time it takes the sonic pulse to
reach a back-end and bounce back.
Because the Echo response is a 5V signal, it is vital to reduce this to 3.3V for the CHIP. A
voltage divider will work.
See also:
Datasheet
YouTube: Arduino Tutorial: Ultrasonic Sensor HC SR04 distance meter with a Nokia 5110 LCD display
nRF24L01 networking
Note: This section is not complete and is likely to be a place holder until we have SPI.
The nRF24L01 is an ultra cheap communication device. It allows one to transmit and receive
data over large distances (hundreds of feet). It transmits and receives packets between like
devices. Do not confuse it with WiFi or Bluetooth. You can transmit from one nRF24 to
another nRF24 but not to anything else. There are a number of reasons to consider this
device for projects over others. First is the extremely low cost. An instance of the device can
be picked up for less than a dollar. Second is the peer-to-peer nature of the devices. There
does not need to be a master/slave relationship in the story. In fact, two or more devices can
exchange packets with each other without any of them acting as controllers. There are
occasions where this is exactly what is needed.
Page 262
The device operates on 3.3V and uses SPI as a protocol. The pin-out on the nRF24L01 is:
Pin Description
Vcc 3.3V
GND Ground
MISO SPI Master In/Slave Out.
MOSI SPI Master Out/Slave In.
SCK SPI System clock.
CSN SPI Chip Select Enable the device (active low).
CE RX or TX mode. The Data sheet calls this Chip Enable which seems an awfully strange name for a communication mode
selector.
IRQ Maskable interrupt.
See also:
Nordic Semi Home Page
Data sheet
Lamp Switch
Tutorial: Ultra Low Cost 2.4 GHz Wireless Transceiver with the FRDM Board
Julian Ilett #1
NRF24L01 - How To
Adding a nRF24L01 to a breadboard or stripboard
Tutorial 0: Everything You Need to Know about the nRF24L01 and MiRF-v2
npm nrf nRF24L01 library for Node.js and Raspberry Pi
Page 263
RF24 radio(CE_PIN, CSN_PIN);
radio.begin();
radio.setPALevel(RF24_PA_LOW);
radio.openWritingPipe(addressTX);
radio.openReadingPipe(1,addressRX);
// To be a receiver ...
radio.startListening();
Since there are only two modes, a receiver and a transmitter, when one stops being a
receiver, one immediately becomes a transmitter.
To determine if there is data available to receive, call:
radio.available()
The Arduino pins for ce and cs need not be fixed and hence we must instruct the library as to
which pins we actually used. Take care that when an instance is created you specify the pins
in the correct order as there is no guard against that.
Before any other functions can be called, we must call begin(). This perform the
initialization of the device plus sets defaults for many of the operational parameters.
Every device has an address associated with it. This allows a transmission from one source
device to be addressed to a specific destination device. When a transmission occurs, devices
other than the one with the matching address will ignore the transmission. The device
requires the size of the address to be supplied and fixed. The allowable address sizes are 3,
4 or 5 bytes. The default is 5 bytes. The size of the address can be changed with the
setAddressWidth() method however you are unlikely going to want to change the default
value. You must remember to supply the full data for the address. For example, don't set an
address of "123" when the address size is 5 bytes as your device address will become
"123<?><?>" and you will likely waste time trying to find out why no data is coming to you.
At this point, we must consider what we are going to do with the device from here on. The
device has two modes of operation. It can transmit data or it can receive data but it can't do
Page 264
both simultaneously.
Let us look first at being a transmitter. First we create a "pipe" over which the data will flow.
We supply the address of the destination.
openWritingPipe(const unit8_t *address)
For example:
unit8_t address = "1Node";
openWritingPipe(address);
This method will transmit the data pointed to by the data pointer for the number of bytes
supplied in length. There is a maximum size as specified by the getPayloadSize()
method and you should not exceed that. The method will block until the data has been
transmitted or the transmission request has timed out. The time out is short, about 70ms so
you won't block for long. Upon return from the method call, we can determine whether the
transmission was successful or if it failed. A return value of true means we succeeded while
false means as failure.
The device has a maximum transmission size that is 32 bytes. You can not transmit packets
larger than this. The default maximum transmission set in the library is 32 bytes which is the
maximum that the device will permit. You can change this with the setPayloadSize()
method but it is unlikely you will need that function. Simply remember that we can not ever
send a payload size greater than 32 bytes.
The other mode of the device is that of a receiver where we actually listen for incoming data.
As mentioned, the device can either be in transmit mode or receive mode.
To receive incoming data, we need to supply the address of the device from which we are
expecting incoming data to arrive. We do this using the openReadingPipe() method.
openReadingPipe(uint8_t number, const uint8_t *address)
Once we have specified the address of the transmitting device we will receive data from, we
can start actively listening using the startListening() method. Should we wish to end
listening mode, we can call the stopListening() method which will return us to
transmission mode.
Once we have entered listening mode, we can ask the library if there is data available for us
to read. The method for this is available(). If there is data available, the return value is true
and false otherwise.
To actually read the data that was received, we can use the method called read().
read(void *buf, uint8_t len)
This method supplies a buffer into which the received data will be stored. The length of the
buffer is also supplied.
A debug file called printDetails() is provided which logs the state of the nRF24 to the
Page 265
stdout. In order to use this one must:
Include <printf.h>
Call printf_begin()
When the printDetails() function is then called, a status of the nRF24 is written to the
Serial port. This can be a powerful tool for debugging problems. Here is an example of the
output:
STATUS = 0x0e RX_DR=0 TX_DS=0 MAX_RT=0 RX_P_NO=7 TX_FULL=0
RX_ADDR_P0-1 = 0x0045444f4e 0x0045444f4e
RX_ADDR_P2-5 = 0xc3 0xc4 0xc5 0xc6
TX_ADDR = 0x0045444f4e
RX_PW_P0-6 = 0x20 0x20 0x00 0x00 0x00 0x00
EN_AA = 0x3f
EN_RXADDR = 0x02
RF_CH = 0x4c
RF_SETUP = 0x03
CONFIG = 0x0f
DYNPD/FEATURE= 0x00 0x00
Data Rate = 1MBPS
Model = nRF24L01+
CRC Length = 16 bits
PA Power = PA_LOW
See also:
Github project
Project Blog
Forum thread
Graphic Equalizer
An IC called the MSGEQ7 is a graphic equalizer IC. It can operated at 3.3V or 5V.
If we provide it an audio input signal, it will analyze the signal into 7 discrete frequency bands
of 63Hz, 160Hz, 400Hz, 1KHz, 2.5KHz and 6.25KHz.
The IC is supplied in an 8 pin configuration with:
Page 266
Pin Function Description
1 VDDA 3.3-5V power supply
2 VSSA Negative
3 OUT Multiplexed DC Output. This is an analog output value.
4 STROBE Channel selection
5 IN Audio input
6 GND
7 RESET Reset. A high resets the multiplexor, low enables the strobe.
8 CKIN Clock
Since the output of this device is an analog signal and our Pi doesn't have an in-built analog
to digital converter, we will have to combine this IC with an ADC if we wish to consume the
generated values back into our Pi.
See also:
Datasheet
Microphone inputs
We can attach a microphone to a sound card but what if we don't have a sound card what
choices do we have? Microphones can be found very cheaply. The going rate is about $0.40
each on eBay. The most useful microphone style for us is the electret style.
FM Radio
The IC called TEA5767 provides FM frequency radio receiver which is digitally controlled via
the I2C bus. The address of the device is 0x60. Modules for this IC can be found on eBay
for about $4. The device can be powered from either 3.3V or 5V. The reception frequency is
from 76MHz to 108MHz.
The pins on it are:
Page 267
Pin Description
1 SDA
2 SCL
3 GND should be grounded
4 NC
5 VCC 3.3V 5V
6 GND should be grounded
7 Left Out
8 Right Out
9 GND
10 Antenna
Page 268
7 SWP2 Default: 0
6 STBY Default: 0
5 BL Band limits:
1 Japanese
0 US/Europe
Default: 0
4 XTAL Default: 1
3 SMUTE Soft mute:
1 Soft mute is on
0 Soft mute is off
Default: 0
2 HCC Default: 0
1 SNC Stereo noise canceling:
1 Stereo noise canceling is on
0 Stereo noise canceling is off
Default: 0
0 SI Default: 0
Byte 5
7 PLLREF Default: 0
6 DTC Default: 0
5:0 N/A Default: 0
See also:
Datasheet
Arduino TEA5767 Radio
Github: andykarpov/TEA5767
Although a seven segment display looks "quite dated" they should not be discounted. If what
one wants to do is show a numeric value that can be read from a distance and which is very
cheap to use, then this device may be just perfect.
Page 269
It is common to see 7-segment displays used in conjunction with the MAX7219 or MAX7221
ICs. These ICs know how to drive up to eight multiplexed 7-segment displays with data
received over an SPI bus. As such, CHIP can send in an SPI signal and these devices can
easily display the results. Unfortunately at the time of writing, CHIP does not yet support SPI.
See also:
Wikipedia Seven-segment display
These are available in both module and discrete component form. The following image
shows a 4 channel version available for under $2. This device is based upon the TLP281 IC.
The switch time for this is less than 5s.
Page 270
The pin-out of the module is:
OUT1 NC
OUT2 IN1
OUT3 IN2
OUT4 IN3
HVCC IN4
HGND GND
The notion is that GND is wired to the Pi GND. IN1, IN2, IN3 and IN4 can be wired to GPIO
output pins of CHIP. This forms the Pi side of the circuit. On the other side of the circuit, we
provide a separate source at HVCC and HGND. Note that HVCC is not to be greater than 24V.
A high on IN1 produces a high on OUT1 and a low on IN1 produces a low on OUT1
effectively OUTx = INx but physically decoupled from each other and hence no noise
leakage between them.
Others are the PC123. The switch time on this device is less than 20s.
Note that the internal LED is driven at between 1.2 and 1.4 volts. This means that it is not
tolerant of either 3.3V or 5V so take care to use a voltage divider to properly control the input.
Page 271
Input R1 (where R2 = 10K)
3.3V (3.3-1.3) x 10000/1.3 = 15K
5V (5-1.3) * 10000/1.3 = 28.4K
The PC123 has a "dot" on its top left surface to mark orientation. The pin out thus aligned is:
LED Anode Transistor Collector
LED Cathode Transistor Emitter
For driving high current load devices, consider using a sufficiently powerful transistor and
having the output of the opto isolator drive the switching of the transistor which is connected
to the final load.
See also:
TLP281 home page
TLP281-4 Data Sheet
Sharp PC123 Data Sheet
Logic Level Shifting
There is a common pin from which data can be either read or written. That pin will be
connected internally to one of the channel pins (0 through 7). The selection of which pin is
governed by the digital signals on A, B and C. The 3 bits of information there select the
Page 272
corresponding channel. Finally there is an inhibit pin which if set will prevent any connection
between the input and the output. The connections between the common pin and the
channel pin is fully analog it can be used to provide a digital signal or an analog signal
in either direction.
With this background, how then might we use such an IC? Imagine we only have one Analog
to digital convertor input on our Pi but we want to take readings from multiple sensors. We
could switch between each of the sensors taking a reading before moving onto the next.
The CD4051 is very cheap, under $0.40 an instance as found on eBay.
Its physical pin layout is:
Pin Description
1 Channel #4
2 Channel #6
3 Common In or Out
4 Channel #7
5 Channel #5
6 Inhibit
7 Vee (Ground)
8 Vss (Ground)
9 Input C
10 Input B
11 Input A
12 Channel #3
13 Channel #0
14 Channel #1
15 Channel #2
16 Vdd (+ve)
See also:
Texas Instruments CD4051 data sheet
YouTube: 4000 Series Logic ICs: The 4051 8-channel analog multiplexer/demultiplexer IC
Page 273
My experience with plugging the device directly into CHIP has been very poor. When I plug it
directly into the USB port, the whole CHIP shuts down. I believe it is because the device is
drawing too much current. When I plugged the receiver into an external USB hub and
plugged the hub into CHIP, all started working as advertized.
ADS-B Decoding
Airplanes transmit transponder signals called ADS-B (Automatic Dependent Surveillance
Broadcast). These can be received and decode with a Software Defined Radio (SDR). The
broadcast frequency is 1090MHz.
After plugging in the DVB-T dongle, we find that it will show up as a USB device with vendor
code 0bda and device code 2838. For example:
$ sudo lsusb
Bus 001 Device 005: ID 0bda:2838 Realtek Semiconductor Corp. RTL2838 DVB-T
Page 274
The drivers for this are part of the package called "rtl-sdr" and should be installed with:
$ sudo apt-get install rtl-sdr
If you are compiling code, you will also need "librtlsdr-dev". This can be installed with:
$ sudo apt-get install librtlsdr-dev
With the environment ready, we now need some software to decode the data. An excellent
application called dump1090 (1090 because of 1090MHz) is available.
Next we download the GIT source for dump1090 and compile it. This will also require the
"libusb-1.0-0-dev" package:
$ sudo apt-get install libusb-1.0-0-dev
If we run the program with the --net flag, then we can point a browser to it at:
http://<hostname>:8080
and we will be presented with a Google map showing the data.
For programatic access, we also have some options. The dump1090 program listens on port
30003 for parsed data. A Node.js module is available that can parse this data (see Github:
wiseman/node-sbs1) it is available as the NPM package called "sbs1".
The format of the data can be seen here:
http://woodair.net/SBS/Article/Barebones42_Socket_Data.htm
One particularly useful mechanism is to use the Node-RED TCP input node to target the
Page 275
output of dump1090. We can ask for streams of strings terminated by a newline. This will
generate one record per event received over ADS-B. The configuration of such a node will
be:
We can then inject that into a CSV node to parse the comma separated data:
The first column is "col1" which will always be the constant "MSG". The next column, "col2"
is the message type (numeric) and this becomes important.
Page 276
An example flow might be:
To make this activity easier, a custom ADS-B Node-RED node has been developed that takes
as input an SBS message and returns a populated JavaScript object. The node can be found
here:
http://flows.nodered.org/node/node-red-contrib-ads-b
If you get the flight number of an aircraft and are curious to know where it is going to or from,
you can look it up at flightradar24 here:
https://www.flightradar24.com/data/flights
See also:
RTL-SDR TUTORIAL: CHEAP ADS-B AIRCRAFT RADAR
dump1090 message data format descriptions
https://github.com/MalcolmRobb/dump1090
https://github.com/antirez/dump1090
If we use Bluetoothctl and scan for new devices with an iTag in range, we will find it. If we
pair with it and then ask for the info of the device, we will find:
Page 277
[bluetooth]# info FF:FF:45:19:14:80
Device FF:FF:45:19:14:80
Alias: FF-FF-45-19-14-80
Appearance: 0x03c1
Icon: input-keyboard
Paired: yes
Trusted: no
Blocked: no
Connected: yes
LegacyPairing: no
UUID: Generic Access Profile (00001800-0000-1000-8000-00805f9b34fb)
UUID: Immediate Alert (00001802-0000-1000-8000-00805f9b34fb)
UUID: Battery Service (0000180f-0000-1000-8000-00805f9b34fb)
UUID: Unknown (0000ffe0-0000-1000-8000-00805f9b34fb)
Now if we disconnect from the device, the device will start beeping. If we reconnect or press
the button on the device, it will stop beeping.
The services exposed by the device include:
Generic Access 0x1800
Immediate Alert 0x1802
Battery Service 0x180f
Now let us turn our attention on how to control this device from CHIP. First, we need to find
the address Bluetooth Low Energy (BLE) address of our iTag. Switch on the iTag (it will beep
once). Now from the terminal, enter:
$ sudo hcitool lescan
LE Scan ...
FF:FF:45:19:14:80 (unknown)
Copy this address into a notepad or other buffer, we will need it later.
Now we can start to perform work against the device. Start up "gatttool".
$ sudo gatttool --interactive
[ ][LE]>
If we now issue a disconnect, we will find out device starts to beep. We can stop the iTag
beeping by either pressing the button on its top or else re-connecting. And we have now
performed the first of our tests. By connecting CHIP to the device and then disconnecting, we
caused the device to beep.
The next thing we can do is to cause the device to beep by sending it a command.
Understanding this recipe assumes that you have studied BLE specifications and it is a bit
deep, however I'll describe the high levels as we go along.
With gatttool running and connected, issue the command called "primary":
[FF:FF:45:19:14:80][LE]> primary
Page 278
attr handle: 0x0001, end grp handle: 0x0005 uuid: 00001800-0000-1000-8000-00805f9b34fb
attr handle: 0x0006, end grp handle: 0x0008 uuid: 0000180f-0000-1000-8000-00805f9b34fb
attr handle: 0x0009, end grp handle: 0x000b uuid: 00001802-0000-1000-8000-00805f9b34fb
attr handle: 0x000c, end grp handle: 0x000e uuid: 0000ffe0-0000-1000-8000-00805f9b34fb
Notice the "UUIDs" on the right. The UUID that starts with "1802" is the specification for an
"Immediate Alert" service. Notice also that it has "handles" in the range of 0x9 to 0xb. Next
run the command to examine the handles:
[FF:FF:45:19:14:80][LE]> char-desc 0x9 0xb
handle: 0x0009, uuid: 00002800-0000-1000-8000-00805f9b34fb
handle: 0x000a, uuid: 00002803-0000-1000-8000-00805f9b34fb
handle: 0x000b, uuid: 00002a06-0000-1000-8000-00805f9b34fb
From here we see that the handle at 0xb corresponds to the characteristic of the service
known as 2a06 which is the command to send an alert. By experimentation, I have found that
sending "10" causes the device to start beeping and sending "00" cause it to stop beeping:
[FF:FF:45:19:14:80][LE]> char-write-cmd 0xb 10
[FF:FF:45:19:14:80][LE]> char-write-cmd 0xb 00
Finally, when the device is silent, if we click the button on the top of the iTag, notice that we
receive an indication/event/notification that something happened:
Notification handle = 0x000e value: 01
Page 279
These can be picked up on eBay for about $1. The RTC uses battery backup when not
powered. This is a standard CR2032 battery type. When inserting the battery, it is +ve down.
When soldering on header pins, think through how you will mount the board when complete.
It may be that you want to solder the header pins such that you can easily access and replace
the battery once attached to your PCB or strip board.
This device provides a real time clock capability that includes hours, minutes, seconds,
month, day of month, day of week and year. Month calculations including leap year support
are also accommodated. Interestingly, it also provides 56 bytes of non-volatile RAM for user
storage. The device uses I2C for communication.
Note: Before plugging your board to the CHIP, realize that this board is a 5V module. If you connect 5V to the CHIP's GPIO pins bad
things can happen because they are 3.3V pins. The I2C protocol uses pull-up resistors to bring the SDA and SCL lines high by default.
If we were to look at the schematic diagram of this module we would find that resistors R2 and R3 are pull-up resistors. Unfortunately,
we have two problems. First is that the CHIP already pulls up the pins and secondly the module pulls them high to a 5V line which is
too much for the CHIP. The solution is to physically remove these two resistors from your module before use. The resistors are clearly
marked on the board as R2 and R3 and carry the mark 332 (3.3k).
The boards have 5 pins on one side 7 on the other. The 5 side is duplicated on the 7 side.
Pin Label Description
1 DS DS18B20 Temp.
2 SCL I2C clock.
3 SDA I2C data line.
4 Vcc Vcc (5V).
5 GND Ground.
Page 280
Here is an example schematic for wiring:
Once plugged in, we will see the RTC at I2C address 0x68:
$ sudo i2cdetect -y 2
0 1 2 3 4 5 6 7 8 9 a b c d e f
00: -- -- -- -- -- -- -- -- -- -- -- -- --
10: -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
20: -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
30: -- -- -- -- -- -- -- -- 38 -- -- -- -- -- -- --
40: -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
50: 50 -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
60: -- -- -- -- -- -- -- -- 68 -- -- -- -- -- -- --
70: -- -- -- -- -- -- -- --
Once connected, it is time to start looking at the software side of the house. CHIP provides a
kernel level module to integrate with the clock that is enabled by default.
Next we must tell the I2C subsystem that a RTC clock is present and tell it what I2C address
it is present on. We do that by echoing "ds1307 0x68" into the file /sys/class/i2c-
adapter/i2c-2/new_device. For example:
$ sudo bash
# echo "ds1307 0x68" > /sys/class/i2c-adapter/i2c-2/new_device
Page 281
Following this we will find some new device driver files up in /dev. Namely we will find
/dev/rtc and /dev/rtc0.
We can now ask the hwclock what time it thinks it is with:
$ sudo hwclock --show
Tue 05 Jan 2016 07:24:22 PM CST -0.156382 seconds
If the clock has not been set, then it will be some odd time back in the past. If so, we can set
the hardware clock from the current date/time with:
$ sudo hwclock --systohc --debug --noadjfile -utc
hwclock from util-linux 2.25.2
Using the /dev interface to the clock.
Assuming hardware clock is kept in UTC time.
1470504124.500000 is close enough to 1470504124.500000 (0.000000 < 0.001000)
Set RTC to 1470504124 (1470504124 + 0; refsystime = 1470504124.000000)
Setting Hardware Clock to 17:22:04 = 1470504124 seconds since 1969
ioctl(RTC_SET_TIME) was successful.
and if we re-run hwclock --show we will see the proper time. At this point, the hwclock is
set and keeping real-time. When next the CHIP reboots we need to instruct it to read the time
from the hwclock which will have kept it for us and sync the system time from that value.
This means that CHIP has to re-associate the hwclock with I2C and then execute:
hwclock --hctosys
Page 282
This will take a few minutes to complete. The result will be a new directory called "arduino-
1.6.11". Change into that directory and we are now ready to launch the IDE. Run the
command:
$ ./arduino
after a minute, the IDE will launch and you are now squarely in the world of building Arduino
based applications.
Electronics
One of the key stories of the CHIP is that we can use it to interact with the real-world through
a variety of electronics devices connected to its headers. Thankfully, we don't have to be
rocket scientists or electrical engineers in order to get this going. There are many free
sources of knowledge on these topics on the Internet including step by step tutorials.
However, there are certain items that one would not normally hear about when working in the
CHIP that are related to electronics that I feel are of real value for you to understand.
See also:
Learning about Electronics
Page 283
Attaching your Chip to circuit boards
CHIP has two headers called U13 and U14 both of which are 40 pin female headers. Now
imagine that you want to build a circuit and attach your CHIP to that circuit. How then do you
achieve that task? One way is to obtain some male/male 40 pin headers. There are
commonly known as "IDE drive gender changer adapters" because of their common usage
pattern.
If we solder a set into our PCB or perf-board then the other end will fit into the CHIP. Be very
careful with polarity. If you set the CHIP backwards into the header, you are very likely to
damage something. Always mark your circuit board with which end is which and check,
check and double check your polarity before powering up.
An alternative to these male to mail adapters are the following "long legged" headers:
These will expose an extension of the female headers for additional plugin.
Page 284
http://fritzing.org/home/
I recommend the use of Fritzing for all but the most trivial of projects. By using Fritzing to
visualize both a schematic and physical layout when it comes time to assemble a project, you
will have a blue print to work from. It is tempting to build a project going straight to the
breadboard but resist that. Invariably, you will spend more time debugging than you would
have spent drawing and then wiring. In addition, you have an artifact in the form of the
Fritzing images that can be referred to again and again over time. Ideally you will share these
images on the Internet for others to use. If nothing else, when you come back to this project a
month later you will have somewhere better to start with than any written notes you may or
may not have taken.
One of the true benefits of this package is that we are not constrained by the availability of
parts.
This takes a little time and practice and one should be handy with the open source SVG
graphics editor called Inkscape.
See also:
Fritzing home page
Inkscape
Page 285
See also:
Scheme-it
schematics.com
Page 286
elegant and performs all the activities that one could sensibly want.
If one searches on eBay, one can also find very cheap versions of analyzers at around the
$10 price point.
Here is an example screen shot captured using the excellent Saleae Logic tool:
Not only does this tool clearly illustrate the wave forms, but it is also able to decode some of
the well known protocols such as I2C and SPI.
See also:
Saleae
Wikipedia Logic analyzer
Page 287
Physically, the ICs that we will be working with come as small, black devices with pins.
Commonly 8 or 16pin but sometimes more and sometimes less. They should be bread-board
friendly meaning that you can push them into a bread-board for testing. The biggest mystery
over the devices is how to connect them properly and this is where the data-sheet of the
device becomes your greatest friend. Each manufacturer of an IC produces a document
called a data-sheet that describes in a lot of detail how to use it. For hobbyists, that can
seem like too much detail. There are charts and graphs showing all kinds of information
about its characteristics. However, from our perspective, what we really are looking for is the
pin configuration. Once we learn which pin performs which function, we are well on our way
to success.
When buying ICs, make sure you understand the difference between DIP (Dual in-line
package) which are ICs with the pins in parallel rows that can be plugged into bread-boards.
These are also called "through the hole" components. The primary alternative format is called
surface-mount. These can not be directly used with bread-boards and for hobbyist and ease
of tinkering considerations, should be discounted in favor of DIP.
When it comes to assembling circuits and soldering them to a PCB or strip board, if the chip is
expensive or you feel you don't want to commit it to just one project, you can solder in an IC
socket into which the IC can be inserted and subsequently removed.
There are sockets for all the different DIP IC dimensions and they can also be found cheaply.
See also:
Wikipedia: Dual in-line package
Wikipedia: List of integrated circuit package dimensions
Page 288
level and provides bi-directional switching.
If you don't have access to such logic level circuits, an alternative solution is to create a
voltage divider. This will protect 3V CHIP inputs from over voltage from a 5V source. This is a
simple circuit made with two resistors. A higher voltage is provided to input than output and
using a correct combination of resistors, the input voltage is proportionally scaled to the
output voltage.
Page 289
Transistors as switches
If we consider that a GPIO pin can produce a signal of high or low we see that the signal can
be used to perform work. However there is a maximum current that can be drawn from any
given I/O pin. If, for example, we wished to attach a device to a pin which would draw more
current than the pin is rated to supply, we may very easily damage our CHIP. This is where a
transistor can come into play. A transistor can be thought of as an electronic switch. A
transistor has three pins labeled emitter, base and collector. If a small current flows between
base and emitter a correspondingly higher current will be allowed to flow between collector
and emitter. There are two types of transistor we will come across called NPN and PNP. The
difference between them is whether or not the "switch" is triggered by a high signal or a low
signal. The following is the schematic symbol for an NPN transistor. Normally we connect
the emitter to ground and a load between the collector and +ve. If we then connect the base
to the output of an I/O pin (through a resistor(, then when the pin goes high, a small current
will flow between B and E and a much higher current will flow between C and E.
For the other type of transistor (PNP) whose symbol is shown in the following diagram, the
emitter is usually connected to +ve and the collector to the load and then ground. With the
base pin connected to a GPIO (through a resistor), when the base goes low, the small current
flowing from the emitter through the base will be greatly amplified as the current flowing
through the emitter to the collector.
By using a transistor, we are no longer directly drawing load through the GPIO pin but instead
simply using the signal present on the pin to energize the transistor.
Common transistors that can easily be obtained include:
Page 290
Name Type Style
2N3904 (Signal transistor) 200mA max NPN TO-92
current
2N3906 (Signal transistor) PNP TO-92
PN2222 NPN TO-92
TIP120 5A max current NPN TO-220
TIP125 5A max current PNP TO-220
BC547 100mA max current NPN TO-92
BC557 100mA max current PNP TO-92
2N3055 15A max current NPN TO-3
TO-3
See also:
2N3904 Data Sheet
2N3906 Data Sheet
PN2222 Data Sheet
TIP120 Data Sheet
TIP125 Data Sheet
BC547 Data Sheet
BC557 Data Sheet
You Tube: Electronic Basics #22: Transistor (BJT) as a switch
Power inputs
Powering devices is important and there are many form factors for supplying input. A common
one is the 2.1 mm barrel type that can mate with input power plugs. These typically output 5V
or higher which are, of course, fatal to a CHIP.
Page 291
An excellent power source for use when working with breadboards is the very cheap MB102.
These can be found for under a dollar on eBay. They take input from either USB or barrel
plug power sources and deliver either 3V or 5V which is configurable through jumpers. The
MB102 is extremely breadboard friendly. The barrel input should be between 6.5V and 12V.
The maximum output current is 700mA.
If we have a voltage input higher than we can use we need to use a voltage regulator device
to bring it down to the desired value. An excellent example of this is the LM317.
Seen from the front, the pins from left to right are Adjust, Output and Input. The LM317 has an
input up to 40V with a regulated output between 1.25V and 7V with a maximum current load
of 2.2A. The LM317 produces a constant voltage output as long as the input voltage is
greater than the desired output voltage.
Page 292
For example, if we want an output of 5V then we have 5V = 1.25V(1+R2/240). This gives R2
as 720.
An alternate solution for providing power to CHIP is to use a "buck step down converter".
These can take inputs of from 6V to 28V and step down to 5V. There is an IC called the
MP1584 for which there are modules available at about the $3 price range.
See also:
The power system
Breadboard power supply MB102 review
LM317 Data Sheet
Learning about Electronics LM317 Voltage Regulator
MP1584 Datasheet
When these devices arrive, there is no documentation with them which is a shame.
Connected to the meter are two sets of wires. One small set with two connections which are
red and black. These are the power input to drive the meter. This is a source from 4.5V
Page 293
30V. The larger set of leads has three taps. These are red, black and blue. The wiring of
these should be as follows:
red Voltage sense. Connect this to where you wish to measure the voltage between
that wiring point and ground.
black Ground. Always connect this to ground.
blue Current sense. Connect this to what would otherwise be ground on your circuit
and pass the current through this wire.
Web programming
There is a school of thought that says the browser will become the universal desktop for all
applications. Instead of applications having to drive the screen locally using a variety of
different UI technologies such as X-Windows, JavaFX, Swing, Windows UI and all the many,
many others, applications will instead serve up HTML that runs within the browser. Any data
need by the UI will be requested by the browser from the app using REST. Any actions that
need to be performed such as data base updates will again be performed by the App on
request of the user through the browser. Not only will the browser become the UI but it opens
up the opportunity for remote interaction with the application where today a thick UI can only
be used locally.
Another important consideration is the variety of devices that can host a browser. Not only
does this include PCs but of course includes phones and tablets.
Browser security
To allow geolocation from files loaded from the file system, we need to start Chrome with the
extra flags "--allow-file-access-from-files".
$ google-chrome --allow-file-access-from-files
To allow yourself to make cross site calls unsecured, start with "--disable-web-
security".
$ google-chrome -disable-web-security
Page 294
REST requests
In the field of computers, the moment that there was more than one in the world, we had the
notion that they could collaborate to get work done. As networked devices have become the
norm, various protocols and techniques have been designed to allow applications to work with
each other. These distributed computing technologies have been widely varied and include
such members as Sockets, DCE RPC, Web Services (SOAP/HTTP), JMS and more. Here
we look at a specific technique called "REST". REST is the usage of the HTTP protocol to
transmit information from a caller to be processed by a server. As such, it is client/server
oriented. The client formulates a REST request by building and sending an HTTP protocol
request over a TCP/IP network. This request is commonly received by a server application
that is listening at a well known port. On receipt of the request, the server analyzes the
incoming data and performs some corresponding task using the data supplied. If needed, the
server can then send back a response message to the original caller.
REST calls are stateless. When a call is made from the client to the server and the call
completes, there is no context maintained about that previous call.
JSON processing in C
When working with REST, we will invariably also be working with JSON data. A C library is
available on github called json-c. The package can be installed on a CHI{ using "libjson-
c-dev".
$ sudo apt-get libjson-c-dev
To use this library, we must include <json-c/json.h> and link with "-l json-c".
We create a new instance of a struct json_object using
json_object_new_object(). From there, we can start populating it. We have primitives
for creating the core types:
Type Function
string json_object_new_string
integer json_object_new_int
boolean json_object_new_boolean
floating json_object_new_double
object json_object_new_object
array json_object_new_array
Page 295
json_object_array_add(existingArrayObject, child);
The options parameter is a structure which contains the details of the request to be made.
The callback parameter is a JavaScript function that will be invoked when a response is
returned.
Page 296
The full details of the function can be found in the documentation so need to repeat them
here. Let us look at a sample scenario. Imagine there is a web service at
https://example.com that when passed a request to /sales with an accessKey query
parameter returns a sales record in JSON. For example:
https://example.com/sales?accessKey=1234
See also:
Node.js http.request() API
Page 297
var http = require('http');
const PORT=8080;
server.listen(PORT, function(){
console.log("Server listening on: http://localhost:%s", PORT);
});
For an HTTP GET request, it may carry query parameters of the form:
<name>=<value>&<name>=<value>...
We can parse these out using the Node.js class called "url". For example:
var params = require('url').parse(request.url, true);
In addition, from the parsed URL, we can also find the pathname property which, in REST, is
equivalent to the name of the service being requested.
See also:
Build your first HTTP server in Node.js
Node.js HTTP class
Node.js URL class
WebSocket
WebSocket is both an API and a protocol introduced in HTML5. Simply put, if we imagine an
HTTP server sitting waiting for incoming HTTP requests, we can convert a current request
into a socket connection between the server and the browser such that either end can send
data to be received by its partner.
Here we see a raw request to upgrade an HTTP connection to a WebSocket connection:
GET / HTTP/1.1
Host: 192.168.1.10
Connection: Upgrade
Pragma: no-cache
Cache-Control: no-cache
Upgrade: websocket
Origin: file://
Sec-WebSocket-Version: 13
User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko)
Chrome/46.0.2490.86 Safari/537.36
Accept-Encoding: gzip, deflate, sdch
Accept-Language: en-US,en;q=0.8
Sec-WebSocket-Key: saim6TzFH+zVb4qY2nrh0Q==
Sec-WebSocket-Extensions: permessage-deflate; client_max_window_bits
Page 298
See also:
html5rocks Introducing WebSockets: Bringing Sockets to the Web
The WebSocket protocol RFC6455
The WebSocket API
websocket.org
Wikipedia WebSocket
Note the URL prefix value of "ws" to indicate that a WebSocket is being requested. If one
needs additional security, the prefix can be changed to "wss" to specify that we wish to use
SSL encryption.
The WebSocket API is mostly event driven and there are a number event types of interest to
us:
open Invoked when the connection to the WebSocket server has been established
and we are now ready to send or receive data. We can define this with the "onopen"
property of the WebSocket as a function reference.
message Receive a message from the server. We can define this with the
"onmessage" property of the WebSocket as a function reference. The callback
function is invoked asynchronously when a new message is sent by the browser. The
event message passed as a parameter to the callback contains:
data The data received over the connection.
For example:
ws.onmessage = function(messageEvent) {
console.log("Data: " + messageEvent.data);
});
error Receive an indication that an error was detected. We can define this with the
"onerror" property of the WebSocket as a function reference.
close Receive an indication that a request to close the connection was detected.
Page 299
We can define this with the "onclose" property of the WebSocket as a function
reference.
Event handlers can be registered either with an "on<event>" mechanism or with an
addEventListener() call. For example, to register to receive a callback when the
connection to the socket server has been completed, we would code:
ws.onopen = function() {
// Do something ...
};
Take care to note the lower case on the event name. The property is "onopen" and not
"onOpen".
There are two methods defined on a Web Socket object. Those are:
send Send data to a Web Socket server. When called, the browser application will
receive a new message over the socket. The signature of the function is:
send(data)
Typically what we send is a String. If we wish to send an object, we would convert it to JSON
first.
close Close the connection to a Web Socket server. The close() method takes
two parameters:
close code An integer close code describing the reason for the close.
1000 CLOSE_NORMAL
1001 CLOSE_GOING_AWAY
status message A string describing the close reason.
Finally, there are a few attributes:
readyState The state of the WebSocket connection. Values include:
WebSocket.CONNECTING
WebSocket.OPEN The connection to the server is open.
WebSocket.CLOSING
WebSocket.CLOSED
bufferedAmount Amount of data that is buffered pending transmission to the
WebSocket server.
protocol The WebSocket server selected protocol being used.
Page 300
will be a server side (CHIP) application that owns the server side half of the socket
connection. That application is one that you will be writing. As such, the nature of that
application should be under your control and, of course, the core nature of the application is
the language in which it is written. As we have seen, we have a rich assortment of languages
available to us and now we need to look at the different technologies we could use to
leverage a WebSocket from that language.
To write an app, we have to include the header file <libwebsockets.h>. We must also link
with libwebsockets.
The high level nature of writing a libwebsocket app is as follows:
Setup the environment
Build a context
Loop around a call to libwebsocket_service() to service requests
The creation of a context is through the use of libwebsocket_create_context(). The
return from this is a struct libwebsocket_context pointer. If we end up no longer
needing the context, we should release it with a call to libwebsocket_context_destory(). The
input to libwebsocket_create_context is a pointer to a struct
lws_context_creation_info. This structure has to be populated before it can be used.
Specifically:
port the port number on which to listen
protocols an array of struct libwebsocket_protocols.
gid, uid Security info, should likely be -1 for our purposes.
An example initialization might be:
struct lws_context_creation_info info;
memset(&info, 0, sizeof(info));
info.port = <port number>;
info.gid = -1;
info.uid = -1;
info.protocols = protocols;
The protocols array defines the protocols that the server will handle. The format of this struct
is:
name The name of the protocol
Page 301
callback A callback function
per_session_data_size
rx_buffer_size
owning_server
protocol_index
The array must be terminated by a NULL entry. The first entry in the array must be an HTTP
protocol handler. A simple way to initialize the protocols is:
struct libwebsocket_protocols protocols[]= {
{
"http-only", httpCallback, 0
},
{
"dumb-increment-protocol", dumbIncrementCallback, 0
},
{
NULL, NULL, 0
}
};
The majority of the work is performed in the callback handlers. The library provides some
assistance with some of the more fundamental tasks.
libwebsockets_serve_http_file Send the content of a named file down the
socket to the partner.
When a callback function is invoked, we are supplied a reason code. There are many
possible reasons and what we do when we receive a callback is key. Some of the more
common reasons are:
LWS_CALLBACK_ESTABLISHED Handshake with the partner has now completed.
LWS_CALLBACK_RECEIVE Incoming data has arrived. The data is in the in buffer
with the supplied length.
LWS_CALLBACK_PROTOCOL_INIT Called once to allow initialization to be performed
(if needed).
To actually write data down a web socket, we employ the libwebsocket_write() function.
Page 302
The signature of the function is:
int libwebsocket_write(
struct libwebsocket *wsi,
unsigned char *buf,
size_t len,
enum libwebsocket_write_protocol protocol);
This is an overloaded function that can send data through HTTP or through a WebSocket.
When sending through a WebSocket, the protocol type should be LWS_WRITE_BINARY or
LWS_WRITE_TEXT. The length is the number of bytes to send. Take care with the buffer
though. When writing through a web socket, the buffer must be larger than expected.
Specifically, it must have LWS_SEND_BUFFER_PRE_PADDING bytes in front of it and
LWS_SEND_BUFFER_POST_PADDING after it.
Here is a complete sample server application:
#include <string.h>
#include <stdio.h>
#include <stdlib.h>
#include <libwebsockets.h>
static int sendWebSocketData(struct libwebsocket *wsi, unsigned char *buf, size_t len);
Page 303
switch(reason) {
case LWS_CALLBACK_PROTOCOL_INIT:
printf("LWS_CALLBACK_PROTOCOL_INIT\n");
break;
}
return 0;
}
static int sendWebSocketData(struct libwebsocket *wsi, unsigned char *buf, size_t len) {
unsigned char *tmpBuf = malloc(LWS_SEND_BUFFER_PRE_PADDING + len + LWS_SEND_BUFFER_POST_PADDING);
memcpy(tmpBuf+LWS_SEND_BUFFER_PRE_PADDING, buf, len);
int rc = libwebsocket_write(wsi, tmpBuf+LWS_SEND_BUFFER_PRE_PADDING, len, LWS_WRITE_TEXT);
free(tmpBuf);
return rc;
}
void initWebSockets() {
struct libwebsocket_protocols protocols[3];
memset(protocols, 0, sizeof(protocols));
protocols[0].name = "http-only";
protocols[0].callback = httpCallback;
protocols[1].name = "dumb-increment-protocol";
protocols[1].callback = dumbIncrementCallback;
while(1) {
libwebsocket_service(context, 50);
}
}
See also
libwebsockets.org
Libwebsockets Introduction
Test drive libwebsockets library a simple server
Once a context has been created, we can then register ourselves as a server:
Page 304
noPollCon *listener = nopoll_listener_new(ctx, "0.0.0.0", "1234");
nopoll_ctx_set_on_msg(ctx, listener_on_message_handler, NULL);
nopoll_loop_wait(ctx, 0);
See also:
noPoll home page
noPoll core library manual
noPoll modules
This will register a connection handler that will be invoked when a client connects to the
server. The ws parameter passed in on the connection event is an instance of a Web Socket
connection.
Upon this we can receive messages with:
ws.on('message, function(message) {
// Code here ...
});
Later on we will also see that the popular WebServer package called "express" is also able to
serve as a WebSocket provider.
See also:
socket.io
Page 305
npm ws
Express WebSocket support
Express has a number of concepts associated with it. Key amongst these are the notions of:
middleware A set of handlers that get executed in a chain when a request arrives.
The handler can read and write both the request and response. Middleware is set up
using the app.use() API.
routing Functions executed conditional on the path of the original request. Routing
is set up using the app.get(), app.post() (etc) API.
See also:
Node.js Express
Build your first HTTP Server in Node.js
Express middleware
Express provides some pre-built middleware components that are especially useful:
express.static Serve static web pages.
Express routing
We add a route using the API of the form
app.get("/<path>", function(request, response) {});
Page 306
function will be executed if the path starts with "/hello" and the remainder of the path will be
available as a property called request.params.planet. For example, browser requests
to:
http://<host>/hello/world
http://<host>/hello/Earth
will both resolve to the same function with different parameters values for
request.params.planet.
The request and response parameters are rich with capabilities. From these we can set and
get input and output values. Here are some examples:
request.query an object that has properties for each of the query parameters
response.end() A function to complete the response for the request
response.status(<code>) Set the status code for the response
See also:
Express request
Express response
Now the environment has additional Web Socket support. To receive a message we can use:
app.ws("/<path>", function(ws, req) {
ws.on('message', function(msg) {
// Handle the message here
}
});
Page 307
Express as an https server
To run express as https server, we need credentials. These can be created with:
openssl req -new -x509 -newkey rsa:2048 -keyout key.pem -out cert.pem -days 365 -subj
"/CN=kolban" -nodes
Using jQuery
The simplest way to use jQuery is to include it within the <head> of your page. For example:
<script src="https://ajax.googleapis.com/ajax/libs/jquery/2.1.4/jquery.min.js"></script>
You can bring it in from a variety of CDN locations or serve a copy of it up from your local
Webserver. If we load "jquery" through "bower", we should use:
<script src="./bower_components/jquery/dist/jquery.js"></script>
Page 308
jqXHR jqXHR
error a JavaScript function that is called when an error return occurs. The
funcation has the signature of:
jqXHR
textStatus
errorThrown
context The JavaScript object (commonly passed as this) that will be the context
used for the callback functions.
method The type of HTTP command. Commonly either "GET" or "POST".
username A username to be passed.
password A password to be passed.
See also:
jQuery.ajax()
Using Bower
Bower is a package manager for web based applications. It has a pre-req of Node.js and can
be installed through npm.
Bower can be installed using:
$ sudo npm install bower -g
Once installed, we can ask bower to install packages for us. For example, to install jquery,
we would run:
$ bower install jquery
This would then create a new directory called "bower_components" and underneath that,
download and install "jquery".
Some of the more common packages we may use include:
jquery
threejs
See also:
Bower home page
Bower packages
Robotics
Robotics is our desire to control movement from our CHIP. This is likely to be motors but
Page 309
could be servos or boat propellers.
Heading in a direction
Let us consider the notion that we wish our robot to head off in a direction. Let us assume we
have control over speed and the ability to turn left or right. As an analogy, imagine sitting in
the drivers seat of your car. You have a compass on a gimbal on your dashboard and hence
you can tell what direction you are facing. Now imagine that instead of being on a road which
contrains the directions you can driver you are instead on a flat car park that is (for our
purposes) infinite in all directions. Let us now consider what has to be done to drive in a
particular direction.
Imagine our goal is to drive in the direction "T" (for target) where "T" is a compass direction
expresses as an angle in degrees with 0 being North, 90 being East, 180 being South and
270 being West. At any given point we can ask ourselves "which direction are we currently
facing". Let us call that direction "C" (for current). Which way should we turn the steering
wheel to get us going in the right direction? The answer is to turn it in the direction which has
the shortest angle between "C" and "T". For example if "T" is 90 and "C" is 10 then by turning
the wheel right we would rotate through 80 to reach our objetive. If we turned the wheel left
we would rotate through 280. The generic answer is that we have two values:
Right turn = | T C |
Left turn = | C T |
We calculate both and choose the turn direction which results is the smaller of the two. As we
turn, we should be recalculating our deviation from desired to direction from actual direction
and continue to turn while the deviation is greater than some threshold. Our specific project
will determine what that threshold is. The chances are high that we should not expect to hit
the target exactly as there can be a myriad of factors affecting the determination. We may
also have to account for other real-world factors such as inertia. If we are turning and exactly
hit our desired target, it may take some (hopefully small) anount of time to "stop turning".
During that interval, we may find that we have now over turned and have to turn back in the
opposite direction which again may cause us to over-steer. A solution might be to change our
rate of turn as a function of how far from our target direction we are. If we are just a small
angle off, then steer a proportionally small angle of turn to come back on course.
In the real world, there may be other forces acting on our robot such as wind, slopes, waster
current or errors in motor control. By incorporating a feedback loop, we can keep adjusting
our steering to keep us on track. If we wanted to become even more sophisticated, we could
measure our turns rate as a function of our current heading and model predictions of effects
of different turns.
So far we have assumed that we can control the turn amount but we have a second
consideration which is the speed of propulsion. If you sit in your stationery car, you can turn
the wheel all day long and not change your heading. Only when you apply the gas and start
moving in your "local forward" will you start to change direction. The time taken to reach your
desired target direction is now a function of both how much you turned the wheel and how
Page 310
fast you are going.
Projects
Wiring-CHIP
For the Raspberry Pi world, there is a fantastic package called "Wiring-Pi" that provides a C
language interface library to C programs to be able to perform hardware interfacing. At the
time of writing (July 2016), there is not an obvious equivalent for CHIP so let us build one.
The APIs we need are:
int wiringCHIPSetup()
void pinMode(int pin, int mode)
void digitalWrite(int pin, int value)
int digitalRead(int pin)
int wiringCHIPI2CSetup(int bus, int devId);
int wiringCHIPI2CRead(int fd);
int wiringCHIPI2CReadReg8(int fd, int reg);
int wiringCHIPI2CReadReg16(int fd, int reg);
int wiringCHIPI2CWrite(int fd, int data);
int wiringCHIPI2CWriteReg8(int fd, int reg, int data);
int wiringCHIPI2CWriteReg16(int fd, int reg, int data);
A GitHub project has been created for this task and can be found here:
https://github.com/nkolban/wiring-CHIP
Building libsoc
Install git, autoconf, libtool, automake, pkg-config, libpython2.7-dev
$ sudo apt-get install git autoconf libtool automake pkg-config libpython2.7-dev
Clone libsoc
$ git clone https://github.com/jackmitch/libsoc.git libsoc.git
Run autoreconf -i
$ autoreconf -i
Page 311
This will take a bit and build an environment
Run ./configure
$ ./configure
If when you compile a program that uses libsoc and run it and you get an error that reads:
error while loading shared libraries: libsoc.so.2: cannot open shared object file: No such
file or directory
Then that means that the installation directory for libsoc libraries is not in your shared library
lookup path. Set or change the LD_LIBRARY_PATH environment variable to include that
directory which is commonly /usr/local/lib.
See also:
libsoc home page
Recommended packages
geany a GUI based text editor
build-essential C compilation tools
nfs-common NFS tools to be able to access remote file systems
pcmanfm File manager
Page 312
? - alias for 'help'
base - print or set address offset
bdinfo - print Board Info structure
bmp - manipulate BMP image data
boot - boot default, i.e., run 'bootcmd'
bootd - boot default, i.e., run 'bootcmd'
bootelf - Boot from an ELF image in memory
bootm - boot application image from memory
bootp - boot image via network using BOOTP/TFTP protocol
bootvx - Boot vxWorks from an ELF image
bootz - boot Linux zImage image from memory
chpart - change active partition
clrlogo - fill the boot logo area with black
cmp - memory compare
coninfo - print console devices and information
cp - memory copy
crc32 - checksum calculation
dfu - Device Firmware Upgrade
dhcp - boot image via network using DHCP/TFTP protocol
dm - Driver model low level access
echo - echo args to console
editenv - edit environment variable
env - environment handling commands
erase - erase FLASH memory
exit - exit script
ext2load- load binary file from a Ext2 filesystem
ext2ls - list files in a directory (default /)
ext4load- load binary file from a Ext4 filesystem
ext4ls - list files in a directory (default /)
ext4size- determine a file's size
false - do nothing, unsuccessfully
fastboot- use USB Fastboot protocol
fatinfo - print information about filesystem
fatload - load binary file from a dos filesystem
fatls - list files in a directory (default /)
fatsize - determine a file's size
fatwrite- write file into a dos filesystem
fdt - flattened device tree utility commands
flinfo - print FLASH memory information
fstype - Look up a filesystem type
go - start application at address 'addr'
gpio - query and control gpio pins
help - print command description/usage
i2c - I2C sub-system
iminfo - print header information for application image
imxtract- extract a part of a multi-image
itest - return true/false on integer compare
load - load binary file from a filesystem
loadb - load binary file over serial line (kermit mode)
loads - load S-Record file over serial line
loadx - load binary file over serial line (xmodem mode)
loady - load binary file over serial line (ymodem mode)
loop - infinite loop on address range
ls - list files in a directory (default /)
md - memory display
mii - MII utility commands
mm - memory modify (auto-incrementing address)
mtdparts- define flash/nand partitions
mw - memory write (fill)
nand - NAND sub-system
Page 313
nboot - boot from NAND device
nfs - boot image via network using NFS protocol
nm - memory modify (constant address)
part - disk partition related commands
ping - send ICMP ECHO_REQUEST to network host
printenv- print environment variables
protect - enable or disable FLASH write protection
pxe - commands to get and boot from pxe files
reset - Perform RESET of the CPU
run - run commands in an environment variable
save - save file to a filesystem
saveenv - save environment variables to persistent storage
setenv - set environment variables
setexpr - set environment variable as the result of eval expression
showvar - print local hushshell variables
size - determine a file's size
sleep - delay execution for some time
source - run script from memory
sysboot - command to get and boot from syslinux files
test - minimal test like /bin/sh
tftpboot- boot image via network using TFTP protocol
true - do nothing, successfully
ubi - ubi commands
ubifsload- load file from an UBIFS filesystem
ubifsls - list files in a directory
ubifsmount- mount UBIFS volume
ubifsumount- unmount UBIFS volume
ums - Use the UMS [USB Mass Storage]
usb - USB sub-system
usbboot - boot from USB device
version - print monitor, compiler and linker version
One of the more interesting commands is "printenv" which lists the environment variables
in effect. It is these variables which control some of how the kernel is started:
baudrate=115200
boot_a_script=load ${devtype} ${devnum}:${bootpart} ${scriptaddr} ${prefix}${script}; source
${scriptaddr}
boot_extlinux=sysboot ${devtype} ${devnum}:${bootpart} any ${scriptaddr} $
{prefix}extlinux/extlinux.conf
boot_prefixes=/ /boot/
boot_script_dhcp=boot.scr.uimg
boot_scripts=boot.scr.uimg boot.scr
boot_targets=fel usb0 pxe dhcp
bootargs=root=ubi0:rootfs rootfstype=ubifs rw earlyprintk ubi.mtd=4
bootcmd=gpio set PB2; if test -n ${fel_booted} && test -n ${scriptaddr}; then echo (FEL
boot); source ${scriptaddr}; fi; mtdparts; ubi part UBI; ubifsmount ubi0:rootfs; ubifsload
$fdt_addr_r /boot/sun5i-r8-chip.dtb; ubifsload $kernel_addr_r /boot/zImage; setenv bootargs
$bootargs $kernelarg_video; bootz $kernel_addr_r - $fdt_addr_r
bootcmd_dhcp=usb start; if dhcp ${scriptaddr} ${boot_script_dhcp}; then source $
{scriptaddr}; fi
bootcmd_fel=if test -n ${fel_booted} && test -n ${scriptaddr}; then echo '(FEL boot)';
source ${scriptaddr}; fi
bootcmd_pxe=usb start; dhcp; if pxe get; then pxe boot; fi
bootcmd_usb0=setenv devnum 0; run usb_boot
bootdelay=2
bootm_size=0xa000000
dip_addr_r=0x43400000
dip_overlay_cmd=if test -n "${dip_overlay_name}"; then ubifsload $dip_addr_r
$dip_overlay_dir/$dip_overlay_name; fi
Page 314
dip_overlay_dir=/lib/firmware/nextthingco/chip/early
distro_bootcmd=for target in ${boot_targets}; do run bootcmd_${target}; done
ethact=usb_ether
ethaddr=02:ce:04:42:72:97
fdt_addr_r=0x43000000
fdtcontroladdr=5ab26520
fdtfile=sun5i-r8-chip.dtb
kernel_addr_r=0x42000000
mtdids=nand0=sunxi-nand.0
mtdparts=mtdparts=sunxi-nand.0:4m(spl),4m(spl-backup),4m(uboot),4m(env),-(UBI)
preboot=usb start
pxefile_addr_r=0x43200000
ramdisk_addr_r=0x43300000
scan_dev_for_boot=echo Scanning ${devtype} ${devnum}:${bootpart}...; for prefix in $
{boot_prefixes}; do run scan_dev_for_extlinux; run scan_dev_for_scripts; done
scan_dev_for_boot_part=part list ${devtype} ${devnum} -bootable devplist; env exists
devplist || setenv devplist 1; for bootpart in ${devplist}; do if fstype ${devtype} $
{devnum}:${bootpart} bootfstype; then run scan_dev_for_boot; fi; done
scan_dev_for_extlinux=if test -e ${devtype} ${devnum}:${bootpart} $
{prefix}extlinux/extlinux.conf; then echo Found ${prefix}extlinux/extlinux.conf; run
boot_extlinux; echo SCRIPT FAILED: continuing...; fi
scan_dev_for_scripts=for script in ${boot_scripts}; do if test -e ${devtype} ${devnum}:$
{bootpart} ${prefix}${script}; then echo Found U-Boot script ${prefix}${script}; run
boot_a_script; echo SCRIPT FAILED: continuing...; fi; done
scriptaddr=0x43100000
serial#=162542ce04427297
splashpos=m,m
stderr=serial
stdin=serial,usbkbd
stdout=serial
usb_boot=usb start; if usb dev ${devnum}; then setenv devtype usb; run
scan_dev_for_boot_part; fi
usbnet_devaddr=de:ad:be:af:00:01
video-mode=sunxi:720x480-24@60,monitor=composite-ntsc,overscan_x=40,overscan_y=20
Page 315
Knowledge Sources
When learning anything new, it is always vital to know where to go to read and find out more.
For the CHIP, there are some great places.
Research Areas
What to have on a test board?
Header for Salaea logic analyzer
Page 316
Level Converter (1 or 2)
Push button
Serial connector
LEDs
NeoPixel
MPU-6050
Arduino Nano with Johnny-Five backpack
Install/Configure PulseAudio Apparently broken on 4.4
Build a Telescope controller
Determine if 3.7 volts can power LEDs
Test a WiFi USB dongle in the USB port
IP network using USB https://bbs.nextthing.co/t/connecting-chip-to-your-network-
over-usb/8935
See if "PiBakery" will work on CHIP
Test out the new driver (9/13) - https://bbs.nextthing.co/t/fix-cdc-composite-gadget-4-4-
driver-issue-on-windows/7458/18
Document the use of BMP180
Page 317