TM

TM

JOURNAL
Jan/Feb 2017

Globally
Accessible FreeBSD
FlightAware
and FreeBSD

Improving
FreeBSD
TRANSLATION
TOOLS
FreeBSD-powered
LCD
Advertising
in China

Also:
The Foundation’s
GLOBAL
EFFORTS
 TM

TM

January/February 2017

3 Foundation Letter
Welcome to FreeBSD Journal 2017!
By George Neville-Neil

24 New Faces
In this installment, the spotlight is on
Nikolai Lifanov (ports) and Konrad
Witaszczyk (src), who became committers
Globally Accessible FreeBSD
in November 2016, and Larry Rosenman
and Jean-Sébastien Pédron, who received
FlightAware and FreeBSD
4
ports commit bits in January 2017.
By Dru Lavigne It is the greatest, mostly. Up until very recent-
28 Book Review ly, FreeBSD was used almost entirely to power the site
Aditya Y. Bhargava’s Grokking and services with only a handful of exceptions. This,
Algorithms: An Illustrated Guide for however, is starting to change and the author explains why.
Programmers and Other Curious By Sean Kelly
People. The goal of this book is to be an
easy on-ramp to learning more about algo-
rithms and to help dispel the myth
that algorithms are an obscure and com-
FreeBSD-powered
LCD Advertising
10
plex subject. By Steven Kreuzer

30 svn Update While activity in Displays (in Some Waiting

base tapered off a bit in December, the Rooms Along the High-speed
author did see a few commits that he Railway of China). In 2011 and 2013, a number of
describes as “interesting.”
By Steven Kreuzer
freestanding LCD advertising displays were installed in
waiting rooms along some of China’s high-speed railways.
32 Events Calendar In this article, the author covers the business use case—
By Dru Lavigne excluding the network server, the user experience on
•••••••••••••••••• ••• stability, and the pain points on the GPU. By Li, Xiao

INTERACTING
with the FreeBSD Project

The Foundation’s Improving the FreeBSD

Global Efforts Translation Tools. A large portion of the 16
22 The FreeBSD Foundation is proud world is inhabited by people who could benefit from
to serve a welcoming international com- using FreeBSD—and can't—because they don't speak
munity and to support and grow English even as a second or third language. There is a
FreeBSD awareness throughout the
huge benefit that we can bring to them.
world. By Anne Dickison
By Warren Block

Jan/Feb 2017 1
 TM

JOURNAL LETTER
Editorial Board from the Board •
John Baldwin • Member of the FreeBSD Core Team
and Co-Chair of FreeBSD Journal
Editorial Board
Brooks Davis • Senior Software Engineer at SRI
International, Visiting Industrial Fellow
at University of Cambridge, and past
Welcome to
FreeBSD Journal 2017!
TM
member of the FreeBSD Core Team
Bryan Drewery • Senior Software Engineer at EMC Isilon,
member of FreeBSD Portmgr Team,

T
and FreeBSD Committer he Editorial Board is excited about the selection of
Justin Gibbs • Founder and President of the
topics planned for our 2017 issues and, of course, the
FreeBSD Foundation and a Senior
Software Architect at Spectra Logic articles in this, the first, issue of the new year. First up is
Corporation Sean Kelly’s article on how FlightAware uses FreeBSD to
Daichi Goto • Director at BSD Consulting Inc. provide accurate flight data. If you travel a lot, then you prob-
(Tokyo)
ably know about flightaware.com and their free app. When
Joseph Kong • Senior Software Engineer at EMC and
author of FreeBSD Device Drivers you use it to find out why you're delayed, yet again, you're
Steven Kreuzer • Member of the FreeBSD Ports Team using FreeBSD. Knowing that you're using FreeBSD may take a
Dru Lavigne • Director of the FreeBSD Foundation, bit of the sting out of that two-hour ground delay!
Chair of the BSD Certification Group,
and author of BSD Hacks
This issue’s other two feature articles relate to FreeBSD's use
Michael W. Lucas • Author of Absolute FreeBSD
Ed Maste • Director of Project Development, in various languages. Li, Xiao writes about FreeBSD and rail-
FreeBSD Foundation way information signs in China, and Warren Block discusses
Kirk McKusick • Director of the FreeBSD the new slate of translation tools in use by the FreeBSD docu-
Foundation and lead author of
The Design and Implementation
mentation and development teams. The international focus is
book series rounded out by a piece from Anne Dickison of the FreeBSD
George V. Neville-Neil • Director of the FreeBSD Foundation, Foundation, outlining how the Foundation is working to make
Chair of FreeBSD Journal Editorial
Board, and coauthor of The Design
sure FreeBSD is known and loved around the world.
and Implementation of the FreeBSD
Operating System As we’re just beginning the year, we do not have a confer-
Hiroki Sato • Director of the FreeBSD ence report for this issue, but in later issues we’ll report on
Foundation, Chair of Asia
BSDCon, member of the FreeBSD FOSDEM 2017, which just finished a couple of weeks ago, as
Core Team, and Assistant well as AsiaBSDCon (https://2017.asiabsdcon.org) planned for
Professor at Tokyo Institute of
Technology Tokyo, Japan, in March; BSDCan (http://www.bsdcan.org/2017/)
Benedict Reuschling • Vice President of the FreeBSD in Ottawa, Canada, in June; as well as BSDCam, VBSDCon,
Foundation and a FreeBSD and EuroBSD, taking place in the months of August and
Documentation Committer
September.
Robert Watson • Director of the FreeBSD Foundation,
Founder of the TrustedBSD Project,
and University Senior Lecturer at In this issue, you’ll enjoy columns from Dru Lavigne, who
the University of Cambridge
profiles New Faces in the FreeBSD community, and Steven
Kreuzer, who reviews Grokking Algorithms: An illustrated
S &W PUBLISHING LLC
P O B O X 4 0 8 , B E L F A S T, M A I N E 0 4 9 1 5
guide for programmers and other curious people, by Aditya Y.
Bhargava, and also delivers a new installment of his svn
Publisher • Walter Andrzejewski update column.
[email protected]

Editor-at-Large • James Maurer 2017 is shaping up to be a great year for the FreeBSD
[email protected] Journal. We are working on issues on security, configuration
Copy Editor • Annaliese Jakimides management, and resource control for the first half of this
Art Director • Dianne M. Kischitz year. And for now, we'll just keep the rest of the year under
[email protected] our horns.
Office Administrator • Michael Davis
[email protected]
Hope to see everyone at a BSD event in 2017!
Advertising Sales • Walter Andrzejewski
[email protected]
Call 888/290-9469

FreeBSD Journal (ISBN: 978-0-615-88479-0) is published 6 times a

year (January/February, March/April, May/June, July/August, George Neville-Neil
September/October, November/December).
Published by the FreeBSD Foundation,
5757 Central Ave., Suite 201, Boulder, CO 80301
Editor in Chief of the FreeBSD Journal
ph: 720/207-5142 • fax: 720/222-2350
email: [email protected]
Director of the FreeBSD Foundation
Copyright © 2017 by FreeBSD Foundation. All rights reserved.

This magazine may not be reproduced in whole or in part without

written permission from the publisher.
Jan/Feb 2017 3
 SEE
TEXT
O N LY

B y Sean Kelly

FlightAware and FreeBSD

I t i s t h e g re a t e s t , m o s t l y.

If you’ve not heard of us, FlightAware is the

world’s largest flight-tracking data company,
providing over 10,000 aircraft operators and
service companies with global flight-tracking solu-
tions. We provide free flight tracking to the general
public on our website and via our mobile apps, but we
also have other commercial products and services used by air-
craft operators around the world. We accomplish this by aggre-
gating data from our own ground stations, government data
sources covering over 55 countries, and other satellite-based
providers. We then take all this data, interpret it, and turn it into
more holistic views of what is going on. As you might expect,
this requires a fair amount of processing and storage.

4 FreeBSD Journal
A Installation: A Server Is Born
t FlightAware, our business to date has
been built on top of FreeBSD, and we are
While bsdinstall is a great improvement on
very strong proponents of it. We have
sysinstall, we still needed something that provid-
four employees who are inactive members of the
ed much more automation. Almost all of our
FreeBSD Project, including our CEO and CTO. Up
servers start with the same base install, so it was
until very recently, FreeBSD was used almost
unnecessary for us to repeatedly punch the same
entirely to power our site and services with only
information into bsdinstall. Instead, we devel-
a handful of exceptions. This, however, is starting
oped a set of scripts that incorporate bsdinstall
to change and I’d like to explain why. But first,
and bsdconfig. These scripts run from a mfsBSD
let’s cover our history with FreeBSD and how
PXE environment and do the following:
we’ve been using it.
Our CEO used to run ftp2.freebsd.org and he
• Destroy any existing hardware RAID configura-
contributed to the ports tree. He has been using
tion and partitions
FreeBSD since the 2.0-ALPHA days and continues
• Wipe any existing ZFS labels on disks
using it both professionally and personally to this
• Create a new zpool on all internal disks
day. Our CTO goes further back, to the begin-
• Create all the appropriate ZFS filesystems to
nings of 386BSD, where he maintained patch
support boot environments
kits and later contributed loopback detection to
• Pull down the FreeBSD installer files and un-
the T1 driver. Going back even further, one of
tar them
our developers did a lot of hacking on 2BSD and
• Create a user
4.1BSD and later helped with the FreeBSD
• Configure all network parameters
Handbook. Finally, I started using FreeBSD at the
release of 4.0 and have contributed some patch-
Once that is complete, the machine reboots
es as well as the initial implementation of the
and we’re prompted with a menu where we can
software watchdog subsystem.
select the components we’d like installed. Each
Along with FreeBSD, we use the Tcl program-
component is managed by a shell function inside
ming language extensively. This includes our
of a shell script, making it easy to manage and
website, which is written in Tcl and served by
modify. When we need a new component, we
Apache’s mod_rivet extension. Several ports of
just create a new function.
our Tcl packages exist in the ports tree, including
All this evolved over time. Ideally, we’d like to
speedtables, casstcl, yajl-tcl, tcllauncher, tclread-
be able to have an installer that will do a PXE
line, and tclbsd. These ports are maintained by
boot, find a configuration file matching a MAC
tcltk@ and Pietro Cerruti. While Pietro is not a
address, and do a full headless installation from
FlightAware employee, he generously updates
that configuration file. It would be even better if
ports to coincide with our software releases.
this was built into FreeBSD so that FreeBSD had
Some of the reasons that we do use FreeBSD
an easy zero-touch deployment system that was
include the stability and robustness of the net-
standardized and each shop didn’t have to roll
work stack, the cohesive community, the holistic
their own.
view of the system from kernel to userland, and
ZFS. All these things combine to create an opti-
mum and trustworthy open-source environment Traditional Databases:
for our servers and services. Make It Fast
While it is less novel than our custom installer,
How We Use It another use for FreeBSD at FlightAware is to
Like many others, we run web servers, Varnish power our PostgreSQL database cluster. We chose
servers, mail servers, and DNS servers on FreeBSD for this especially due to the data integri-
FreeBSD. While not entirely unusual, there are ty features provided by ZFS. Another nice benefit
several other things we do on the FreeBSD plat- is the LZ4 compression which yields a compression
form that may interest others a bit more. These ratio of around 2.15x with a negligible perform-
include a custom FreeBSD installer, high transac- ance impact. The cluster itself is a standard
tion RDBMS, a few different in-memory databas- PostgreSQL 9.4 setup using the native streaming
es using shared memory, and, of course, all of replication support. We use pgpool to load bal-
these utilize the fantastic ZFS filesystem. ance read-only transactions while all write trans-
actions go to the master. This allows us to spread
queries out over a larger number of servers,

Jan/Feb 2017 5
 FlightAware and FreeBSD
increasing our transaction processing potential due As you can see, shared memory is an impor-
to most transactions being read-only. tant technology for us. We use it quite extensively
In October 2015, we transitioned our with our internal tools as well as with third party
PostgreSQL servers away from 24 hard disks in ZFS projects like PostgreSQL. We’ve found that the
mirror pairs to use four NVMe SSDs in mirrored overall performance of SHM has gotten better in
pairs. This was a huge performance gain once we the last several years and hope that it continues
got past issues with TRIM that eventually resulted to do so. We noticed significant improvements
in us disabling it. The NVMe subsystem timed out with FreeBSD 9 and a bit again with FreeBSD 10.
upon zpool create since ZFS was trying to TRIM
the entire pool and NVMe gave up waiting on the Pain Points:
SSD to do it. We also had issues during normal
operation when TRIM was executed on Samsung
Not Everything Is Perfect
SSDs that caused several outages. When Now that we’ve covered how FreeBSD works for
PostgreSQL would clean up pg_xlog files, it would us, we’d be remiss to not discuss our pain points.
essentially briefly stall I/O due to TRIM. We’ve since FlightAware is rapidly growing, as is our need to
switched to Intel SSDs which don’t seem to exhibit be able to deploy new systems and services at an
the problem, but we left TRIM off anyway. ever-increasing rate. Further, data growth and
To back up all of this, we developed a custom increasing ingestion rates are causing us to reeval-
toolset in Tcl that manages ZFS snapshots. The uate how we store, process, and analyze our
tool takes a snapshot on a database server, sends datasets. This is where our choice of FreeBSD
it to another host where it is staged, and from becomes harder to continue and justify.
there the snapshot is dispersed to various sites.
These same tools also manage the retention poli- Installation:
cy for the snapshots, keeping a certain amount of
hourly, daily, and weekly snapshots on hand.
An Arm of Servers
One amazing thing about FreeBSD is that it can
be installed on a system and then mostly forgot-
In-Memory Databases: ten. You obviously want to install updates and so
Making Fast Faster forth, but the system is generally rock solid and
Along with our PostgreSQL databases, we also requires very little ongoing maintenance and care.
have internal software called Birdseye. Birdseye is As a result, we tend to treat our FreeBSD systems
an in-memory database written in Tcl and C, and as unique rather than as a fleet of entirely dispen-
based on our open-source speedtables project. sable resources. We’ll add and remove software,
Birdseye uses shared memory to store approxi- but rarely do we find the need to do a clean
mately 24 hours of positions for all aircraft that install. The problem with this is that the approach
we know about. Multiple Birdseye processes on a doesn’t scale well. As a result, we’re starting to
single machine share the same shared memory treat operating system installs as a commodity.
segments so that more clients can be serviced As I discussed earlier, we’ve built our own
concurrently. Clients, such as the website, can FreeBSD installer that mostly works for us to
then connect to these Birdseye servers and make address this change. It isn’t great and we’d like to
all sorts of queries to very quickly discern what a add more features, such as the ability for it to
plane is doing, has done, or even what is happen- TFTP a configuration file for entirely headless
ing within an arbitrary geographical box. installs. Maybe there would be a base configura-
Along with Birdseye, we have another in-mem- tion file and then there could be a MAC address-
ory database of sorts called Superbird. Superbird, based or SMBIOS system serial number-based
too, is based on our speedtables project and is override file. But instead of us implementing this
written in Tcl with some generated C. Superbird for our own tools, it’d be fantastic if this was just
will periodically read in any changes that have part of the FreeBSD installer.
been made to PostgreSQL tables and keep them The FreeBSD installer should support entirely
in an in-memory speedtables database. This is headless installations with documented scripting
leveraged by the website so that it is possible to functionality. This is something that the Linux
make faster queries of busy tables by accessing a world conquered long ago with the Red Hat kick-
local in-memory cached copy rather than needing start system or the Debian preconfiguration file.
to access an off-host PostgreSQL server. In other The installer could be PXE booted and then use
words, Superbird is a database caching layer uti- DHCP options to direct it to where it can down-
lizing update polling. load configuration data. This would make it trivial
to deploy or redeploy hundreds of servers at once.

6 FreeBSD Journal
Containers and Isolation: 64-bit Java:
When Jail Isn’t Enough Twice as Good as Before
Now that I’ve shared my vision for how the As I already mentioned, our datasets are growing
FreeBSD installation process could be improved, as is the amount of data we ingest. This has
let me explain what it is we want to do with caused us to turn to newer technologies like
fleets of heedlessly installed servers. Cassandra, Kafka, and Spark to store, move, and
We are rapidly moving away from deploying make sense of it all. All these have one thing in
services on a standard system installation. Like a common: they run on the Java Runtime
lot of cloud scale companies, we are adopting a Environment.
containerization approach that will allow us to There are not a lot of options for Java on
easily deploy many services on a single piece of FreeBSD. You can build and use the OpenJDK
hardware without the software and library open-source implementation of Java which can be
dependencies that this traditionally introduces. compiled into a native 64-bit implementation. You
We’d like to have every base installation be identi- can also use the 32-bit Linux JRE from Oracle
cal and serve as nothing but a platform for host- through the FreeBSD Linux emulation. However,
ing application containers. This simplifies installa- only until very recently have 64-bit Linux system
tion and management. calls been supported and we’ve not successfully
As one potential path toward this, we did gotten the 64-bit Oracle JRE to run on FreeBSD.
some evaluation of jails and found them mostly Finally, The FreeBSD Foundation used to offer a
serviceable. However, we opted not to move in FreeBSD native binary for the Oracle JRE, but that
that direction due to several issues we faced. One is no longer available.
glaring issue is the lack of per-jail shared memory. So you’re probably wondering what our prob-
FreeBSD jails’ shared memory is not isolated from lem is and why we don’t just use OpenJDK. The
each other or from the base system. The lack of answer is that we’ve encountered issues with it
separation means services like PostgreSQL and that were hard to pin down. With Kafka, we saw
Birdseye can’t be securely deployed in adjacent periodic message corruption using OpenJDK that
jails without special care to avoid conflict or we didn’t see with the Oracle JRE. Not to mention
exploitation. For example, running PostgreSQL in that the Oracle JRE is the authoritative implemen-
adjacent jails is risky due to the pgsql user having tation of Java which matters in a production envi-
the same UID and thus each jail having access to ronment. After that, we chose to use the Oracle
the other jail’s IPC resources. JRE at least until we can circle back and retest and
Despite all of this, a lot of progress seems to help fix OpenJDK.
have been made by adding RCTL to GENERIC, but We want 64-bit and we’ve chosen Oracle’s JRE
we’d also like to see more focus in this area. as the path forward. That means that we had to
VIMAGE should be made stable and added, as turn to Linux in order to run our Kafka,
well. Further, we’d like to see more namespacing Cassandra, and Spark environments. As we con-
for things like shared memory implemented to tinue to build out these services, this will result in
provide true isolation. This is one area where us having more and more Linux.
Linux is ahead and has resulted in technologies
like Docker and Kubernetes thriving. System Tuning:
The actual container mechanism is important,
but how it is managed is also a big deal. As it is, My Server Isn’t That Wimpy
FreeBSD has many ways to manage jails that are Another area that could use more attention is
in various states of support and development. out-of-the-box tuning for a FreeBSD install. While
We’ve got ezjail, iocage, CBSD, jail(8)’s many defaults seem to be tuned for the absolute
/etc/jail.conf, and many other homegrown tools least common denominator of support, it’d be
and derivatives. While choice is almost always a great to implement some way to have profiles
good thing, this diverse ecosystem also makes it that can be selected during install. Let me explain
harder for automation and orchestration tools to with concrete examples.
target support for jails. It would be fantastic if Right now, we compile our Tcl interpreter with
there was a way as part of base to bring all of the an additional CFLAGS value of -DFD_SETSIZE=4096
diverse features under one roof—one set of APIs to override the default of 1024 from <sys/select.h>.
and commands to rule them all. We know that select() isn’t ideal and we’ve put out
a bounty for Tcl kqueue support, but at the same
time 1024 seems really small for modern day net-

Jan/Feb 2017 7
FlightAware and FreeBSD
work applications. It’d be great to see this and so forth, as well as GigE, 10GigE, or 40GigE
increased so that FreeBSD systems could be ready networking. As it is, it is sort of a dark art to get it
to handle all of the concurrent connections that it set up and fully understand it.
can otherwise effortlessly support without further
tweaking by the user. Debugging:
Default log rotation sizes are also something that
needs some attention. According to the
Because Things Break
/etc/newsyslog.conf on a fresh FreeBSD install, most What would you do if the zpool command
log files should be rotated after they grow to be dumped core? For me, I’d fire up the debugger
100KB in size. I don’t know about you, but my and hope to see where it crashed by loading up
/var/log/messages and my /var/log/auth.log don’t the core file. Unfortunately, the base system lacks
take very long to reach that. In an era where you debugging symbols when you install a release ver-
can get 10TB hard disks and 1TB SSDs, this seems sion of FreeBSD. Why is this? Are they too big, just
like an obvious candidate for a profile system with like those 100KB log files? It would be nice if base
options for embedded, server, desktop, etc., so that binaries weren’t stripped, or at least an optional
the target rotation size is more reasonable. Even system component included symbol files for all of
100KB seems small for a SD card-based system. the base binaries and libraries.
While I’m complaining about logging, this Similarly, the ports tree defaults to building soft-
seems like the opportune time to suggest some ware without symbols. Most ports have a DEBUG
features. Can we get an include directive for option you can enable that will cause the software
/etc/syslog.conf? newsyslog.conf has one, so it to be built with symbols but generally without any
seems reasonable that syslog should as well. Some compiler optimizations enabled. This isn’t ideal for
improvements to the ! program specifier in sys- production. I’d like to see a universal option to
log.conf would be nice too. As it is, in order to log build a port with optimization but also with
all messages from postgres to debugging symbols, ideally by default. I under-
/var/log/postgres.log, I have to do something like: stand that the debugging will be a bit harder with
something built using -O or -O2, but it is sure bet-
# Capture PostgreSQL logs ter than nothing.
!postgres Finally, everything should have DTRACE by
*.* /var/log/postgres.log default. DTRACE is an amazing tool that
FlightAware has used on more than one occasion
# Capture all other logs to understand a performance problem or system
!-postgres failure. Both PostgreSQL and Tcl conveniently have
*.* /var/log/all.log extensive DTRACE support. However, ports does
not enable this by default. In our opinion, every
It would be nice if there was a way to have a port should default to building with DTRACE. It
program specifier that matched anything except isn’t needed on a day-to-day basis but, like debug-
other defined program specifiers. Then you can ging symbols, it is invaluable when a problem
have a catch-all section without having to list the eventually does appear.
exclusions out specifically. This will also work
around another issue I encountered: the maximum
line length of syslog.conf is too short for listing Wrapping Up
these out. I hope I’ve been able to make it clear that
Finally, another area that could be improved is FlightAware really does prefer and advocate for
overall network stack tuning. If you search FreeBSD. That being said, there is always room for
Google, you can find lots of different pages that improvement and I set out to explain here what
tell you sysctls and tunables that you should be could be improved in FreeBSD to make it work
setting to maximize your network stack and better for us and presumably others. It is time to
10GigE NIC throughput. It would be great if there start looking at ways to make it easier to increase
was a profile selection during install that could the number of installs in the wild and make it easi-
tune things to be great for Varnish, nginx, Apache, er to treat FreeBSD as a commodity platform. •

SEAN KELLY has been a member of the FlightAware team since 2012. He serves as the Director of IT
Operations. In this role, Sean designs and oversees the growth and maintenance of both the worldwide infra-
structure that powers FlightAware as well as all of the IT systems powering FlightAware's two corporate offices.
Sean is also a former committer of the FreeBSD Project and has been using FreeBSD since at least the 4.0 days.

8 FreeBSD Journal
 Support
FreeBSD
®

Donate to the Foundation!

You already know that FreeBSD is an internationally
recognized leader in providing a high-performance,
secure, and stable operating system. It’s because of
you. Your donations have a direct impact on the Project.
Please consider making a gift to support FreeBSD for the
coming year. It’s only with your help that we can continue
and increase our support to make FreeBSD the high-
performance, secure, and reliable OS you know and love!

Your investment will help:

 Funding Projects to Advance FreeBSD
 Increasing Our FreeBSD Advocacy and
   
 Providing Additional Conference
Resources and Travel Grants
 Continued Development of the FreeBSD
Journal
 Protecting FreeBSD IP and Providing
Legal Support to the Project
 Purchasing Hardware to Build and 5.

Improve FreeBSD Project Infrastructure

Making a donation is quick and easy.

freebsdfoundation.org/donate
 SEE
TEXT
O N LY

By Li, Xiao

FreeBSD-powered
LCD Advertising Displays
In 2011, 101 freestanding LCD advertising displays were installed in 11 waiting rooms
along the high-speed railway between Shanghai and Xuzhou. In 2013, seven LCD
advertising display towers were installed in four waiting rooms along the high-speed
railway between Shenyang and Dalian. In these devices, tailored FreeBSD images
with Xorg run with Intel CPUs and flash memories. The strong body of FreeBSD
ensures that the advertising pictures and videos are stably played and
that the electric motors in the towers behave correctly
under control of the advertising programs.

TM

Above: Freestanding LCD displays. Only the lower LCDs

were driven by hosts running FreeBSD. They were playing
advertising pictures customized by Mega-info Media.

10 FreeBSD Journal
 Harbin
•

Shenyang •
Beijing • • Dalian

• Shanghai
ince 2000, electronic displays such as LCD and LED have

S
Geographical
been widely used in advertising all over the world. Inside those positions of
Beijing-
advertising devices, open-source software compiled for embedded Shanghai and
Harbin-Dalian
systems supports playing audio and video, accessing an SD card or USB high-speed
lines on China’s
flash drive, and networking and remote control. mainland.
Since 2000, China has built the longest in the FreeBSD ports tree.
high-speed railway lines in the world [1]. In At that time, Xorg had imported a kernel
order to develop the economic value of wait- mode setting (KMS) mechanism in the
ing rooms in railway stations along those high- GNU/Linux community. But FreeBSD hadn’t
speed lines, LCD and LED displays have been kept up with them. Fortunately, x11-
installed to play commercial advertising pic- drivers/xf86-video-intel29 worked fine with
tures and videos. Xorg 7.5.1 and the onboard GPU in user
From 2011 to 2013, I was an advertising mode. The display mode could be smoothly set
equipment provider for some of those loca- to 1920×1080p@60Hz, which fully conformed
tions. And I adopted FreeBSD with Xorg as the to the monitors’ best performance. And the X-
operating system in my products. The stability Video extension of Xorg also worked fine with
of FreeBSD and my software product surprised
many customers with their very rare software
failures. G = Railway station with my product
H = Railway station without my product
Products and Installations Beijing-Shanghai High-speed Railway Line
I secured two equipment supply contracts in
H Beijing South
H Langfang
H Tianjin South
H Cangzhou West
H
H
H
H
H Tengzhou East
H Zaozhuang
G Xuzhou East
G Sùzhōu East
G Bengbu South
G Dingyuan
G Chuzhou
G Nanjing South
G Zhengjiang South
G Danyang North
G Changzhou North
G Wuxi East
G Sūzhōu North
G Kunshan South
H Shanghai Hongqiao
2011 and 2013, respectively for a section of
Dezhou East
Jinan West
Tai’an
Qufu East

the Beijing-Shanghai high-speed line and a

section of the Harbin-Dalian high-speed line.
Map above shows the geographical positions
of the two lines on China’s mainland [17].

Beijing-Shanghai High-speed Fig. 1. The main line of the Beijing-Shanghai high-speed railway
Railway Line line. Since 2016, some of my products have been removed from
some of those railway stations such as Nanjing South.
In the section between Shanghai and Xuzhou
(about 626 kilometers [6]) of the Beijing-
Shanghai high-speed railway line [2-3] (about
Motherboard Intel Desktop Board D410PT or D425KT
1318 kilometers [6]), the customer ordered the
freestanding LCD advertising displays shown in CPU Intel Atom D410 or D425
the inset photo on page 10 [9]. They were GPU Intel GMA 3150
installed in waiting rooms of the railway sta-
Chipset Intel NM10
tions listed in Figure 1.
The hardware configuration to run FreeBSD Ethernet Realtek 8103EL or 8105E
is shown in Table 1 [7-8]. In 2011, the ethernet Audio Realtek ALC662
and the audio controllers had been fully sup-
ported by FreeBSD’s drivers re(4) and RAM 1GB or 2GB
snd_hda(4). But the GPU was somewhat of a Storage 8GB, USB flash drive
newbie for FreeBSD, and required a newer
Xorg driver xf86-video-intel29, which didn’t Table 1. Hardware configuration of my products for the
belong to the main line of Xorg dependencies Beijing-Shanghai high-speed railway line.

Jan/Feb 2017 11
 G = Railway station with my product
H = Railway station without my product the driver, which prevented the CPU from being
overloaded when playing video.
The Shenyang-Dalian Section of
Harbin-Dalian High-speed Railway Line
The Harbin-Dalian High-speed
G Shenyang North

H Shenyang

H Liaoyang

G Anshan West

H Haicheng West

H Yingkou East

H Gaizhou West

G Bayuquan

H Wafangdian West

H Puwan

G Dalian North
Railway Line
In the section between Shenyang and Dalian
(about 380 kilometers [6]) of the Harbin-Dalian
high-speed line (about 918 kilometers [6]), the
customer ordered the LCD advertising display
towers shown in photo at left [16]. They were
Fig. 2. The Shenyang-Dalian section of the Harbin-Dalian
high-speed railway line.
installed in waiting rooms of the railway stations
listed in Figure 2 [4-5].
The hardware configuration to run FreeBSD is
shown in Table 2 [10-11]. In 2013, the ethernet
controller Atheros GbE hadn’t been supported by
FreeBSD. But I didn’t have enough time to hack
the driver on my own. I had to install an addition-
al PCI ethernet card with RTL8139 onto the
motherboard.
In this application case, every host was designed
to drive 5 LCD displays via VGA ports. Besides
onboard GPU, I had to install two PCI-E graphics
cards with VGA and DVI-I ports onto every moth-
erboard. At that time, the GPU of NVIDIA worked
fine with NVIDIA’s closed source driver [12], but
the Intel GPU could only be driven by xf86-video-
vesa. Fortunately again, since the Xinerama exten-
sion of Xorg was applied to combine five LCD dis-
plays as a single virtual display, the XVideo exten-
sion wasn’t necessary (actually, Xinerama disables
most hardware rendering); xf86-video-vesa was
sufficient for this case.
LCD advertising display tower. It is playing advertising
contents customized by Xinhua News Agency. Technical Points
Tailored FreeBSD images with Xorg and my
software were written into a USB flash drive or
Motherboard GIGABYTE GA-B75-D3V SATA SSD.

CPU Intel Core i5 (LGA1155) Tailored FreeBSD File Set

GPU Intel HD Graphics 2500 (onboard) To simplify the work, an MBR partition table was
NVIDIA GeForce 210 (PCI-E, doubled) created in every USB flash drive or SATA SSD.
Only one partition was created in every drive, and
Chipset Intel B75
a BSD label “a” was created in the partition. The
Ethernet Atheros GbE (onboard, unused) slice /dev/da0s1a (for USB flash drive) or
Realtek RTL8139 (PCI) /dev/ada0s1a (for SATA SSD) was formatted into
FreeBSD FFS 2 with soft updates and journaling.
Audio Realtek ALC887
In actual use, writing operations to drives was
RAM 2GB not frequent, at no more than about 100 MB a
day. But abnormal power-off might occur 10
Storage SATA SSD 120GB, Intel
times a day. Thus, the robust filesystem could
effectively reduce booting failure. Having benefit-
Table 2. Hardware configuration of my products for the
ed from FreeBSD FFS 2 with soft updates and
Harbin-Dalian high-speed railway line.
journaling, booting failures of my products have

12 FreeBSD Journal
 /boot/ FreeBSD kernel and driver modules,
loader(8) and configuration files
been caused by hardware damage only recently.
The slice /dev/da0s1a or /dev/ada0s1a is used as /libexec/ Run time link-editor, ld-elf.so.1
the root filesystem of FreeBSD. The hierarchy is /bin/, /sbin/, Command line utilities (e.g. /bin/sh,
shown in Table 3. Since I found “mfsBSD” made /usr/bin/, /bin/rm, /usr/bin/killall, /sbin/ifconfig
by Martin Matuška [13], I have looked through his /usr/sbin/ and /sbin/mount)
work for references. /dev/ An empty directory for devfs(5)

Tailored Xorg File Set /var/, /tmp/ Multipurpose files (e.g. /var/empty)

To run the GUI, a file set of Xorg had to be writ- /etc/ Initializing scripts and configuration files
ten into the filesystem. The hierarchy is shown in
/lib/, /usr/lib/ Shared objects
Table 4.
I hadn’t integrated any typical X11 toolkit such /usr/local/ Xorg
as GTK into the flash drive. /root/ My own software files

Use of wxWidgets Table 3. The outlined layout of the filesystem in the USB flash
wxWidgets [14] has been collected in the FreeBSD drive or SATA SSD.
ports tree, which is compiled with GTK 2, but
wxWidgets can work directly with X11. The port
wxWidgets with X11 is called wxX11. Although
Networking
wxX11 is incomplete and buggy so far, it can well SSH daemon configured with public key authenti-
support picture file handling and basic sub-win- cation was started in system. SSH can supply secure
dow management which is required for playing and multipurpose channels for various remote
advertising contents. accessing. An rsh(1)-like remote command execut-
For my product, wxWidgets 2.8.x can be con- ing via ssh(1) simplifies networking into writing
figured as: some small /bin/sh scripts, avoiding the designing
of a tiring and buggy transport protocol.
./configure --with-x11 --with-opengl\ SSH can also act as a secure transport mechanism
--disable-shared --enable-unicode of rsync(1), which is used to transfer large files.

After compiling wxX11 (GNU make is required), A Customized Script /etc/rc

the script wx-config can be used to get C++ com- After the FreeBSD kernel is started, usually
piling and linking parameters for the header and /sbin/init is executed, which tries to run /etc/rc to
library files of wxX11. The commands look like: do most of the essential initialization [15]. My cus-
tomized script /etc/rc contained initializing steps
# To compile C++ source files
~/wxWidgets-2.8.12/wx-config --cxxflags
# To link object files bin/ Standalone executable files
~/wxWidgets-2.8.12/wx-config –-libs (e.g. Xorg, xrandr and xkbcomp)
~/wxWidgets-2.8.12/wx-config --gl_libs etc/ xorg.conf and configuration files for
font-config and pango

Motor Control via RS-232-C Port lib/ Shared objects

To simplify my program, stty(1) was used against lib/X11/fonts/ Basic fonts

/dev/ttyu0.init to set default baud rate, bit count,
lib/dri/ DRI drivers
and other options before my program opened
/dev/ttyu0. lib/pango/ Pango modules
The onboard serial port was connected with a
lib/xorg/modules/ Drivers
printed circuit board (PCB) I designed to set the
electric motors’ behaviors: start and stop, rotating share/X11/xkb/ Resource files concerning keyboard
direction, and rotating speed. At the same time,
some physical quantities (e.g., ambient temperature Table 4. The outlined file layout of tailored Xorg under /usr/local/.
and angular displacement) captured by sensors on I hadn’t integrated any typical X11 toolkit such as GTK into the
my PCB were sent to FreeBSD via the serial port. flash drive.

Jan/Feb 2017 13
 reduced from a complete FreeBSD base and a few of the network devices. Actually, in reality,
complete Xorg distribution. At the end of the although FreeBSD has a very robust and well-
script, it started my own program. The main line designed kernel, it leaves people close to
of the script is: GNU/Linux because of its shortcomings.
1. Waiting for the USB device to get ready in sev- For the future of FreeBSD, my proposal has to
eral seconds. do with the device driver:
2. Finding and checking the storage device in a 1. Move the Windows kernel compatibility layer
possible set of /dev/da0s1a and /dev/ada0s1a and of ndis(4) from the FreeBSD kernel to user mode,
mounting it in read-write mode. which means the layer and device driver will run
3. Finding and configuring the network interface in a process. In user mode, all programs are easier
in a possible set of re(4), rl(4), and em(4). Also to hack and debug than in kernel. At least most
setting network routing. The driver em(4) is used faults in user mode don’t cause kernel crash.
when debugging the file set in VirtualBox. 2. Extend the coverage of the compatibility layer,
4. Adjusting the time zone setting of the kernel which means covering more network device driv-
by adjkerntz(8). ers and trying to cover other types of device driv-
5. Tuning the sound volume. ers such as that of the GPU.
6. Starting the SSH daemon.
7. Starting the Xorg server. Acknowledgment
8. Starting my own program. Thanks to Liang, Li (also known as Kylie Liang) for
her help with this paper. •
Proposals
In the source tree of FreeBSD, there is a great
thing—ndis(4). It can run the network device driv- Li, Xiao is a software and hardware engineer living
er written for Microsoft Windows inside the in China. He runs his tiny company in Beijing. He is
FreeBSD kernel. an experienced FreeBSD developer. He worked on
But the module lacks maintenance. It often does- ports of LaTeX-CJK and Linux compatibility in 2006.
n’t work with up-to-date device drivers for He is enthusiastically involved in the FreeBSD com-
Windows, especially wireless ethernet device drivers. munity of China. He is interested in design of print-
What’s more, ndis(4) can only cover quite a ed circuit board and cross-platform software.

REFERENCES
[1] “High-speed rail in China,” Wikipedia, online: https://en.wikipedia.org/wiki/High-speed_rail_in_China.
[2] “Beijing–Shanghai High-Speed Railway,” Wikipedia, online: https://en.wikipedia.org/wiki/Beijing%E2%80%93Shanghai_High-Speed_Railway.
[3] (In Chinese Only) “Beijing–Shanghai High-speed Railway,” Baidu Encyclopedia, online: http://baike.baidu.com/view/141756.htm.
[4] “Harbin-Dalian High-Speed Railway,” Wikipedia, online: https://en.wikipedia.org/wiki/Harbin%E2%80%93Dalian_High-Speed_Railway.
[5] (In Chinese Only) “Harbin-Dalian High-speed Railway,” Baidu Encyclopedia, online: http://baike.baidu.com/view/1213823.htm.
[6] (In Chinese Only) High-speed Railway Web, online: http://www.gaotie.cn/.
[7] “Technical Product Specification for the Intel® Desktop Board D410PT,” Intel, online:
http://www.intel.com/content/www/us/en/support/boards-and-kits/desktop-boards/000021877.html.
[8] “Technical Product Specification for Intel Desktop Board D425KT/D425KTW,” Intel, online:
http://www.intel.com/content/www/us/en/support/boards-and-kits/desktop-boards/000020992.html.
[9] Mega-info Media Co. Ltd., the official website: http://www.megainfomedia.com/.
[10] GA-B75-D3V User’s Manual, GIGA-BYTE Technology Co. Ltd., online:
http://download.gigabyte.cn/FileList/Manual/mb-manual_ga-b75-d3v_v1.2_e.pdf.
[11] Introduction to Intel Core i5-3450, Intel, online: http://ark.intel.com/products/65511/Intel-Corei5-3450-Processor-6M-Cache-up-to-3_50-GHz.
[12] GeForce driver for FreeBSD, NVIDIA, online: http://www.geforce.com/drivers.
[13] Martin Matuška, “mfsBSD”, online: http://mfsbsd.vx.sk/.
[14] wxWidgets, a cross-platform C++ GUI library, online: http://www.wxwidgets.org/.
[15] “Kernel Initialization,” “FreeBSD Architecture Handbook,” online:
https://www.freebsd.org/doc/en_US.ISO8859-1/books/arch-handbook/boot-kernel.html.
[16] Xinhua News Agency, China, the official website: http://www.news.cn/english/.
[17] A reworked profile map of China’s mainland, referenced from the governmental digital map website “Tianditu” (The Chinese meaning of
this name is “Sky-land Map” or “Sky-earth Map”), online: Chinese: http://map.tianditu.com/ English: http://en.tianditu.com/.

14 FreeBSD Journal
THE INTERNET
NEEDS YOU
GET CERTIFIED AND GET IN THERE!
Go to the next level with
Getting the most out of
BSD
CERTIFICATION

. ... .
BSD operating systems requires a
serious level of knowledge
and expertise . . NEED
AN EDGE?
..
SHOW
YOUR STUFF!
Your commitment and
. . BSD Certification can
make all the difference.
Today's Internet is complex.
Companies need individuals with
proven skills to work on some of
the most advanced systems on
dedication to achieving the the Net. With BSD Certification
BSD ASSOCIATE CERTIFICATION YOU’LL HAVE
can bring you to the WHAT IT TAKES!
attention of companies
that need your skills.

BSDCERTIFICATION.ORG
Providing psychometrically valid, globally affordable exams in BSD Systems Administration
 SEE
TEXT
O N LY

Improving the FreeBSD

Translation
The first question that
Tools
TM

comes up about translation is B y Wa r re n B l o c k

“Why bother?” Or maybe, to
put it more politely, is the effort
of translating worth the result?
A large portion of the world is
inhabited by people who could
benefit from using FreeBSD—and
can’t—because they don’t speak
English even as a second or third
S
econdly, translators are usually the ambassadors
language. There is a huge benefit into a new area, region, or country. They are not
just translators; they use whatever they have trans-
that we can bring to them. lated. If you can make the job easier for translators,
you can also increase the number of people who are
using your system in those areas.
18 Finally, you can enlarge the community. Metcalfe’s
16 Law, roughly paraphrased, states that the value of a
14
telecommunications network is proportional to the
number of nodes in that network. In other words, the
12
more nodes, the more valuable it is. With the Internet,
10 of course, if you had only two nodes on it, it would
8 not be that useful. But if you have a million, it’s much
more useful. And I say that FreeBSD is a telecommuni-
6
cations network, and people are the nodes in that net-
4 work. More users mean more people discovering bugs,
2 submitting patches, adding features, enhancing docu-
0
mentation, and all this increases the value of FreeBSD
2005 2006 2007 2008 2009 2010 2011 2012 2013 2014 2015 2016 for the very same people who are doing that work. It’s
a feedback loop. That is the value of translation.
Year New translations Year New translations
2005 ...............9 2011 ...............0
2006 ...............12
2007 ...............7
2012 ...............11
2013 ...............3
The FreeBSD Documentation
2008 ..............17 2014 ..............0 There are several different categories of FreeBSD docu-
2009 ...............8 2015 ...............6 mentation. There are the books and articles, which are
2010 ...............2 2016 ...............11

16 FreeBSD Journal
marked up in DocBook XML. That is not just the commit; then a second commit changes white-
famous FreeBSD Handbook, but also other books space—and only whitespace—to modify format-
like the Porter’s Handbook, which describes how ting. Whitespace-only changes to documents usu-
to port programs to FreeBSD. It’s a large, detailed, ally have a commit message that points out that
continuously updated book that is well worth a content has not changed and the document does
read. There are also a number of stand-alone arti- not need any new translation.
cles marked up with DocBook XML. Using Separating content and whitespace commits
DocBook XML gives us the ability to render that helps translators, but adds a serious amount of
source into a number of output formats like work for people editing the original documents.
HTML, PDF, ePUB, PostScript, or even plain ASCII. Only content or whitespace can be changed at a
Secondly, we have our man pages. Ten or single time, so documents become ragged and
twenty years ago, some of the open-source world poorly formatted. When working on the follow-
didn’t quite see the advantage of man pages, or up, whitespace-only change, errors in content are
didn’t understand it, and kind of took a wrong often found. These cannot be fixed immediately,
turn. But for the BSD community, this is a major but require another round of content change and
feature. A man page is not a tutorial; it’s meant to then yet another whitespace-only change. So,
be a quick reference for when the exact format of whitespace is a problem for pretty much everyone
a command or some configuration detail can’t be working on documentation.
remembered. You can look in the man page and
there it is. We have a huge resource in those man The Old Translation Method
pages, and they are continuously updated and
The traditional FreeBSD document translation
improved. They use the mdoc(7) markup lan-
method is simple in concept: translators work by
guage, mostly, and there are some old ones that
commits. Translators work through the changes
use roff or troff.
made to the English documents, translating those
There is other documentation. Part of the
changes into the target language.
FreeBSD Documentation Project’s domain includes
In practice, this method is difficult and time-
the source. If there is an error in spelling or gram-
consuming for translators. It is fully manual, giving
mar or clarity in source files, we are allowed to
no assistance to the translator. Quite the opposite,
edit those. This also holds for text files, or any file,
in fact: translators must work through commits in
really. That is the scope of the FreeBSD
order, essentially having no control over the size of
Documentation Project.
a change to the original document. A single sen-
tence might have changed, or a whole chapter
Whitespace rewritten. Until that change has been translated,
One surprisingly difficult problem we encounter subsequent changes cannot be translated, and
with translation is whitespace. These are the blank this can prevent other translators from working on
areas between text: spaces, tabs, line feeds, car- the document. The scale of a change is also a
riage returns, and a number of others. Being invisi- problem with volunteer translators, who might see
ble, these characters are hard to see, and this a large change as too much work for the time
makes them difficult to explain. The reason they they are willing to contribute.
matter in translation is that when a document is There is no formal method, but translation
edited, translators have no easy way to tell if the teams typically keep track of the latest translated
content changed and a new translation is required, version of a document with a comment noting the
or if only whitespace changed and the existing last commit number translated from the English
translation is still valid. A typical example is when document. This, too, is completely manual.
someone rewraps a paragraph to fit a particular To locate and work by commits requires a non-
line-length in an editor. The content is still the trivial familiarity with the version control system.
same, but that is difficult for translators to see Because translators are working directly with
without rereading the whole thing. At worst, the DocBook XML source files, a non-trivial familiarity
translator might retranslate the document unneces- with DocBook is also often required. The quality
sarily. That is very demotivating and a serious prob- and quantity of translations created with this old
lem when we depend on volunteer translators. method is a testament to the dedication of the
Whitespace changes have traditionally been translation teams.
managed in the same way as is usually done with After all this overhead, the translator has finally
source code. Changes to content are done in one located the next change to the English document

Jan/Feb 2017 17
 Translation Tools
that must be added to the translated version. neither as good nor as bad as people tend to
This change could be anything: new text added, imagine, but it can help reduce the translator’s
old text removed, or both. At some point, the workload by perhaps 5–15%.
translator will compare the previous translated The new translation method has only three
document with the diff file containing the steps:
changes. Because this method is fully manual,
there is no assistance in locating the place in the 1. Run ‘make po’ to extract translatable strings
document source where the changes occurred. from the English document. If a translation
Translations do not match the English documents already exists, it is preserved and updated with
line-for-line, so the translator is forced to locate strings that have changed.
the place where changes must be made. This can
end up taking a lot of time when the change to 2. Run a PO editor and enter translations next to
the English document was non-trivial. the English strings.
Finally, the translator can get to the one thing
they wanted to do: creating a translation of the 3. Run ‘make tran’ to build the translated version
English document in the target language. of the original document.
Prospective translators do not respond well to
this demotivating, labor-intensive workflow. That is the entire process, all of it. The transla-
tor does not need extensive knowledge of the
The New Translation version control system or DocBook markup. They
do not have to translate a particular change or
Method entire large changes. Instead, they can do as
Something had to change. In 2012, Thomas much or as little translation work as they want
Abthorpe and Benedict Reuschling were talking without impeding other translators. Automation
about a new translation method using "PO assists with the translation where possible, and
files." Even though I am a monolingual PO editors typically show how much of the docu-
American, this work was obviously very impor- ment has been translated. It is a huge change
tant, and I slowly learned about ways to use this from the old translation method and lets volun-
new translation method. In 2015, we finally had teer translators actually contribute rather than
a working system. driving them away.
The new translation method uses gettext, a
program developed so vendors would not have
to recompile programs so that their messages
Implementation
could be in a different language. Instead, a There are several programs that can extract trans-
Portable Object (PO) file was created that con- latable strings from XML files. The one chosen for
tained both the English strings and their trans- this project was itstool (http://itstool.org),
lated equivalents. After the user defined their because of its simplicity and standards compli-
locale, the strings were shown in the local ance. Some small utilities from the gettext-tools
language. package are also used. itstool requires Python,
At first glance, this does not seem like a sys- which is already a required dependency of the
tem that would lend itself to translating entire documentation port (textproc/docproj). The get-
documents, but it does turn out to work surpris- text-tools port (devel/gettext-tools) is also likely
ingly well for that. A program is used to extract already installed on a system used by a documen-
strings from the original English DocBook XML tation writer or translator, so the overhead of
source into a PO file. Translators use a PO editor these tools is very small.
to edit these files. The English string is shown,
and the translator enters the equivalent translat- Outcome
ed string next to it. There is a 1:1 correspondence The new PO translation system went live at the
between the original and the translation. The end of August 2015. In the remainder of 2015,
translator does not have to look through source there were six newly-created translations of arti-
files to determine where the changes must be cles and books into German, Dutch, Spanish,
made. Whitespace goes from being a serious Chinese, and Korean. This compares favorably to
problem to mostly not being an issue. Finally, the numbers for 2013, when there were only
there is a "translation memory" that offers to three new translations, and 2014, when there
reuse translations known from earlier uses. This is were no new translations at all. In 2016, there

18 FreeBSD Journal
have been another 10 new translations, including even showing them these strings. But we also
two very large translations of the Handbook and want a single source for this type of informa-
Porter’s Handbook into traditional Chinese. These tion. If these strings were translated, they would
numbers admittedly disregard the size of new be copied into the translated file, making multi-
translations and ongoing maintenance on existing ple copies of them and guaranteeing that some
translations. However, they demonstrate that sim- will always be out of date in the translated ver-
plified translation methods can be an enabling sion. So, we need a way to mark up the source
technology, and that given better tools, volun- to say "this string should not be translated."
teers are capable of producing large quantities of Ideally, the translator would never even see that
useful work. string. Instead of seeing 600 pages of XML
source, they would only see the two sentences
Challenges of introduction.
Searches for standard ways of marking strings
We do still have some challenges, and I call these
that should not be translated did not result in any
"challenges" because when you have a problem,
obvious way of accomplishing that. There were a
it’s just a problem. When you have a challenge,
few organizations that had used toolchain-specif-
you are challenged to come up with a solution.
ic methods that appeared to not be applicable to
We have things that should not be translated.
our documentation. A messy hack of using XML
A prime example is our article that contains
processing instructions might be workable with
developer PGP encryption keys. That is all it con-
our toolchain, but a better way is still preferable.
tains, two sentences of introduction (essentially,
The search is still on, and suggestions are always
"Our developers have PGP keys. Here they are.")
welcome.
and then 600 pages of PGP keys. These are just
Another challenge is preserving the huge
numbers. We don’t want these translated,
amount of work put into translations that were
because the translated form of them is identical
created with the old translation method. Some of
to the original. It wastes the translator’s time by

Premier VPS Hosting

RootBSD has multiple datacenter locations,
and offers friendly, knowledgeable support staff.
Starting at just $20/mo you are granted access to the latest
FreeBSD, full Root Access, and Private Cloud options.

www.rootbsd.net
Jan/Feb 2017 19
 Translation Tools

these are very current, and we want to preserve fully translated set of FreeBSD man pages would
as much of that work as possible. Ideally, we be incredibly valuable. With the barriers to trans-
would convert those translations into PO transla- lation lowered by the PO system, this becomes
tions without losing any of that work. The possible.
GNOME xml2po program can take an English New translators often become FreeBSD docu-
original document and a fully translated version mentation contributors and committers. PO
and produce a PO file from them. However, the translation offers an easy way to begin contribut-
two documents must correspond line to line ing and an introduction to the community which
exactly. But our translation teams have their own can lead to increased participation.
rules for line wrapping, so the original and trans- New translations of documentation will bring
lation do not match. There might be other pro- in new users from new regions and countries,
grams out there to do this based on matching offering their own unique perspective on prob-
the XML elements of the two files. It is likely that lems and opportunities for the community. Some
not much effort has been put into conversion of those people will continue on to become con-
programs because conversion for most organiza- tributors, making FreeBSD better for everyone
tions is a one-time occurrence. who uses it.
Documentation translators would benefit from
seeing a few lines of text from before and after What We Learned
the source string to get an idea of the context in
The journey to make PO translation tools usable
which it is used. Many PO editors do not show
on FreeBSD taught us some related things.
much context. The original use of gettext was for
The value of cross-pollinating with other proj-
translating program prompts where there was lit-
ects should not be underestimated. Some of the
tle or no context to show. With documentation,
most valuable insights were gained when Ryan
that is very different, and the job of translating is
Lortie of the GNOME project and I were talking
easier if surrounding context is shown. This is not
before or after presentations at BSDCan.
a technical problem, because the PO files already
Different projects have different experiences,
have a comment with each line showing its line
and two or more projects can benefit by sharing
number from the original XML file.
their knowledge. We should not look at other
There are many PO editors, but only a few
projects as competitors, but as rich gold mines
have been ported to FreeBSD. Currently, there
of information.
are three: editors/poedit, devel/gtranslator, and
The old translation method is essentially a
devel/lokalize. Having more PO editors in ports
commit-by-commit porting effort, porting an
will make it easier for translators to choose an
English document to another language. When
editor that is well-suited to translating the specif-
the X11 team asked about commit-by-commit
ic document or target language. A nearly func-
versus file-by-file porting, we were able to identi-
tional port of the well-regarded Virtaal (http://
fy some of the problems inherent to the commit-
virtaal.translatehouse.org/) is available and prob-
by-commit method:
ably does not need much more effort to com-
plete. Java-based PO editors are available, and • commits
commits must
must be
be done
done in
in order,
order, even
even ifif aa criti-
the web-based online Pootle system is also in critically important
cally important later later
commitcommit is needed
is needed
ports as textproc/pootle. PC-BSD had used Pootle
for their translations. PC-BSD has now been • usually
usually the
the porter
porter isis forced
forced to
to work
work in
in units
units of
renamed TrueOS, and currently uses Weblate
anofentire
an entire source
source commit,
commit, regardless
regardless of how
of how large
(https://weblate.org/en/) for web-based transla-
itlarge
mightit have
mightbeen,
haveand
been, theand the project
project can be can
tion. There are also commercially hosted sites be stalled
stalled whilewhile one porter
one porter struggles
struggles with with a
a large
with similar operation, like Transifex large commit
commit
(https://www.transifex.com/), which is free for
open-source use. • even
even though
though earlier
earlier commits
commits can
can be
be entirely
entirely
erasedbybylater
erased laterones,
ones,the
thework
worktotoport
portthem
them
Potential muststill
must stillbebedone
donebecause
becauselater
latercommits
commitsdepend
The most obvious and compelling possibility with depend
on on their presence.
their presence.
PO-based translations is translation of the
FreeBSD man pages. While itstool is strictly for Technical debt can be a serious problem.
XML files, the Debian po4a package is capable of When we tell people how we include chapters in
extracting strings from man pages to PO files. A XML files by defining them as entities, and they

20 FreeBSD Journal
pause and say, “We didn’t think anyone was still
doing it that way,” it is a warning sign. Even if Thomas Abthorpe Koop Mast
we follow standards, there is a danger that the Glen Barber Shaun McCance
new tool might not support the old methods. Mark Felder Chris Petrik
Ultimately, this could leave documentation or Hiroki Sato Benedict Reuschling
translation work stranded while new methods are René Ladan ...and many others.
implemented. Keeping up-to-date with industry
Ryan Lortie
standards is usually less painful if it is done grad-
ually rather than as a sudden, urgent need.
Related to the previous point about technical Links
debt: our documents need to be switched to
Material discussed in this article, including the
UTF-8. It is long past time, and I have written a
presentations, scripts used to collect statistics,
program to do this. All that remains is the hard
graphs, and the Virtaal port, is here:
work of verifying that the conversion works
correctly.
http://wonkity.com/~wblock/translation/
Acknowledgments
The work of bringing the PO translation tools to WARREN BLOCK has been using FreeBSD since
the FreeBSD documentation has been under way 1998 for such diverse things as servers, documen-
for several years. Many people have contributed, tation creation, network monitoring, and enterprise
both from within and outside the FreeBSD computing. He has been a FreeBSD documentation
Project. This list is not complete, although anyone committer since 2011 and a member of the
missing is due to forgetfulness on my part. My
Documentation Engineering team since 2014. In
sincere thanks to everyone for their contribution
to making FreeBSD more available to people 2016, he began working on the FreeNAS documen-
around the world! tation for iXsystems Inc.

Jan/Feb 2017 21
 SEE
® TEXT
INTERACTING with the FreeBSD Project O N LY
by Anne Dickison

The Foundation’s Global Efforts

he FreeBSD Foundation is proud to serve Linux Distro” to around 400+ attendees. Next,

T a welcoming international community.

Part of our mission is to support and
grow FreeBSD awareness throughout the
world. To that end, our Board of
Directors is made up of members based around
the globe, and they are dedicated to helping us
spread the word. One of the ways we do this is by
Foundation Board Director Hiroki Sato presented
an “Introduction to FreeBSD” to around 300
attendees at COSCon 2016: The China Open
Source Conference. This was the first annual con-
ference sponsored by the open-source organiza-
tion Kaiyuanshe. Following the presentation, 30
people joined Hiroki in a WeChat group for BSD.
sponsoring and attending conferences. In addition In addition, Director Robert Watson delivered a
to sponsoring BSD-related events such as presentation on “Cambridge L41: Teaching
EuroBSDcon, AsiaBSDcon, BSDCan, and the Advanced Operating Systems with FreeBSD”
Cambridge FreeBSD Developers Summit (BSDCam), through audio conference, and FreeBSD src com-
the Foundation also participates in non-BSD events mitter Yanmin Qiao presented “FreeBSD and BSD-
throughout the world, including womENcourage in based Virtual Appliance in Microsoft Azure” at a
Austria, Chemnitz Linux Days in Germany, FOS- BSD meetup in Shanghai. Finally, approximately
DEM in Brussels, and many more. 45 people attended a BSD meetup in Beijing.
Last year, with the help of new board member George Neville-Neil again delivered his “FreeBSD
Kylie Liang, we were able to begin extending our is Not a Linux Distro,” Zheng Fu of Array
reach to new regions. Kylie arranged for mem- Networks shared “FreeBSD development @
bers of the Foundation Board to present at a Array,” and Yanmin Qiao talked about “FreeBSD
number of Chinese conferences and meetups to and BSD-based Virtual Appliance in Microsoft
increase the awareness and adoption of FreeBSD Azure.” More on our recent efforts in China
in China. First, at OSC 2016: The Annual Open can be found in our December 2016 newsletter:
Source Conference, Foundation Board Director https://www.freebsdfoundation.org/wp-content/
George Neville-Neil presented “FreeBSD is Not a uploads/2016/12/FreeBSD-Foundation-December

Not only does the

Foundation’s own team
attend events, but our
TRAVEL GRANT
PROGRAM
allows for other community
members to do so as well.

Photo by Ollivier Robert

22 FreeBSD Journal
2016-Update.pdf
This year, we’re working with new board mem-
ber Philip Paeps to bring a greater FreeBSD pres-
ence to India by sponsoring Rootconf 2017, taking
place in Bangalore in May. In addition to having a
7
developer’s room at FOSDEM in February, Director
Benedict Reuschling will be heading up a booth as
part of the event’s exhibition, allowing us to better
introduce the FOSDEM attendees to FreeBSD.
George and Benedict will be teaching a two-week
FreeBSD OS class in Darmstadt, Germany, and
Benedict will also be teaching a course in Oulu,
Finland.
Not only does the Foundation’s own team attend
events, but our Travel Grant Program allows for
to AsiaBSDCon 2017!
other community members to do so as well. The
program is designed to both bring members of the
FreeBSD community face-to-face for further develop-
ment of the Project, and to spread the word about
FreeBSD. Travel grants are available to community
members who need assistance with expenses to
attend conferences related to FreeBSD development
March 9–12, 2017
and advocacy. Due to the limited size of the
Foundation board and staff, we rely on the FreeBSD
community to attend the events that we cannot.
You can find out more about our travel grant pro-
gram here: https://www.freebsdfoundation.org/
what-we-do/grants/travel-grants/. Tokyo University of Science
Attending events or speaking at meetups are
just some of the ways the FreeBSD Foundation Tokyo, Japan
helps to increase global awareness of the FreeBSD
Project. But, we can’t do this without you. If there’s https://2017.asiabsdcon.org
an event you’d like to present at but are not sure
how to go about it, please let us know. We can
help. If you’re looking for a speaker for your local
meetup, we can help with that too. We’ve provid-
ed PDFs of our advocacy materials here:
https://www.freebsdfoundation.org/what-we-
do/education-advocacy/.
We can also provide raw files should you be
interested in getting them translated into your pre-
ferred language. Finally, if you are a FreeBSD user in
an area you feel lacking FreeBSD advocacy, contact
us ([email protected]).
Thank you for your continued support of the
FreeBSD Foundation. •

Anne Dickison is the Marketing Director for the

FreeBSD Foundation. She spent over 15 years in
technology-focused marketing and communica-
tions enjoying roles in biotech, publishing, and
the nonprofit industry. When not promoting
FreeBSD and the importance of open-source
technologies, you’ll find her enjoying game
nights with friends and family and chasing after
her wee dog Finnegan.

Jan/Feb 2017 23
 SEE
TEXT
O N LY

new faces
of FreeBSD BY DRU LAVIGNE

This column aims to shine a spotlight on contributors who recently

received their commit bit and to introduce them to the FreeBSD
community. This month, the spotlight is on Nikolai Lifanov (ports)
and Konrad Witaszczyk (src), who became committers in
November 2016, and Larry Rosenman and Jean-Sébastien Pédron,
who both received a ports commit bit in January 2017.

Tell us a bit about yourself, your back- PostgreSQL, and others. My hobby and my
ground, and your interests. employment coincide, as computers are my
hobby. I also do volunteer work for the
Nikolai: I am originally from Moscow but American Red Cross on their Disaster Services
have made a life for myself in North Carolina. Technology and IT End-User-Support teams.
I’m a systems/ops guy by trade. I enjoy collect-
ing computers and playing mostly open- Jean-Sébastien: Hi! I’m
source rogue-like games. I also enjoy chess Jean-Sébastien Pédron, 36
and origami. years old. I come from a
small village in Brittany, in
Konrad: My name’s Konrad Witaszczyk. I’m the west of France. I now live
25 years old and I’m a software developer at in Paris where I moved to
Wheel Systems where I work on Fudo PAM, a work in IT in 2000. While in
product based on FreeBSD. I studied Brittany, I became interested in computers
Theoretical Computer Science at the about the age of 14 or 15. Before that, I was
Jagiellonian University. Currently, I live in spending (and still do) my spare time playing
Copenhagen, but in a few months I’ll be back music (organ and percussions), fishing for
in Warsaw. My main interests are algorithms, carp, and observing planets and the deep sky
cryptography, and security. in my local astronomy society. Now that I’m in
Paris, astronomy or fishing has proved to be
more difficult to do, so I started to practice
Larry: I’ve been a computer “nerd” since I rollerskating to compensate.
was in 10th grade in high school, which was My nickname, “Dumbbell,” comes from the
1971. I’ve worked on all classes of computers following: Dumbbell, or M27 (27th object in the
(mainframes, minicomputers, and now micro- Messier catalog), is a nebula which is among
computers) up and down the entire software the first deep sky objects you learn to locate
stack. I’m 59 years old, married, with one and observe in the Northern Hemisphere. The
married 24-year-old daughter. As of February first time I tried to find it on my own with a tel-
13, 2017, I’ll be working for MobileIron, a escope, I needed an hour and a half, the crappi-
mobile device management company. Prior to est performance of all time. The second time, it
this move, I worked for Alcatel-Lucent, which took me seconds, which is way more common.
was bought by Nokia in 2016. I’ve also Ever since that time, the other members gave
worked for IBM, Lombardi Software, Surgient, me that name and I kept it outside of the
Pervasive Software, and others in various astronomy society as well.
places around the U.S. I’m a big fan of open- When my family bought a computer, an
source software, including FreeBSD, IBM powered by an Intel 8086 with a 20 MiB

24 FreeBSD Journal
hard disk and a 3.5-inch floppy disk, I started to and the ports system all being engineered as a
play with DOS and Basic. But I didn’t do a lot. It’s cohesive system, unlike Linux where it’s which dis-
only when I began university in 1998 that I started tribution, which version of which library, etc.
to learn things. It’s not at the university that I actu- Jean-Sébastien: I don’t remember exactly when I
ally learned computer science, but with two first heard about FreeBSD. I think it was through
friends I met there who were already deep into either Linux Mag France (a paper magazine) or
computer science and the Free Software world. http://linuxfr.org. I tried FreeBSD 4.6 on my home
That’s the time I was introduced to C program- server. It maintained my Internet connectivity and
ming and Slackware Linux. hosted various services such as a website, a mail
service, or a DNS. The system was so easy to use
How did you first learn about FreeBSD and and comfortable with the unique /etc/rc.conf con-
what about FreeBSD interested you? figuration file, and had such great documentation.
Nikolai: I learned about FreeBSD from a poll on a The Ports system was great, and I wasn’t afraid of
Linux forum about which Linux distribution people compiling applications because I was still using
like best. FreeBSD was one of the options, and Slackware Linux on my workstation. I really liked
someone helpfully mentioned that FreeBSD is not FreeBSD. In early 2004, my workstation died.
Linux. I tried it out and it took me long enough to When I replaced it, I decided to give FreeBSD a try
get my wireless card working to appreciate the as a desktop. The problem was that my previous
unity of base system and documentation, the puri- disk was formatted using ReiserFS and there was
ty and simplicity of the BSD license, which focuses no way to read that on FreeBSD. As I couldn’t
on protecting the code rather than the product, have a second computer to have Linux and
and that the community as a whole is more capa- FreeBSD in parallel, I had this crazy idea of adding
ble and excited than the Linux community. Perhaps ReiserFS support to FreeBSD. Even today, I don’t
unique to FreeBSD, developers have a wide range understand how I could think this was the best
of skills and collectively care about the quality of solution to get back my data from that disk. But I
the product as a whole. Have you written a cool eventually ported that code.
utility, but mandoc markup needs some love?
There is probably someone who can help with
that. This is a sharp contrast to Linux systems
where, for example, iproute2 maintainers may not
care what the device mapper stack looks like.
Konrad: I first heard about FreeBSD from my
brother. He experimented with various UNIX-like
“ FreeBSD is a great project to learn
things. It covers many, many topics in a single
big project. It is both very advanced and still
has room for improvement in many areas.”
operating systems during his studies. Once he —JEAN-SÉBASTIEN
bought some magazines that included FreeBSD 5.4
CDs and decided to use it as his desktop. By the
time I got interested in computers, he was running
FreeBSD 7.0 on his laptop. I was 16 back then and How did you end up becoming a committer?
really liked that he was managing his OS using a Nikolai: I started out with a few bug reports here
console and that he searched and installed pro- and there. Sometime later, I took the games/wtf
grams using ports. The directory structure was port under my care. It comes from the NetBSD
clear and the documentation was easy for me to base system and I over-engineered a process that
read. I liked that, more or less, a user knew what crawls the NetBSD CVS repo and uploads changes
was happening in its OS instead of using a black to SourceForge. I scaled up my contribution from
box. Since then I’ve been using FreeBSD as my there, and 600-something PRs later, Matthew
desktop and server. Seaman agreed to take me under his mentorship.
Larry: I learned about FreeBSD back during the Konrad: During my bachelor’s degree I had some
Internet .com boom/bust in the late 1990s. I was courses about OS development. They were
intrigued by the engineering of kernel, userland, based on Minix and Linux, and I enjoyed them a lot.

Jan/Feb 2017 25
 I think I was lucky to have such good teachers. As mitters! In October 2004, Scott Long (scottl@)
a FreeBSD user, I wanted to contribute to this OS, offered me an src commit bit, and Maxime
but I didn’t know where to start. In 2013, I decided agreed to mentor me. I accepted and joined the
to apply for Google Summer of Code. Having a Project in November 2004. I finished the work on
mentor who could help you find the way to solve a read-only ReiserFS and committed it to the tree.
problem sounded great. It was not easy for me to It was first published in FreeBSD 6.0-RELEASE.
come up with something that I could do for Since then, I have done a few things in
FreeBSD. Helpfully, FreeBSD had a wiki page with FreeBSD: improvements to the Synaptics touch-
ideas for GSoC students. I picked some of them pad support in psm(4) and some bug fixes here
and asked Edward Tomasz Napierała (trasz@) for and there, including the Ports tree. It’s only since
his opinion. Finally I sent a proposal for Unattended January 2013 that I started to contribute serious-
Encrypted Kernel Crash Dumps. It was accepted ly again. My laptop had a Mobility Radeon HD
and I was mentored by Gleb Kurtsou (gleb@). The 5870 GPU which was barely supported and it
same year, I went to Malta for EuroBSDcon as a was getting worse with the drop of “user mode-
GSoC student to give a short talk about my proj- setting” for AMD GPUs. I contacted Konstantin
ect. The first day, I met other FreeBSD developers, Belousov (kib@) and Alexander Kabaev (kan@) to
including Paweł Jakub Dawidek (pjd@) during a see if and how I could port the Radeon kernel
DevSummit. I found them extremely welcoming driver from Linux to FreeBSD. I finished that port
and helpful. A half year later, Paweł asked me if I around August 2013.
would like to work with him on Fudo. The idea of In the process, I became more involved with
playing with FreeBSD during my free and work the graphics stack ports: mainly the X.Org server
time seemed great. and Mesa. I worked with Koop Mast (kwm@)
During the next three years, I followed the and Niclas Zeising (zeising@). They guided me
FreeBSD Project and attended FreeBSD into the Ports tree. With all those newly acquired
DevSummits. Last year, I finally finished encrypt- skills, I submitted more contributions to the Ports
ed kernel crash dumps and Paweł decided to tree—outside the graphics stack area. I even took
propose me for a source commit bit. The core maintainership of graphics/darktable, a digital
approved it, and I could commit EKCD myself. photo development application. And I submitted
many updates to the ports packaging tools from

“
my current employer, Pivotal (tools around
If you want to become a committer, Cloud Foundry, Bosh, and Concourse) and
created the lang/rust-nightly and
submit high-quality PRs, be responsive to devel/cargo (Rust package manager and
PRs against ports you maintain, and pick up build tool) ports. In January 2017, Baptiste
Daroussin (bapt@) and Antoine Brodin
unmaintained ports to maintain. ” —LARRY
(antoine@) offered to mentor me for a
ports commit bit.
Larry: I’ve been maintaining sysutils/lsof and
mail/dovecot2-pigeonhole for a while, as well as How has your experience been since joining
contributing lots of PRs with bugs and issues, the FreeBSD Project? Do you have any advice
including a few release showstoppers. I’ve been for readers who may be interested in also
in the community for a long time, and Adam becoming a FreeBSD committer?
Weinberger suggested I become a committer Nikolai: I became more active since joining the
after he and I worked on the latest Dovecot ver- FreeBSD Project. There is no shortage of useful
sion together—as we worked well together. work to do and my involvement became an
Jean-Sébastien: At the time I ported ReiserFS, I always-present way to spend any chunk of time.
was working for a French telco, Cegetel. When I Have an hour to spare? Pick up some PRs and
posted the patch to freebsd-current@, two work on them! Recognizing this, it’s fun to find
Cegetel coworkers came to me, surprised: contributors with neglected patches and keep
Maxime Henrion (mux@) and Olivier Houchard them hooked and on track to contribute more.
(cognet@). I was as surprised as they were The pain point of contributing to a project run by
because I had no idea they were FreeBSD com- volunteers (or it has been for me) is that PRs

26 FreeBSD Journal
might sit neglected for months at a time. The The FOSS world taught me everything I know
trick is to not get discouraged and to find and about computers and influenced my life outside
prod people who might have spare cycles to of IT in a positive way. I’m thankful I can pay my
work on these and get them committed. debt today.
Konrad: During the last four years, I had the The advice I would give is to contribute to
pleasure of using or working with audit, bhyve, whatever area and topic that interests you. The
Capsicum, jails, pkg, ZFS, and more, and I also apparent “difficulty” of the task is unimportant.
met a lot of nice people during six FreeBSD There are people more than knowledgeable who
DevSummits. If you want to become a FreeBSD will help you. Don’t hesitate to discuss. You will
committer, do the same. Meet people, even via learn whatever you need and you’ll meet great
FreeBSD mailing lists, and run FreeBSD as your people. So don’t be afraid! Just go!
desktop to find interesting tools that you could FreeBSD is a great project to learn things. It
develop yourself. covers many, many topics in a single big project.
It is both very advanced and still has room for
Larry: If you want to become a committer, sub- improvement in many areas. If at some point you
mit high-quality PRs, be responsive to PRs find networking is boring, you have the opportu-
against ports you maintain, and pick up unmain- nity to work on something else. I didn’t know
tained ports to maintain. I’ve only been a com- the graphics stack, for example. •
mitter for three weeks as I write this, so I don’t
have much more experience.
Jean-Sébastien: I enjoy being part of the DRU LAVIGNE is a doc committer for the
FreeBSD Project every day, as being able to con- FreeBSD Project and Chair of the BSD
tribute to a critical project is very rewarding. Certification Group.

TM TM

The Browser-based Edition

(DE) is now available for
JOURNAL viewing as a PDF with
printable option.
NEiW
ng &
The Broswer-based DE format offers sub-
View ble
O
ver time, Unix operating systems have evolved a variety of
mechanisms to launch programs and influence their execution.
scribers the same features as the App, but
Printa ns
To put this into perspective, let’s take a quick tour of the dif-
ferent kinds of programs that run on a typical FreeBSD system.

Optio When the computer first boots up, the

rc(8) mechanism launches programs called
“services” that can perform one-time system
age falls below a certain threshold.
There are many other mechanisms for
launching programs. Inetd(8) launches
permits viewing the Journal through your
initialization tasks, or execute daemons to programs when an incoming socket is creat-

favorite browser. Unlike the Apps, you can

run in the background. These services can ed. nice(1) runs programs with a modi-
also be stopped and started while the com- fied priority level. chroot(8) launches pro-
puter is running. grams with an alternate root directory.
Every minute the computer is running, the service(8) launches programs with a
cron(8) daemon wakes up and launches “supervisor” process that can restart the
scheduled tasks. Cron also supports the abil-
ity to run at(1) jobs after a certain time
interval has passed. It can also run
batch(1) jobs when the system load aver-
program if it crashes. Jail managers like
jail(8) launch programs within a jailed
environment. Virtual machine managers like
iohyve(8) launch programs that execute a
view the DE as PDF pages and print them.

To order a subscription, or get

back issues, and other
$1
The DE, like the App, is an YEAR
9.99 Foundation interests, go to
SU
individual product. You will $6 99 B
get an email notification each S
.
INGL www.freebsdfoundation.org
E COP
time an issue is released. Y

Jan/Feb 2017 27
 SEE
TEXT
O N LY

BOOK review by Steven Kreuzer

GROKKING ALGORITHMS
An illustrated guide for programmers and
other curious people
......................................................................By Aditya Y. Bhargava
Publisher.............................................Manning Publications (2016)
Print List Price ...........................................$44.99 (including digital)
Digital List Price .......................................................................$35.99
ISBN ..........................................................................9781617292231
Pages ............................................................................................256

I
purchased a copy of Grokking Algorithms the only way to make this material accessible
on a whim based on a comment I happened was to have someone with a strong back-
to stumble upon on some random post on ground in math and computer science help
Hacker News. Because I don’t have a comput- guide you through.
er science background, from time to time I like What I truly love about this book is that the
to challenge myself with small programming author does a fantastic job of taking material
exercises, and I figured this book might have that on the surface can seem complex and
some tips and tricks that I could use down the overwhelming to the casual reader, and making
line. When my copy finally arrived, I started to it extremely accessible regardless of your techni-
skim through it, and right in the first chapter I cal background. This is accomplished by taking
found a much better solution to a program- real-world problems and mixing them in with
ming problem I had been working on. I sat plenty of illustrated explanations. The result is
down at my computer, replaced my code to very unique and engaging, especially if you
search through an array with the simple binary consider yourself to be a visual learner.
search algorithm described in the book, and Throughout the book the example code pro-
finally managed to get this small application to vided for each algorithm is presented in Python,
run fast enough that it wasn’t getting killed but the implementations are easy enough to
due to excessive execution time when I submit- translate to whatever language you would pre-
ted the code to the programming challenge fer to work in. Considering the popularity of
website for grading. I was thrilled when I finally the algorithms the author chose, I am sure that
got a green checkmark next to each of the test examples in your preferred language can be
cases, and I was immediately hooked on this easily found. Because of that, instead of just
book. explaining the algorithm, the author spends
When you compare Grokking Algorithms most chapters detailing the trade-offs you
to other books on the same subject, the first would encounter with each algorithm and
thing you will notice is how thin this book is focuses more on showing you techniques for
when you compare it with all the others. This is problem solving. As a result, a large chunk of
by design, because the goal of this book is to Chapter 1 is spent helping the reader get a bet-
be an easy on-ramp to learning more about ter understanding about the running time of
algorithms and to help dispel the myth different algorithms. Big O notation has always
that algorithms are an obscure and complex been a little confusing to me, and so the parts
subject. In the past, these types of topics were of this book that provide the most value to
only covered in thick and heavy textbooks, and me are the seven pages covering algorithm run-

28 FreeBSD Journal
time using very simple explanations and illustra- “Very unique and engaging,
tions. The author also included a practical example especially if you consider
on how you could visualize Big O runtimes with a yourself to be a visual learner.”
few pieces of paper and a pen.
Another interesting feature of this book is that
if you buy a hard copy of it, the publisher will of this book. While Grokking Algorithms is a
throw in a free digital copy as well. For a variety of quick read—you can get through it in an after-
reasons, I still prefer physical copies of books, noon—I have a strong feeling this will be one of
especially ones on technical topics. However, from those books that you will keep around in your col-
time to time you might find yourself at home lection and pull off the shelf for a quick refresher
wanting to look up something in a book you left every once in a while. While I think I will get plenty
at work, or you just might want to give your back of value out of this book as the years go by, what
a break, and that is when having a digital copy most excites me is anticipating when my daughter
comes in handy. With most other publishers, the is old enough to take my dog-eared copy so that
cost of the physical and digital copies is almost the she can work though the chapters in the book,
same, which makes it hard to justify buying both, take that knowledge, and apply the same solutions
and fortunately with Manning Publications, you to problems she might be working on. •
don’t have to make that tough decision. Hopefully,
this is something we will see other publishers start
STEVEN KREUZER is a FreeBSD Developer
to do as well.
If you are new to data structures and algorithms and Unix Systems Administrator with an
and would like to learn more or just want to expe- interest in retro-computing and air-cooled
rience a very fresh and unique perspective on Volkswagens. He lives in Queens, New
them, I strongly encourage you to pick up a copy York, with his wife, daughter, and dog.

ZFS experts
By Brooks make
Davis, Robert Norton, Jonathantheir
Woodruff &servers
Robert N. M. Watson

Now you can too. Get a copy of.....

Choose ebook, print or combo. You’ll learn:
• Use boot environments to make the riskiest sysadmin
tasks boring.
• Delegate filesystem privileges to users.
• Containerize ZFS datasets with jails.
• Quickly and efficiently replicate data between machines
• Split layers off of mirrors.
• Optimize ZFS block storage.
• Handle large storage arrays.
• Select caching strategies to improve performance.
• .Manage next-generation storage hardware.
• Identify and remove bottlenecks.
• Build screaming fast database storage.
• Dive deep into pools, metaslabs, and more!

Link to:

WHETHER YOU MANAGE A SINGLE SMALL SERVER OR INTERNATIONAL DATACENTERS,

SIMPLIFY YOUR STORAGE WITH FREEBSD MASTERY: ADVANCED ZFS. GET IT TODAY!

Jan/Feb 2017 29
 SEE
TEXT

svn UPDATE
O N LY

by Steven Kreuzer
•
This past year was an incredible one for the FreeBSD Project. Over 10,000
commits were made to src, the ports tree is getting close to 27,000 pack-
ages, 23 new developers were granted commit bits, and both FreeBSD 10.3
and 11 were released. For a multitude of reasons, I have a strange feeling
that when we look back at 2016, the word “interesting” will be used quite
a bit, and the FreeBSD Project was not immune. While activity in base
tapered off a bit in December, keeping with the theme, I did see a few
commits that I would describe as “interesting.”
clang, llvm, lldb, compiler-rt and libc++ Support for Ingenic XBurst JZ4780
have been upgraded to 3.9.0, and lld and X1000 systems on chips (https://
has been imported (https://svnweb.freebsd. svnweb.freebsd.org/changeset/base/308857).
org/changeset/base/309124).

T he most exciting part of this commit is that it

also brings lld into the base system, which is
I ngenic Semiconductor designs CPU
Microarchitectures based around the MIPS
instruction set. This change introduces support for
a major milestone for the llvm project. It is the first the Imgtec CI20 and Ingenic CANNA single board
release that can link real-world large userland pro- computers.
grams, including LLVM/Clang/LLD themselves. In
fact, it can now be used to produce most userland Capsicumize some trivial stdio programs
programs distributed as part of FreeBSD. (https://svnweb.freebsd.org/changeset/base/
308432).
Add WITH_LLD_AS_LD build knob (https://
svnweb.freebsd.org/changeset/base/309142). W e are starting to see more and more applica-
tions take advantage of Capsicum, a light-

I f set, it installs LLD as /usr/bin/ld. LLD (as of

version 3.9) is not capable of linking the
world and kernel, but can self-host and link
weight OS capability and sandbox framework. A
recent commit added capsicum support to
the echo, sleep, basename, dc, dirname, fold,
many substantial applications. GNU ld continues getopt, locate, logname, printenv, and yes, since
to be used for the world and kernel build, regard- these userland utilities only interact with stdio.
less of how this knob is set. It is on by default for
arm64, and off for all other CPU architectures. Performance improvements for pw
operations that edit /etc/group or /etc/
Use buffer pager for NFS (https:// passwd (https://svnweb.freebsd.org/
svnweb.freebsd.org/changeset/base/308980). changeset/base/308806).

T he pager, due to its construction, implements

clustering for the page-ins. In particular,
buildworld load demonstrates reduction of the
r 285050 fixed a bug in pw that could lead to
/etc/passwd or /etc/group corruption on
power loss. However, it was fixed by opening
READ RPCs from 39k down to 24k. No change in those files with O_SYNC, which is very slow, espe-
real or CPU time was observed. cially on ZFS. This change replaces O_SYNC with

30 FreeBSD Journal
appropriately placed fsync()s instead, which is much
faster. Using a ZFS tmpdir, the time to run pw's
kyua tests drops from 245s to 35s.

Concurrency Kit is now part of the base

system (https://svnweb.freebsd.org/
changeset/base/309266).

C K is a BSD-licensed toolkit providing

concurrency primitives, safe memory reclama-
tion mechanisms, and non-blocking data structures
for the research, design, and implementation of
high performance concurrent systems.
TM
TM

Additional updates in contrib/ JOURNAL

• Subversion has been updated to version 1.9.5
(https://svnweb.freebsd.org/changeset/base/309356)

• ntp has been upgraded to version 4.2.8p9

(https://svnweb.freebsd.org/changeset/base/308957) WRITE FOR US!
• ACPICA has been upgraded to 20161117
(https://svnweb.freebsd.org/changeset/base/308953) Contact Jim Maurer
• am-utils has been updated to version 6.2 ( [email protected])
(https://svnweb.freebsd.org/changeset/base/308493)
with your article ideas.
• jemalloc has been updated to version 4.3.1
(https://svnweb.freebsd.org/changeset/base/308473)

• file has been updated to version 5.29

(https://svnweb.freebsd.org/changeset/base/308420)

• tzdata has been updated to version 2016i

(https://svnweb.freebsd.org/changeset/base/308270)

• libarchive has been updated to 3.2.2 (https://

svnweb.freebsd.org/changeset/base/307861)

STEVEN KREUZER is a FreeBSD

Developer and Unix Systems Admin-
istrator with an interest in retro-comput-
ing and air-cooled Volkswagens. He lives
in Queens, New York, with his wife,
daughter, and dog.

Jan/Feb 2017 31
 THROUGH MARCH 2017 BY DRU LAVIGNE

Events Calendar
The following BSD-related conferences will take
place in March 2017.

SCALE • Mar 2 – 5 • Pasadena, CA

https://www.socallinuxexpo.org/scale/15x/ • The 15th annual Southern
California Linux Expo will once again provide several FreeBSD-related presenta-
tions and a FreeBSD booth in the expo area. This event requires registration at a
nominal fee.

AsiaBSDCon • Mar 9 – 12 • Tokyo, Japan

http://2017.asiabsdcon.org/ • This is the annual BSD technical conference for
users and developers on BSD-based systems. It provides several days of workshops,
presentations, a Developer Summit, and an opportunity to take the BSDA certifica-
tion exam in either English or Japanese. Registration is required for this event.

Uranium

Thank you! Iridium

The Koum Family
Anonymous

Platinum
The FreesBSD Foundation would like to
acknowledge the following companies for
their continued support of the Project.
Because of generous donations such as
these we are able to continue moving the Gold

Project forward.
TM

TM
Silver

Are you a fan of FreeBSD? Help us give back to the

Project and donate today! freebsdfoundation.org/donate/

Please check out the full list of generous community investors at

freebsdfoundation.org/donate/sponsors

32 FreeBSD Journal