Data Science at the Command Line: Obtain, Scrub,

Explore, and Model Data with Unix Power Tools

2nd Edition Jeroen Janssens pdf download

https://ebookmeta.com/product/data-science-at-the-command-line-
obtain-scrub-explore-and-model-data-with-unix-power-tools-2nd-
edition-jeroen-janssens/

Download more ebook from https://ebookmeta.com

We believe these products will be a great fit for you. Click
the link to download now, or visit ebookmeta.com
to discover even more!

Cambridge IGCSE and O Level History Workbook 2C - Depth

Study: the United States, 1919-41 2nd Edition Benjamin
Harrison

https://ebookmeta.com/product/cambridge-igcse-and-o-level-
history-workbook-2c-depth-study-the-united-states-1919-41-2nd-
edition-benjamin-harrison/

Python Data Science Handbook: Essential Tools for

Working with Data, 2nd Edition Jake Vanderplas

https://ebookmeta.com/product/python-data-science-handbook-
essential-tools-for-working-with-data-2nd-edition-jake-
vanderplas/

Python Data Science Handbook: Essential Tools for

Working with Data, 2nd Edition Jake Vanderplas

https://ebookmeta.com/product/python-data-science-handbook-
essential-tools-for-working-with-data-2nd-edition-jake-
vanderplas-2/

The Rough Guide to Andalucía 10th ed 2023 10th Edition

Rough Guides

https://ebookmeta.com/product/the-rough-guide-to-andalucia-10th-
ed-2023-10th-edition-rough-guides/
Doing Good in the World The Inspiring Story of The
Rotary Foundation s First 100 Years 1st Edition David
Forward

https://ebookmeta.com/product/doing-good-in-the-world-the-
inspiring-story-of-the-rotary-foundation-s-first-100-years-1st-
edition-david-forward/

My Eyes Are Black Holes 1st Edition Logan Ryan Smith

https://ebookmeta.com/product/my-eyes-are-black-holes-1st-
edition-logan-ryan-smith/

Cross Technology Communication for Internet of Things

Fundamentals and Key Technologie 1st Edition Xiuzhen
Guo

https://ebookmeta.com/product/cross-technology-communication-for-
internet-of-things-fundamentals-and-key-technologie-1st-edition-
xiuzhen-guo/

Complete checkers 2 revised Edition Richard Pask

https://ebookmeta.com/product/complete-checkers-2-revised-
edition-richard-pask/

Edexcel A level Mathematics Pure Mathematics Year 2 A

level Maths and Further Maths 2017 1st Edition Greg
Attwood

https://ebookmeta.com/product/edexcel-a-level-mathematics-pure-
mathematics-year-2-a-level-maths-and-further-maths-2017-1st-
edition-greg-attwood/
IoT and AI Technologies for Sustainable Living: A
Practical Handbook 1st Edition Abid Hussain

https://ebookmeta.com/product/iot-and-ai-technologies-for-
sustainable-living-a-practical-handbook-1st-edition-abid-hussain/
Praise for Data Science at the Command Line
Traditional computer and data science curricula all too often
mistake the command line as an obsolete relic instead of teaching
it as the modern and vital toolset that it is. Only well into my
career did I come to grasp the elegance and power of the
command line for easily exploring messy datasets and even
creating reproducible data pipelines for work. The first edition of
Data Science at the Command Line was one of the most
comprehensive and clear references when I was a novice in the
art, and now with the second edition, I’m again learning new tools
and applications from it.
—Dan Nguyen, data scientist, former news
application developer at ProPublica, and former Lorry
I. Lokey Visiting Professor in Professional Journalism
at Stanford University
The Unix philosophy of simple tools, each doing one job well, then
cleverly piped together, is embodied by the command line. Jeroen
expertly discusses how to bring that philosophy into your work in
data science, illustrating how the command line is not only the
world of file input/output, but also the world of data manipulation,
exploration, and even modeling.
—Chris H. Wiggins, associate professor in the
department of applied physics and applied
mathematics at Columbia University, and chief data
scientist at The New York Times
This book explains how to integrate common data science tasks
into a coherent workflow. It’s not just about tactics for breaking
down problems, it’s also about strategies for assembling the
pieces of the solution.
—John D. Cook, consultant in applied mathematics,
statistics, and technical computing
Despite what you may hear, most practical data science is still
focused on interesting visualizations and insights derived from flat
files. Jeroen’s book leans into this reality, and helps reduce
complexity for data practitioners by showing how time-tested
command-line tools can be repurposed for data science.
—Paige Bailey, principal product manager code
intelligence at Microsoft, GitHub
It’s amazing how fast so much data work can be performed at the
command line before ever pulling the data into R, Python, or a
database. Older technologies like sed and awk are still incredibly
powerful and versatile. Until I read Data Science at the Command
Line, I had only heard of these tools but never saw their full
power. Thanks to Jeroen, it’s like I now have a secret weapon for
working with large data.
—Jared Lander, chief data scientist at Lander
Analytics, organizer of the New York Open Statistical
Programming Meetup, and author of R for Everyone
The command line is an essential tool in every data scientist’s
toolbox, and knowing it well makes it easy to translate questions
you have of your data to real-time insights. Jeroen not only
explains the basic Unix philosophy of how to chain together single-
purpose tools to arrive at simple solutions for complex problems,
but also introduces new command-line tools for data cleaning,
analysis, visualization, and modeling.
—Jake Hofman, senior principal researcher at
Microsoft Research, and adjunct assistant professor
in the department of applied mathematics at
Columbia University
Data Science at the Command
Line
SECOND EDITION

Obtain, Scrub, Explore, and Model Data with Unix

Power Tools

Jeroen Janssens
Data Science at the Command Line
by Jeroen Janssens
Copyright © 2021 Jeroen Janssens. All rights reserved.
Printed in the United States of America.
Published by O’Reilly Media, Inc., 1005 Gravenstein Highway North,
Sebastopol, CA 95472.
O’Reilly books may be purchased for educational, business, or sales
promotional use. Online editions are also available for most titles
(http://oreilly.com). For more information, contact our
corporate/institutional sales department: 800-998-9938 or
[email protected].

Acquisitions Editor: Jessica Haberman

Development Editor: Sarah Grey

Production Editor: Kate Galloway

Copyeditor: Arthur Johnson

Proofreader: Shannon Turlington

Indexer: nSight, Inc.

Interior Designer: David Futato

Cover Designer: Karen Montgomery

Illustrator: Kate Dullea

October 2014: First Edition

August 2021: Second Edition
Revision History for the Second Edition
2021-08-17: First Release

See http://oreilly.com/catalog/errata.csp?isbn=9781492087915 for

release details.
The O’Reilly logo is a registered trademark of O’Reilly Media, Inc.
Data Science at the Command Line, the cover image, and related
trade dress are trademarks of O’Reilly Media, Inc.
The views expressed in this work are those of the author, and do not
represent the publisher’s views. While the publisher and the author
have used good faith efforts to ensure that the information and
instructions contained in this work are accurate, the publisher and
the author disclaim all responsibility for errors or omissions,
including without limitation responsibility for damages resulting from
the use of or reliance on this work. Use of the information and
instructions contained in this work is at your own risk. If any code
samples or other technology this work contains or describes is
subject to open source licenses or the intellectual property rights of
others, it is your responsibility to ensure that your use thereof
complies with such licenses and/or rights.
Data Science at the Command Line is available under the Creative
Commons Attribution NonCommercial-No Derivatives 4.0
International License. The author maintains an online version at
https://github.com/jeroenjanssens/data-science-at-the-command-
line.
978-1-492-08791-5
[LSI]
Once again to my wife, Esther. Without her continued
encouragement, support,
and patience, this second edition would surely have ended up in
/dev/null.
Foreword
It was love at first sight.
It must have been around 1981 or 1982 that I got my first taste of
Unix. Its command-line shell, which uses the same language for
single commands and complex programs, changed my world, and I
never looked back.
I was a writer who had discovered the joys of computing, and
regular expressions were my gateway drug. I’d first tried them in the
text editor in HP’s RTE operating system, but it was only when I
came to Unix and its philosophy of small cooperating tools with the
command-line shell as the glue that tied them together that I fully
understood their power. Regular expressions in ed, ex, vi (now
vim), and emacs were powerful, sure, but it wasn’t until I saw how
ex scripts unbound became sed, the Unix stream editor, and then
AWK, which allowed you to bind programmed actions to regular
expressions, and how shell scripts let you build pipelines not only out
of the existing tools but out of new ones you’d written yourself, that
I really got it. Programming is how you speak with computers, how
you tell them what you want them to do, not just once, but in ways
that persist, in ways that can be varied like human language, with
repeatable structure but different verbs and objects.
As a beginner, other forms of programming seemed more like
recipes to be followed exactly—careful incantations where you had
to get everything right—or like waiting for a teacher to grade an
essay you’d written. With shell programming, there was no
compilation and waiting. It was more like a conversation with a
friend. When the friend didn’t understand, you could easily try again.
What’s more, if you had something simple to say, you could just say
it with one word. And there were already words for a whole lot of
the things you might want to say. But if there weren’t, you could
easily make up new words. And you could string together the words
you learned and the words you made up into gradually more
complex sentences, paragraphs, and eventually get to persuasive
essays.
Almost every other programming language is more powerful than
the shell and its associated tools, but for me at least, none provides
an easier pathway into the programming mindset, and none provides
a better environment for a kind of everyday conversation with the
machines that we ask to help us with our work. As Brian Kernighan,
one of the creators of AWK as well as the coauthor of the marvelous
book The Unix Programming Environment, said in an interview with
Lex Fridman, “[Unix] was meant to be an environment where it was
really easy to write programs.” [00:23:10] Kernighan went on to
explain why he often still uses AWK rather than writing a Python
program when he’s exploring data: “It doesn’t scale to big programs,
but it does pretty darn well on these little things where you just
want to see all the somethings in something.” [00:37:01]
In Data Science at the Command Line, Jeroen Janssens
demonstrates just how powerful the Unix/Linux approach to the
command line is even today. If Jeroen hadn’t already done so, I’d
write an essay here about just why the command line is such a
sweet and powerful match with the kinds of tasks so often
encountered in data science. But he already starts out this book by
explaining that. So I’ll just say this: the more you use the command
line, the more often you will find yourself coming back to it as the
easiest way to do much of your work. And whether you’re a shell
newbie, or just someone who hasn’t thought much about what a
great fit shell programming is for data science, this is a book you will
come to treasure. Jeroen is a great teacher, and the material he
covers is priceless.

Tim O’Reilly
May 2021
Preface

Data science is an exciting field to work in. It’s also still relatively
young. Unfortunately, many people, and many companies as well,
believe that you need new technology to tackle the problems posed
by data science. However, as this book demonstrates, many things
can be accomplished by using the command line instead, and
sometimes in a much more efficient way.
During my PhD program, I gradually switched from using Microsoft
Windows to using Linux. Because this transition was a bit scary at
first, I started with having both operating systems installed next to
each other (known as a dual-boot). The urge to switch back and
forth between Microsoft Windows and Linux eventually faded, and at
some point I was even tinkering around with Arch Linux, which
allows you to build up your own custom Linux machine from scratch.
All you’re given is the command line, and it’s up to you what to
make of it. Out of necessity, I quickly became very comfortable using
the command line. Eventually, as spare time got more precious, I
settled down with a Linux distribution known as Ubuntu because of
its ease of use and large community. However, the command line is
still where I’m spending most of my time.
It actually wasn’t too long ago that I realized that the command line
is not just for installing software, configuring systems, and searching
files. I started learning about tools such as cut, sort, and sed.
These are examples of command-line tools that take data as input,
do something to it, and print the result. Ubuntu comes with quite a
few of them. Once I understood the potential of combining these
small tools, I was hooked.
After earning my PhD, when I became a data scientist, I wanted to
use this approach to do data science as much as possible. Thanks to
a couple of new, open source command-line tools including
xml2json, jq, and json2csv, I was even able to use the
command line for tasks such as scraping websites and processing
lots of JSON data.
In September 2013, I decided to write a blog post titled “7
Command-Line Tools for Data Science”. To my surprise, the blog post
got quite some attention, and I received a lot of suggestions of other
command-line tools. I started wondering whether the blog post
could be turned into a book. I was pleased that, some 10 months
later, and with the help of many talented people (see the
acknowledgments), the answer was yes.
I am sharing this personal story not so much because I think you
should know how this book came about, but because I want to you
know that I had to learn about the command line as well. Because
the command line is so different from using a graphical user
interface, it can seem scary at first. But if I could learn it, then you
can as well. No matter what your current operating system is and no
matter how you currently work with data, after reading this book
you will be able to do data science at the command line. If you’re
already familiar with the command line, or even if you’re already
dreaming in shell scripts, chances are that you’ll still discover a few
interesting tricks or command-line tools to use for your next data
science project.

What to Expect from This Book

In this book, we’re going to obtain, scrub, explore, and model data—
a lot of it. This book is not so much about how to become better at
those data science tasks. There are already great resources available
that discuss, for example, when to apply which statistical test or how
data can best be visualized. Instead, this practical book aims to
make you more efficient and productive by teaching you how to
perform those data science tasks at the command line.
While this book discusses more than 90 command-line tools, it’s not
the tools themselves that matter most. Some command-line tools
have been around for a very long time, while others will be replaced
by better ones. New command-line tools are being created even as
you’re reading this. Over the years, I have discovered many amazing
command-line tools. Unfortunately, some of them were discovered
too late to be included in the book. In short, command-line tools
come and go. But that’s OK.
What matters most is the underlying idea of working with tools,
pipes, and data. Most command-line tools do one thing and do it
well. This is part of the Unix philosophy, which makes several
appearances throughout the book. Once you have become familiar
with the command line, know how to combine command-line tools,
and can even create new ones, you have developed an invaluable
skill.

Changes for the Second Edition

While the command line as a technology and as a way of working is
timeless, some of the tools discussed in the first edition have either
been superseded by newer tools (e.g., csvkit has largely been
replaced by xsv) or abandoned by their developers (e.g., drake),
or they’ve been suboptimal choices (e.g., weka). I have learned a lot
since the first edition was published in October 2014, either through
my own experience or as a result of the useful feedback from my
readers. Even though the book is quite niche because it lies at the
intersection of two subjects, there remains a steady interest from
the data science community, as evidenced by the many positive
messages I receive almost every day. By updating the first edition, I
hope to keep the book relevant for at least another five years. Here’s
a nonexhaustive list of changes I have made:

I replaced csvkit with xsv as much as possible. xsv is a

faster alternative to working with CSV files.
 In Chapters 2 and 3, I replaced the VirtualBox image with a
Docker image. Docker is a faster and more lightweight way
of running an isolated environment.
I now use pup instead of scrape to work with HTML.
scrape is a Python tool I created myself. pup is much
faster, has more features, and is easier to install.
Chapter 6 has been rewritten from scratch. Instead of
drake, I now use make to do project management. drake
is no longer maintained, and make is much more mature and
very popular with developers.
I replaced Rio with rush. Rio is a clunky Bash script I
created myself. rush is an R package that is a much more
stable and flexible way of using R from the command line.
In Chapter 9 I replaced Weka and BigML with Vowpal Wabbit
(vw). Weka is old, and the way it is used from the command
line is clunky. BigML is a commercial API that I no longer
want to rely on. Vowpal Wabbit is a very mature machine
learning tool that was developed at Yahoo! and is now at
Microsoft.
Chapter 10 is an entirely new chapter about integrating the
command line into existing workflows, including Python, R,
and Apache Spark. In the first edition I mentioned that the
command line can easily be integrated with existing
workflows but never delved into the topic. This chapter fixes
that.

How to Read This Book

In general, I advise you to read this book in a linear fashion. Once a
concept or command-line tool has been introduced, chances are that
I employ it in a later chapter. For example, in Chapter 9, I make
heavy use of parallel, which is discussed extensively in
Chapter 8.
Data science is a broad field that intersects many other fields such
as programming, data visualization, and machine learning. As a
result, this book touches on many interesting topics that
unfortunately cannot be discussed at great length. At the end of
each chapter, I provide suggestions for further exploration. It’s not
required that you read this material in order to follow along with the
book, but if you are interested, just know that there’s much more to
learn.
Who This Book Is For
This book makes just one assumption about you: that you work with
data. It doesn’t matter which programming language or statistical
computing environment you’re currently using. The book explains all
the necessary concepts from the beginning.
It also doesn’t matter whether your operating system is Microsoft
Windows, macOS, or some flavor of Linux. The book comes with a
Docker image, which is an easy-to-install virtual environment. It
allows you to run the command-line tools and follow along with the
code examples in the same environment as this book was written.
You don’t have to waste time figuring out how to install all the
command-line tools and their dependencies.
The book contains some code in Bash, Python, and R, so it’s helpful
if you have some programming experience, but it’s by no means
required to follow along with the examples.

Conventions Used in This Book

The following typographical conventions are used in this book:
Italic
Indicates new terms, URLs, directory names, and filenames.

Constant width
Used for code and commands, as well as within paragraphs to
refer to command-line tools and their options.

Constant width bold

Shows commands or other text that should be typed literally by
the user.
Exploring the Variety of Random
Documents with Different Content
 IV

A Siege and its Heroine

T HE population of the region, friends and foes, were now up in

alarm. Reports met us of the outrages of the Ruffians upon Free
State settlers the night previous.
Here is the story of one of the depredations, detailed to us at one of
our halts.
It was upon a stanch old German and his family, settled near the
junction of the North and South branches of the Pottawatomie. Old
Kepler, as he was nicknamed, had not taken any leading or even
active part in the "troubles" (as they were termed), but his strong
anti-slavery sentiments had cropped out and were known to the
enemy.
They now made directly for his cabin, evidently resolved, as the
opportunity might offer, to force him to declare himself for one side
or the other. No man, in fact, in those days of the Kansas conflict,—
partisan, bitter, bloody,—could long occupy anything like neutral
ground. If one undertook to "sit on the fence," he soon became a
target for both parties and was relentlessly dislodged.
It was not the nature of the old German to dissemble, when the trial
came. On the approach of the Ruffians he prepared for the worst, as
he expected no favor. He barricaded his cabin door and refused their
demand for admittance. They burned his wheat and hay stacks, and
all his outbuildings, and then called upon the besieged to surrender.
It was believed, probably rightly, by the assailants, that the old man
was possessed of considerable money, brought with him from the
old country. This lent incitement to their attack; while, if true, the
fact was undoubtedly an additional motive on his part for keeping
the invaders at a distance.
Brave old Kepler was quite advanced in years. He was about three
score and ten, but all the old valorous Teutonic blood in his veins
was aroused, and he prepared to resist the spoilers even to the
death, if need be. His wife, partner of his New World adventures and
toils, had succumbed not long before to the frontier hardships and
had passed on. He had one son, a chip of the old block, brave,
strong, and inured to the rough Western life, equally interested with
the father in carving out their fortunes in this new country, and in
the making of their Western prairie home.
And there was an only daughter, alike the support and solace of both
father and brother;—the light, indeed, of the household and of the
neighborhood.
I must interpolate a word here, in passing, descriptive of this
daughter,—the worthy heroine of the event, as we shall see. She
was a light-haired, blond-complexioned young girl, with all the
proverbial German fairness,—bright and handsome as a prairie
flower. And she had the German habit of taking a share in the work
in the open field. Often was she seen by the passers up and down
the creek, "chopping in corn" (as they call it in the West),—keeping
even step in the row with her robust brother; or now driving the
cattle while he held the plough; then changing work with him,
guiding the share while he drove the oxen.
Her household duties, however, were not neglected meanwhile.
Doubtless the brother, in return, here gave her a helping hand.
Nowhere else on the road (as the writer can testify from personal
experience) did the weary and hungry traveler find such bread as
when thrown upon the Keplers' hospitality,—bread of this young
girl's manufacture.
Besides all this,—and appropriately to be said in this connection,—
this fair maiden could handle a rifle on occasion, as we shall
presently see. Such ability was often a quite useful accomplishment
for the gentler sex on our wild Western border. It proved eminently
so in the case before us.
The yelling, hooting, and now drunken mob began at length to fire
upon the cabin at its vulnerable points. The heroic inmates returned
the shots through the holes between the logs in the loft, and not
without effect. One of the assailants was seriously wounded and
several others less so. The battle grew warm, the effusion of blood
thus far serving only to increase the wild fury of the besiegers.
The father and son stood with their guns at the openings, while the
young girl loaded the pieces for them as fast as they were emptied.
At length the baffled and maddened crowd changed their tactics.
They managed to pile wood, logs, and rubbish against the cabin,
hoping to fire the building. There was danger that the dastardly
effort would prove only too successful. The flames began to crackle.
All now seemed lost, when suddenly the brave daughter unbarred
the cabin door and sprang forth with a bucket of water in her hand
to dash out the newly kindled flames. This was done from the girl's
own impulse at the moment. Had they divined her intention, the
father and brother would not have allowed it. The feat, however,
strange to say, was as successful as it was heroic and perilous.
The surprised besiegers were not actually cowardly and base enough
to fire upon the unarmed, defenseless girl. However, one of them
sprang from his covert behind a tree to seize her. But the old
backwoodsman father, watching breathlessly the scene below from
his post in the loft,—his hand and eye steadied to perfect accuracy
by the imminent danger,—sent a rifle-bullet straight to the heart of
the venturesome wretch, and he fell forward dead at the maiden's
feet.
The girl regained the door and, with the aid of her brother, who
hastened to her assistance, rebarred it securely. All was now again
safe for the time being,—and permanently, as it proved. The
marauders, overawed by this episode and by the generally
unexpected course of affairs,—one of their number being actually
killed and several others more or less severely wounded,—hastily fell
back to a safe distance and finally beat a retreat from the
neighborhood.
 V

The March Resumed

IT did not require the narration of this stirring tale to nerve our
forward movement, but it certainly increased our determination to
proceed at all hazard.
Our next halt was made at the cabin, some miles further on, from
which, as mentioned in the first chapter, the young man whom we
all knew and counted as one of us had been borne off a prisoner. As
soon as it was made known, by the usual signs, that we were
friends, we were joyfully if tearfully greeted. The family, consisting of
aged parents, sister, brother's wife and little children, were in
despair. Dreadful anxiety filled their minds. It was an illustration of
the saying that "to know the worst is better than suspense." If in the
great cause then firing their hearts this family had seen that son and
brother shot down before their eyes, they would have borne the
affliction silently and with submission. But the terrible uncertainty as
to his fate wrought upon them. A price had previously been set upon
the young man's head, and they had reason to fear the worst for
him.
It must be added, in passing, that his beloved ones never saw him
again alive. The good fortune fell to us to liberate him the next day
from his captors, when we found him bound upon his horse, with his
hands lashed behind him and his feet tied together under the
animal; but, alas! his liberation gave him only a short respite from
death. He fell, only a few days after, heroically fighting at the battle
of Osawatomie.
Some miles beyond we had to make that ford of the Pottawatomie
river of unenviable fame, and which we looked upon as the danger-
point of all others in our journey; for there our enemy, we thought,
would most likely be in ambush. But we swam the swift, dark,
muddy stream, swelled by recent rains to a flood, with the water up
to our horses' backs, luckily without hindrance or serious mishap.
That ford was the notorious Dutch Henry's crossing, so-called,—
surely a gloomy, gruesome, and dreaded spot at that dark midnight
hour. There, close by, had been enacted, just two months prior, the
rightly named Pottawatomie tragedy, which made that locality, on
account of this bloody event, verily for the time the "storm center"
of the Kansas conflict. But, terrible as it was, it served a great
purpose and was speedily followed by good.
The hero of our sketch was the central figure in this tragic act of the
Kansas drama, as he was in most others at this trying period. Brown
was the cyclonic force, the lightning's flash in the darkness, that
cleared and lighted the way for the men of that day.
Despite all delays on the way, we made our forced night-march of
twenty-two or more miles in remarkably good time, and arrived at
our destination about two o'clock in the morning, as weary,
exhausted, and hungry a set of troopers as ever drew rein and
slipped stirrup to seek rest and refreshment.
 The Adair Log Cabin.
It will be of interest to our readers to learn here that, a couple of
miles from the town,—our halting place,—we passed the log cabin of
the Adair family, which has such historic interest gathered about it,
and which we shall have occasion to mention again later.
It so happened, as we learned afterward, that the hero of our story
lodged under that roof that night. He was aroused from his slumbers
and watched us from the window as we marched past,—having been
reliably assured, by our advanced guard, that we were no
threatening foe, but his firmest and safest friends.
A photographic view of the cabin's exterior is given on the opposite
page, as it appears to-day; and nearly the same as it existed at that
early date, now almost fifty years ago.
The town referred to was Osawatomie, soon to be made famous by
the man who is the principal subject of these sketches.
We were challenged by friendly pickets on guard, who escorted us to
the old "block-house" reared for town defense, where we were glad
to find shelter, and especially to find food, for hungry we were
indeed.
To what a sumptuous feast were we welcomed on that occasion!
And yet, strange to relate, the recollection of it is not calculated to
make one's mouth water. It so happened that a side of bacon and a
barrel of hardtack were stored there, for just such emergencies as
the present one, and these were now pressed into our service.
Their edible condition was such as naturally to suggest certain
Scripture phrases as descriptive thereof;—of the bacon, "ancient of
days"; and of the biscuit, "fullness of life." As we crunched the latter
between our teeth, the peculiar, fresh, sweet-and-bitter taste,
commingling at every mouthful, told us too well of the "life"
ensconced therein. No comments were made, however, except the
ejaculation occasionally, by one and another, "Wormy!" " Wormy!"
However, nothing daunted, we paused not in our eating till our
ravenous hunger was appeased. And then, on the bare floor of
boards, rived roughly out of forest trees,—though it was a little
difficult to fit our forms to their ridges and hollows,—we gained a
few hours of as sweet and refreshing slumber as ever visited mortal
eyes.
 VI

Seeking the Enemy

IT will be asked, perhaps, why we came to this particular place. In

this little town were encamped, at this particular time, Captain
John Brown and his daring and trusty band of men.
"Old Brown," as he was most often called, was a tower of strength in
time of need. He had become by that time a veritable terror to the
enemy. Tell a Border Ruffian: "John Brown is coming," and he would
shake in his shoes, or would run away had he strength enough left
for locomotion. Missouri mothers frightened their babies to sleep or
to quietude by the sound of his name.
If our information were correct, the foe we sought largely
outnumbered us. What more natural than that we should, under the
circumstances, desire the counsel of the stanch old man, and his
help, if needed.
He had not looked for an invasion from the direction at present
threatened, but was daily expecting one from another quarter. He
detailed two small companies, Captain Shore's and Captain Cline's,—
two-thirds of his own command,—to join our force; then bade us
seek the enemy, with the direction, if we found them too strong for
us, to send back word to him, whereupon he would come to our aid.
Meanwhile, he said, he would stay with the remainder of his men
and guard the town.
We set out in the morning, early and hopefully. Scouts with fleet
horses were dispatched in advance, and we rapidly followed after.
Rumors of all wild and exaggerated sorts met us as we went. First, it
was said, there were three hundred of the enemy, well armed and
mounted; then there were five hundred men, strongly intrenched to
receive our attack; later, there were a thousand, coming to meet us.
At last we began to be a little apprehensive, possibly a grain
frightened. In the uncertainty, a messenger was sent back to
Captain Brown to say that probably we should need his help.
But we resolutely pushed on, if with somewhat slackened speed.
Presently a scout returned bearing reliable tidings. The position and
strength of the invaders had been quite accurately ascertained. They
were about three hundred in number, quietly encamped, and as yet
unaware of our approach.
Our officers decided not to wait for Captain Brown to come up, but
to press forward to the attack and by celerity of movement gain
what advantage was possible.
One point was, nevertheless, taken into consideration. We were but
about sixty in number, all told. We were prepared and determined to
do some hard fighting if necessary; but, it was argued, if we could
take the enemy by surprise, victory would be more fully assured us,
and much needless spilling of blood might be avoided.
We therefore proceeded cautiously till we arrived within two miles of
the hostile force, where our advanced scouts had taken up position
and were actually looking down with spy-glasses into the enemy's
camp and watching their every movement. The foe seemed wholly
unconscious of any impending danger.
 VII

The Battle

I N less time than it takes to relate it, the plan of battle was
arranged.
Our men were divided into three companies. Two divisions were to
make flank movements, one on the right and the other on the left of
the foe, while the third was to assault directly in front. The plan of
attack was well conceived and as successfully executed.
We had a circuit of some miles to make to gain the flank positions. It
was quickly and silently traveled. In our division, detailed on the left
flank, hardly a word was spoken during a two hours' march. Each
man was busy with his own thoughts. It is said that persons in
critical situations will sometimes have their whole lives pass before
them. I believe that most of us, during this march, recalled nearly all
we had ever done or seen, known or felt.
We were suddenly awakened, at length, from such reveries, by the
crack of rifles and the clash of musketry, and by bullets actually
whizzing about our ears. So closely had we stolen the march on
them that when we opened fire we were actually more in danger
from the guns of our friends than from those of our foes.
The enemy were taken completely by surprise. As prisoners whom
we took told us afterward, they thought that "Old Brown" was surely
upon them; and their next and only thought was of escape. They left
all, and ran for dear life, some on foot, shoeless and hatless; others
springing to their horses, and, even without bridle or saddle,
desperately making the trial of flight. Perfectly bewildered, they ran
this way and that; and naturally, as our forces were positioned,
many ran directly into our hands.
The one thing they did not do well was to fight, except in the case of
a few desperate ones and of the leaders, who called in vain upon
their men to rally. Then they gave up all for lost, and each looked
out for himself. Many discharged their pieces at the first onslaught,
but so much at random that not a man of our number was fatally
injured, though several were more or less severely wounded. We
took many prisoners, and captured some thirty horses, all the
enemy's wagons and luggage, and much ammunition and arms. The
victory was complete.
Not until all was over did Captain Brown and his reserve come up,
though they had ridden hard to lend us a helping hand. He warmly
congratulated us, however, upon our good success, saying that he
could not have done it better himself, and that he was just as glad
and proud of our victory as though he had won it.
 VIII

A Scene for a Painter

T HERE were incidents not a few, connected with the day and with
the central figure of our sketch, which would add interest to our
pages. One there was which especially impressed itself upon all
witnesses of it.
This relates to one of the enemy who was fatally wounded in the
battle. He desired very much, he said, to see "Old Brown" before he
died.
Captain Brown was informed of the wish, whereupon he rode up to
the wagon which served as ambulance, and, with somewhat of
sternness in his manner, said to the prisoner, "You wish to see me.
Here I am. Take a good look at me, and tell your friends, when you
get back to Missouri, what sort of man I am."
Then he added in a gentler tone, "We wish no harm to you or to
your companions. Stay at home, let us alone, and we shall be
friends. I wish you well."
The prisoner meanwhile had raised himself with great difficulty, and
viewed the old man from head to foot as if feasting his eyes on a
great curiosity. Then he sank back, pale and exhausted, as he
answered, "I don't see as you are so bad. You don't talk like it."
The countenance of Brown as he viewed the sufferer had changed
to a look of commiseration. The wounded man saw it, and, reaching
out his hand, said, "I thank you." Brown tenderly clasped it, and
replied, "God bless you," while he turned with tears in his eyes and
rode away.
The present writer was standing within a few feet of Brown at the
time, and naturally drank in the scene with a boy's eager curiosity
and susceptibility to impression.
It was a scene for a painter, and the artist could with
appropriateness have called his work, "The Conqueror Conquered."
But it was perfectly illustrative of the man and of the hero. Brown
was as brave as a lion. He seemed absolutely not to know fear. Yet
withal he possessed a heart tender as a child's or as the tenderest
woman's.
 IX

Brown's Night Appointment

W E gathered together the spoils and took up our march on the

backward track toward home, discussing the exciting events of
the day and recounting to each other our individual experiences,
adventures, and "hairbreadth escapes." When we had thus
proceeded some three miles, it was nearing sundown, and we halted
for supper and to determine our course for the night.
Meanwhile we had learned an important fact from our prisoners,
namely: that we had not met all of our enemies. A part of them,
quite a large force, had gone north that morning, and might be at
that very moment ravaging our own homes which we had left behind
the evening before. Naturally, these unwelcome tidings cast a cloud
across our rejoicings. They might after all be turned to mourning!
Having nearly finished our meal, and while we were yet speculating
on the situation, Captain Brown hastily rose to his feet and called
upon all those, who were ready to go with him, to mount their
horses. Forty or more men instantly sprang into their saddles, and
others were about to do the same, when the old man cried, "Enough
—and too many." He thanked them for their readiness, and then
selected thirty of the number, tried and trusted men who had
followed him before, and without asking why or whither. In the
present instance also they ventured not a question.
Brown seldom disclosed his intention or plans to any one. He wished
no man with him who was not absolutely reliable. He required the
implicit confidence of his followers and unquestioning obedience to
his commands. Whoever put himself under his leadership took his
life in his hand and followed whithersoever he was led.
On this occasion some not acquainted with his habits plied him with
queries as to where he was going and what he would do. He only
answered, characteristically, that he "had an appointment with some
Missourians and must not disappoint them." One ventured jocosely
to ask further, concerning the appointed place of meeting. He
replied, they had not been kind enough to fix upon the precise spot,
but he felt bound, out of courtesy, inasmuch as they came from a
distance, to hold himself in readiness when wanted. This left us, of
course, wholly in the dark as to his movements.
With some words of advice to those of us remaining,—that we would
better seek our homes, be prepared to defend them, and ready for
any action when needed,—he gave the command, "Ready! Forward!"
and, with a wave of his hand, led his Knights Errant away.
After they had departed it was decided that it would be advisible for
us to return to the camping-ground of the enemy and pitch our tents
there for the night; because, it was argued, when the detached force
gone north returned, they would naturally seek their friends in the
camp where they left them.
Accordingly, though weary near to exhaustion, we returned and
camped there, threw out our pickets, and made every preparation to
give the marauders a warm reception should they appear. We slept
on our arms, ready for any emergency, but the night passed and we
were undisturbed.
The next morning dawned on us clear and beautiful. All our
apprehensions of danger had passed with the darkness. Our pickets
were withdrawn. The scouts, who had been sent out to gather news
of the scattered settlers, had come back with no tidings of the foe
we had awaited. Consequently, relieved of all military restraint, we
gave ourselves up for the time to the preparation and enjoyment of
an early breakfast.
The wagons were unpacked of their provisions. The horses were
picketed, or were turned loose for grazing. The prisoners, disarmed,
were allowed comparative freedom. Fires were lighted here and
there for cooking. And thus we were spread out over a large area,
forgetful of the enemy, without a thought of an attack, and bent only
on making ready to satisfy the cravings of hunger.
 X

An Intrepid Charge

T HEN occurred the scene which gives us one of the glimpses of

John Brown for the sake of which these reminiscences have been
written.
Suddenly, over the hill or rising ground some half or third of a mile
away, two horsemen came up at full speed.
"Look! look!" was whispered in suppressed voices from one to
another of our party, and all eyes were upturned in that direction.
Observing us, the horsemen as suddenly turned on their heels, and
disappeared the way they came, leaving us stupefied with doubt and
wonder.
In a moment more, however, the heads of a whole troop rose in
sight, and the cry, "The Missourians! the Missourians!" rang through
our camp in startling accents.
We were in dismay, for we were entirely unprepared for attack and
there was no time to make ready. We were apparently caught just as
our enemy had been surprised by ourselves. Men sprang, some for
their arms, some for their horses. Whether to fight or to try to
escape was uppermost in their minds,—each could settle that
question only for himself. At any rate, every one felt that a daring
and determined foe, apparently numbering a hundred, which was
double our own number, could, in the condition in which we were,
utterly cut us to pieces and destroy us at a blow.
What grave emotions that thought aroused! It is difficult for one,
never thrown into any such situation, to realize or in any degree
even imagine the feelings that may surge through the bosom of men
thus placed. Accounts have been given of what panic-stricken
crowds or armies will sometimes do, but a description of what they
feel on such occasions of disaster was never yet fully penned or
painted by man.
Meanwhile, some of our number, who had been cool enough to
observe the fiercely advancing cavaliers, perceived that they were
friends, not foes. It was old Captain Brown himself and his trusty
band. With joy, this news rang through our ranks. All eyes were then
directed toward them, enchained and enchanted. It was a splendid
sight.
They at first, naturally, took us for enemies, not dreaming but that
we were miles away, where they left us the evening before. They
suspected us to be the force, encamped there, which they had been
riding all night to overtake,—the same force we had awaited.
They came swiftly up over the brow of the hill, in full view, with
Brown at their head, and, without halting or even slackening their
speed, swung into line of battle. Only thirty men! yet they presented
a truly formidable array. The line was formed two deep, and was
stretched out to give the men full room for action. Brown sprang his
horse in front of the ranks, waving his long broadsword, and on they
came, sweeping down upon us with irresistible fury.
It was indeed a splendid and fearful sight, never to be forgotten by
the beholders. Only thirty men! yet they seemed a host. In their
every action, in their entire movements, seemed emblazoned, as in
their determined souls it was written, "Victory or death!"
Their leader looked the very impersonation of Battle. Many of us had
seen John Brown before, some of us a number of times, and under
trying circumstances. But now all felt that the real man we had
never before beheld. The daring, the intrepidity, the large resources
of the man, none of us had imagined till that moment.
Not a gun was discharged, their commander having given to his men
the same strict orders that were given at Bunker Hill of old, that they
should "reserve their fire till they could see the whites of their
enemy's eyes." But before they had quite gained that very
dangerous proximity to us, we succeeded in making them
understand that we were their friends.
Then such a glad shout as rent the air from both sides was seldom
ever heard, we believe, on any field even of victory. They were as
glad to find that we were their friends, as we, in our helpless
condition, were glad to learn that they were not our enemies.
The full intrepidity of Brown and his men, though it appeared to us
astounding, was not fully appreciable till we came to look at it
somewhat from their own view-point.
We were actually about eighty men, prisoners and all. But, spread
out as we were, with the many horses grazing, the scattered and
unpacked wagons, numerous camp-fires,—widely separated for
convenience,—arms stacked in some places, and men gathered in
groups in others, we presented altogether a formidable appearance.
What was more, this was enhanced by our peculiar position, so that,
to them, our numbers and strength were exaggerated, while our
weakness and confusion were concealed. Brown admitted to us
himself, afterward, that he thought he was undertaking to whip a
force of two or three hundred, while his men declared that they
believed they were actually charging upon not less than a thousand.
Brown's quick military eye took in, at the first, the supposed
situation; and, as in a flash, he decided what to do. All depended, he
concluded, upon rapidity of action. His only hope lay in striking a
sudden and crushing blow, for which we were unprepared, and from
which we could not recover till he had made victory sure. From the
time Brown's forces came in sight over the hill, till they were within
gunshot of us, hardly thirty seconds elapsed,—a very short notice in
which to prepare for action, even if an attack were expected.
 XI

Brown to Our Prisoners

A FTER mutual congratulations over the bloodless and happy

conclusion of the adventure, we set our friends down with us to
eat the interrupted breakfast, to which they were prepared to do
ample justice. They had ridden all night, some forty or fifty miles, in
pursuit of the enemy,—had ridden all night, without rest or food,
from the time they left us, at dusk of evening, till they surprised us
that morning with their dauntless charge.
Another incident in connection with the events described it seems
fitting to mention, as affording a very interesting side-glance at the
character of our hero. After the meal, Captain Brown was asked by
our officers to give a talk to the prisoners taken the day before, who
were now drawn up in line for parole. He responded without an
instant's hesitation or a moment to think what he should say.
He spoke to them in a plain, simple, unpretentious way, but with a
directness, a force, and an eloquence withal, which doubtless
wonderfully impressed those addressed, as certainly it held spell-
bound all others who listened. Such vivid and indelible impression did
this speech of Brown make on the mind of the present writer that,
even after the lapse of these many years, he is able to reproduce it,
not only in substance, but almost word for word; and he has no
doubt of its exceptional character. Perhaps it was second only to that
immortal address which the hero made three years later to the court
at his trial in Virginia, which Emerson pronounced one of the three
most remarkable addresses in the world.
On the latter occasion, however, instead of a few plain, simple, rough
and ready, but intensely admiring followers, he had almost the whole
civilized world eagerly to hear and sacredly to preserve his utterance.
Brown's speech to the prisoners was probably not over five minutes
long in its delivery, but it lasted those forty trembling men a lifetime.
It was not known that one of them ever afterward ventured over the
Missouri border into the Kansas territory.
The address was as follows:
"Men of Missouri, one of your number has asked to see John Brown.
Here he is. Look at him, and hereafter remember that he is the
enemy of all evil-doers.
"And what of you yourselves, men! You are from a neighboring State.
What are you here for? You are invaders of this territory,—and for
evil purposes, you know as well as we know. You have been killing
our men, terrorizing our women and children, and destroying our
property,—houses, crops, and animals. So you stand here as
criminals.
"You are fighting for slavery. You want to make or keep other people
slaves. Do you not know that your wicked efforts will end in making
slaves of yourselves? You come here to make this a slave State. You
are fighting against liberty, which our Revolutionary fathers fought to
establish in this Republic, where all men should be free and equal,
with the inalienable rights of life, liberty, and the pursuit of
happiness. Therefore, you are traitors to liberty and to your country,
of the worst kind, and deserve to be hung to the nearest tree.
"But we shall not touch a hair of your heads. Have no fear. You are
deluded men. You have been deceived by men who are your elders
but not your betters. You have been misled into this wrong, by those
your leaders; thus, they are the real criminals and worse than
traitors, and, if we had them here instead of you, they would not find
such mercy at our hands.
"You we forgive. For, as you yourselves have confessed, we believe it
can be said of you that, as was said of them of old, you being
without knowledge, 'you know not what you do.' But hereafter you
will be without excuse.
"Go in peace. Go home and tell your neighbors and friends of your
mistake. We deprive you only of your arms, and do that only lest
some of you are not yet converted to the right. We let you go free of
punishment this time; but, do we catch you over the border again
committing depredations, you must not expect, nor will you receive,
any mercy.
"Go home, and become liberty-loving citizens of your State and
country, and your mistakes and misdeeds, as also the injuries which
you have inflicted upon us, will not have been in vain."
 XII

Hard Lines

T HE personal experiences here related are of interest and have a

value mainly as they throw somewhat of fresh light upon the
character of the subject of this work, Captain Brown, and upon the
events and times in which he was the leading actor.
Those were troublous times,—times that indeed "tried the men's
souls" who experienced them. The hardships were severe. Danger
and disease, death by ruthless hands, and even death from
starvation, often stared us in the face. At one time we lived six
weeks solely on Indian-meal mixed with water and dried before the
fire, and that without even a condiment. This was our common fare
in times of scarcity. Bacon and molasses, and tea without milk or
sugar, were our luxuries in times of plenty.
For months, in the summer of '56, the men in our settlement never
had their clothes off, day or night, unless torn or worn off. On a trip
early in the summer mentioned, made by a companion and myself to
Kansas City for provisions, we chanced to come across John Brown
and his company encamped in the woods on a river-bank. After we
made ourselves known as friends we were invited into their camp. A
more ragged set of men than we found were rarely, we believe, ever
seen,—Brown worst off of all, for he would not fare better than his
men. They had no shirts to their backs, and their outer clothing was
worn or torn to tatters. While in camp, they were going barefoot to
save the remnants of their worn-out shoes for emergencies. And
withal, they were, they said, on short rations, having no bread, but
only Indian-meal and water. They were glad of the opportunity to
engage us to bring them provisions on our return, but they
confessed they were as short of money as they were of provisions,
which simply meant that we must share ours with them.
The men of our company worked hard by day to raise crops, with
their rifles near at hand, and slept in the "bush" at night to avoid
surprise and capture in their cabins. Only the women and children
ran the risk of remaining in the houses, in their defenselessness
trusting to the mercy of the enemy. That border life invited sickness,
especially the malaria of the low prairie. Our cabins were roughly
made, and so open that when it rained it was about as wet inside of
them as outside.
We had not time to dig wells, and in mid-summer the rivers were
low and the water so stagnant that we had to brush the green scum
from the surface when we dipped the water to drink or for other
uses. Every man, woman, and child of the settlement was ill with the
"fever and ague," so termed. There came near being an exception to
the rule. One man kept so full of whiskey, continuously, that the
ague didn't seem to have even a fighting chance; but at length the
liquor fell short, and the ague then found its opportunity and even
made up for lost time.
As for fire-arms with which to defend ourselves, we were not well
off. The famous Sharpe's rifles—"Beecher's Bibles," so-called, from
the great preacher's contribution of them—won Kansas to freedom
in large measure; but more by their terrible name than by virtue of
any large number of the weapons themselves. The Free State men
in Kansas actually had few of them.
When my older brother, with whom I went to the territory, and
myself called on Theodore Parker in Boston,—for one thing to ask
him if those going to Kansas would be helped to fire-arms,—he said