Hands-on Guide to Apache Spark 3: Build Scalable

Computing Engines for Batch and Stream Data

Processing 1st Edition Alfonso Antolínez García
install download
https://ebookmeta.com/product/hands-on-guide-to-apache-
spark-3-build-scalable-computing-engines-for-batch-and-stream-
data-processing-1st-edition-alfonso-antolinez-garcia/

Download more ebook from https://ebookmeta.com

We believe these products will be a great fit for you. Click
the link to download now, or visit ebookmeta.com
to discover even more!

Modern Data Engineering with Apache Spark: A Hands-On

Guide for Building Mission-Critical Streaming
Applications 1st Edition Scott Haines

https://ebookmeta.com/product/modern-data-engineering-with-
apache-spark-a-hands-on-guide-for-building-mission-critical-
streaming-applications-1st-edition-scott-haines/

Chess Explained The c3 Sicilian 1st Edition Sam Collins

https://ebookmeta.com/product/chess-explained-
the-c3-sicilian-1st-edition-sam-collins/

Starting Out The c3 Sicilian 1st Edition John Emms

https://ebookmeta.com/product/starting-out-the-c3-sicilian-1st-
edition-john-emms/

Savage Club Curve Series 1st Edition Nell Alexander

https://ebookmeta.com/product/savage-club-curve-series-1st-
edition-nell-alexander-2/
Otherness and Identity in the Gospel of John 1st
Edition Sung Uk Lim

https://ebookmeta.com/product/otherness-and-identity-in-the-
gospel-of-john-1st-edition-sung-uk-lim/

The Impact of Corporate Culture and CMS A Cross

Cultural Analysis on Internal and External Preventive
Effects on Corruption 1st Edition Kai-D Bussmann

https://ebookmeta.com/product/the-impact-of-corporate-culture-
and-cms-a-cross-cultural-analysis-on-internal-and-external-
preventive-effects-on-corruption-1st-edition-kai-d-bussmann/

Adoption and Law: The Unique Personal Experiences of

Birth Mothers in Adoption Proceedings 1st Edition
Lisamarie Deblasio

https://ebookmeta.com/product/adoption-and-law-the-unique-
personal-experiences-of-birth-mothers-in-adoption-
proceedings-1st-edition-lisamarie-deblasio/

Machine Learning for Knowledge Discovery with R:

Methodologies for Modeling, Inference and Prediction
1st Edition Kao-Tai Tsai

https://ebookmeta.com/product/machine-learning-for-knowledge-
discovery-with-r-methodologies-for-modeling-inference-and-
prediction-1st-edition-kao-tai-tsai/

Learning to Trust (The Trust Series Book 1) Susan

Horsnell

https://ebookmeta.com/product/learning-to-trust-the-trust-series-
book-1-susan-horsnell/
Stem Cell Niche Kursad Turksen

https://ebookmeta.com/product/stem-cell-niche-kursad-turksen/
Alfonso Antolínez García

Hands-on Guide to Apache Spark 3

Build Scalable Computing Engines for Batch and
Stream Data Processing
Alfonso Antolínez García
Madrid, Spain

ISBN 978-1-4842-9379-9 e-ISBN 978-1-4842-9380-5

https://doi.org/10.1007/978-1-4842-9380-5

© Alfonso Antolínez García 2023

Apress Standard

The use of general descriptive names, registered names, trademarks,

service marks, etc. in this publication does not imply, even in the
absence of a specific statement, that such names are exempt from the
relevant protective laws and regulations and therefore free for general
use.

The publisher, the authors and the editors are safe to assume that the
advice and information in this book are believed to be true and accurate
at the date of publication. Neither the publisher nor the authors or the
editors give a warranty, expressed or implied, with respect to the
material contained herein or for any errors or omissions that may have
been made. The publisher remains neutral with regard to jurisdictional
claims in published maps and institutional affiliations.

This Apress imprint is published by the registered company APress

Media, LLC, part of Springer Nature.
The registered company address is: 1 New York Plaza, New York, NY
10004, U.S.A.
To my beloved family
Any source code or other supplementary material referenced by the
author in this book is available to readers on GitHub
(https://github.com/Apress). For more detailed information, please
visit http://www.apress.com/source-code.
Table of Contents
Part I: Apache Spark Batch Data Processing
Chapter 1:​Introduction to Apache Spark for Large-Scale Data
Analytics
1.​1 What Is Apache Spark?​
Simpler to Use and Operate
Fast
Scalable
Ease of Use
Fault Tolerance at Scale
1.​2 Spark Unified Analytics Engine
1.​3 How Apache Spark Works
Spark Application Model
Spark Execution Model
Spark Cluster Model
1.​4 Apache Spark Ecosystem
Spark Core
Spark APIs
Spark SQL and DataFrames and Datasets
Spark Streaming
Spark GraphX
1.​5 Batch vs.​Streaming Data
What Is Batch Data Processing?​
What Is Stream Data Processing?​
Difference Between Stream Processing and Batch
Processing
 1.​6 Summary
Chapter 2:​Getting Started with Apache Spark
2.​1 Downloading and Installing Apache Spark
Installation of Apache Spark on Linux
Installation of Apache Spark on Windows
2.​2 Hands-On Spark Shell
Using the Spark Shell Command
Running Self-Contained Applications with the spark-submit
Command
2.​3 Spark Application Concepts
Spark Application and SparkSession
Access the Existing SparkSession
2.​4 Transformations, Actions, Immutability, and Lazy
Evaluation
Transformations
Narrow Transformations
Wide Transformations
Actions
2.​5 Summary
Chapter 3:​Spark Low-Level API
3.​1 Resilient Distributed Datasets (RDDs)
Creating RDDs from Parallelized Collections
Creating RDDs from External Datasets
Creating RDDs from Existing RDDs
3.​2 Working with Key-Value Pairs
Creating Pair RDDs
Showing the Distinct Keys of a Pair RDD
 Transformations on Pair RDDs
Actions on Pair RDDs
3.​3 Spark Shared Variables:​Broadcasts and Accumulators
Broadcast Variables
Accumulators
3.​4 When to Use RDDs
3.​5 Summary
Chapter 4:​The Spark High-Level APIs
4.​1 Spark Dataframes
Attributes of Spark DataFrames
Methods for Creating Spark DataFrames
4.​2 Use of Spark DataFrames
Select DataFrame Columns
Select Columns Based on Name Patterns
Filtering Results of a Query Based on One or Multiple
Conditions
Using Different Column Name Notations
Using Logical Operators for Multi-condition Filtering
Manipulating Spark DataFrame Columns
Renaming DataFrame Columns
Dropping DataFrame Columns
Creating a New Dataframe Column Dependent on Another
Column
User-Defined Functions (UDFs)
Merging DataFrames with Union and UnionByName
Joining DataFrames with Join
4.​3 Spark Cache and Persist of Data
 Unpersisting Cached Data
4.​4 Summary
Chapter 5:​Spark Dataset API and Adaptive Query Execution
5.​1 What Are Spark Datasets?​
5.​2 Methods for Creating Spark Datasets
5.​3 Adaptive Query Execution
5.​4 Data-Dependent Adaptive Determination of the Shuffle
Partition Number
5.​5 Runtime Replanning of Join Strategies
5.​6 Optimization of Unevenly Distributed Data Joins
5.​7 Enabling the Adaptive Query Execution (AQE)
5.​8 Summary
Chapter 6:​Introduction to Apache Spark Streaming
6.​1 Real-Time Analytics of Bound and Unbound Data
6.​2 Challenges of Stream Processing
6.​3 The Uncertainty Component of Data Streams
6.​4 Apache Spark Streaming’s Execution Model
6.​5 Stream Processing Architectures
The Lambda Architecture
The Kappa Architecture
6.​6 Spark Streaming Architecture:​Discretized Streams
6.​7 Spark Streaming Sources and Receivers
Basic Input Sources
Advanced Input Sources
6.​8 Spark Streaming Graceful Shutdown
6.​9 Transformations on DStreams
 6.​10 Summary
Part II: Apache Spark Streaming
Chapter 7:​Spark Structured Streaming
7.​1 General Rules for Message Delivery Reliability
7.​2 Structured Streaming vs.​Spark Streaming
7.​3 What Is Apache Spark Structured Streaming?​
Spark Structured Streaming Input Table
Spark Structured Streaming Result Table
Spark Structured Streaming Output Modes
7.​4 Datasets and DataFrames Streaming API
Socket Structured Streaming Sources
Running Socket Structured Streaming Applications Locally
File System Structured Streaming Sources
Running File System Streaming Applications Locally
7.​5 Spark Structured Streaming Transformations
Streaming State in Spark Structured Streaming
Spark Stateless Streaming
Spark Stateful Streaming
Stateful Streaming Aggregations
7.​6 Spark Checkpointing Streaming
Recovering from Failures with Checkpointing
7.​7 Summary
Chapter 8:​Streaming Sources and Sinks
8.​1 Spark Streaming Data Sources
Reading Streaming Data from File Data Sources
Reading Streaming Data from Kafka
 Reading Streaming Data from MongoDB
8.​2 Spark Streaming Data Sinks
Writing Streaming Data to the Console Sink
Writing Streaming Data to the File Sink
Writing Streaming Data to the Kafka Sink
Writing Streaming Data to the ForeachBatch Sink
Writing Streaming Data to the Foreach Sink
Writing Streaming Data to Other Data Sinks
8.​3 Summary
Chapter 9:​Event-Time Window Operations and Watermarking
9.​1 Event-Time Processing
9.​2 Stream Temporal Windows in Apache Spark
What Are Temporal Windows and Why Are They Important
in Streaming
9.​3 Tumbling Windows
9.​4 Sliding Windows
9.​5 Session Windows
Session Window with Dynamic Gap
9.​6 Watermarking in Spark Structured Streaming
What Is a Watermark?​
9.​7 Summary
Chapter 10:​Future Directions for Spark Streaming
10.​1 Streaming Machine Learning with Spark
What Is Logistic Regression?​
Types of Logistic Regression
Use Cases of Logistic Regression
 Assessing the Sensitivity and Specificity of Our Streaming
ML Model
10.​2 Spark 3.​3.​x
Spark RocksDB State Store Database
10.​3 The Project Lightspeed
Predictable Low Latency
Enhanced Functionality for Processing Data/​Events
New Ecosystem of Connectors
Improve Operations and Troubleshooting
10.​4 Summary
Bibliography
Index
About the Author
Alfonso Antolínez García
is a senior IT manager with a long
professional career serving in several
multinational companies such as
Bertelsmann SE, Lafarge, and TUI AG. He
has been working in the media industry,
the building materials industry, and the
leisure industry. Alfonso also works as a
university professor, teaching artificial
intelligence, machine learning, and data
science. In his spare time, he writes
research papers on artificial intelligence,
mathematics, physics, and the
applications of information theory to
other sciences.
About the Technical Reviewer
Akshay R. Kulkarni
is an AI and machine learning evangelist
and a thought leader. He has consulted
several Fortune 500 and global
enterprises to drive AI- and data
science–led strategic transformations.
He is a Google Developer Expert, author,
and regular speaker at major AI and data
science conferences (including Strata,
O’Reilly AI Conf, and GIDS). He is a
visiting faculty member for some of the
top graduate institutes in India. In 2019,
he has been also featured as one of the
top 40 under-40 data scientists in India.
In his spare time, he enjoys reading,
writing, coding, and building next-gen AI products.
Part I
Apache Spark Batch Data Processing
© The Author(s), under exclusive license to APress Media, LLC, part of Springer
Nature 2023
A. Antolínez García, Hands-on Guide to Apache Spark 3
https://doi.org/10.1007/978-1-4842-9380-5_1

1. Introduction to Apache Spark for

Large-Scale Data Analytics
Alfonso Antolínez García1
(1) Madrid, Spain

Apache Spark started as a research project at the UC Berkeley AMPLab

in 2009. It became open source in 2010 and was transferred to the
Apache Software Foundation in 2013 and boasts the largest open
source big data community.
From its genesis, Spark was designed with a significant change in
mind, to store intermediate data computations in Random Access
Memory (RAM), taking advantage of the coming-down RAM prices that
occurred in the 2010s, in comparison with Hadoop that keeps
information in slower disks.
In this chapter, I will provide an introduction to Spark, explaining
how it works, the Spark Unified Analytics Engine, and the Apache Spark
ecosystem. Lastly, I will describe the differences between batch and
streaming data.

1.1 What Is Apache Spark?

Apache Spark is a unified engine for large-scale data analytics. It
provides high-level application programming interfaces (APIs) for Java,
Scala, Python, and R programming languages and supports SQL,
streaming data, machine learning (ML), and graph processing. Spark is
a multi-language engine for executing data engineering, data science,
and machine learning on single-node machines or clusters of
computers, either on-premise or in the cloud.
Spark provides in-memory computing for intermediate
computations, meaning data is kept in memory instead of writing it to
slow disks, making it faster than Hadoop MapReduce, for example. It
includes a set of high-level tools and modules such as follows: Spark
SQL is for structured data processing and access to external data
sources like Hive; MLlib is the library for machine learning; GraphX is
the Spark component for graphs and graph-parallel computation;
Structured Streaming is the Spark SQL stream processing engine;
Pandas API on Spark enables Pandas users to work with large datasets
by leveraging Spark; SparkR provides a lightweight interface to utilize
Apache Spark from the R language; and finally PySpark provides a
similar front end to run Python programs over Spark.
There are five key benefits that make Apache Spark unique and
bring it to the spotlight:
Simpler to use and operate
Fast
Scalable
Ease of use
Fault tolerance at scale
Let’s have a look at each of them.

Simpler to Use and Operate

Spark’s capabilities are accessed via a common and rich API, which
makes it possible to interact with a unified general-purpose distributed
data processing engine via different programming languages and cope
with data at scale. Additionally, the broad documentation available
makes the development of Spark applications straightforward.
The Hadoop MapReduce processing technique and distributed
computing model inspired the creation of Apache Spark. This model is
conceptually simple: divide a huge problem into smaller subproblems,
distribute each piece of the problem among as many individual solvers
as possible, collect the individual solutions to the partial problems, and
assemble them in a final result.
Fast
On November 5, 2014, Databricks officially announced they have won
the Daytona GraySort contest.1 In this competition, the Databricks team
used a Spark cluster of 206 EC2 nodes to sort 100 TB of data (1 trillion
records) in 23 minutes. The previous world record of 72 minutes using
a Hadoop MapReduce cluster of 2100 nodes was set by Yahoo.
Summarizing, Spark sorted the same data three times faster with ten
times fewer machines. Impressive, right?
But wait a bit. The same post also says, “All the sorting took place on
disk (HDFS), without using Spark’s in-memory cache.” So was it not all
about Spark’s in-memory capabilities? Apache Spark is recognized for
its in-memory performance. However, assuming Spark’s outstanding
results are due to this feature is one of the most common
misconceptions about Spark’s design. From its genesis, Spark was
conceived to achieve a superior performance both in memory and on
disk. Therefore, Spark operators perform regular operations on disk
when data does not fit in memory.

Scalable
Apache Spark is an open source framework intended to provide
parallelized data processing at scale. At the same time, Spark high-level
functions can be used to carry out different data processing tasks on
datasets of diverse sizes and schemas. This is accomplished by
distributing workloads from several servers to thousands of machines,
running on a cluster of computers and orchestrated by a cluster
manager like Mesos or Hadoop YARN. Therefore, hardware resources
can increase linearly with every new computer added. It is worth
clarifying that hardware addition to the cluster does not necessarily
represent a linear increase in computing performance and hence linear
reduction in processing time because internal cluster management,
data transfer, network traffic, and so on also consume resources,
subtracting them from the effective Spark computing capabilities.
Despite the fact that running in cluster mode leverages Spark’s full
distributed capacity, it can also be run locally on a single computer,
called local mode.
 If you have searched for information about Spark before, you
probably have read something like “Spark runs on commodity
hardware.” It is important to understand the term “commodity
hardware.” In the context of big data, commodity hardware does not
denote low quality, but rather equipment based on market standards,
which is general-purpose, widely available, and hence affordable as
opposed to purpose-built computers.

Ease of Use
Spark makes the life of data engineers and data scientists operating on
large datasets easier. Spark provides a single unified engine and API for
diverse use cases such as streaming, batch, or interactive data
processing. These tools allow it to easily cope with diverse scenarios
like ETL processes, machine learning, or graphs and graph-parallel
computation. Spark also provides about a hundred operators for data
transformation and the notion of dataframes for manipulating semi-
structured data.

Fault Tolerance at Scale

At scale many things can go wrong. In the big data context, fault refers
to failure, that is to say, Apache Spark’s fault tolerance represents its
capacity to operate and to recover after a failure occurs. In large-scale
clustered environments, the occurrence of any kind of failure is certain
at any time; thus, Spark is designed assuming malfunctions are going to
appear sooner or later.
Spark is a distributed computing framework with built-in fault
tolerance that takes advantage of a simple data abstraction named a
RDD (Resilient Distributed Dataset) that conceals data partitioning and
distributed computation from the user. RDDs are immutable collections
of objects and are the building blocks of the Apache Spark data
structure. They are logically divided into portions, so they can be
processed in parallel, across multiple nodes of the cluster.
The acronym RDD denotes the essence of these objects:
Resilient (fault-tolerant): The RDD lineage or Directed Acyclic Graph
(DAG) permits the recomputing of lost partitions due to node failures
from which they are capable of recovering automatically.
 Distributed: RDDs are processes in several nodes in parallel.
Dataset: It’s the set of data to be processed. Datasets can be the result
of parallelizing an existing collection of data; loading data from an
external source such as a database, Hive tables, or CSV, text, or JSON
files: and creating a RDD from another RDD.
Using this simple concept, Spark is able to handle a wide range of
data processing workloads that previously needed independent tools.
Spark provides two types of fault tolerance: RDD fault tolerance and
streaming write-ahead logs. Spark uses its RDD abstraction to handle
failures of worker nodes in the cluster; however, to control failures in
the driver process, Spark 1.2 introduced write-ahead logs, to save
received data to a fault-tolerant storage, such as HDFS, S3, or a similar
safeguarding tool.
Fault tolerance is also achieved thanks to the introduction of the so-
called DAG, or Directed Acyclic Graph, concept. Formally, a DAG is
defined as a set of vertices and edges. In Spark, a DAG is used for the
visual representation of RDDs and the operations being performed on
them. The RDDs are represented by vertices, while the operations are
represented by edges. Every edge is directed from an earlier state to a
later state. This task tracking contributes to making fault tolerance
possible. It is also used to schedule tasks and for the coordination of the
cluster worker nodes.

1.2 Spark Unified Analytics Engine

The idea of platform integration is not new in the world of software.
Consider, for example, the notion of Customer Relationship
Management (CRM) or Enterprise Resource Planning (ERP). The idea of
unification is rooted in Spark’s design from inception. On October 28,
2016, the Association for Computing Machinery (ACM) published the
article titled “Apache Spark: a unified engine for big data processing.” In
this article, authors assert that due to the nature of big data datasets, a
standard pipeline must combine MapReduce, SQL-like queries, and
iterative machine learning capabilities. The same document states
Apache Spark combines batch processing capabilities, graph analysis,
and data streaming, integrating a single SQL query engine formerly split
up into different specialized systems such as Apache Impala, Drill,
Storm, Dremel, Giraph, and others.
Spark’s simplicity resides in its unified API, which makes the
development of applications easier. In contrast to previous systems that
required saving intermediate data to a permanent storage to transfer it
later on to other engines, Spark incorporates many functionalities in
the same engine and can execute different modules to the same data
and very often in memory. Finally, Spark has facilitated the
development of new applications, such as scaling iterative algorithms,
integrating graph querying and algorithms in the Spark Graph
component.
The value added by the integration of several functionalities into a
single system can be seen, for instance, in modern smartphones. For
example, nowadays, taxi drivers have replaced several devices (GPS
navigator, radio, music cassettes, etc.) with a single smartphone. In
unifying the functions of these devices, smartphones have eventually
enabled new functionalities and service modalities that would not have
been possible with any of the devices operating independently.

1.3 How Apache Spark Works

We have already mentioned Spark scales by distributing computing
workload across a large cluster of computers, incorporating fault
tolerance and parallel computing. We have also pointed out it uses a
unified engine and API to manage workloads and to interact with
applications written in different programming languages.
In this section we are going to explain the basic principles Apache
Spark uses to perform big data analysis under the hood. We are going to
walk you through the Spark Application Model, Spark Execution Model,
and Spark Cluster Model.

Spark Application Model

In MapReduce, the highest-level unit of computation is the job; in
Spark, the highest-level unit of computation is the application. In a job
we can load data, apply a map function to it, shuffle it, apply a reduce
function to it, and finally save the information to a fault-tolerant storage
device. In Spark, applications are self-contained entities that execute
the user's code and return the results of the computation. As
mentioned before, Spark can run applications using coordinated
resources of multiple computers. Spark applications can carry out a
single batch job, execute an iterative session composed of several jobs,
or act as a long-lived streaming server processing unbounded streams
of data. In Spark, a job is launched every time an application invokes an
action.
Unlike other technologies like MapReduce, which starts a new
process for each task, Spark applications are executed as independent
processes under the coordination of the SparkSession object running in
the driver program. Spark applications using iterative algorithms
benefit from dataset caching capabilities among other operations. This
is feasible because those algorithms conduct repetitive operations on
data. Finally, Spark applications can maintain steadily running
processes on their behalf in cluster nodes even when no job is being
executed, and multiple applications can run on top of the same
executor. The former two characteristics combined leverage Spark
rapid startup time and in-memory computing.

Spark Execution Model

The Spark Execution Model contains vital concepts such as the driver
program, executors, jobs, tasks, and stages. Understanding of these
concepts is of paramount importance for fast and efficient Spark
application development. Inside Spark, tasks are the smallest execution
unit and are executed inside an executor. A task executes a limited
number of instructions. For example, loading a file, filtering, or applying
a map() function to the data could be considered a task. Stages are
collections of tasks running the same code, each of them in different
chunks of a dataset. For example, the use of functions such as
reduceByKey(), Join(), etc., which require a shuffle or reading a dataset,
will trigger in Spark the creation of a stage. Jobs, on the other hand,
comprise several stages.
Next, due to their relevance, we are going to study the concepts of
the driver program and executors together with the Spark Cluster
Model.
Spark Cluster Model
Apache Spark running in cluster mode has a master/worker
hierarchical architecture depicted in Figure 1-1 where the driver
program plays the role of master node. The Spark Driver is the central
coordinator of the worker nodes (slave nodes), and it is responsible for
delivering the results back to the client. Workers are machine nodes
that run executors. They can host one or multiple workers, they can
execute only one JVM (Java Virtual Machine) per worker, and each
worker can generate one or more executors as shown in Figure 1-2.
The Spark Driver generates the SparkContext and establishes the
communication with the Spark Execution environment and with the
cluster manager, which provides resources for the applications. The
Spark Framework can adopt several cluster managers: Spark’s
Standalone Cluster Manager, Apache Mesos, Hadoop YARN, or
Kubernetes. The driver connects to the different nodes of the cluster
and starts processes called executors, which provide computing
resources and in-memory storage for RDDs. After resources are
available, it sends the applications’ code (JAR or Python files) to the
executors acquired. Finally, the SparkContext sends tasks to the
executors to run the code already placed in the workers, and these
tasks are launched in separate processor threads, one per worker node
core. The SparkContext is also used to create RDDs.
Figure 1-1 Apache Spark cluster mode overview
In order to provide applications with logical fault tolerance at both
sides of the cluster, each driver schedules its own tasks and each task,
running in every executor, executes its own JVM (Java Virtual Machine)
processes, also called executor processes. By default executors run in
static allocation, meaning they keep executing for the entire lifetime of
a Spark application, unless dynamic allocation is enabled. The driver, to
keep track of executors’ health and status, receives regular heartbeats
and partial execution metrics for the ongoing tasks (Figure 1-3).
Heartbeats are periodic messages (every 10 s by default) from the
executors to the driver.

Figure 1-2 Spark communication architecture with worker nodes and executors

This Execution Model also has some downsides. Data cannot be

exchanged between Spark applications (instances of the SparkContext)
via the in-memory computation model, without first saving the data to
an external storage device.
As mentioned before, Spark can be run with a wide variety of
cluster managers. That is possible because Spark is a cluster-agnostic
platform. This means that as long as a cluster manager is able to obtain
executor processes and to provide communication among the
architectural components, it is suitable for the purpose of executing
Spark. That is why communication between the driver program and
worker nodes must be available at all times, because the former must
acquire incoming connections from the executors for as long as
applications are executing on them.
Figure 1-3 Spark’s heartbeat communication between executors and the driver

1.4 Apache Spark Ecosystem

The Apache Spark ecosystem is composed of a unified and fault-
tolerant core engine, on top of which are four higher-level libraries that
include support for SQL queries, data streaming, machine learning, and
graph processing. Those individual libraries can be assembled in
sophisticated workflows, making application development easier and
improving productivity.

Spark Core
Spark Core is the bedrock on top of which in-memory computing, fault
tolerance, and parallel computing are developed. The Core also
provides data abstraction via RDDs and together with the cluster
manager data arrangement over the different nodes of the cluster. The
high-level libraries (Spark SQL, Streaming, MLlib for machine learning,
and GraphX for graph data processing) are also running over the Core.

Spark APIs
Spark incorporates a series of application programming interfaces
(APIs) for different programming languages (SQL, Scala, Java, Python,
and R), paving the way for the adoption of Spark by a great variety of
professionals with different development, data science, and data
engineering backgrounds. For example, Spark SQL permits the
interaction with RDDs as if we were submitting SQL queries to a
traditional relational database. This feature has facilitated many
transactional database administrators and developers to embrace
Apache Spark.
Let’s now review each of the four libraries in detail.

Spark SQL and DataFrames and Datasets

Apache Spark provides a data programming abstraction called
DataFrames integrated into the Spark SQL module. If you have
experience working with Python and/or R dataframes, Spark
DataFrames could look familiar to you; however, the latter are
distributable across multiple cluster workers, hence not constrained to
the capacity of a single computer. Spark was designed to tackle very
large datasets in the most efficient way.
A DataFrame looks like a relational database table or Excel
spreadsheet, with columns of different data types, headers containing
the names of the columns, and data stored as rows as shown in Table 1-
1.

Table 1-1 Representation of a DataFrame as a Relational Table or Excel Spreadsheet

firstName lastName profession birthPlace

Antonio Dominguez Bandera Actor Málaga
Rafael Nadal Parera Tennis Player Mallorca
Amancio Ortega Gaona Businessman Busdongo de Arbas
Pablo Ruiz Picasso Painter Málaga
Blas de Lezo Admiral Pasajes
Miguel Serveto y Conesa Scientist/Theologist Villanueva de Sigena

On the other hand, Figure 1-4 depicts an example of a DataFrame.

Figure 1-4 Example of a DataFrame
A Spark DataFrame can also be defined as an integrated data
structure optimized for distributed big data processing. A Spark
DataFrame is also a RDD extension with an easy-to-use API for
simplifying writing code. For the purposes of distributed data
processing, the information inside a Spark DataFrame is structured
around schemas. Spark schemas contain the names of the columns, the
data type of a column, and its nullable properties. When the nullable
property is set to true, that column accepts null values.
SQL has been traditionally the language of choice for many business
analysts, data scientists, and advanced users to leverage data. Spark
SQL allows these users to query structured datasets as they would have
done if they were in front of their traditional data source, hence
facilitating adoption.
On the other hand, in Spark a dataset is an immutable and a strongly
typed data structure. Datasets, as DataFrames, are mapped to a data
schema and incorporate type safety and an object-oriented interface.
The Dataset API converts between JVM objects and tabular data
representation taking advantage of the encoder concept. This tabular
representation is internally stored in a binary format called Spark
Tungsten, which improves operations in serialized data and improves
in-memory performance.
Datasets incorporate compile-time safety, allowing user-developed
code to be error-tested before the application is executed. There are
several differences between datasets and dataframes. The most
important one could be datasets are only available to the Java and Scala
APIs. Python or R applications cannot use datasets.

Spark Streaming
Spark Structured Streaming is a high-level library on top of the core
Spark SQL engine. Structured Streaming enables Spark’s fault-tolerant
and real-time processing of unbounded data streams without users
having to think about how the streaming takes place. Spark Structured
Streaming provides fault-tolerant, fast, end-to-end, exactly-once, at-
scale stream processing. Spark Streaming permits express streaming
computation in the same fashion as static data is computed via batch
processing. This is achieved by executing the streaming process
incrementally and continuously and updating the outputs as the
incoming data is ingested.
With Spark 2.3, a new low-latency processing mode called
continuous processing was introduced, achieving end-to-end latencies
of as low as 1 ms, ensuring at-least-once2 message delivery. The at-
least-once concept is depicted in Figure 1-5. By default, Structured
Streaming internally processes the information as micro-batches,
meaning data is processed as a series of tiny batch jobs.

Figure 1-5 Depiction of the at-least-once message delivery semantic

Spark Structured Streaming also uses the same concepts of datasets

and DataFrames to represent streaming aggregations, event-time
windows, stream-to-batch joins, etc. using different programming
language APIs (Scala, Java, Python, and R). It means the same queries
can be used without changing the dataset/DataFrame operations,
therefore choosing the operational mode that best fits our application
requirements without modifying the code.
Spark’s machine learning (ML) library is commonly known as MLlib,
though it is not its official name. MLlib’s goal is to provide big data out-
of-the-box, easy-to-use machine learning capabilities. At a high level, it
provides capabilities such as follows:
Machine learning algorithms like classification, clustering,
regression, collaborative filtering, decision trees, random forests, and
gradient-boosted trees among others
 Featurization:
Term Frequency-Inverse Document Frequency (TF-IDF) statistical
and feature vectorization method for natural language processing
and information retrieval.
Word2vec: It takes text corpus as input and produces the word
vectors as output.
StandardScaler: It is a very common tool for pre-processing steps
and feature standardization.
Principal component analysis, which is an orthogonal
transformation to convert possibly correlated variables.
Etc.
ML Pipelines, to create and tune machine learning pipelines
Predictive Model Markup Language (PMML), to export models to
PMML
Basic Statistics, including summary statistics, correlation between
series, stratified sampling, etc.
As of Spark 2.0, the primary Spark Machine Learning API is the
DataFrame-based API in the spark.ml package, switching from the
traditional RDD-based APIs in the spark.mllib package.

Spark GraphX
GraphX is a new high-level Spark library for graphs and graph-parallel
computation designed to solve graph problems. GraphX extends the
Spark RDD capabilities by introducing this new graph abstraction to
support graph computation and includes a collection of graph
algorithms and builders to optimize graph analytics.
The Apache Spark ecosystem described in this section is portrayed
in Figure 1-6.
Figure 1-6 The Apache Spark ecosystem
In Figure 1-7 we can see the Apache Spark ecosystem of connectors.

Figure 1-7 Apache Spark ecosystem of connectors

1.5 Batch vs. Streaming Data

Nowadays, the world generates boundless amounts of data, and it
continues to augment at an astonishing rate. It is expected the volume
of information created, captured, copied, and consumed worldwide
from 2010 to 2025 will exceed 180 ZB.3 If this figure does not say much
to you, imagine your personal computer or laptop has a hard disk of 1
TB (which could be considered a standard in modern times). It would
be necessary for you to have 163,709,046,319.13 disks to store such
amount of data.4
Presently, data is rarely static. Remember the famous three Vs of big
data:
Volume
The unprecedented explosion of data production means that
storage is no longer the real challenge, but to generate actionable
insights from within gigantic datasets.
Velocity
Data is generated at an ever-accelerating pace, posing the
challenge for data scientists to find techniques to collect, process,
and make use of information as it comes in.
Variety
Big data is disheveled, sources of information heterogeneous, and
data formats diverse. While structured data is neatly arranged
within tables, unstructured data is information in a wide variety of
forms without following predefined data models, making it
difficult to store in conventional databases. The vast majority of
new data being generated today is unstructured, and it can be
human-generated or machine-generated. Unstructured data is
more difficult to deal with and extract value from. Examples of
unstructured data include medical images, video, audio files,
sensors, social media posts, and more.
For businesses, data processing is critical to accelerate data insights
obtaining deep understanding of particular issues by analyzing
information. This deep understanding assists organizations in
developing business acumen and turning information into actionable
insights. Therefore, it is relevant enough to trigger actions leading us to
improve operational efficiency, gain competitive advantage, leverage
revenue, and increase profits. Consequently, in the face of today's and
tomorrow's business challenges, analyzing data is crucial to discover
actionable insights and stay afloat and profitable.
It is worth mentioning there are significant differences between
data insights, data analytics, and just data, though many times they are
used interchangeably. Data can be defined as a collection of facts, while
data analytics is about arranging and scrutinizing the data. Data
insights are about discovering patterns in data. There is also a
hierarchical relationship between these three concepts. First,
information must be collected and organized, only after it can be
analyzed and finally data insights can be extracted. This hierarchy can
be graphically seen in Figure 1-8.

Figure 1-8 Hierarchical relationship between data, data analytics, and data insight
extraction

When it comes to data processing, there are many different

methodologies, though stream and batch processing are the two most
common ones. In this section, we will explain the differences between
these two data processing techniques. So let's define batch and stream
processing before diving into the details.

What Is Batch Data Processing?

Batch data processing can be defined as a computing method of
executing high-volume data transactions in repetitive batches in which
the data collection is separate from the processing. In general, batch
jobs do not require user interaction once the process is initiated. Batch
processing is particularly suitable for end-of-cycle treatment of
information, such as warehouse stock update at the end of the day, bank
reconciliation, or monthly payroll calculation, to mention some of them.
Batch processing has become a common part of the corporate back
office processes, because it provides a significant number of
advantages, such as efficiency and data quality due to the lack of human
intervention. On the other hand, batch jobs have some cons. The more
obvious one could be they are complex and critical because they are
part of the backbone of the organizations. As a result, developing
sophisticated batch jobs can be expensive up front in terms of time and
resources, but in the long run, they pay the investment off.
Another disadvantage of batch data processing is that due to its
large scale and criticality, in case of a malfunction, significant
production shutdowns are likely to occur. Batch processes are
monolithic in nature; thus, in case of rise in data volumes or peaks of
demand, they cannot be easily adapted.

What Is Stream Data Processing?

Stream data processing could be characterized as the process of
collecting, manipulating, analyzing, and delivering up-to-date
information and keeping the state of data updated while it is still in
motion. It could also be defined as a low-latency way of collecting and
processing information while it is still in transit. With stream
processing, the data is processed in real time; thus, there is no delay
between data collection and processing and providing instant response.
Stream processing is particularly suitable when data comes in as a
continuous flow while changing over time, with high velocity, and real-
time analytics is needed or response to an event as soon as it occurs is
mandatory. Stream data processing leverages active intelligence,
owning in-the-moment consciousness about important business events
and enabling triggering instantaneous actions. Analytics is performed
instantly on the data or within a fraction of a second; thus, it is
perceived by the user as a real-time update. Some examples where
stream data processing is the best option are credit card fraud
detection, real-time system security monitoring, or the use of Internet-
of-Things (IoT) sensors. IoT devices permit monitoring anomalies in
machinery and provide control with a heads-up as soon as anomalies or
outliers5 are detected. Social media and customer sentiment analysis
are other trendy fields of stream data processing application.
One of the main disadvantages of stream data processing is
implementing it at scale. In real life data streaming is far away from
being perfect, and often data does not flow regularly or smoothly.
Imagine a situation in which data flow is disrupted and some data is
missing or broken down. Then, after normal service is restored, that
missing or broken-down information suddenly arrives at the platform,
flooding the processing system. To be able to cope with situations like
this, streaming architectures require spare capacity of computing,
communications, and storage.

Difference Between Stream Processing and Batch

Processing
Summarizing, we could say that stream processing involves the
treatment and analysis of data in motion in real or almost-real time,
while batch processing entails handling and analyzing static
information at time intervals.
In batch jobs, you manipulate information produced in the past and
consolidated in a permanent storage device. It is also what is commonly
known as information at rest.
In contrast, stream processing is a low-latency solution, demanding
the analysis of streams of information while it is still in motion.
Incoming data requires to be processed in flight, in real or almost-real
time, rather than saved in a permanent storage. Given that data is
consumed as it is generated, it provides an up-to-the-minute snapshot
of the information, enabling a proactive response to events. Another
difference between batch and stream processing is that in stream
processing only the information considered relevant for the process
being analyzed is stored from the very beginning. On the other hand,
data considered of no immediate interest can be stored in low-cost
devices for ulterior analysis with data mining algorithms, machine
learning models, etc.
A graphical representation of batch vs. stream data processing is
shown in Figure 1-9.
Figure 1-9 Batch vs. streaming processing representation

1.6 Summary
In this chapter we briefly looked at the Apache Spark architecture,
implementation, and ecosystem of applications. We also covered the
two different types of data processing Spark can deal with, batch and
streaming, and the main differences between them. In the next chapter,
we are going to go through the Spark setup process, the Spark
application concept, and the two different types of Apache Spark RDD
operations: transformations and actions.

Footnotes
1 www.databricks.com/blog/2014/11/05/spark-officially-sets-a-
new-record-in-large-scale-sorting.html
2 With the at-least-once message delivery semantic, a message can be delivered
more than once; however, no message can be lost.

3 www.statista.com/statistics/871513/worldwide-data-created/

4 1 zettabyte = 1021 bytes.

5 Parameters out of defined thresholds.

© The Author(s), under exclusive license to APress Media, LLC, part of Springer Nature 2023
A. Antolínez García, Hands-on Guide to Apache Spark 3
https://doi.org/10.1007/978-1-4842-9380-5_2

2. Getting Started with Apache Spark

Alfonso Antolínez García1
(1) Madrid, Spain

Now that you have an understanding of what Spark is and how it works, we can get you set up to start using
it. In this chapter, I’ll provide download and installation instructions and cover Spark command-line utilities
in detail. I’ll also review Spark application concepts, as well as transformations, actions, immutability, and
lazy evaluation.

2.1 Downloading and Installing Apache Spark

The first step you have to take to have your Spark installation up and running is to go to the Spark download
page and choose the Spark release 3.3.0. Then, select the package type “Pre-built for Apache Hadoop 3.3 and
later” from the drop-down menu in step 2, and click the “Download Spark” link in step 3 (Figure 2-1).

Figure 2-1 The Apache Spark download page

This will download the file spark-3.3.0-bin-hadoop3.tgz or another similar name in your case,
which is a compressed file that contains all the binaries you will need to execute Spark in local mode on your
local computer or laptop.
What is great about setting Apache Spark up in local mode is that you don’t need much work to do. We
basically need to install Java and set some environment variables. Let’s see how to do it in several
environments.

Installation of Apache Spark on Linux

The following steps will install Apache Spark on a Linux system. It can be Fedora, Ubuntu, or another
distribution.

Step 1: Verifying the Java Installation

Java installation is mandatory in installing Spark. Type the following command in a terminal window to
verify Java is available and its version:

$ java -version

If Java is already installed on your system, you get to see a message similar to the following:

$ java -version
java version "18.0.2" 2022-07-19
Java(TM) SE Runtime Environment (build 18.0.2+9-61)
Java HotSpot(TM) 64-Bit Server VM (build 18.0.2+9-61, mixed mode, sharing)

Your Java version may be different. Java 18 is the Java version in this case.
If you don’t have Java installed
1. Open a browser window, and navigate to the Java download page as seen in Figure 2-2.

2. Click the Java file of your choice and save the file to a location (e.g., /home/<user>/Downloads).

Figure 2-2 Java download page

Step 2: Installing Spark

Extract the Spark .tgz file downloaded before. To unpack the spark-3.3.0-bin-hadoop3.tgz file in Linux, open
a terminal window, move to the location in which the file was downloaded

$ cd PATH/TO/spark-3.3.0-bin-hadoop3.tgz_location

and execute

$ tar -xzvf ./spark-3.3.0-bin-hadoop3.tgz

Step 3: Moving Spark Software Files

You can move the Spark files to an installation directory such as /usr/local/spark:

$ su -
Password:

$ cd /home/<user>/Downloads/
$ mv spark-3.3.0-bin-hadoop3 /usr/local/spark
$ exit

Step 4: Setting Up the Environment for Spark

Add the following lines to the ~/.bashrc file. This will add the location of the Spark software files and the
location of binary files to the PATH variable:

export SPARK_HOME=/usr/local/spark
export PATH=$PATH:$SPARK_HOME/bin

Use the following command for sourcing the ~/.bashrc file, updating the environment variables:

$ source ~/.bashrc

Step 5: Verifying the Spark Installation

Write the following command for opening the Spark shell:

$ $SPARK_HOME/bin/spark-shell

If Spark is installed successfully, then you will find the following output:

Setting default log level to "WARN".

To adjust logging level use sc.setLogLevel(newLevel). For SparkR, use
setLogLevel(newLevel).
22/08/29 22:16:45 WARN NativeCodeLoader: Unable to load native-hadoop
library for your platform... using builtin-java classes where applicable
22/08/29 22:16:46 WARN Utils: Service 'SparkUI' could not bind on port 4040.
Attempting port 4041.
Spark context Web UI available at http://192.168.0.16:4041
Spark context available as 'sc' (master = local[*], app id = local-
1661804206245).
Spark session available as 'spark'.
Welcome to
____ __
/ __/__ ___ _____/ /__
_\ \/ _ \/ _ `/ __/ '_/
/___/ .__/\_,_/_/ /_/\_\ version 3.3.0
/_/

Using Scala version 2.12.15 (Java HotSpot(TM) 64-Bit Server VM, Java 18.0.2)
Type in expressions to have them evaluated.
Type :help for more information.

scala>
You can try the installation a bit further by taking advantage of the README.md file that is present in the
$SPARK_HOME directory:

scala> val readme_file = sc.textFile("/usr/local/spark/README.md")

readme_file: org.apache.spark.rdd.RDD[String] = /usr/local/spark/README.md
MapPartitionsRDD[1] at textFile at <console>:23

The Spark context Web UI would be available typing the following URL in your browser:

http://localhost:4040

There, you can see the jobs, stages, storage space, and executors that are used for your small application.
The result can be seen in Figure 2-3.

Figure 2-3 Apache Spark Web UI showing jobs, stages, storage, environment, and executors used for the application running on
the Spark shell

Installation of Apache Spark on Windows

In this section I will show you how to install Apache Spark on Windows 10 and test the installation. It is
important to notice that to perform this installation, you must have a user account with administrator
privileges. This is mandatory to install the software and modify system PATH.

Step 1: Java Installation

As we did in the previous section, the first step you should take is to be sure you have Java installed and it is
accessible by Apache Spark. You can verify Java is installed using the command line by clicking Start, typing
cmd, and clicking Command Prompt. Then, type the following command in the command line:

java -version

If Java is installed, you will receive an output similar to this:

openjdk version "18.0.2.1" 2022-08-18

OpenJDK Runtime Environment (build 18.0.2.1+1-1)
OpenJDK 64-Bit Server VM (build 18.0.2.1+1-1, mixed mode, sharing)
 If a message is instead telling you that command is unknown, Java is not installed or not available. Then
you have to proceed with the following steps.
Install Java. In this case we are going to use OpenJDK as a Java Virtual Machine. You have to download the
binaries that match your operating system version and hardware. For the purposes of this tutorial, we are
going to use OpenJDK JDK 18.0.2.1, so I have downloaded the openjdk-18.0.2.1_windows-
x64_bin.zip file. You can use other Java distributions as well.
Download the file, save it, and unpack the file in a directory of your choice. You can use any unzip utility
to do it.

Step 2: Download Apache Spark

Open a web browser and navigate to the Spark downloads URL and follow the same instructions given in
Figure 2-1.
To unpack the spark-3.3.0-bin-hadoop3.tgz file, you will need a tool capable of extracting .tgz
files. You can use a free tool like 7-Zip, for example.
Verify the file integrity. It is always a good practice to confirm the checksum of a downloaded file, to be
sure you are working with unmodified, uncorrupted software. In the Spark download page, open the
checksum link and copy or remember (if you can) the file’s signature. It should be something like this (string
not complete):

1e8234d0c1d2ab4462 ... a2575c29c spark-3.3.0-bin-hadoop3.tgz

Next, open a command line and enter the following command:

certutil -hashfile PATH\TO\spark-3.3.0-bin-hadoop3.tgz SHA512

You must see the same signature you copied before; if not, something is wrong. Try to solve it by
downloading the file again.

Step 3: Install Apache Spark

Installing Apache Spark is just extracting the downloaded file to the location of your choice, for example,
C:\spark or any other.

Step 4: Download the winutils File for Hadoop

Create a folder named Hadoop and a bin subfolder, for example, C:\hadoop\bin, and download the winutils.​
exe file for the Hadoop 3 version you downloaded before to it.

Step 5: Configure System and Environment Variables

Configuring environment variables in Windows means adding to the system environment and PATH the
Spark and Hadoop locations; thus, they become accessible to any application.
You should go to Control Panel ➤ System and Security ➤ System. Then Click “Advanced system settings”
as shown in Figure 2-4.
Figure 2-4 Windows advanced system settings
You will be prompted with the System Properties dialog box, Figure 2-5 left:
1. Click the Environment Variables button.

2. The Environment Variables window appears, Figure 2-5 top-right:

a. Click the New button.

3. Insert the following variables:

a. JAVA_HOME: \PATH\TO\YOUR\JAVA-DIRECTORY

b. SPARK_HOME: \PATH\TO\YOUR\SPARK-DIRECTORY

c. HADOOP_HOME: \PATH\TO\YOUR\HADOOP-DIRECTORY

You will have to repeat the previous step twice, to introduce the three variables.

4. Click the OK button to save the changes.

5. Then, click your Edit button, Figure 2-6 left, to edit your PATH.

6. After that, click the New button, Figure 2-6 right.

And add these new variables to the PATH:
a. %JAVA_HOME%\bin

b. %SPARK_HOME%\bin

c. %HADOOP_HOME%bin
 Configure system and environment variables for all the computer’s users.

7. If you want to add those variables for all the users of your computer, apart from the user performing this
installation, you should repeat all the previous steps for the System variables, Figure 2-6 bottom-left,
clicking the New button to add the environment variables and then clicking the Edit button to add them
to the system PATH, as you did before.

Verify the Spark installation.

If you execute spark-shell for the first time, you will see a great bunch of informational messages. This is
probably going to distract you and make you fear something went wrong. To avoid these messages and
concentrate only on possible error messages, you are going to configure the Spark logging parameters.

Figure 2-5 Environment variables

Figure 2-6 Add variables to the PATH
Go to your %SPARK_HOME%\conf directory and find a file named log4j2.properties.template,
and rename it as log4j2.properties. Open the file with Notepad or another editor.
Find the line

rootLogger.level = info

And change it as follows:

rootLogger.level = ERROR

After that, save it and close it.

Open a cmd terminal window and type spark-shell. If the installation went well, you will see
something similar to Figure 2-7.
Figure 2-7 spark-shell window
To carry out a more complete test of the installation, let’s try the following code:

val file =sc.textFile("C:\\PATH\\TO\\YOUR\\spark\\README.md")

This will create a RDD. You can view the file’s content by using the next instruction:

file.take(10).foreach(println)

You can see the result in Figure 2-8.

Figure 2-8 spark-shell test code

Another Random Scribd Document
with Unrelated Content
 “Do you know,” said Jack, while they were impatiently awaiting his
return, “I fancy I've seen that fellow before, though I can't for the
life of me recall where.”
The guide not returning, they at length went in search of him. But
Pug's “bery nice black man” was nowhere to be seen.
“Looks as if he meant to leave us in the lurch,” Jack began, when
a shout of “Him here got, sa'b!” from Puggles, brought them back to
the footpath at a run.
The new-comer, however, was not the missing guide, but a
stranger. He had been belated at the bazaar, he told them, and was
now making his way home to the village close by. In answer to
inquiries concerning the shark-charmer, he imparted a startling piece
of news.
The shark-charmer had indeed taken his departure from the
bazaar, but not to this village. He had, the stranger asserted,
embarked in a coasting vessel bound for the opposite side of the
Strait.
Don uttered an exclamation of impatience and dismay.
“He will be safe on the Madras coast by daybreak!” he cried.
“Him there coming from, sa'b,” put in Puggles.
“And that lying guide,” added Jack savagely, “was an accomplice,
left behind to throw us off the scent. Don't you remember you saw
some one swimming after the boat? I'll lay any odds 'twas the lascar.
He got to the bazaar ahead of us—he could easily manage that, you
know, by running along the sands—muffled himself up so that I
shouldn't recognise him, and then led us on this fool's errand while
his master made off. Well, good-bye to the golden pearl!”
“Not a bit of it!” cried Don resolutely. “I, for one, shan't relinquish
the quest, come what may. Back we go to the schooner! Then, with
the governor's consent, we'll go further. Point, Pug!”
Jack seconding this proposal heartily, they rewarded the
communicative native, and with unflagging determination retraced
their steps. By four o'clock they had traversed something more than
half the distance. The dawn star was now high above the eastern
horizon. A rosy flush in the same quarter warned them that day was
rapidly approaching. Suddenly, out of the gray distance ahead, a dull
booming sound floated to their ears.
“The schooner's signal gun!” exclaimed Don. “Why, it's too early
yet by a good hour for the boats to put out. What's the governor
about, I wonder?”
“There it goes again!” cried Jack. “I never knew it to be fired twice
of a morning, did you?”
“Never,” said Don uneasily. “Come, let us get on!”
Off again at their best speed, until at length the heavy path was
exchanged for the smooth, hard sand of the beach. On this it was
possible to make better time, and by five o'clock they were within
half a mile or so of the bazaar. It was now daylight; but a sharp
bend in the coast-line, and the sand-hills which here rose steeply
from the beach on their left, as yet concealed both the landing-place
and the schooner from view.
Puggles, who in spite of his shortness of limb had throughout
maintained the lead by several rods, suddenly stopped, and fell to
shouting and gesticulating wildly. Breaking into a run, Don and Jack
speedily came up with him.
“Look, sa'b, look!” gasped Puggles, pointing down the coast with
shaking hand.
Far away on the horizon appeared the white canvas of a vessel
bowling along before the fresh land breeze, with a fleet of fishing-
boats spreading their fustian-hued wings in her wake.
The spot where our adventurers had last seen the schooner at
anchor was deserted. She was gone!
CHAPTER IV.—INTRODUCES BOSIN,
AND TELLS HOW CAPTAIN MANGO
PROVED HIMSELF A TRUMP.

T
he schooner had sailed!
When the dismay caused by this unlooked-for turn of events
had somewhat abated, Jack, catching sight of the black boy's
lugubrious face, fell to laughing heartily.
“After all,” said Don, following his chum's example, “it's no use
crying over spilt milk. I'm not sure but this is the best thing that
could have happened, Jack.”
“My opinion exactly. We began the quest without the guv's
knowledge, and nolens volens we must continue it without his
consent. What's the next piece on the programme, old fellow?”
Don pondered for a moment.
“Why, first,” said he, “we must ascertain whether that fellow told
us the truth about the shark-charmer's having gone across the Strait.
If it turns out that he has, then I'm not exactly clear yet as to what
our next move will be, though I've an idea. You shall hear what it is
later on.”
“All right,” said Jack “whatever course you decide on, I'm with you
heart and fist, anyhow.”
Arrived in the vicinity of the bazaar, Puggles was at once
despatched to learn what he could of the shark-charmer's
movements. In half an hour he returned. His report confirmed that
which they had already heard. The shark-charmer had undoubtedly
sailed for the opposite side of the Strait.
Throwing himself upon his back in the shade of the banyan tree
which had witnessed the discovery of the pearls, Don drew his
helmet over his eyes, and pondered long and deeply.
“Jack,” said he at length, “how much money have you?”
Jack turned out his pockets.
“Barely a rupee and a half,” said he,
“And I,” added Don, turning out his own, “have four and a half.”
“Here one rupee got, sa'b,” cried Puggles, tugging at his waist-
cloth. “Me giving him heart and fist, anyhow.”
“That makes seven rupees, then,” said his master, laughing; “not
much to continue the quest on, eh, Jack?”
“We'll manage,” said Jack hopefully. “But, I say, you haven't told
us your plans yet, old fellow.”
“Oh, our course is as plain as a pikestaff. We'll hire a native boat,
and follow the shark-charmer across the Strait. The only question is,
where's enough money to come from?”
“Don't know,” said Jack, “unless we try to borrow it in the bazaar.”
At this juncture there occurred an interruption which, unlikely
though it may seem, was destined to lead to a most satisfactory
solution of this all-important and perplexing question.
While this conversation was in progress Puggles had seated
himself at a short distance behind his master, and throwing his
turban aside, proceeded to untie and dress the one tuft of hair which
adorned the back of his otherwise cleanly shaven head.
Directly above the spot where he sat there extended far out from
the trunk of the banyan a branch of great size, from which dangled
numerous rope-like air-roots, which, reaching to-within a few feet of
the ground, swayed to and fro in the morning breeze. Out along this
branch crept a large black monkey, which, after taking a cautious
survey of Puggles and his unconscious neighbours, glided noiselessly
down one of the swinging roots, and from its extremity dropped
lightly to the ground within a yard of the discarded turban.
Cautiously, with his cunning ferret-eyes fastened on the preoccupied
Puggles, the monkey approached the coveted prize, snatched it up,
and with a shrill cry of triumph turned tail and fled.
 Looking quickly round at the cry, Puggles took in the situation at a
glance.
“Sa'b! Sar!” he shouted, invoking the aid of both his master and
Jack in one breath, “one black debil monkey me turban done
hooking;” and leaping to his feet he gave chase.
“Why,” said Jack, “the little beast is making a bee-line for the old
fort. It must be Bosin, Captain Mango's pet monkey.”
“Captain Mango!” cried Don, as though seized with some sudden
inspiration. “Never thought of him until this minute!” and, clapping
on his helmet, he set off at a run after Puggles and the monkey.
Away like the wind went the monkey, the stolen turban trailing
after him through the sand like a great serpent; and away went
Puggles, his back hair flying. But while Puggles was short of wind,
the monkey was nimble of foot. The race was, therefore, unequal
from the start, its finish more summary than satisfactory; for as
Puggles ran, with his eyes glued upon the scurrying monkey, and his
mouth wide-stretched, his foot unluckily came in contact with a tree-
root, which lay directly across his path. Immediately beyond was a
bed of fine soft sand, and into this he pitched, head foremost. Just
then his master came up, with Jack at his heels.
“Sa'b! Sar!” spluttered Puggles, knuckling his eyes and spitting
sand right and left, “debil monkey done stole turban. Where him
going, sa'b?”
“Come on, Pug,” his master called out as he ran past; “your
headgear's all right—the monkey's taken it into the fort.”
The structure known as “the fort” occupied the summit of a sandy
knoll, about which grew a thick plantation of cocoanut palms,
seemingly as ancient as the fort itself. The walls of the enclosure had
so crumbled away in places as to afford glimpses of the buildings
within. These were two in number—one an ancient godown, as
dilapidated as the surrounding wall; the other, a bungalow in
excellent repair, blazing in all the glory of abundant whitewash.
Towards this building, after passing the tumble-down gateway,
with its turreted side-towers alive with pigeons, Don and his
companion shaped their course; for this was by no means their first
visit to the fort. A broad, low-eaved verandah shaded the front of
the bungalow, and upon this opened two or three low windows and
a door. As they drew near a shadow suddenly darkened the doorway,
and there emerged upon the verandah an individual whose pea-
jacket and trousers of generous nautical cut unmistakably
proclaimed him to be a seafaring man. About his throat a
neckerchief of a deep marine blue was tied in a huge knot; while
from beneath the left leg of his wide pantaloons there projected the
end of a stout wooden substitute for the real limb.
On catching sight of his visitors an expression of mingled
astonishment and pleasure overspread his honest, bronzed features.

“Shiver my binnacle!” roared he, advancing with a series of hitches

and extended hand to meet them. “Shiver my binnacle if it ain't
Master Don and Master Jack made port again! An' split my topsails,
yonder's the little nigger swab a-bearin' down under full sail out o'
the offin! Lay alongside the old hulk, my hearties, an' tell an old
shipmate what may be the meaning of it all. Where away might the
schooner be, I axes?”
 “To tell you the truth, Captain Mango,” said Don, shaking the old
sailor by the hand in hearty fashion, “on that point we're as much at
sea as yourself. We pulled ashore last night on a little matter of
business of our own—without the skipper's knowledge, you
understand—and when we returned here this morning the schooner
had sailed.”
“Shiver my figger-head if ever I hear'd any yarn to beat that!”
roared the captain, gripping Jack by the hand in turn. “An' d'ye mean
to say now, as ye ain't atween decks, sound asleep in your bunks,
when the wessel gets under weigh?”
“Not we,” cried Jack, laughing at the captain's puzzled face and
earnest manner; “we were miles down the coast just then.”
“Belay there!” sang out the captain, rubbing his stubbly chin in
greater perplexity than ever. “Blow me if I'm able to make out what
tack you're on, lad. For, d'ye see, I lays alongside o' the wessel
somewheres about eight bells—arter they fires the signal gun, d'ye
see—to pay my 'specks to the master like, and shiver my bulk-head,
when I axes what might your bearin's be, lads, he ups an' says, 'The
younkers be below decks,' says he; an' so he weighs anchor, an'
shapes his course for Colombie.”
“It's plain there's been a double misunderstanding,” said Don; “we
knew nothing of the guv's intention to sail this morning, and he
knew nothing of our absence from the schooner. He, of course,
thought we were below, and so sailed without us. As I hinted just
now, we're ashore on business of our own. Fact is, we're in a fix,
and we want your advice.”
“Adwice is it?” cried the captain, leading his visitors indoors; “fire
away, lads, till I hears what manner o' stuff you wants, and the wery
best a water-logged old seaman can give ye, ye shall have—shiver
my figger-head if ye shan't! Howsomedever, afore we lays our heads
together like, I'll pipe the cook and order ye some wittles.” This
hospitable duty performed, the captain threw himself into a chair
with his “main-brace,” as he jocosely termed his wooden leg,
extended before him, and, bidding Don proceed with what he had to
say, composed himself to listen. Whereupon Don recounted the
cause and manner of the shark-charmer's punishment, the discovery
and subsequent loss of the pearls, together with their reasons for
suspecting the shark-charmer of the theft, as well as how they had
been tricked by the latter's supposed accomplice, and on making
their way back to the beach had found, not the schooner as they
expected, but a deserted roadstead.
“The thief has crossed the Strait, there's no doubt about that,” he
concluded. “We want to hire a boat and go in pursuit of him; but the
governor's sudden departure has placed us in a dilemma. The fact is,
captain, we haven't enough cash to——”
“Belay there!” roared the captain, stumping across the room to a
side-table. “Hold hard, lads, till I has a whiff o' the fragrant! Shiver
my maintop! there's nothing like tobackie for ilin' up a seaman's
runnin' gear, says you!”
Filling a meerschaum pipe of high colour and huge dimensions
from a pouch almost as large as a sailor's bag, the captain reseated
himself, and for some minutes puffed away in silence.
“Shiver my smokestack!” cried he at last, slapping his thigh
energetically with his disengaged hand, “the thing's as easy as boxin'
the compass, lads! You axes me for adwice: my adwice is, up anchor
and away as soon as ye can. Supplies is low, says you. What o' that?
I axes. There's a canvas bag in the old sea-chest yonder as'll charter
all the boats hereabouts, if so be as they're wanted, which they ain't,
d'ye mind me. Ye can dror on the canvas bag, lads, an' welcome—
why not? I axes. An' there's as tight a leetle cutter in the boat-house
below as ever ye clapped eyes on—which the Jolly Tar's her name—
what's at your sarvice, shiver my main-brace if it ain't! An' blow me,
as the fog-horn says to the donkey-engine, I'll ship along with ye,
lads!”

“An' a-sailin' we'll go, we'll go;

An' a-sailin' we will go-o-o!”
 he concluded, with a stave of a rollicking old sea-song.
“Hurrah! You're a trump, captain, and no mistake!” cried Jack,
while Don sprang forward and gripped the old sailor's hand with a
heartiness that showed how thoroughly he appreciated this
generous offer.
“Why, y'see, lads,” explained the captain apologetically, “'twould
be ekal to a-sendin' of ye to Davy Jones if I was to let ye go pokin”
round this 'ere Strait alone. Now me—rope-yarn an' marlin-spikes!—
there ain't a reef, nor a shool, nor yet a crik atween Colombie an'
Jafna P'int but what's laid down on this 'ere old chart o' mine,”
tapping his forehead significantly. “An' besides I'm a-spilin' for a bit
o' the briny, so with you I ships—an' why not? I axes.”
“And right glad of your company and assistance we'll be, captain,”
said Don. “The main difficulty will be, of course, to discover to what
part of the Indian coast the thief has gone.”
The captain puffed thoughtfully at his pipe.
“Why, as for that,” said he at length, “I've an idee as I knows his
reckonin', shiver my binnacle if I ain't! But that's neither here nor
there at this present speakin'. Ballast's the first consideration, lads;
so dror up your cheers an' tackle the perwisions.”
When they had complied with this welcome invitation to the entire
satisfaction of the captain and their own appetites, “Now, lads,” said
the old sailor gaily, “do ye turn in an' snatch a wink o' sleep, whiles I
goes an' gets the cutter ready for puttin' to sea. For, says you, look
alive's the word if so be as we wants to overhaul the warmint as
took the treasure in tow. Spike my guns!—we'll make him heave to
in no time!

“For all things is ready, an' nothing we want,

To fit out our ship as rides so close by;
Both wittles an' weapons, they be nothing scant,
Like worthy sea-dogs ourselves we will try!”
 Trolling this ditty, the captain stumped away, while his guests
made themselves as comfortable as they could, and sought the
slumber of which they stood so much in need.
It was late in the afternoon when they woke. Puggles had
disappeared. Proceeding to the beach, they found the captain,
assisted by a small army of native servants, busily engaged in
putting the-finishing touches to his preparations for the proposed
voyage. Just above the surf-line lay the Jolly Tar—a trim little craft,
fitted with mast-and sprit, whose sharp, clean-cut lines betokened
possibilities in the way of speed that promised well for the issue of
their enterprise. In the cuddy, amid a bewildering array of pots,
pans, and pannkins, Puggles had already installed himself, his
shining face a perfect picture of self-complacent good-nature, whilst
Bosin, newly released from durance vile, sat in the stern-sheets,
cracking nuts-and jabbering defiance at his black rival.
“A purty craft!” chuckled the captain, checking for a moment the
song that was always on his lips, as he led his visitors to the cutters
side; “stave my water-butt if there's anything can pull ahead of her
in these 'ere parts. Everything shipshape 'an' ready to hand, d'ye see
—wittles for the woyage, an' drink for the woyagers. Likewise
ammunitions o' war,” cried he proudly, pointing out a number of
muskets and shining cutlasses, which a servant just then brought up
and placed on board.

“Bath, wittles an' weapons, they be nothing scant,

So like worthy sea-dogs ourselves we will try.”

“What with the cutlasses and guns, and the captain's wooden leg,
to say nothing of our small-arms, Don,” said Jack, “we'd better set
up for buccaneers at once.”
“Shiver my main-brace! a wooden leg ain't sich a bad article arter
all,” rejoined the captain; “specially when a seaman falls overboard.
With a life-buoy o' that nater rove on to his starn-sheets, he's sartin
to keep one leg above water, says you.”
 “No doubt of that, even if he goes down by the head,” assented
Don, laughing. “But, I say, captain, what's in the keg—spirits?”
“Avast there!” replied the captain, half shutting one eye and
contemplating the keg with the other, “that 'ere keg, lads, has stuff
in its hold what's a sight better'n spurts. Gunpowder, lads, that's
what it is; and spike my guns if we don't broach the same to the
health of old Salambo when we falls in with him. What say you,
lads?

“We always be ready,

Steady, lads, steady;
We'll fight an' we'll conquer agin an' agin.”

“I hope we shan't have to do that, captain,” said Jack gravely. “But

powder or no powder, we'll pay the beggar out, anyhow.”
“Right, lad; so we'll just take the keg along with us in case of
emargencies like. Shiver my compass, there's no telling aforehand
what this 'ere wenture may lead to.”
To whatever the venture was destined to lead, preparations for its
successful inception went on apace, and by nightfall all was in
readiness. The captain declaring that he “couldn't abide the ways o'
them 'ere jabbering nigger swabs when afloat,” the only addition to
their numbers was a single trusty servant of the old sailor's, who
was taken along rather with a view to the cutter's safety when they
should be ashore than because his assistance was required in sailing
her.
Don having despatched an overland messenger with a letter to his
father, explaining their absence and proposed undertaking, as the
full moon rose out of the eastern sea the cutter was launched.
Half an hour later, with her white sails bellying before the
freshening land-breeze, she bore away for the opposite shore of the
Strait, on that quest from which one at least of those on board was
destined never to return.
 While her sails were yet visible in the moonlit offing, a native crept
down to the deserted beach. He was a dark-skinned, evil-featured
fellow; and the moonlight, falling upon his face, showed his left
temple to be swollen and discoloured as from a recent blow. On his
shoulder he carried a paddle-and a boathook.
“The wind will drop just before dawn,” he muttered, as he stood a
moment noting the strength and direction of the breeze. “Then, you
white-devil, then!” and he patted the boathook affectionately, as if
between him and it there existed some secret, dark understanding.
Selecting a ballam or “dug-out” from amongst a number that lay
there, he placed the boathook carefully in the bottom of the frail
skiff, and launched it almost in the furrow which the cutter's keel
had ploughed in the yielding sand. Then springing in, and plying his
paddle with rapid strokes, he quickly disappeared in the cutter's
wake.
CHAPTER V.—THE LASCAR GETS HIS
KNIFE BACK.

H
er light sails winged to catch every breath of the light but
steady breeze that chased her astern, the cutter for some
hours bowled through the water merrily. In the cabin Puggles
and the captain's Black servant snored side by side; whilst Don and
Jack lolled comfortably just abaft the mast-, where the night wind,
soft and spicy as the breath of Eden, would speedily have lulled
them to slumber but for the excitement that fired their blood. The
Captain was at the tiller, Bosin curled up by his side.
“If this 'ere wind holds, lads,” exclaimed the old sailor abruptly,
after a prolonged silence on his part, “we'd orter make the island
agin sunrise, shiver my forefoot if we don't!”
Don looked up with half-sleepy interest. “Island, captain? I
thought we were heading straight for the Indian coast.”
“Ay, so we be, straight away. But, y'see, lad, as I hinted a while
back, I has a sort o' innard idee, so to say, as the old woman ain't
on the mainland.”
“What old woman?” queried Jack, yawning. “Didn't know there
was one in the case, captain.”
The old sailor burst into a roar of laughter. “An' no more there
ain't, lad,” chuckled he; “an' slit my hammock if we wants one, says
you. Forty odd year has I sailed the seas, an' hain't signed articles
with any on 'em yet. A tight leetle wessel's the lass for me, lads; for,
unship my helm! she never takes her own head for it, says you.”
“Then what about the old woman you mentioned captain?” said
Don banteringly.
“Avast there now! An' d'ye mean to say,” demanded the captain
incredulously, “as you ain't ever hear'd tell o' the fish what sails
under that 'ere name? And a wicious warmint he is, too, shiver my
keelson! Hysters is his wittles, an' pearls his physic; he lives on 'em,
so to say; an' so I calls the cove as took them pearls o' your'n in tow
an old woman; an' why not, I axes?”
“But what about the island you spoke of just now, captain?”
“Why, d'ye see, it's this way, lads; there's an island off the coast
ahead, a sort o' holy place like, where them thievin' natives goes
once a year an' gets salwation from their sins. Howsomedever, that's
neither here nor there, says you; the p'int's this, lads: Somewheres
about the month o' March, which is this same month, says you, here
the priests flocks from all parts, an' here they stays until they gets a
purty pocketful o' cash. Now, my idee's this, d'ye see: the old
woman—which I means Salambo—lays alongside the schooner an'
takes them pearls o' your'n in tow. What for? says you. Cash, says I.
An' so, shiver my main-brace, he shapes his course for this 'ere
island, an' sells 'em to the priests.”
“Very likely,” assented Don. “He's bound to carry them to the best
market, of course.”
“And equally of course the best market is where the most priests
are. By Jove, you have a headpiece, captain!” put in Jack.
“I'm afraid, though,” resumed Don, after a moment's silence, “I'm
afraid it's not going to be so easy to come at the old fellow as we
think. You say this island's a sort of holy place; well, it's bound to be
packed with natives to the very surf-line in that case. Rather ticklish
work, I should think, taking the old fellow among so many pals.
There's the getting ashore, too; what's to prevent their sighting us?”
“Belay there!” roared the captain, vigorously thumping the bottom
of the boat with his wooden leg. “Shiver my main-brace! what sort o'
craft do ye take me for, I axes? A island's a island the world over—a
lump o' land what's floated out to sea. Wery good, that bein' so—
painters an' boathooks!—ain't it as easy a-boardin' of her through
the starn-ports as along o' the forechains?”
“Oh, you mean to make the back of the island, and steal a march
on old Salambo from the rear, then?” cried Don. “A capital idea!”
 “You're on the right tack there, lad,” assented the captain.
“There's as purty a leetle cove at the backside o' that island as ever
wessel cast anchor in, an' well I knows it, shiver my binnacle!
Daylight orter put us into it, if so be—— Split my sprit-sail, lads, if it
ain't a-fallin' calm!”

An ominous flapping of the cutter's sails confirmed the captain's

words. During the half-hour over which this conversation extended
the wind had gradually died away until scarcely a movement of the
warm night air could be felt. The cutter, losing her headway, rolled
lazily to the motion of the long, glassy swell. Consulting his watch,
Don announced it to be three o'clock.
“This 'ere's the lull at ween the sea-breeze an' the land-breeze,”
observed the captain complacently, working the tiller from side to
side as if trying to coax renewed life into the cutter. “How-
somedever, it hadn't orter last long. Stow my sea-chest!—we'll turn
in an' catch a wink o' sleep atween whiles. Here, Master Jack, lad!
take a turn at the tiller, will 'ee?”
Settling himself in the captain's place, with instructions to call that
worthy sea-dog should the wind freshen, Jack began his first watch.
Becalmed as they were, the tiller was useless, so he let it swing,
contenting himself with keeping a bright look-out. But soon he
concluded even this to be an unnecessary precaution. Not a sail was
to be seen on the moonlit expanse of ocean; and even had a score
been in sight, there would still have been no danger whatever, in the
absence of wind, of their interfering with the cutter. In fine, so
secure did he consider their position, and so soporific an influence
did the comfortable snoring of Don and the captain exercise upon
him, that in a very short time his head sank upon his breast, and he
fell asleep.
He had slept soundly for perhaps an hour, when a cold, touch
upon the cheek startled him into consciousness.
Rousing himself, he found Bosin at his elbow. The monkey for
some reason had left his masters side, and it was his clammy paw,
Jack now perceived, that had awakened him. It almost looked as if
the monkey had purposely interrupted his slumber. But what had
roused the monkey? Jack rose to his feet, stretched himself, and
looked about him.
The night was, if anything, more breathlessly calm than when he
had relieved the captain. Upon the unruffled, deserted sea the
moonlight shimmered with a brilliancy uncanny in its ghostliness.
From the cutter straight away to and around the horizon not an
object, so far as he could make out, darkened the surface of the
water, except under the cutter's larboard bow, where the moon-cast
shadow of the sail fell. He fancied he saw something move there,
close under the bow where the shadow lay blackest. The next
instant it had disappeared.
“All right, Bosin, old chap,” said he, stroking the monkeys back; “a
false alarm this time—back to your quarters, old fellow!”
The monkey, as if reassured by these words, crept away to his
master's side, whilst Jack resumed his seat, and again dozed off.
Not for long, however. It was not the monkey this time, but a
sudden and by no means gentle thud against the cutters side that
roused him. Awake in an instant, he sprang to his feet with a
startled exclamation. Close under the cutter's quarter lay a canoe,
and in the canoe there stood erect a native, with what appeared to
be a boathook poised above his head. All this Jack took in at a
glance.
“Boat ahoy! Who's that?” he cried sharply, his hand instinctively
seeking the knife at his belt.
For answer came a savage, muttered imprecation; and the
boathook, impelled with all the strength of the native's muscular
arms, descended swiftly through the air. Starting aside, Jack
received the blow' upon his left arm, off which the heavy, iron-shod
weapon glanced, striking the gun'le of the boat with a resounding
crash.
“The lascar!” muttered Jack between his teeth, as he stepped back
a pace and whipped out his knife in anticipation of a renewal of the
attack.
But the lascar, baffled in his attempt to take his enemy by
surprise, did not repeat the blow. Instead, he drew off, and with all
his strength drove the iron point of the boathook through the
cutter's side below the water-line.
“By Heaven!” cried Jack, as he perceived his intention, “I'll soon
settle scores with you, my fine fellow.”
Springing lightly upon the gunle, at a single bound he cleared the
few yards of open water intervening between the cutter and the
canoe, and with all the impetus of his leap drove the knife into the
lascar's shoulder up to the very hilt.
The lascar went overboard like a log. The canoe overturning at the
same instant, Jack followed him.
The noise of the scuffle having roused the sleepers, all was now
wild commotion on board the cutter; Captain Mango roaring out his
strange nautical oaths, and stumping hither and thither in search of
something with which to stop the leak; Don shouting wildly at Jack,
as he hastily threw off shoes and coat to swim to his assistance.
Before either well knew what had actually happened, Jack was
alongside.
“What's the matter? Are you hurt?” Don inquired anxiously, giving
him a hand over the side.
“Hurt? No, not a scratch,” said Jack lightly, scrambling inboard,
and proceeding to wring the water from his dripping garments. “A
narrow squeak, though. That lascar villain has got his knife back,
anyhow.”
“Who?” cried Don in amazement; for, amid the confusion, neither
he nor the captain had seen the native.
“The lascar. What else do you suppose I went over the side for? I
dozed off, you see, captain,” said Jack, as the old sailor came
stumping up with extended hand, “and that lascar dog, who must
have seen us sail and paddled after us, stole a march on me, and
tried to crack my nut with a boathook. Lucky for me, he ran his
canoe against the side and woke me up. Got on my feet just in time
to dodge the blow. Then he smashed the boathook through the side.
By Jove! I forgot that. We must stop the leak, or we'll fill in no time.”
“Stave my quarter!” roared the captain, detaining him as he was
about to rush aft. “The leak's stopped, lad; but blow me if ever I
hear'd anything to beat this 'ere yarn o' your'n, so spin us the rest
on it.”
“That's soon done,” resumed Jack. “When I found the fellow
wouldn't give me a fair show, I boarded him, captain, and treated
him to a few inches of cold steel. He won't trouble us again, I
reckon!”
Scarcely had he finished speaking when Don gripped his arm and
pointed to where, a dozen yards away, the bottom of the canoe
glistened in the moonlight. A dark object had suddenly appeared
alongside the overturned skiff. Presently a surging splash was heard.
“Shiver my keelson if he ain't righted the craft!” roared the
captain, snatching up one of the muskets as the lascar was seen to
scramble into the canoe and paddle slowly away.
Don laid a quick hand upon the old sailor's arm.
“Let the beggar go,” said he. “He'll never reach land with that
knife in him.”
“Maybe not, lad,” replied the captain, shaking off the hold upon his
arm and taking the best aim he could, considering the motion of the
boat. “Bloodshed's best awoided, says you. Wery good; all' the best
way to awoid it, d'ye mind me, is to send yon warmint to Davy Jones
straight away. Consequential, the quality o' marcy shan't be strained
on this 'ere occasion, as the whale says when he swallied the school
o' codlings.” And with that he fired.
The lascar was seen to discontinue the use of his paddle for a
moment, and then to make off faster than before.
The old sailor's face fell.
“Spike my guns, I've gone and missed the warmint!” said he.
“Howsomedever, we'll meet again, as the shark's lower jaw says to
the upper 'un when they parted company to accomidate the sailor.
An' blow me, lads, here comes the wind!

“Ay, here's a master excelleth in skill,

An' the master's mate he is not to seek;
An' here's a Bjsin ull do our good will,
An' a ship, d'ye see, lads, as never had leak.
So lustily, lustily, let us sail forth!
 Our sails he right trim an' the wind's to the north!”

It was now five o'clock, and as day broke the cutter, with a
freshening breeze on her starboard quarter, bore away for the island,
now in full view. When about a mile short of it, however, the captain
laid the boat's head several points nearer the wind, and shaped his
course as though running past it for the mainland, which lay like a
low bank of mist on the horizon. In the cuddy Puggles was busy with
preparations for breakfast, whilst Don lolled on the rail, watching the
shore, and idly trailing one hand in the water.
“Hullo! what's this?” he exclaimed suddenly, examining with
interest a fragment of dripping cloth that had caught on his hand.
“Jack, come here!”
Jack happened to be forward just then, hanging out his drenched
clothes to dry upon an improvised line, but hearing Don's
exclamation, he sprang aft. Somehow he was always expecting
surprises now.
“Look here,” said Don, rapidly spreading out the soaked cloth upon
his knee, “have you ever seen this before?”
“Not likely!—a mere scrap of rag that some greasy native——”
Jack began, eyeing the said scrap of rag contemptuously. But
suddenly his tone changed, and he gasped out: “By Jove, old fellow,
it's not the handkerchief, is it?”
“The very same!” replied-Don, rising and hurrying aft to where the
captain stood at the tiller. “I say, captain, you remember my telling
you how I tied a handkerchief round that bag of pearls? Well, here's
the identical 'wipe.' with my initials on it as large as life. Just fished it
out of the water.”
For full a minute the old sailor stared at him open-mouthed. Then:
“Flush my scuppers,” roared he, “if this 'ere ain't the tidiest piece o'
luck as ever I run agin. We've got the warmint safe in the maintop,
so to say, where he can't run away—shiver my main-brace if we
ain't!”
 “Thanks to your clear head, captain,” said Don. “It certainly does
look as if he had come straight to the island here.”
“We'll purty soon know for sartin; we're a-makin' port hand over
fist,” rejoined the captain, bringing the cutter's head round, and
running under the lee of the island.
This side, unlike the wind-swept seaward face, was thickly clad in
jungle, above which at intervals towered a solitary palm like a
sentinel on duty. No traces of human habitation were to be seen; for
a rocky backbone or ridge, running lengthwise of the island, isolated
its frequented portion from this jungly half. Midway between the
extremities of this ridge rose two hills: one a symmetrical, cone-
shaped elevation, clad in a mantle of jungle green; the other a vast
mass of naked rock, towering hundreds of feet in air, and in its
general-outline somewhat resembling a colossal kneeling elephant.
As if to heighten the resemblance, there was perched upon the lofty
back a native temple, which looked for all the world like a gigantic
howdah.
“D'ye see them elewations, lads?” cried the captain, heading the
cutter straight for what-appeared to be an unbroken line of jungle.
“A. brace o' twins, says you. Wery good; atween 'em lies as purty a
leetle cove as wessel ever cast anchor in—slip my cable if it ain't!”
“Are you sure you're not out of your reckoning, captain?” said
Jack, scanning the shore-line with dubious eye. “It's no
thoroughfare, so far as I can see.”
“Avast there! What d'ye say to that, now?” chuckled the captain,
as the cutter, in obedience to a movement of the tiller, swept round
a tiny eyot indistinguishable in its mantle of green from, the shore
itself, and entered a narrow, land-locked creek, whose precipitous
sides were completely covered from summit to water-line with a rank
growth of vegetation. “Out with the oars, lads! a steam-whistle
couldn't coax a wind into the likes o' this place, says you.”
The oars run out, they pulled for some distance through this
remarkable rift in the hills, the cutter's mast in places sweeping the
overhanging jungle; until at last a spot was reached where a side
ravine cleft the cliff upon their left, terminating at the water's edge
in a strip of sandy beach, thickly shaded with cocoa-nut palms.
“Stow my cargo!” chuckled the captain, as he ran the cutter bow-
on into the sand, “a nautical sea-sarpent himself couldn't smell us
out here, says you. So here we heaves to, and here we lies until——
swabs an' slush-buckets, what's this?”
For the captain had already scrambled ashore, and as he uttered
these words he stooped and intently examined the sand at his feet.
In it were visible recent footprints, and a long trailing furrow that
started from the water's edge and ran for several yards straight up
the beach. Where the furrow terminated there lay a native ballam.
Jack was first to espy the canoe. Guessing the cause of the
captain's sudden excitement, he ran up the sands to the spot where
the rude vessel lay. The ballam was still dripping sea-water; and in
it, amid a pool of blood, lay a sailors sheath-knife.
“The lascar!” he shouted, snatching up the blood-stained weapon,
and holding it out at arms length, as Don and the captain hurried
up; “we've landed in his very tracks!”
 CHAPTER VI.—IN THE THICK OF IT.

E
ither the lascar's wound had not proved as serious as Jack
surmised, or the fellow was endowed with as many lives as a
cat. At all events, he had reached land before them, and in
safety.
“Sharks an' sea-sarpents!” fumed the captain, Stumping excitedly
round and round the canoe. “The warmint had orter been sent to
Davy Jones as I ad wised. Howsomedever, bloodshed's best
awoided, says you, Master Don, lad; an' so, shiver my keelson! here
we lies stranded. What's the course to be steered now, I axes?
That's a matter o' argyment, says you; so here's for a whiff o' the
fragrant!”
Bidding his servant fetch pipe and tobacco, the captain seated
himself upon the canoe and fell to puffing meditatively, his
companions meanwhile discussing the situation and a project of their
own, with many anxious glances in the direction of the adjacent
jungle, where, for anything they knew to the contrary, the lascar
might even then be stealthily watching their movements.
“Shiver my smokestack! d'ye see that, now?” exclaimed the
captain at last, following with half-closed eye and tarry finger the
ascent of a perfect smoke-ring that had just left his lips. “An' what's
a ring o' tobackie smoke? says you. A forep'intin' to ewents to come,
says I. A ring means surrounded, d'ye see; an'—grape-shot an' gun-
swabs!—surrounded means fightin', lads!”
“Fun or fighting, I'm ready, anyhow!” cried Jack, flourishing his
knife.
“Ay, ay, lad; an' me, too, for the matter o' that,” replied the old
sailor, presenting his pipe at an imaginary foe like a pistol; “but when
our situation an' forces is beknownst to the enemy, we're sartin to
be surprised, d'ye mind me. An' so I gets an idee!
 “Go palter to lubbers an' swabs, d'ye see?
'Bout danger, an' fear, an' the like;
A tight leetle boat an' good sea-room give me,
An' it ain't to a leetle I'll strike!”

“Out with the idea then, captain!” cried Don.

“Shiver my cutlass, lads!—we must carry the war into the camp o'
the enemy, dye see'. Wery good, that bein' so, what we wants, d'ye
mind me, is a safe, tidy place to fall back on, as can't be took, or
looted, or burnt, like the cutter here, whiles we're away on the
rampage, so to say.”
“Why not entrench ourselves on the hill just above?” suggested
Jack.
“Stow my sea-chest!—the wery identical plan I perposes,”
promptly replied the captain. “An' why? you naterally axes. Because
it's ha'nted, says I.”
“Because it's what?” cried the two young men in chorus.
“Haunted?”
“Ay, the abode o' spurts,” continued the captain. “There's a old
ancient temple aloft on yon hill, d'ye see, as they calls the 'Ha'nted
Pagodas'—which they say as it's a tiger-witch or summat inhabits it,
d'ye see—an' shiver my binnacle if a native'll go a-nigh it day or
night!”
“Admirable! But what about the cutter, captain?” said Don.
The captain sucked for a moment at his pipe as if seeking to draw
a suitable idea therefrom.
“What o' the cutter? you axes,” said he presently. “Why, we'll
wrarp her down the crik a bit, d'ye see, an' stow her away out o'
sight where the wegitation's thickish-like on the face o' the cliff;
copper my bottom if we won't!”
 “The stores, of course, must be carried up the hill,” said Jack,
entering readily into the captain's plans. “We should set about the
job at once.”
“Avast there, lad! What's to perwent the jungle hereabouts a-usin'
of its eyes? I axes. The wail o' night, says you. So, when the wail o'
night unfurls, as the poic says, why, up the hill they goes.”
This being unanimously agreed to, and Puggles at that moment
announcing breakfast, our trio of adventurers adjourned to the
cutter.
“Captain,” said Don, after delighting the black boy's heart by a
ravenous attack upon the eatables, “like you, I've got an idee—Hullo,
you, Pug! What are you grinning at?”
“Nutting, sa'b,” replied Puggles, clapping his hand over his mouth;
“only when marster plenty eating, he sometimes bery often one idee
getting. Plenty food go inside, he kicking idee out!”
“Just double reef those lips of yours, Pug, and tell us where do
your ideas come from?” said Jack, laughing.
“Me tinking him here got, sar,” said Puggles, gravely patting his
waistband, at which the old sailor nearly choked.
“And a pretty stock of them you have, too, judging by the size of
your apple-cart!” said his master, shying a biscuit at his head. “Well,
as I was saying, captain, I have an idea——”
“Flush my scuppers!” gasped the old sailor, swallowing a brimming
pannikin of coffee to clear his throat. “Let's hear more on it then,
lad.”
“Well, it's this. Jack and I are going over to the town—where the
temples are, you understand—to see if we can't sight old Salambo. A
bit of reconnoitring may be of use to us later, you see.”
“A-goin'—over—to—the—town!” roared the captain in amazement,
separating the words as though each were a reluctant step in the
direction proposed. “Scuttle my cutter, lads! ye'll have the whole
pack o' waimints down on ye in a brace o' shakes!”
 “You won't say so when you see us in full war-paint,” retorted
Jack, as he and Don rose and disappeared in the cuddy.
In the course of half an hour the cuddy door was thrown open,
and two stalwart young natives, in full country dress, confronted the
old sailor. With the assistance of Puggles and the captain's “boy,” not
to mention soot from the cuddy pots, the two young fellows had
cleverly “made up” in the guise of Indian pilgrims. At first sight of
them, the captain, thinking old Salambo's crew were upon him,
seized a musket and threw himself into an attitude of defence.
“Blow me!” he roared, when a loud burst of laughter apprised him
of his mistake, “if this ain't the purtiest go as ever I see. Scrapers an'
holystones, ye might lay alongside the old woman himself, lads, an'
him not know ye from a reglar, genewine brace o' lying niggers.
What tack are ye on now, lads? I axes.”
“Off to the town, captain,” replied Don, “to search for old Salambo
among his idols. That is, if you'll let Spottie here come with us as
pilot.”
“Spottie” was the nickname with which they had dubbed the
captain's black servant, whose face was deeply pitted from smallpox.
“Right, lads; he's been here afore, an' knows the lay o' the land;
so take him in tow, and welcome,” was the captain's hearty
rejoinder. “An' stow your knives away amidships, in case of
emargency like; though blow me if they ever take ye for aught but
genewine lying niggers!”
Concealing their knives about their persons in accordance with this
advice, they launched the lascar's ballam upon the creek—which the
captain assured them expanded a little further inland into a broad
lagoon, too deep to ford—and so set out.. The paddle had been
removed; but as the creek appeared to have nowhere, in its upper
reaches at any rate, a greater depth than half-a-dozen feet, the
boathook served admirably as, a substitute for propelling the canoe.
“What's the line for, Spottie?” Jack asked, seeing their guide throw
a coil of small rope into the canoe, which he afterwards boarded in
person and shoved off.
 “Turkle, sar,” replied Spottie. “Plenty time me catching big turkle
asleep on sand. He no come in ballam, so me taking rope to tow him
astern. Him bery nice soup making, sar,” said Spottie, who had
always an eye to anything.
Little as they guessed it then, this line was to play a more unique
and serviceable part in the day's adventures than that indicated by
the soup-loving Spottie.
The creek, as the captain had intimated, presently expanded into
a lagoon fully a quarter of a mile wide, and so shallow in parts that
the canoe almost touched the amber-coloured sands over which it
passed. Arrived at the further side, they drew the canoe upon the
beach, and continued their route to the town by way of a steep
jungle-path, which, in the course of some fifteen minutes' hard
climbing, led them to the crest of the rocky ridge. Here they paused
a moment to look about them.
To the left lay Haunted Pagoda Hill; on their right the colossal
Elephant Rock; and, nestling at its base, the native town, with its
sea of dun roofs and gleaming white temples. The stirring ramp of
tom-toms, and the hoarse roar of the multitude, floated up to them
as they stood contemplating the scene.
“Now for it!” cried Jack, heading the descent. “We'll soon be in the
thick of it, anyhow.”
A few minutes more and they stood on the outskirts of the town.
“Make for the chief temple, Spottie,” said Don to their guide; “and
whatever you do, don't call us sahib or sir. We're only pilgrims like
yourself, you understand. And say, Spottie, do you know old
Salambo, the shark-charmer, when you see him?”
By a nod Spottie intimated that he did.
“Good! He's the chap we're after, you understand. Keep a sharp
look-out, and if you happen to get your eye on him——”
“Or on a lascar with a knife-wound in his shoulder,” put in Jack.
“Just pull my cloth, will you?” concluded Don.
 Again the trusty Spottie nodded, and at a signal led the way into
the main-street, where they immediately found themselves in the
midst of a noisy, surging crowd of natives.
So perfect was their disguise, however, that Don could not detect
a single suspicious glance directed towards them.
The natives who thronged the street were, to a man, heading for
the temples. Into these, if nothing was seen of the shark-charmer
outside, Don was resolved to penetrate.
As no English foot is ever allowed—in Southern India, at least—to
cross the threshold of a Hindu shrine, this was a step attended with
tremendous risk. Detection would mean fighting for their lives
against overwhelming odds.
“We'll do it, however,” said Don resolutely. “The temple's the place
to look for him, since he's a priest, and in this disguise the pearls are
worth the risk.”
That this was also Jack's opinion was plain from the resolute,
nonchalant manner in which he pressed forward.
Owing to the congested state of the thoroughfare, progress was
necessarily slow. They were more than an hour in gaining the open
maidan in which the street terminated.
In the centre of this open space lay a sacred tank, flanked, on that
side nearest the Elephant Rock, by a vast semicircle of temples.
Midway in this line stood the chief temple. Here, if at all, the shark-
charmer would most likely be found.
But to reach the chief temple was no easy task. Vast crowds of
pilgrims surrounded the sacred tank, awaiting their turn to bathe in
its stagnant green waters.
At last, after much elbowing and pushing, they reached the steps
of the chief temple. Thus far they had seen nothing of Salambo. As
they had already made the entire circuit of the tank, there was
nothing for it but to seek him in the sacred edifice itself.
Spottie led the way, since for him there was absolutely no risk.
Following close upon his heels, past the hideous stone monsters
which flanked the entrance, the mock pilgrims found themselves in
the temple court. Here the crush was even greater than without.
They had now reached the crucial point of their adventure.
A single unguarded word or action on their part, and each man of
these teeming thousands would instantly become a mortal enemy!
Don strove to appear unconcerned, but his pulses throbbed madly
at the mere thought of detection. As for Jack, the careless poise of
his right hand at his belt showed him to be on his guard, though he
looked as cool as a sea-breeze.
Over the heads of the multitude, on the opposite side of the court,
could be seen an inner shrine, where offerings were being made.
Selecting this as his goal, Don began to edge his way slowly but
steadily towards it, closely followed by Spottie and the undaunted
Jack.
Suddenly he felt a hand tugging at his cloth. Unable to turn
himself about in the crush, he twisted his head round and caught
Spottie's eye. By a quick, almost imperceptible movement of hand
and head, the black directed his attention towards the left. Looking
in the direction thus indicated, Don saw, but a few yards away, the
portly person of the shark-charmer.
By dint of persistent pushing, he presently succeeded in
approaching so near to his man that, had he so wished, he could
have laid a hand upon his shoulder.
The shark-charmer was evidently bent upon gaining the inner
shrine at the opposite side of the court. Inch by inch he pummelled
his way through the dense crowd, unconscious that the sahibs whom
he had robbed were dogging his steps. Once when he turned his
head his eyes actually rested upon Don's face. But he failed to
recognise him, and so went on again, greatly to Don's relief.
Then of a sudden the limit of the crush was reached, and they
emerged upon a comparatively clear space immediately in front of
the shrine. This the shark-charmer crossed without hesitation, but
Don hung back, uncertain whether it would be prudent to venture
further. However, seeing a group of natives about to approach the
shrine with offerings, he joined them, and in company with Jack
ascended the steps.
The shark-charmer had already disappeared within.
Fumbling in his cloth for some small coin, to present as an
offering, Don crossed the threshold, and was in the very act of
penetrating the dimly lighted, incense-clouded chamber just beyond,
when a guarded exclamation from Jack caused him to glance quickly
over his shoulder.
Following them with the stealthy tread of a panther was a swarthy,
evil-looking native.
“The lascar!” said Jack, in a low, breathless whisper. “Back, old
fellow, for your life! Once in the crowd, we're safe.”

Back they darted towards the entrance, but the lascar, anticipating
this manouvre, was on his guard. As Jack dashed past, the cunning
spy thrust out his foot and sent him sprawling on the flagstones.
Don, hearing the noise, turned back to his friend's assistance, and
by the time Jack regained his feet the lascar had reached the
entrance mid raised the hue-and-cry.
 “This way!” cried Don, making for a narrow side door, as the
lascar's shouts began to echo through the precincts of the temple.
“Get your knife ready, he's raised the alarm!”
Through the door they dashed, only to find themselves in the
court, hemmed in on every side. The frenzied cries of the lascar
continued to ring through the enclosure; but, fortunately for the
mock pilgrims, so vast was the concourse of natives, and so
deafening the uproar, that only those nearest the shrine understood,
his words, while even they failed, as yet to penetrate the clever
disguise of the intruders. This gave them time to draw breath, and
look about them.
Close, on their left Jack's quick eye discovered an exit, about
which the crowd was less dense than elsewhere. The great doors
stood wide open, disclosing a narrow street. Between this exit and
the spot where they stood at bay, a number of sacred bulls were
quietly feeding off a great heap of corn which the devotees had
poured out upon the flags of the court. All this Jack's eyes took in at
a glance.
A roar, terrific as that of ten thousand beasts of prey, burst from
the surging multitude. The lascars words were understood. Glancing
quickly over his shoulder, Jack saw that this man, from his place
upon the steps of the shrine, was pointing them out.
Another instant, and their disguise would avail them nothing; the
maddened, fanatical crowd would be upon them.
“Don,” he said, in rapid, husky tones, as he grasped his friend's
hand for what he believed to be the last time, “there's but one
chance left us, and that's a slim one. You see the door on our left,
and those bulls? Do you take one of the two big fellows feeding side
by side, and I'll take the other. Use your knife to guide the brute,
and with God's help——”
A tremendous roar of voices and a thunderous rush-of feet cut his
words short.
“Now for it, old fellow!”
 With one swift backward glance at the furious human wave
sweeping down upon them, they darted towards the bulls, of which
the two largest, accustomed to the daily tumult of town and temple,
were still composedly feeding, their muzzles buried deep in the
mound of corn.
Before the animals had time to lift their heads, the mock pilgrims
were on their backs and plying knives and heels upon their sleek
flanks.
Bellowing with pain and terror, the bulls, with tails erect and heads
lowered, charged the throng about the doorway, bowling them over
in all directions like so many ninepins. Before the infuriated crowd in
their rear understood the meaning of this unexpected manoeuvre,
the mock pilgrims were in the street.
It was a side street, fortunately, separated from the densely-
packed maidan by a high brick wall, and but few natives were about.
Those who followed them out of the temple, too, they soon
distanced, for their ungainly steeds made capital time.
But now a new, if less serious, danger menaced them. Apart from
the difficulty of clinging to the round, arched backs of the bulls, once
started, the maddened animals could not be stopped. Fortunately,
they took the direction of the hill-path.
On they tore, bellowing madly, and scattering showers of foam
and sand right and left, until, in an amazingly brief space of time,
they reached the outskirts of the town. Here, as if divining that their
services were no longer required, the bulls stopped abruptly,
shooting their riders off their backs into the sand with scant
ceremony.
“Regular buck-jumpers!” groaned Jack, rubbing his lacerated shins
ruefully. “Glad we're safe out of it, anyhow.”
“So am I. But I wonder where Spottie is?” said Don, fanning
himself with the loosened end of his turban.
Jack started up. “Never once thought of Spottie since we entered
the shrine,” cried he. “Come, we must go back and look him up.”
 Their uneasiness on Spottie's account, however, was at that
instant set at rest by the precipitate appearance on the scene of
Spottie himself. Seeing his masters charge the crowd on the bulls'
backs, he had extricated himself from the crush, and followed them
with all possible speed.
“Dey coming, sar!” he panted, as he ran up, “Lascar debil done
fetching plenty black man!”
And there swelled up from the street below a tumult of voices that
left no doubt as to the accuracy of his statement.