Step 1: Verifying Java Installation: Download Scala

Download as pdf or txt
Download as pdf or txt
You are on page 1of 3

Apache Spark Installation Page 1 of 3

https://www.tutorialspoint.com/apache_spark/apache_spark_installation.htm Copyright © tutorialspoint.com

Spark is Hadoop’s sub-project. Therefore, it is better to install Spark into a Linux based system. The
following steps show how to install Apache Spark.

Step 1: Verifying Java Installation


Java installation is one of the mandatory things in installing Spark. Try the following command to verify
the JAVA version.

$java -version

If Java is already, installed on your system, you get to see the following response −

java version "1.7.0_71"


Java(TM) SE Runtime Environment (build 1.7.0_71-b13)
Java HotSpot(TM) Client VM (build 25.0-b02, mixed mode)

In case you do not have Java installed on your system, then Install Java before proceeding to next step.

Step 2: Verifying Scala installation


You should Scala language to implement Spark. So let us verify Scala installation using following
command.

$scala -version

If Scala is already installed on your system, you get to see the following response −

Scala code runner version 2.11.6 -- Copyright 2002-2013, LAMP/EPFL

In case you don’t have Scala installed on your system, then proceed to next step for Scala installation.

Step 3: Downloading Scala


Download the latest version of Scala by visit the following link Download Scala. For this tutorial, we are
using scala-2.11.6 version. After downloading, you will find the Scala tar file in the download folder.

Step 4: Installing Scala


Follow the below given steps for installing Scala.

Extract the Scala tar file


Type the following command for extracting the Scala tar file.

$ tar xvf scala-2.11.6.tgz

Move Scala software files

https://www.tutorialspoint.com/cgi-bin/printpage.cgi 9/22/2017
Apache Spark Installation Page 2 of 3

Use the following commands for moving the Scala software files, to respective directory
.

$ su – /usr/local/scala
Password:
# cd /home/Hadoop/Downloads/
# mv scala-2.11.6 /usr/local/scala
# exit

Set PATH for Scala


Use the following command for setting PATH for Scala.

$ export PATH = $PATH:/usr/local/scala/bin

Verifying Scala Installation


After installation, it is better to verify it. Use the following command for verifying Scala installation.

$scala -version

If Scala is already installed on your system, you get to see the following response −

Scala code runner version 2.11.6 -- Copyright 2002-2013, LAMP/EPFL

Step 5: Downloading Apache Spark


Download the latest version of Spark by visiting the following link Download Spark. For this tutorial, we
are using spark-1.3.1-bin-hadoop2.6 version. After downloading it, you will find the Spark tar file in
the download folder.

Step 6: Installing Spark


Follow the steps given below for installing Spark.

Extracting Spark tar


The following command for extracting the spark tar file.

$ tar xvf spark-1.3.1-bin-hadoop2.6.tgz

Moving Spark software files


The following commands for moving the Spark software files to respective directory /usr/local/spark .

$ su –
Password:

# cd /home/Hadoop/Downloads/
# mv spark-1.3.1-bin-hadoop2.6 /usr/local/spark
# exit

https://www.tutorialspoint.com/cgi-bin/printpage.cgi 9/22/2017
Apache Spark Installation Page 3 of 3

Setting up the environment for Spark


Add the following line to ~/.bashrc file. It means adding the location, where the spark software file are
located to the PATH variable.

export PATH = $PATH:/usr/local/spark/bin

Use the following command for sourcing the ~/.bashrc file.

$ source ~/.bashrc

Step 7: Verifying the Spark Installation


Write the following command for opening Spark shell.

$spark-shell

If spark is installed successfully then you will find the following output.

Spark assembly has been built with Hive, including Datanucleus jars on classpath
Using Spark's default log4j profile: org/apache/spark/log4j-defaults.properties
15/06/04 15:25:22 INFO SecurityManager: Changing view acls to: hadoop
15/06/04 15:25:22 INFO SecurityManager: Changing modify acls to: hadoop
15/06/04 15:25:22 INFO SecurityManager: SecurityManager: authentication disabled;
ui acls disabled; users with view permissions: Set(hadoop); users with modify
permissions: Set(hadoop)
15/06/04 15:25:22 INFO HttpServer: Starting HTTP Server
15/06/04 15:25:23 INFO Utils: Successfully started service 'HTTP class server' on
port 43292.
Welcome to
____ __
/ __/__ ___ _____/ /__
_\ \/ _ \/ _ `/ __/ '_/
/___/ .__/\_,_/_/ /_/\_\ version 1.4.0
/_/

Using Scala version 2.10.4 (Java HotSpot(TM) 64-Bit Server VM, Java 1.7.0_71)
Type in expressions to have them evaluated.
Spark context available as sc
scala>

https://www.tutorialspoint.com/cgi-bin/printpage.cgi 9/22/2017

You might also like