Scribd
Upload
75 views3 pages

Step 1: Verifying Java Installation: Download Scala

Apache Spark Install guide

Uploaded by

eabernstein
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
75 views3 pages

Step 1: Verifying Java Installation: Download Scala

Apache Spark Install guide

Uploaded by

eabernstein
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 3

Apache Spark Installation Page 1 of 3

https://www.tutorialspoint.com/apache_spark/apache_spark_installation.htm Copyright © tutorialspoint.com

Spark is Hadoop’s sub-project. Therefore, it is better to install Spark into a Linux based system. The
following steps show how to install Apache Spark.

Step 1: Verifying Java Installation


Java installation is one of the mandatory things in installing Spark. Try the following command to verify
the JAVA version.

$java -version

If Java is already, installed on your system, you get to see the following response −

java version "1.7.0_71"


Java(TM) SE Runtime Environment (build 1.7.0_71-b13)
Java HotSpot(TM) Client VM (build 25.0-b02, mixed mode)

In case you do not have Java installed on your system, then Install Java before proceeding to next step.

Step 2: Verifying Scala installation


You should Scala language to implement Spark. So let us verify Scala installation using following
command.

$scala -version

If Scala is already installed on your system, you get to see the following response −

Scala code runner version 2.11.6 -- Copyright 2002-2013, LAMP/EPFL

In case you don’t have Scala installed on your system, then proceed to next step for Scala installation.

Step 3: Downloading Scala


Download the latest version of Scala by visit the following link Download Scala. For this tutorial, we are
using scala-2.11.6 version. After downloading, you will find the Scala tar file in the download folder.

Step 4: Installing Scala


Follow the below given steps for installing Scala.

Extract the Scala tar file


Type the following command for extracting the Scala tar file.

$ tar xvf scala-2.11.6.tgz

Move Scala software files

https://www.tutorialspoint.com/cgi-bin/printpage.cgi 9/22/2017
Apache Spark Installation Page 2 of 3

Use the following commands for moving the Scala software files, to respective directory
.

$ su – /usr/local/scala
Password:
# cd /home/Hadoop/Downloads/
# mv scala-2.11.6 /usr/local/scala
# exit

Set PATH for Scala


Use the following command for setting PATH for Scala.

$ export PATH = $PATH:/usr/local/scala/bin

Verifying Scala Installation


After installation, it is better to verify it. Use the following command for verifying Scala installation.

$scala -version

If Scala is already installed on your system, you get to see the following response −

Scala code runner version 2.11.6 -- Copyright 2002-2013, LAMP/EPFL

Step 5: Downloading Apache Spark


Download the latest version of Spark by visiting the following link Download Spark. For this tutorial, we
are using spark-1.3.1-bin-hadoop2.6 version. After downloading it, you will find the Spark tar file in
the download folder.

Step 6: Installing Spark


Follow the steps given below for installing Spark.

Extracting Spark tar


The following command for extracting the spark tar file.

$ tar xvf spark-1.3.1-bin-hadoop2.6.tgz

Moving Spark software files


The following commands for moving the Spark software files to respective directory /usr/local/spark .

$ su –
Password:

# cd /home/Hadoop/Downloads/
# mv spark-1.3.1-bin-hadoop2.6 /usr/local/spark
# exit

https://www.tutorialspoint.com/cgi-bin/printpage.cgi 9/22/2017
Apache Spark Installation Page 3 of 3

Setting up the environment for Spark


Add the following line to ~/.bashrc file. It means adding the location, where the spark software file are
located to the PATH variable.

export PATH = $PATH:/usr/local/spark/bin

Use the following command for sourcing the ~/.bashrc file.

$ source ~/.bashrc

Step 7: Verifying the Spark Installation


Write the following command for opening Spark shell.

$spark-shell

If spark is installed successfully then you will find the following output.

Spark assembly has been built with Hive, including Datanucleus jars on classpath
Using Spark's default log4j profile: org/apache/spark/log4j-defaults.properties
15/06/04 15:25:22 INFO SecurityManager: Changing view acls to: hadoop
15/06/04 15:25:22 INFO SecurityManager: Changing modify acls to: hadoop
15/06/04 15:25:22 INFO SecurityManager: SecurityManager: authentication disabled;
ui acls disabled; users with view permissions: Set(hadoop); users with modify
permissions: Set(hadoop)
15/06/04 15:25:22 INFO HttpServer: Starting HTTP Server
15/06/04 15:25:23 INFO Utils: Successfully started service 'HTTP class server' on
port 43292.
Welcome to
____ __
/ __/__ ___ _____/ /__
_\ \/ _ \/ _ `/ __/ '_/
/___/ .__/\_,_/_/ /_/\_\ version 1.4.0
/_/

Using Scala version 2.10.4 (Java HotSpot(TM) 64-Bit Server VM, Java 1.7.0_71)
Type in expressions to have them evaluated.
Spark context available as sc
scala>

https://www.tutorialspoint.com/cgi-bin/printpage.cgi 9/22/2017

You might also like