Scribd
Upload
9 views4 pages

Hadoop Hive

Uploaded by

viveksingh67067
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
9 views4 pages

Hadoop Hive

Uploaded by

viveksingh67067
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 4

Installation of HIVE

NOTE: My Folder Structure

\ (root)
|--home
| |
| |--main
| | | ...
| | | ...
| |
| |--hadoop
| | | Downloads
| | | hadoop
| | | .bashrc
| | | ...

1. Install Verify hadoop and java

hadoop@vm:~$ which hadoop


/home/hadoop/hadoop/bin/hadoop
hadoop@vm:~$ which java
/usr/bin/java

2. Download Hive tar.gz file and extract

hadoop@vm:~/Downloads$ wget https://archive.apache.org/dist/hive/hive-3.1.2/apache-


hive-3.1.2-bin.tar.gz
hadoop@vm:~/Downloads$ tar xzf apache-hive-3.1.2-bin.tar.gz
hadoop@vm:~/Downloads$ mv apache-hive-3.1.2-bin ~/hive

3. Edit and prepare your .bashrc

hadoop@vm:~$ nano .bashrc

# ====== IN NANO APPEND ==============

#Hive
export HIVE_HOME=/home/hadoop/hive
export PATH=$PATH:$HIVE_HOME/bin

# ====================================

hadoop@vm:~$ source .bashrc

4. Create directories for hive in hdfs.

hadoop@vm:~$ hdfs dfs -mkdir -p /user/hive/warehouse


hadoop@vm:~$ hdfs dfs -ls /
Found 2 items
drwxr-xr-x - hadoop supergroup 0 2022-01-27 09:21 /Venkat_Sir
drwxr-xr-x - hadoop supergroup 0 2022-01-27 09:55 /user
hadoop@vm:~$ hdfs dfs -ls /user/
Found 1 items
drwxr-xr-x - hadoop supergroup 0 2022-01-27 09:55 /user/hive
hadoop@vm:~$ hdfs dfs -ls /user/hive/
Found 1 items
drwxr-xr-x - hadoop supergroup 0 2022-01-27 09:55 /user/hive/warehouse
hadoop@vm:~$ hdfs dfs -ls /user/hive/warehouse
hadoop@vm:~$
hadoop@vm:~$ hdfs dfs -mkdir /tmp
hadoop@vm:~$ hdfs dfs -chmod g+w /tmp
hadoop@vm:~$ hdfs dfs -chmod g+w /user/hive/warehouse

5. Edit and prepare configuration files like hive-site.xml in conf


directory and hive-env.sh

6. We can create metastore using derby or Using MYSQL or any other


database

hadoop@vm:~$ cd $HIVE_HOME/conf
hadoop@vm:~/hive/conf$ cp hive-default.xml.template hive-site.xml
hadoop@vm:~/hive/conf$ nano hive-site.xml

Remove the  in the line 3215 use CTRL+SHIT+- to move cursor to a line.

Make sure the guava is same for the hadoop and hive

hadoop@vm:~$ ls $HIVE_HOME/lib | grep guava-


guava- 19.0.jar
hadoop@vm:~$ ls $HADOOP_HOME/share/hadoop/hdfs/lib | grep guava-
guava-27.0-jre.jar
# if they both are not same copy the one in hadoop to hive and delete the
original in hive
hadoop@vm:~$ rm $HIVE_HOME/lib/guava-19.0.jar
hadoop@vm:~$ cp $HADOOP_HOME/share/hadoop/hdfs/lib/guava-27.0-jre.jar
$HIVE_HOME/lib/

Paste the following at the start of the configuration

<property>
<name>system:java.io.tmpdir</name>
<value>/tmp/hive/java</value>
</property>
<property>
<name>system:user.name</name>
<value>${user.name}</value>
</property>

You can change the location of warehouse in hadoop by editing the values in the
below property in hive-site.xml
<property>
<name>hive.metastore.warehouse.dir</name>
<value>/user/hive/warehouse</value>
<description>location of default database for the warehouse</description>
</property>

You can also change the meta store to mysql by replacing the DERBY with
required service

<property>
<name>hive.metastore.db.type</name>
<value>DERBY</value>
<description>
Expects one of [derby, oracle, mysql, mssql, postgres].
Type of database used by the metastore. Information schema &amp;
JDBCStorageHandler depend on >
</description>
</property>

7. Verify derby is there, if not download and setup derby

hadoop@vm:~$ $HIVE_HOME/bin/schematool -dbType derby -initSchema


...
hadoop@vm:~$ hive
...

Sample Commands in Hive - Create,Insert


1. Create a csv file and add text to the file
2. Add the file to hdfs

hadoop@vm:~$ hdfs dfs -put ~/textfile.csv /


hadoop@vm:~$ hdfs dfs -mv /textfile.csv /Venkat_Sir/
hadoop@vm:~$ hdfs dfs -ls /Venkat_Sir/
Found 2 items
-rw-r--r-- 1 hadoop supergroup 104 2022-01-27 09:21 /Venkat_Sir/file1
-rw-r--r-- 1 hadoop supergroup 42 2022-01-27 10:32 /Venkat_Sir/textfile.csv

# Venkat_sir is a folder in the hadoop that we created earlier.

3. Code in Hive

hive> show databases;


OK
default
krishna
Time taken: 0.065 seconds, Fetched: 2 row(s)
hive> use krishna;
OK
Time taken: 0.073 seconds
hive> create table Rama(id int,name String)
> ROW FORMAT DELIMITED
> FIELDS TERMINATED BY ',';
OK
Time taken: 1.472 seconds
hive> describe Rama;
OK
id int
name string
Time taken: 0.482 seconds, Fetched: 2 row(s)
hive> LOAD DATA INPATH '/Venkat_Sir/textfile.csv' INTO TABLE Rama;
Loading data to table krishna.rama
OK
Time taken: 1.084 seconds
hive> SELECT * FROM Rama;
OK
1 'Mohith Kumar'
2 'Venkat'
3 'Babu Garu'
Time taken: 7.285 seconds, Fetched: 3 row(s)
hive>

You might also like