Webserver setup : to create webserver

sudo yum install httpd -y

sudo chkconfig httpd on
sudo service httpd status
sudo service httpd start

Hit on browser with ip address for centos ip

cd /var/www/html/
ls ---> blank
cd

this is for CM
wget http://archive.cloudera.com/cm5/repo-as-tarball/5.4.0/cm5.4.0-centos6.tar.gz
---> downlod to home folder

wget http://archive.cloudera.com/cm5/repo-as-tarball/5.4.0/cm5.4.0-centos6.tar.gz

sudo tar -zxvf cm5.4.0-centos6.tar.gz -C /var/www/html/

sudo tar -zxvf cm5.4.0-centos6.tar.gz -C /var/www/html/ --> move to this folder

Hit on browser with ip address for centos ip like

10.0.0./cm in browser --> its means it will make this untar & place it on his
server

Click on cloudera-manager-repo -->> this is repo of CM & jenkins comply dont want
this

cd /var/www/html/
ls
cm
cd cm
ls
cat cm.repo

cd /var/www/html/

===================================================================================
=====
===================================================================================
=====
For CDH -> both cm & CDH folder are as same place - > cd /var/www/html/
===================================================================================
==
create folder for CDH
cd /var/www/html/
sudo mkdir cdh5.4.0

ls
cd cdh5.4.0
For CDH download this in this cd cdh5.4.0

sudo wget http://archive.cloudera.com/cdh5/parcels/5.4.1.6/CDH-5.4.1-

1.cdh5.4.1.p0.6-el6.parcel

sudo wget http://archive.cloudera.com/cdh5/parcels/5.4.1.6/CDH-5.4.1-

1.cdh5.4.1.p0.6-el6.parcel

ls
du -h cdh version --> Download all the repos in this floders

sudo wget http://archive.cloudera.com/cdh5/parcels/5.4.1.6/manifest.json

changing the repo address

cd /etc/yum.repos.d/

ls --->there is no cm repo available for CM

sudo nano cloudera-manager.repo --> Create the file for CM with change the
ip/fqdn address as your CM machine

[cloudera-manager]
name=cloudera manager
baseurl=http://ip-10-0-1-135.ec2.internal/cm/5.4.0/
gpgcheck = 0
gpgkey =http://ip-10-0-1-135.ec2.internal/cm/RPM-GPG-KEY-cloudera

Refest the repolist & check for cm in list

sudo yum clean all --> clean all the data

sudo yum makecache --> to make cache for repo & tell server to our repo as local
server
sudo yum repolist all -->. update & available all updated repo list

-->> Ensure the list has the cloudera manager repo <<--

cd
clear

Setup MYSQL SERVER ON local machine -->>MYSQL SERVER ON SAME LOCAL MACHINE

sudo yum install mysql-server -y --> yum will take mysql server from his own
repo

sudo service mysqld status

sudo service mysqld start
sudo service mysqld status Status as OK

sudo yum install -y mysql-connector-java -->jdbc driver installed for onw machine

INSTALLATION OF SQL:

mysql_secure_installation --> To make sceure TO DATABASE

Enter current password for root (enter for none):[ENTER]

Set root password? [Y/n] y
New password: 123
Re-enter new password: 123
Remove anonymous users? [Y/n] y
Disallow root login remotely? [Y/n] n
Remove test database and access to it? [Y/n] n
Reload privilege tables now? [Y/n] y

===============================================

login to mysql :-

mysql -u root -p

password- 123

select host, user, password from mysql.user;

create database hive DEFAULT CHARACTER SET utf8;

grant all on hive.* TO 'hive'@'%' IDENTIFIED BY 'P@ssw0rd';

create database hue DEFAULT CHARACTER SET utf8;

grant all on hue.* TO 'hue'@'%' IDENTIFIED BY 'P@ssw0rd';

create database rman DEFAULT CHARACTER SET utf8;

grant all on rman.* TO 'rman'@'%' IDENTIFIED BY 'P@ssw0rd';

create database navs DEFAULT CHARACTER SET utf8;

grant all on navs.* TO 'navs'@'%' IDENTIFIED BY 'P@ssw0rd';

create database navms DEFAULT CHARACTER SET utf8;

grant all on navms.* TO 'navms'@'%' IDENTIFIED BY 'P@ssw0rd';

create database oozie DEFAULT CHARACTER SET utf8;

grant all on oozie.* TO 'oozie'@'%' IDENTIFIED BY 'P@ssw0rd';

create database actmo DEFAULT CHARACTER SET utf8;

grant all on actmo.* TO 'actmo'@'%' IDENTIFIED BY 'P@ssw0rd';

create database sentry DEFAULT CHARACTER SET utf8;

grant all on sentry.* TO 'sentry'@'%' IDENTIFIED BY 'P@ssw0rd';

grant all on *.* to 'temp'@'%' identified by 'P@ssw0rd' with grant option;

show databases;

select host, user, password from mysql.user;

Exit

INSTALL CM SERVER AGENT DEAMONS:

sudo yum install cloudera-manager-server cloudera-manager-daemons

sudo yum install cloudera-manager-agent cloudera-manager-daemons

-----------------------------------------------------------------

sudo yum install cloudera-manager-server cloudera-manager-daemons

sudo yum install cloudera-manager-agent cloudera-manager-daemons

--------------------------------------------------------------------

tail -f /var/log/cloudera-scm-server/cloudera-scm-server.logs
sudo /usr/share/cmf/schema/scm_prepare_database.sh mysql -h ip-10-0-
1.122.ec2.internal -p 3306 -u temp -p --scm-host ip-10-0-0.152.ec2.internal scmdb
scmuser

password - p@ssw0rd

login to databases

mysql -u root -p password - 123

select host, user, password from mysql.user;

drop user 'temp'@'%';

select host, user, password from mysql.user;

exit;
================================================================
================================================================

INSTALL CM SERVER AND AGENT DAEMONS (Run both)

sudo yum install cloudera-manager-server cloudera-manager-daemons

sudo yum install cloudera-manager-agent cloudera-manager-daemons

sudo less /etc/cloudera-scm-server/db.properties ---> to know all the downloads

to be from local repo

-->> to check logs

FOR LOCAL MYSQL SERVER

tail -f /var/log/cloudera-scm-server/cloudera-scm-server.log

sudo /usr/share/cmf/schema/scm_prepare_database.sh mysql -h ip-10-0-0-

119.ec2.internal -P 3306 -u temp -p --scm-host ip-10-0-0-119.ec2.internal scmdb
scmuser

Password should be - P@ssw0rd P@ssw0rd

mysql -u root -p

select host, user, password from mysql.user;

drop user 'temp'@'%';

select host, user, password from mysql.user;

exit

=============================
in another duplicate shall
sudo less /etc/cloudera-scm-server/db.properties --> to verify DB is create or
not.

netstat -nap | grep 7180

9. START CM AGENT AND CM SERVER

sudo service cloudera-scm-agent start

sudo service cloudera-scm-agent status
sudo service cloudera-scm-server start
sudo service cloudera-scm-server status

After 2 min goto yourcmip:7180 Log in paste the host IPS, Your CM host will appear
in Managed hosts and won’t be
included in the cluster.

Take more machines same like this as prerequisite is completed as 4 machine 1 for
SG & 3 for datanodes

On the Cluster INSTALLATION page for CDH version go to more options and remove all
genkins parcel address and at the local parcel repo box add
and continue

sudo cp /etc/yum.repos.d/cloudera-manager.repo --> To see the CM version from

local repo
cd /var/www/html/cdh -->put cdh version here

pwd

http://10.0.1.135/cdh5.4.0/ only change ip address & paste this in to config

for CM
http://10.0.1.135/cm/5 put proper ip in custome repo adresss & paste

for key - > http://10.0.1.135/cm/RPM-GPG-KEY-cloudera

sh clustercmd.sh sudo yum install -y mysql-connector-java

Set up sql with user name & password

test conection

Use powerWindow
cd /Downloads

.\pscp.exe -i .\vtnew.PPK key .\clustercmd.sh [email protected]:/home/centos

cat clustercmd.sh

to put jdbc connector set to all machines us script with create file as cluster
with all ip addresses

send .pem file on cluster

.\pscp.exe -i .\.PPK key .\.pem centos@ip adresss:/home/centos

chmod 400 .pem

sh clustercmd.sh sudo yum install -y mysql-connector-java

sh clustercmd.sh sudo yum install -y mysql-connector-java

====================================================
====================================================
No kernel versions that are known to be bad are running.
Cloudera recommends setting /proc/sys/vm/swappiness to at most 10. Current setting
is 60. Use the sysctl command
to change this setting at runtime and edit /etc/sysctl.conf for this setting to be
saved after a reboot.
You may continue with installation, but you may run into issues with Cloudera
Manager reporting that
your hosts are unhealthy because they are swapping. The following hosts are
affected:
No performance concerns with Transparent Huge Pages settings.