3/29/24, 4:29 PM MapReduce Word Count Example - javatpoint

Home C Java Hadoop PHP HTML CSS JavaScript jQuery XML JSON Ajax

https://www.javatpoint.com/mapreduce-word-count-example 1/12
3/29/24, 4:29 PM MapReduce Word Count Example - javatpoint

MapReduce Word Count Example

In MapReduce word count example, we find out the frequency of each word. Here, the role of Mapper is
to map the keys to the existing values and the role of Reducer is to aggregate the keys of common
values. So, everything is represented in the form of Key-value pair.

Pre-requisite

Java Installation - Check whether the Java is installed or not using the following command.
java -version

Hadoop Installation - Check whether the Hadoop is installed or not using the following
command.
hadoop version

If any of them is not installed in your system, follow the below link to install it.

www.javatpoint.com/hadoop-installation

Steps to execute MapReduce word count example

Create a text file in your local machine and write some text into it.
$ nano data.txt

https://www.javatpoint.com/mapreduce-word-count-example 2/12
3/29/24, 4:29 PM MapReduce Word Count Example - javatpoint

Check the text written in the data.txt file.

$ cat data.txt

In this example, we find out the frequency of each word exists in this text file.

https://www.javatpoint.com/mapreduce-word-count-example 3/12
3/29/24, 4:29 PM MapReduce Word Count Example - javatpoint

Create a directory in HDFS, where to kept text file.

$ hdfs dfs -mkdir /test

Upload the data.txt file on HDFS in the specific directory.

$ hdfs dfs -put /home/codegyani/data.txt /test

Write the MapReduce program using eclipse.

File: WC_Mapper.java

package com.javatpoint;

import java.io.IOException;
import java.util.StringTokenizer;
import org.apache.hadoop.io.IntWritable;
import org.apache.hadoop.io.LongWritable;
import org.apache.hadoop.io.Text;
import org.apache.hadoop.mapred.MapReduceBase;

https://www.javatpoint.com/mapreduce-word-count-example 4/12
3/29/24, 4:29 PM MapReduce Word Count Example - javatpoint

import org.apache.hadoop.mapred.Mapper;
import org.apache.hadoop.mapred.OutputCollector;
import org.apache.hadoop.mapred.Reporter;
public class WC_Mapper extends MapReduceBase implements Mapper<LongWritable,Text,Text,IntWrit
{
private final static IntWritable one = new IntWritable(1);
private Text word = new Text();
public void map(LongWritable key, Text value,OutputCollector<Text,IntWritable> output,
Reporter reporter) throws IOException{
String line = value.toString();
StringTokenizer tokenizer = new StringTokenizer(line);
while (tokenizer.hasMoreTokens()){
word.set(tokenizer.nextToken());
output.collect(word, one);
}
}

File: WC_Reducer.java

package com.javatpoint;
import java.io.IOException;
import java.util.Iterator;
import org.apache.hadoop.io.IntWritable;
import org.apache.hadoop.io.Text;
import org.apache.hadoop.mapred.MapReduceBase;
import org.apache.hadoop.mapred.OutputCollector;
import org.apache.hadoop.mapred.Reducer;
import org.apache.hadoop.mapred.Reporter;

public class WC_Reducer extends MapReduceBase implements Reducer<Text,IntWritable,Text,IntWr

public void reduce(Text key, Iterator<IntWritable> values,OutputCollector<Text,IntWritable> outpu
Reporter reporter) throws IOException {
int sum=0;
https://www.javatpoint.com/mapreduce-word-count-example 5/12
3/29/24, 4:29 PM MapReduce Word Count Example - javatpoint

while (values.hasNext()) {
sum+=values.next().get();
}
output.collect(key,new IntWritable(sum));
}
}

File: WC_Runner.java

package com.javatpoint;

import java.io.IOException;
import org.apache.hadoop.fs.Path;
import org.apache.hadoop.io.IntWritable;
import org.apache.hadoop.io.Text;
import org.apache.hadoop.mapred.FileInputFormat;
import org.apache.hadoop.mapred.FileOutputFormat;
import org.apache.hadoop.mapred.JobClient;
import org.apache.hadoop.mapred.JobConf;
import org.apache.hadoop.mapred.TextInputFormat;
import org.apache.hadoop.mapred.TextOutputFormat;
public class WC_Runner {
public static void main(String[] args) throws IOException{
JobConf conf = new JobConf(WC_Runner.class);
conf.setJobName("WordCount");
conf.setOutputKeyClass(Text.class);
conf.setOutputValueClass(IntWritable.class);
conf.setMapperClass(WC_Mapper.class);
conf.setCombinerClass(WC_Reducer.class);
conf.setReducerClass(WC_Reducer.class);
conf.setInputFormat(TextInputFormat.class);
conf.setOutputFormat(TextOutputFormat.class);
FileInputFormat.setInputPaths(conf,new Path(args[0]));
FileOutputFormat.setOutputPath(conf,new Path(args[1]));

https://www.javatpoint.com/mapreduce-word-count-example 6/12
3/29/24, 4:29 PM MapReduce Word Count Example - javatpoint

JobClient.runJob(conf);
}
}

Download the source code.

Create the jar file of this program and name it countworddemo.jar.

Run the jar file

hadoop jar /home/codegyani/wordcountdemo.jar com.javatpoint.WC_Runner /test/data.txt
/r_output

The output is stored in /r_output/part-00000

Now execute the command to see the output.

hdfs dfs -cat /r_output/part-00000

https://www.javatpoint.com/mapreduce-word-count-example 7/12
3/29/24, 4:29 PM MapReduce Word Count Example - javatpoint

← Prev Next →

https://www.javatpoint.com/mapreduce-word-count-example 8/12
3/29/24, 4:29 PM MapReduce Word Count Example - javatpoint

For Videos Join Our Youtube Channel: Join Now

Feedback

Send your Feedback to [email protected]

Help Others, Please Share

Learn Latest Tutorials

Splunk SPSS Swagger Transact-SQL

Tumblr ReactJS Regex Reinforcement

Learning

R Programming RxJS tutorial React Native Python Design

tutorial tutorial Patterns
RxJS
R Programming React Native Python Design
Patterns

Python Pillow Python Turtle Keras tutorial

tutorial tutorial
Keras
Python Pillow Python Turtle

https://www.javatpoint.com/mapreduce-word-count-example 9/12
3/29/24, 4:29 PM MapReduce Word Count Example - javatpoint

Preparation

Aptitude Logical Verbal Ability Interview

Reasoning Questions
Aptitude Verbal Ability
Reasoning Interview Questions

Company
Interview
Questions
Company Questions

Trending Technologies

Artificial AWS Tutorial Selenium Cloud

Intelligence tutorial Computing
AWS
Artificial Selenium Cloud Computing
Intelligence

Hadoop tutorial ReactJS Data Science Angular 7

Tutorial Tutorial Tutorial
Hadoop
ReactJS Data Science Angular 7

Blockchain Git Tutorial Machine DevOps

Tutorial Learning Tutorial Tutorial
Git
Blockchain Machine Learning DevOps

https://www.javatpoint.com/mapreduce-word-count-example 10/12
3/29/24, 4:29 PM MapReduce Word Count Example - javatpoint

B.Tech / MCA

DBMS tutorial Data Structures DAA tutorial Operating

tutorial System
DBMS DAA
Data Structures Operating System

Computer Compiler Computer Discrete

Network tutorial Design tutorial Organization and Mathematics
Architecture Tutorial
Computer Network Compiler Design
Computer Discrete
Organization Mathematics

Ethical Hacking Computer Software html tutorial

Graphics Tutorial Engineering
Ethical Hacking Web Technology
Computer Graphics Software
Engineering

Cyber Security Automata C Language C++ tutorial

tutorial Tutorial tutorial
C++
Cyber Security Automata C Programming

Java tutorial .Net Python tutorial List of

Framework Programs
Java Python
tutorial
Programs
.Net

Control Data Mining Data

Systems tutorial Tutorial Warehouse
Tutorial
Control System Data Mining
Data Warehouse
https://www.javatpoint.com/mapreduce-word-count-example 11/12
3/29/24, 4:29 PM MapReduce Word Count Example - javatpoint

https://www.javatpoint.com/mapreduce-word-count-example 12/12