Scribd
Upload
29 views2 pages

Final Manual Practical 10 de

This document provides instructions for an experiment to stream data with Apache Kafka. It describes downloading Kafka and Zookeeper, setting up directories and configuration files, starting Zookeeper servers, creating a Kafka topic, and using a Kafka producer and consumer to send and receive messages. Key steps include running Zookeeper, starting the Kafka broker, creating a topic, using the Kafka console producer to send messages to the topic, and viewing the messages with the Kafka console consumer. The goal is to understand how to read and write data using Kafka and Zookeeper.

Uploaded by

Rohan Akhare
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as DOCX, PDF, TXT or read online on Scribd
29 views2 pages

Final Manual Practical 10 de

This document provides instructions for an experiment to stream data with Apache Kafka. It describes downloading Kafka and Zookeeper, setting up directories and configuration files, starting Zookeeper servers, creating a Kafka topic, and using a Kafka producer and consumer to send and receive messages. Key steps include running Zookeeper, starting the Kafka broker, creating a topic, using the Kafka console producer to send messages to the topic, and viewing the messages with the Kafka console consumer. The goal is to understand how to read and write data using Kafka and Zookeeper.

Uploaded by

Rohan Akhare
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as DOCX, PDF, TXT or read online on Scribd
You are on page 1/ 2

GS Mandal’s

Maharashtra Institute of Technology, Aurangabad


Laboratory manual -Practical Experiment Instruction sheet

Department of Emerging Sciences and Technology


Class: TY-AI&DS Subject Code: AID321 Subject: Data Engineering
Div: Roll No: Name of Student:

EXPERIMENT NO :10
AIM: Stream data with Apache Kafka
OBJECTIVE: Understand Steps to read and write data using Kafka and Zookeeper.
PER-REQUISITES: Zookeeper and Apache kafka
Theory:
Kafka was developed at LinkedIn and is now an Apache project. You can find Kafka on the web at
http://kafka.apache.org. Kafka requires another application, ZooKeeper, to manage information.You can
install and build a ZooKeeper cluster on your own, but for this example, you will use the ZooKeeper
scripts provided by Kafka. To learn more about ZooKeeper, you can find it at
http://zookeeper.apache.org.

Downloading Kafka and setting up the environment:

From your home directory, run the following commands:

you will need to create three separate folders for Kafka. Use the following commands to create the
directories:

Each ZooKeeper instance needs an ID. It will look for a file named myid with an integer
value in it. In each folder, create the corresponding myid file with the correct value. The
following commands will create the file:
Configuring ZooKeeper and Kafka:
The configuration files for both ZooKeeper and Kafka are in the Kafka directory in the
conf folder. We have three Kafka directories, we will first using Kafka_1 and the steps will need to be
applied to every other directory.

1.Command for starting Zookeeper:


Start cmd from kafka folder and type below command
C:\kafka>.\bin\windows\zookeeper-server-start.bat .\config\zookeeper.properties

2.Open New cmd from kafka folder and type following command to connect to Kafka.
C:\kafka>.\bin\windows\kafka-server-start.bat .\config\server.properties

3. Open new cmd from kafka folder and type following command for topic creation: Here we are
creating a topic named topic-example as shown below.
C:\kafka>.\bin\windows\kafka-topics.bat --create --topic topic-example --bootstrap-server
localhost:9092
Created topic topic-example.

4. In same cmd window start producer and type message to send to consumer as listed below.
C:\kafka>.\bin\windows\kafka-console-producer.bat --topic topic-example --bootstrap-server
localhost:9092
>hello world
>kafka topic event
>demo1
>demo2
(Press CTRL C to exit from Batch file)
>Terminate batch job (Y/N)? y

5. In same cmd window start consumer and read a message from producer listed below:
C:\kafka>.\bin\windows\kafka-console-consumer.bat --topic topic-example --from-beginning --
bootstrap-server localhost:9092
hello world
kafka topic event
demo1
demo2

Conclusion: Kafka and zookeeper are used generate topics. Topics are used used to stream data
between consumer and producer and send data is retrieved at consumer side.

You might also like