Scribd
Upload
11 views

How to install Kafka on Windows machine

Uploaded by

Saadaoui Mayssa
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as DOCX, PDF, TXT or read online on Scribd
11 views

How to install Kafka on Windows machine

Uploaded by

Saadaoui Mayssa
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as DOCX, PDF, TXT or read online on Scribd
You are on page 1/ 7

How to install Kafka on Windows machine

Preface
Kafka is an open-source tool used for building real-time data processing pipelines.

Here’s how to download and set it up on your Windows machine.

In this cpurse, I will show you how to download and set up Kafka on your Windows machine.

Steps
1. Install Java & JDK
2. Set up environment variables
3. Install Kafka
4. Set up configuration files
5. Test Kafka

1. Install Java & JDK


To use Kafka, you must install Java and a Java Development Kit (JDK) on your machine.
This is because Kafka is built using the Java programming language, and therefore requires a
JDK to run.

A. Java
 Enter “download Java” in your browser
 Click on the first link from the official website
 Click on Download Java then click on the downloaded file
 Follow instructions from the Java installation wizard

B. JDK (Java Development Kit)


 Enter “download Java Development Kit” in your browser
 Click on the first link from the official website
 Scroll to Java SE Development Kit 11 and then click on the Windows tab
 Under the Download pane, click on the link for the x64 installer
 You will need an Oracle account to advance with the installation step. Register if you
don’t have one
 Resume this step to tick the terms and condition checkbox and click on Download
jdk-11xxx-windows-x64.exe
 Click on the download file and follow the instructions from the JDK installation
wizard

2. Set up environment variables


 Search for edit the system environment variables page
 Click on the Environment Variables button
 Under System variables, click New
 For the Variable name, enter JAVA_HOME
 For the Variable value, enter the directory path where the JDK is stored

 Once it looks like the above, click all the OK buttons until you exit the environment
variables page
 Enter the command (cmd) prompt and type java -version. You should get the latest
version of java displayed like so:

3. Install Kafka
 Enter “download Kafka” into your browser
 Click on the first link from the official website. You should see the website is sorted
by versions (the latest on top, earlier releases towards the bottom)
 Under the latest version, click on the .tgz file next to Scala 2.13 under Binary
downloads, which should download the latest Kafka version as a compressed file.
 Unzip the .tgz file to get a folder named kafka_2.13-3.4.0.
 Unzip the tar file in this folder too. Once you have this folder, copy and paste it to the
root of your C drive

 Rename the folder to kafka

4. Set up configuration files


A. Server file

 Enter the kafka folder created from the previous step


 In the config folder, open the server.properties file in a notepad

 In the file replace “log.dirs=/tmp/kafka-logs” with


“log.dirs=C:/kafka/kafka-logs/server_logs”
 Save and exit the file.

B. Zookeeper file

 Return to the config folder and open the zookeeper.properties file in a notepad
 In the file replace “dataDir=/tmp/zookeeper” with
“dataDir=C:/kafka/kafka_logs/zookeeper_logs”
 Save and exit the file.

5. Test Kafka
Let’s confirm the installation and configurations worked by starting the Zookeeper and Kafka
servers.

Start Zookeeper server

To start the Zookeeper server,

 Open the command prompt


 Type the following into the terminal:

C:\\kafka\\bin\\windows\\zookeeper-server-start.bat C:\\kafka\\config\\
zookeeper.properties

This should activate the Zookeeper server like so:

Start Kafka server

To start the Kafka server,


 Open a new command prompt
 Type the following into the terminal:

C:\\kafka\\bin\\windows\\kafka-server-start.bat C:\\kafka\\config\\
server.properties

The Kafka server should be running in its terminal like so:

Once the servers are running, we can test if Kafka works by:

1. Creating a topic
2. Creating a producer
3. Creating a consumer

We’ll create a producer to send messages to the consumer. The consumer will subscribe to a
topic we’ll name demo_topic, and the consumer will display the messages from the topic to
the terminal as a validation check that the installation and configuration were successful.

In the commands below, we’ll replace <host>:<port> for localhost:9092 and <topic-name>
for demo_topic.

1. Create a topic

 Open a new command prompt


 Enter the following command into the terminal:

C:\\kafka\\bin\\windows\\kafka-topics.bat --create --bootstrap-server


127.0.0.1:9092 --topic <topic-name> --replication-factor <replication-
factor-no> --partitions <partition-no>

The replications factor specifies the number of copies of each message which will be stored
across multiple broker instances. If the main broker crashes or fails to send a message for any
reason, another broker can step in to ensure the message is delivered to the appropriate topic
without interruption. This is one of the features of Kafka that make it a fault-tolerance
streaming tool.
The partitions are the number of chunks the messages are split into. Each topic can have
multiple partitions, and each partition can be stored on different brokers, which adds to the
scalability of the event processor.

This should return:

2. Create a producer

 Open a new command prompt


 Enter the following command into the terminal:

C:\\kafka\\bin\\windows\\kafka-console-producer.bat --broker-list
127.0.0.1:9092 --topic <topic-name>

 Enter messages to send to the topic specified in the last step:

3. Create a consumer

 Open the command prompt


 Enter the following command into the terminal:

C:\\kafka\\bin\\windows\\kafka-console-consumer.bat --topic <topic-name> --


bootstrap-server 127.0.0.1:9092 --from-beginning

This will return all messages from the start of the topic’s inception. You can consume only
new messages from the topic by removing the --from-beginning flag.

Here is what the producer (left) and consumer (right) look like from a terminal perspective:
Note: the terminals can be opened in an IDE like VS Code too, which makes it easier to
manage multiple terminals in one session.

Congrats! You have successfully installed and configured Kafka on your machine🎉!

Delete a topic

If you want to delete a Kafka topic, you can also run this command:

C:/kafka/bin/windows/kafka-topics.bat --bootstrap-server <host>:<port> --


delete --topic <topic_name>

Replace the <host>:<port> with localhost:9092 and <topic_name> with demo_topic once
you’re done. Once you’re comfortable with the whole process, you can configure the host,
port, topic name and other variables to your preferences.

You might also like