0% found this document useful (0 votes)

7 views

ConfluentKafka-LinuxCommands

The document provides a step-by-step guide to enable Windows Subsystem for Linux (WSL) and install Java Development Kit (JDK) on Ubuntu. It also details the installation and setup of Confluent Kafka, including commands to start various components and manage Java versions. Additionally, it includes links for further resources and examples related to Kafka and Python integration.

Uploaded by

tvr

Available Formats

Download as PDF, TXT or read online on Scribd

0% found this document useful (0 votes)

7 views

ConfluentKafka-LinuxCommands

Uploaded by

tvr

Available Formats

Download as PDF, TXT or read online on Scribd

You are on page 1/ 8

Enable WSL in windows

Step1 : Turn on windows features -->

windows subsystems fir Linux
Step2: Go to command prompt and
enter "wsl --list". It is should show the
list.
Step3: wsl --install -d Ubuntu
Step4: To verify the WSL, open new
command prompt using cmd in run and type wsl
--list and enter

Install JDK in Ubuntu

$ sudo apt update
$ sudo apt upgrade
$ sudo apt install default-jre -->
Installing the Default JRE
$ sudo apt install default-jdk -->
Installing the Default JDK
$ sudo apt install openjdk-11-jdk -->
install your desired OpenJDK version, such
as OpenJDK 11
$ java -version --> to verify the java
installation is successful or not

$ export
JAVA_HOME=/usr/lib/jvm/java-21-openjdk-amd64
$ export PATH=$PATH:$JAVA_HOME/bin

https://docs.confluent.io/platform/current/i
nstallation/installing_cp/zip-tar.html -->
to download the confluent kafka development
environment

To download the confluent kafka in ubuntu

$ curl -O
https://packages.confluent.io/archive/7.8/co
nfluent-community-7.8.0.tar.gz
$ wget
https://packages.confluent.io/archive/7.8/co
nfluent-community-7.8.0.tar.gz --> when the
above one is not working use this command

$ tar xzf confluent-7.8.0.tar.gz --> to

unzip the tar files
$ mv confluent-7.8.0 confluent --> to
change the folder name

Set the path

$ export CONFLUENT_HOME=~/confluent
$ export PATH=$PATH:$CONFLUENT_HOME/bin

If you have multiple Java versions installed

on your system, you have to execute the
following command to list all the versions
installed:
$ sudo update-alternatives --config java

if you want to remove OpenJDK 11 from the

Ubuntu system, you can use the following
command.
$ sudo apt remove openjdk-11-* -y

To remove any residual packages,

dependencies, and cached package files
related to Java, it is recommended to run
the following command.
$ sudo apt autoremove -y && sudo apt clean
-y

To start the kafka in Kraft mode

Start a Kafka controller. Run this command
in its own terminal.
./kafka-server-start
/home/kannan/confluent/etc/kafka/kraft/contr
oller.properties

Start a Kafka broker.

./kafka-server-start
/home/kannan/confluent/etc/kafka/kraft/broke
r.properties

Start Schema Registry. Run this command in

its own terminal.
./schema-registry-start
/home/kannan/confluent/etc/schema-registry/s
chema-registry.properties
Start other Confluent Platform components as
desired. For KRaft mode, make sure the
bootstrap server
points to the Kafka controller (port
9092) and not ZooKeeper.

Control Center
./control-center-start
/home/kannan/confluent/etc/confluent-control
-center/control-center.properties

Kafka Connect
./connect-distributed
/home/kannan/confluent/etc/schema-registry/c
onnect-avro-distributed.properties

Confluent REST Proxy

./kafka-rest-start
/home/kannan/confluent/etc/kafka-rest/kafka-
rest.properties

ksqlDB
./ksql-server-start
/home/kannan/confluent/etc/ksqldb/ksql-serve
r.properties

To start the kafka server in Zookeeper mode:

Start ZooKeeper. Run this command in its own
terminal.
./zookeeper-server-start
/home/kannan/confluent/etc/kafka/zookeeper.p
roperties

Start Kafka.
./kafka-server-start
/home/kannan/confluent/etc/kafka/server.prop
erties

Start Schema Registry. Run this command in

its own terminal.
./schema-registry-start
/home/kannan/confluent/etc/schema-registry/s
chema-registry.properties

Start other Confluent Platform components as

desired.
Control Center
./control-center-start
/home/kannan/confluent/etc/confluent-control
-center/control-center.properties

Kafka Connect
./connect-distributed
/home/kannan/confluent/etc/schema-registry/c
onnect-avro-distributed.properties

Confluent REST Proxy

./kafka-rest-start
/home/kannan/confluent/etc/kafka-rest/kafka-
rest.properties

ksqlDB
./ksql-server-start
/home/kannan/confluent/etc/ksqldb/ksql-serve
r.properties

To Uninstall
Remove the Confluent directory. For
example, if you have Confluent Platform
7.8.0 installed:
rm -rf $CONFLUENT_HOME

Remove the Confluent Platform data

files.
rm -rf
/var/lib/<confluent-platform-data-files>

https://docs.confluent.io/platform/current/i
nstallation/installing_cp/scripted-install.h
tml#installing-systemd-unit --> please
check this

https://developer.confluent.io/courses
1. Very simple producer and consumer using
python
https://www.youtube.com/watch?v=NlSEm8JgrZo

https://developer.confluent.io/courses/data-
streaming-systems/overview/

To install Confluent Kafka in Ubuntu please

follow the link -->
https://www.eddycyu.dev/blog/how-to-install-
apache-kafka-on-ubuntu-20.04-lts

sudo /home/kannan/confluent/bin/kafka-topics
--create --topic test --bootstrap-server
localhost:9092 --replication-factor 1
--partitions 1
sudo
/home/kannan/confluent/bin/kafka-avro-consol
e-producer --topic test \
--broker-list localhost:9092 \
--property
value.schema='{"type":"record","name":"testr
ecord","fields":[{"name":"message","type":"s
tring"}]}'
Supply the below one by one
{"message": "this is a testA"}
{"message": "this is a testB"}
{"message": "this is a testC"}
In another terminal
sudo
/home/kannan/confluent/bin/kafka-avro-consol
e-consumer --topic test --bootstrap-server
localhost:9092 --from-beginning

To connect KSQLDB server -->

https://luppeng.wordpress.com/2022/11/20/acc
ess-ksqldb-cli-inside-a-browser-using-ttyd/

Kafka - MySQLDB connect

-->https://hevodata.com/learn/mysql-kafka-co
nnector/

for python code reference

https://github.com/confluentinc/demo-scene
https://github.com/apostolos1927/KafkaDataba
seSink/blob/main/database_sink_consumer.py

--------------------------------------------
--------------------------------------------
--------------------------
https://linuxgenie.net/create-python-virtual
-environment-ubuntu-24-04/ --> To install
python virtual environment in ubuntu