python confluent_kafka

Confluent Kafka is a Python client library for Apache Kafka, developed by Confluent. It provides an easy-to-use interface for interacting with Kafka clusters, allowing you to produce and consume messages from Kafka.

To use the confluent_kafka library in, you first need to install it. You can do this by running the following command:

pip install confluent-kafka

Once installed, you can import the library in your Python code as follows:

from confluent_kafka import Producer, Consumer

To produce messages to a Kafka topic, you can create a `Producer` instance and use its `produce()` method. Here's an example:

producer = Producer({'bootstrap.servers': 'localhost:9092'})

topic = 'my_topic'
message = 'Hello, Kafka!'

producer.produce(topic, message.encode('utf-8'))
producer.flush()

To consume messages from a Kafka topic, you can create a `Consumer` instance and use its `subscribe()` and `poll()` methods. Here's an example:

consumer = Consumer({
    'bootstrap.servers': 'localhost:9092',
    'group.id': 'my_consumer_group',
    'auto.offset.reset': 'earliest'
})

topic = 'my_topic'

consumer.subscribe([topic])

while True:
    msg = consumer.poll(1.0)

    if msg is None:
        continue

    if msg.error():
        print(f"Consumer error: {msg.error()}")
        continue

    print(f"Received message: {msg.value().decode('utf-8')}")

consumer.close()

These are just basic examples to get you started with the confluent_kafka library. You can refer to the official documentation for more advanced usage and configuration options.

Please note that you need a running Kafka cluster to use the confluent_kafka library.