Have you installed the correct package kafka-python?
*pip install kafka-python*
Collecting kafka-python
Downloading kafka_python-2.0.2-py2.py3-none-any.whl (246 kB)
|████████████████████████████████| 246 kB 1.9 MB/s
Installing collected packages: kafka-python
Successfully installed kafka-python-2.0.2
*pip list|grep kafka*
*kafka-python 2.0.2*
*python3*
Python 3.7.3 (default, Apr 3 2021, 20:42:31)
[GCC 4.8.5 20150623 (Red Hat 4.8.5-39)] on linux
Type "help", "copyright", "credits" or "license" for more information.
*>>> from kafka import KafkaProducer*
*>>>*
view my Linkedin profile
<https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/>
https://en.everybodywiki.com/Mich_Talebzadeh
*Disclaimer:* Use it at your own risk. Any and all responsibility for any
loss, damage or destruction of data or any other property which may arise
from relying on this email's technical content is explicitly disclaimed.
The author will in no case be liable for any monetary damages arising from
such loss, damage or destruction.
On Fri, 18 Feb 2022 at 07:45, karan alang <karan.al...@gmail.com> wrote:
> Hello All,
>
> I've a GCP Dataproc cluster, and i'm running a Spark StructuredStreaming
> job on this.
> I'm trying to use KafkaProducer to push aggregated data into a Kafka
> topic, however when i import KafkaProducer (from kafka import
> KafkaProducer), it gives error
>
> ```
>
> Traceback (most recent call last):
>
> File
>
> "/tmp/7e27e272e64b461dbdc2e5083dc23202/StructuredStreaming_GCP_Versa_Sase_gcloud.py",
> line 14, in <module>
>
> from kafka.producer import KafkaProducer
>
> File "/opt/conda/default/lib/python3.8/site-packages/kafka/__init__.py",
> line 23, in <module>
>
> from kafka.producer import KafkaProducer
>
> File
>
> "/opt/conda/default/lib/python3.8/site-packages/kafka/producer/__init__.py",
> line 4, in <module>
>
> from .simple import SimpleProducer
>
> File
> "/opt/conda/default/lib/python3.8/site-packages/kafka/producer/simple.py",
> line 54
>
> return '<SimpleProducer batch=%s>' % self.async
> ```
>
> As part of the initialization actions, i'm installing the following :
> ---
>
> pip install pypi
> pip install kafka-python
> pip install google-cloud-storage
> pip install pandas
>
> ---
>
> Additional details in stackoverflow :
>
> https://stackoverflow.com/questions/71169869/gcp-dataproc-getting-error-in-importing-kafkaproducer
>
> Any ideas on what needs to be to fix this ?
> tia!
>
