Re: KafkaUtils module not found on spark 3 pyspark

I got similar question recently so had to find some history I missed. If I understand correctly the class is "intentionally" removed in Spark 3, because the class refers "kafka 0.8" module which isn't guaranteed to work with recent Kafka version. And looks like there was another decision to not add

KafkaUtils module not found on spark 3 pyspark

I use hadoop 3.3.0 and spark 3.0.1-bin-hadoop3.2. And my python ide is eclipse version 2020-12. I try to develop python application with KafkaUtils pyspark module. My configuration reference of pyspark and eclipse is this site