from the bin directory of spark
*./spark-submit --master spark://localhost:7077
/home/anas/spark/historical_streamer.py*
On 1 May 2016 at 16:30, Ted Yu <yuzhih...@gmail.com> wrote:
> According to
> examples/src/main/scala/org/apache/spark/examples/streaming/KafkaWordCount.scala
> :
>
> import org.apache.kafka.clients.producer.{KafkaProducer, ProducerConfig,
> ProducerRecord}
>
> Can you give the command line you used to submit the job ?
>
> Probably classpath issue.
>
> On Sun, May 1, 2016 at 5:11 AM, fanooos <dev.fano...@gmail.com> wrote:
>
>> I have a very strange problem.
>>
>> I wrote a spark streaming job that monitor an HDFS directory, read the
>> newly
>> added files, and send the contents to Kafka.
>>
>> The job is written in python and you can got the code from this link
>>
>> http://pastebin.com/mpKkMkph
>>
>> When submitting the job I got that error
>>
>> *ImportError: cannot import name KafkaProducer*
>>
>> As you see, the error is very simple but the problem is that I could
>> import
>> the KafkaProducer from both python and pyspark shells without any problem.
>>
>> I tried to reboot the machine but the situation remain the same.
>>
>> What do you think the problem is?
>>
>>
>>
>> --
>> View this message in context:
>> http://apache-spark-user-list.1001560.n3.nabble.com/Can-not-import-KafkaProducer-in-spark-streaming-job-tp26857.html
>> Sent from the Apache Spark User List mailing list archive at Nabble.com.
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: user-unsubscr...@spark.apache.org
>> For additional commands, e-mail: user-h...@spark.apache.org
>>
>>
>
--
Anas Rabei
Senior Software Developer
Mubasher.info
anas.ra...@mubasher.info
