Thanks, yeah, I understand that I couldn't truly consume from multiple
threads. I was hoping I could get away with just commits from the thread
(to signal that processing of the data it was passed was complete).
It's just a little extra complication. A SMOP.
Skip
On Dec 29, 2017 10:46 AM, "Stephen Durfey" <sjdur...@gmail.com> wrote:
Hey Skip, the Kafka consumer is NOT thread safe. If you want to share
across threads you will need to properly lock. Alternatively you can create
a consumer per thread. Check out the class javadoc under the "multithreaded
processing" for more details and suggestions.
________________________________
From: Skip Montanaro <skip.montan...@gmail.com>
Sent: Friday, December 29, 2017 11:21:52 AM
To: users@kafka.apache.org
Subject: Can I commit from a thread?
I subscribe to topics and receive input in my main thread (this
happens to be in Python):
consumer = kafka.KafkaConsumer(...)
for data in consumer:
do_stuff(data)
do_stuff(...) can hand messages off to a separate thread for
processing. Is it okay to call commit() from that thread or do I need
to do the complete lock dance for that as well?
Thanks,
Skip Montanaro
