Re: How to use gevent with python

2013-11-10 Thread David Montgomery
I am ready to use low level.. I do have a gevent worker server that can export that data I need to send as json. What about command line? I do over 10K qps can will be at 25K soon. Thanks On Mon, Nov 11, 2013 at 4:25 AM, Marc Labbe wrote: > @Kane, Let's be clear, you could send mult

Re: How to use gevent with python

2013-11-10 Thread Marc Labbe
@Kane, Let's be clear, you could send multiple consumer requests in parallel, but in terms of handling events, it would make things a little harder to manage. E.g. the importance of reading messages in order, are you ready to handle duplicates, ... For consumers, gevent may be useful when handling

Re: How to use gevent with python

2013-11-10 Thread David Montgomery
I am using kafka0. 7.x I should mention and is nit by choice because druid, the app I am using only supports 0.7.x. Brod is failing with gevent with 0.7.2. Brod totally does not work with gevent. Thanks On Mon, Nov 11, 2013 at 3:06 AM, Marc Labbe wrote: > Hi David, > > check for mahendra's

Re: How to use gevent with python

2013-11-10 Thread Kane Kane
New client rewrite proposal includes async Producer, but not async Consumer, i think there is a reason. You can not send new consume request before previous one is finished? On Sun, Nov 10, 2013 at 11:42 AM, Marc Labbe wrote: > Kane, you can probably achieve async consumer using the client direct

Re: How to use gevent with python

2013-11-10 Thread Marc Labbe
Kane, you can probably achieve async consumer using the client directly (kafka.client.send_fetch_request) which a parameter to pass in a callback. I haven't used it myself. On Sun, Nov 10, 2013 at 2:33 PM, Marc Labbe wrote: > I was just browsing in the issues list and found the original issue f

Re: How to use gevent with python

2013-11-10 Thread Marc Labbe
I was just browsing in the issues list and found the original issue from mahendra for gevent support, https://github.com/mumrah/kafka-python/issues/37. I suggest moving the discussion there. On Sun, Nov 10, 2013 at 2:06 PM, Marc Labbe wrote: > Hi David, > > check for mahendra's fork of kafka-py

Re: How to use gevent with python

2013-11-10 Thread Kane Kane
How it's possible to have async consumer? On Sun, Nov 10, 2013 at 11:06 AM, Marc Labbe wrote: > Hi David, > > check for mahendra's fork of kafka-python, he has implemented gevent > support in a branch (https://github.com/mahendra/kafka-python/tree/gevent) but > it hasn't made it to the main repo

Re: How to use gevent with python

2013-11-10 Thread Marc Labbe
Hi David, check for mahendra's fork of kafka-python, he has implemented gevent support in a branch (https://github.com/mahendra/kafka-python/tree/gevent) but it hasn't made it to the main repo (yet). I am not sure why and I haven't tested it myself. For brod, AFAIK, it doesn't support 0.8 so you

Re: How to use gevent with python

2013-11-09 Thread David Montgomery
Hi, Even even further...using/kafka-python ( https://github.com/mumrah/kafka-python) I get the below error. There is nothing wrong with my kafka server. I can telnet to 9092 just fine. Again..hoping for a python client that works with gevent. kafka-python did nok event make it past the connect

How to use gevent with python

2013-11-09 Thread David Montgomery
Hi, I asked this question on StackOverflow. No response. No solution seems tp work...even for a python lib that was designed for gevent and kafka, Is there any otehr libary that works with gevent wit python that works? am trying to use gevent to write to kafka using brod on python 2.7. Her