Re: python/avro example producer

2012-12-19 Thread David Arthur
I'll try to give an unbiased answer :) I believe my library has more comprehensive coverage of the APIs for 0.7.x (e.g., MultiProduce, MultiFetch), as well as more test coverage (including integration tests). However, it's newer and has had less real world usage, so there may be undiscovered b

Re: python/avro example producer

2012-12-19 Thread Joseph Crotty
Thanks for the insights. Another developer where I work started on a Python producer, previously attached, a few weeks ago initially using your kafka-python lib, but for some reason switched to pykafka. Is one preferred over the other? On Wed, Dec 19, 2012 at 8:36 AM, David Arthur wrote: > The

Re: python/avro example producer

2012-12-19 Thread David Arthur
The fundamental unit of Kafka is a Message. A Message contains a few bytes of metadata (a magic number, a crc32 checksum, some attributes) and a payload of bytes. For the most part these details are obscured from the end-user, so all you have to concern yourself with sending the actual data (pa

Re: python/avro example producer

2012-12-19 Thread Joseph Crotty
What exactly does a "payload" mean? Sorry, fairly new to Kafka. Is there a payload method that needs to be called by the python producer? Thanks for any insights. Attached some sample code if you have time to lead us to the water! Probably something simple we are missing. Joe On Wed, Dec 19, 20

Re: python/avro example producer

2012-12-19 Thread David Arthur
Do you mean a Python producer that sends Avro payloads? There are a couple of Python clients floating around, including mine: https://github.com/mumrah/kafka-python The Avro package is in pypi (http://pypi.python.org/pypi/avro/1.7.3), with official docs and getting started with Python on the