Thanks for the insights. Another developer where I work started on a Python producer, previously attached, a few weeks ago initially using your kafka-python lib, but for some reason switched to pykafka. Is one preferred over the other?
On Wed, Dec 19, 2012 at 8:36 AM, David Arthur <mum...@gmail.com> wrote: > The fundamental unit of Kafka is a Message. A Message contains a few bytes > of metadata (a magic number, a crc32 checksum, some attributes) and a > payload of bytes. For the most part these details are obscured from the > end-user, so all you have to concern yourself with sending the actual data > (payload). In Java the payload is simply a byte array, in Python it's just > a string. > > I'd suggest reading through the Quick Start (http://kafka.apache.org/** > quickstart.html <http://kafka.apache.org/quickstart.html>), and Design ( > http://kafka.apache.org/**design.html<http://kafka.apache.org/design.html>) > if you're really interested in how things work. > > As for sending data with my Python producer, just check out the README on > the project page: https://github.com/mumrah/**kafka-python#send-a-message- > **to-a-topic<https://github.com/mumrah/kafka-python#send-a-message-to-a-topic> > > Cheers > > > On 12/19/12 10:21 AM, Joseph Crotty wrote: > >> What exactly does a "payload" mean? Sorry, fairly new to Kafka. Is there >> a payload method that needs to be called by the python producer? >> >> Thanks for any insights. Attached some sample code if you have time to >> lead us to the water! Probably something simple we are missing. >> >> Joe >> >> >> On Wed, Dec 19, 2012 at 7:46 AM, David Arthur <mum...@gmail.com <mailto: >> mum...@gmail.com>> wrote: >> >> Do you mean a Python producer that sends Avro payloads? >> >> There are a couple of Python clients floating around, including >> mine: >> https://github.com/mumrah/**kafka-python<https://github.com/mumrah/kafka-python> >> >> The Avro package is in pypi >> >> (http://pypi.python.org/pypi/**avro/1.7.3<http://pypi.python.org/pypi/avro/1.7.3>), >> with official docs and >> getting started with Python on the Avro project page >> >> (http://avro.apache.org/docs/**1.7.3/gettingstartedpython.**html<http://avro.apache.org/docs/1.7.3/gettingstartedpython.html> >> ) >> >> Good luck! >> >> >> On 12/18/12 10:38 PM, Joseph Crotty wrote: >> >> Anyone have a python/avro producer that slurps up records from >> a flat file >> (i.e., mix of string and binary data) and publishes to Kafka >> they would be >> willing to share? >> >> Starting to think this might be a whole lot faster to do in >> Java, but maybe >> someone has a Python solution already in hand. >> >> Joe >> >> >> >> >
