Thanks a lot Keith for detailed explanation, that will surely help me.
Another question, I can see AsyncProducer is not yet implemented, any idea
by when it can be available?
Thanks,
Vijay
On 10 June 2015 at 18:42, Keith Bourgoin <keith.bourg...@gmail.com> wrote:
> Hi Vijay,
>
> PyKafka support custom partitioners so that you can control how the
> producer distributes messages. Looking at it now, I realize the
> documentation around this is a bit lacking, so I'll add that. In the
> meantime, here's an example of how to use the hashing partitioner which is
> included with PyKafka.
>
> client = pykafka.KafkaClient()
>
> # Producer with random_partitioner
> prod = client.topics['foo'].get_producer()
> prod.produce(['foo', 'bar', 'baz'])
>
> # Producer with hashing_partitioner
> prod =
> client.topics['foo'].get_producer(partitioner=pykafka.partitioners.hashing_partitioner)
> prod.produce([('p_key1', 'foo'), ('p_key2', 'foo'), ('p_key1', 'baz')])
>
>
>
> Cheers,
>
> Keith
>
>
> On Wed, Jun 10, 2015 at 6:35 AM Vijay Patil <vijay2110.t...@gmail.com>
> wrote:
>
> > Thanks,
> > a question, does it support keyed Producer?
> > I mean how it works when I have multiple partitions? Will it be able to
> > identify partition based of the key which I pass?
> >
> > On 9 June 2015 at 00:54, Keith Bourgoin <keith.bourg...@gmail.com>
> wrote:
> >
> > > Hi Kafka folks,
> > >
> > > I'm happy to announce the 1.0 release of PyKafka
> > > <https://github.com/parsely/pykafka>, a new Python driver for Kafka!
> > >
> > > This is an upgrade of an older library, named samsa, which was only
> > > compatible with Kafka 0.7.x. We've spent the last few months upgrading
> it
> > > to work with Kafka 0.8.2.1 and revamping the codebase to be cleaner and
> > > more efficient.
> > >
> > > The biggest difference between PyKafka and other Python drivers is the
> > > inclusion of a balanced consumer implementation. We implemented the
> same
> > > balancing algorithm as the Java driver and PyKafka handles consumer
> > > registration via Zookeeper like the Java driver.
> > >
> > > We at Parse.ly have been using this driver in production for a few
> weeks
> > > and are confident in its stability. Even so, we're looking to add more
> > > features over the next couple months and are always trying to find ways
> > to
> > > improve performance.
> > >
> > > You can find the project on PyPI <
> https://pypi.python.org/pypi/pykafka/
> > >,
> > > Github <https://github.com/parsely/pykafka> and our mailing list
> > > <https://groups.google.com/forum/#%21forum/pykafka-user>. We also
> wrote
> > a
> > > blog
> > > post
> > > <
> > >
> >
> http://blog.parsely.com/post/2006/announcing-pykafka-python-support-for-kafka-0-8-x/
> > > >
> > > with a bit more info.
> > >
> > > Cheers,
> > >
> > > Keith.
> > >
> >
>
