Sorry for wrong php code. It should be like that
3. In php code, Instead of using partition = 0, Use App1 = 0, App2 = 1,
App3 = 2
In App 1 Producer Script :
$producer->setMessages($queue, 0, array($data));
In App 2 Producer Script :
$producer->setMessages($queue, 1, array($data));
In Consumer 1 Script :
$consumer->setPartition($queue, 0);
In Consumer 2 Script :
$consumer->setPartition($queue, 1);
On Fri, Apr 29, 2016 at 5:17 PM, Kuldeep Kamboj <kuldeep.kam...@osscube.com>
wrote:
> I restarted brainstroming for Kafka setup. I found link
> http://www.michael-noll.com/blog/2013/03/13/running-a-multi-broker-apache-kafka-cluster-on-a-single-node/
> for kafka cluster setup (Thanks Lohith Samaga for valuable suggestion) .
> After try on local machine, I got this is what I need.
>
> Now I am implementing following approach.
>
> 1. First create 3 brokers
> cd /opt/kafka
> cp config/server.properties config/broker1.properties
> broker.id=1
> port=9092
> log.dir=/tmp/kafka-logs-1
>
> cp config/server.properties config/broker2.properties
> broker.id=2
> port=9093
> log.dir=/tmp/kafka-logs-2
>
> cp config/server.properties config/broker3.properties
> broker.id=3
> port=9094
> log.dir=/tmp/kafka-logs-3
>
>
> 2. Create topics with 3 partitions & 3 replications
> /opt/kafka/bin/kafka-topics.sh --zookeeper localhost:2181
> --create --topic category --partitions 3 --replication-factor
> 3
> ...
>
> 3. In php code, Instead of using partition = 0, Use App1 = 1, App2 = 2,
> App3 = 3
>
> In App 1 Producer Script :
> $producer->setMessages($queue, 0, array($data));
>
> In App 2 Producer Script :
> $producer->setMessages($queue, 2, array($data));
>
> In Consumer 1 Script :
> $consumer->setPartition($queue, 0);
>
> In Consumer 2 Script :
> $consumer->setPartition($queue, 1);
>
> I expect, it should work as my requirements.
>
>
> On Fri, Apr 22, 2016 at 3:34 PM, Lohith Samaga M <
> lohith.sam...@mphasis.com> wrote:
>
>> Hi,
>> Please set up a Kafka cluster. So, you can get high throughput as
>> well as high availability.
>>
>> Best regards / Mit freundlichen Grüßen / Sincères salutations
>> M. Lohith Samaga
>>
>>
>> -----Original Message-----
>> From: Gaurav Agarwal [mailto:gaurav130...@gmail.com]
>> Sent: Friday, April 22, 2016 12.44
>> To: users@kafka.apache.org
>> Subject: Re: How to build strategy for particular setup of kafka
>>
>> Hi
>> You can have one or two instances of Kafka but you can have one or two
>> Kafka topic dedicated to each application according to the need. Partition
>> will have u in increasing the throughput and consumer group id can help u
>> to make queue as topic or queue.
>> On Apr 22, 2016 12:37 PM, "Kuldeep Kamboj" <kuldeep.kam...@osscube.com>
>> wrote:
>>
>> > Thanks for reply,
>> >
>> > I understand the your point, But my whole strategy depend on first
>> > issue and that is how I can integrate Apps in architecture. Partition
>> > / Consumer groups have different purpose. If I need to setup three
>> > kafka instances each for App ?
>> >
>> > On Fri, Apr 22, 2016 at 12:30 PM, Lohith Samaga M <
>> > lohith.sam...@mphasis.com
>> > > wrote:
>> >
>> > > Hi,
>> > > It is better NOT to share topics among applications. You may
>> > > have a wrapper application reading from the queue/topic and routing
>> > > it to the correct application, but it is simpler for each
>> > > application to read from its own topic.
>> > >
>> > > Best regards / Mit freundlichen Grüßen / Sincères salutations M.
>> > > Lohith Samaga
>> > >
>> > >
>> > >
>> > > -----Original Message-----
>> > > From: Kuldeep Kamboj [mailto:kuldeep.kam...@osscube.com]
>> > > Sent: Friday, April 22, 2016 12.08
>> > > To: users@kafka.apache.org
>> > > Subject: How to build strategy for particular setup of kafka
>> > >
>> > > I have three applications let named AppA, AppB, AppC. All could have
>> > > several some message queue named like
>> > >
>> > > AppA
>> > > MsgQueueA
>> > > MsgQueueB
>> > > AppB
>> > > MsgQueueA
>> > > MsgQueueB
>> > > MsgQueueC
>> > > AppC
>> > > MsgQueueA
>> > > MsgQueueB
>> > > MsgQueueD
>> > >
>> > > I have very little exposure for Apache Kafka and even for message
>> > > queue concepts.
>> > >
>> > > I just thinking to create topics for MsgQueueA, MsgQueueB and so on.
>> > > But
>> > I
>> > > am confused how to integrate AppA, AppB, AppC. Partition and
>> > > Consumer
>> > group
>> > > look like wrong choices for them.
>> > >
>> > > Also If separation of queue/topic for Apps is better idea instead of
>> > > re-using them for different Apps for better debugging Like below
>> > >
>> > >
>> > > AppA
>> > > AppAMsgQueueA
>> > > AppAMsgQueueB
>> > > AppB
>> > > AppBMsgQueueA
>> > > AppBMsgQueueB
>> > > AppBMsgQueueC
>> > > AppC
>> > > AppCMsgQueueA
>> > > AppCMsgQueueB
>> > > AppCMsgQueueD
>> > >
>> > >
>> > >
>> > >
>> > > --
>> > > Kuldeep Kamboj
>> > > Information transmitted by this e-mail is proprietary to Mphasis,
>> > > its associated companies and/ or its customers and is intended for
>> > > use only by the individual or entity to which it is addressed, and
>> > may
>> > > contain information that is privileged, confidential or exempt from
>> > > disclosure under applicable law. If you are not the intended
>> > > recipient or it appears that this mail has been forwarded to you
>> > > without proper authority, you are notified that any use or
>> > > dissemination of this information in any manner is strictly
>> > > prohibited. In such cases, please notify us immediately at
>> > > mailmas...@mphasis.com and delete this mail from your records.
>> > >
>> >
>> >
>> >
>> > --
>> > Kuldeep Kamboj
>> > Mob. 9871662849
>> >
>> Information transmitted by this e-mail is proprietary to Mphasis, its
>> associated companies and/ or its customers and is intended
>> for use only by the individual or entity to which it is addressed, and
>> may contain information that is privileged, confidential or
>> exempt from disclosure under applicable law. If you are not the intended
>> recipient or it appears that this mail has been forwarded
>> to you without proper authority, you are notified that any use or
>> dissemination of this information in any manner is strictly
>> prohibited. In such cases, please notify us immediately at
>> mailmas...@mphasis.com and delete this mail from your records.
>>
>
>
>
> --
> Kuldeep Kamboj
> Mob. 9871662849
>
--
Kuldeep Kamboj
Mob. 9871662849
