Hi Gwen,
I like the approach of converting to general forms in most cases, which is
an early bound design.
You can also take a late bound approach of leaving data in its original
form but adding metadata to enable translation if needed at a later time.
This is necessary if you have homogeneous c
Have you looked at Pinterest Secor? (
http://engineering.pinterest.com/post/84276775924/introducing-pinterest-secor
)
Cheers, Robert
On Mon, Jun 16, 2014 at 5:17 AM, Mark Godfrey wrote:
> There is Bifrost, which archives Kafka data to S3:
> https://github.com/uswitch/bifrost
>
> Obviously tha
+1 for detailed examination of metrics. You can see the main metrics here:
https://kafka.apache.org/documentation.html#monitoring
Jconsole is very helpful for looking quickly at what is going on.
Cheers, Robert
On Sun, Jun 15, 2014 at 7:49 AM, pushkar priyadarshi <
priyadarshi.push...@gmail.c
Hi Riyaz,
There are a number of reasons that you may be getting low performance.
Here are some questions to get started:
1. How big are your messages? To meet your throughput requirement you need
a minimum of 10K messages per second continuously. You specified a
replication factor of 3 so at a
;
> Thanks,
> Neha
>
>
> On Mon, Jun 9, 2014 at 8:55 AM, Robert Hodges
> wrote:
>
> > Hi Gouzhang,
> >
> > Thanks for the response. Answers interpolated below.
> >
> > Cheers, Robert
> >
> > On Mon, Jun 9, 2014 at 8:15 AM, Guozhang Wan
ges but keep all of them (for
example) in a single partition.
If I have time next weekend I might try to create an example of this to see
what problems pop up.
Cheers, Robert
> Thanks,
> Guozhang
>
>
> On Sat, Jun 7, 2014 at 11:30 PM, Robert Hodges
> wrote:
>
> &g
Hi Bogdan,
It sounds as if you could implement a form of signaling between the
consumers using a distributed barrier. This can be implemented using Kafka
topics. For example you could create a control thread that posts the
current high-water mark for all consumers into a special topic, which give
rotocol that
works on Kafka.
Cheers, Robert Hodges
On Thu, Jun 5, 2014 at 7:48 AM, Jun Rao wrote:
> It sounds like that you want to write to a data store and a data pipe
> atomically. Since both the data store and the data pipe that you want to
> use are highly available, the only
ort.
>
> Is there any way to avoid/prevent shutdown of all brokers ?
>
> Thanks for you kind response!
>
>
>
>
> On Mon, Jun 2, 2014 at 10:37 PM, Robert Hodges
> wrote:
>
> > Hi Rafeeq,
> >
> > With respect to question #2, are you stopping brokers usi
Hi Rafeeq,
With respect to question #2, are you stopping brokers using
kafka-server-stop.sh and are they all on a single host? If so, the script
finds anything that looks like a Kafka server and should knock out all the
brokers at once. If your cluster runs across multiple hosts something else
i
Hi Chengwei,
What do you see in the Kafka server logs? This may help you diagnose the
failure.
Cheers, Robert
On Thu, May 29, 2014 at 2:38 AM, Chengwei Yang
wrote:
> Hi List,
>
> I'm new to kafka, feel sorry if this has been asked, I didn't figure out
> my answer by googling, so asked here.
11 matches
Mail list logo
