Hello,
Specified below is my code base where I am attempting to marshall a complex
type and receiving the follow error.
Am I missing anything here?
Sending Encoded Messages ..
[error] (run-main) java.lang.ClassCastException: java.lang.String cannot be
cast to com.test.groups.MemberRecord
java.la
I noticed that ProducerData has been removed in 0.8 branch
If I'd wish to send a complex message type (encoded message) ,
how would I do it in 0.8?
In 0.7.X (a snippet from Neha's example)
val producer = new Producer[Message, MemberRecord](config);
// send a single message
val mes
filed: https://issues.apache.org/jira/browse/KAFKA-978
I can try to take a look, can you point me to where/how you generate the
pom file?
On Tue, Jul 16, 2013 at 7:26 AM, Joe Stein wrote:
> Hi Jason, there are two different answers in regards to Scala version in my
> opinion as there are clea
thanks Joel for looking into it. I will try to reproduce it. I don't think
the second zookeeper is needed because i ran into it the first time just by
shutting down the topic leaders.
Cal
On Tue, Jul 16, 2013 at 2:38 AM, Joel Koshy wrote:
> Hey Calvin,
>
> I apologize for not being able to get
Joel,
For #1, I meant multiple consumer configs. Please excuse me for the typo.
For #2, turns out i started the mirror before i brought up the kafka
cluster, hence all the messages failed to send to the remote cluster.
thanks,
Cal
On Tue, Jul 16, 2013 at 3:16 AM, Joel Koshy wrote:
> Calv
1. I don't think this is something kafka can do. We never rename
directories once we create them. I'm a little at a loss as to how/why the
os would do it either, though.
2. Can you check if it ran recovery or not (you would see a bunch of
messages about recovering each topic). This process goes thr
Thanks for the response Jay,
1. Do you think the corrupt partition directory names is happening at the
kafka level, or the file system level? If it's at the file system level,
perhaps we can investigate either changing FS options or selecting a
different filesystem for the log files. Is there a re
Interesting. Yes it will respect whatever setting it is given for new
segments created from that point on.
-Jay
On Tue, Jul 16, 2013 at 11:23 AM, Jason Rosenberg wrote:
> Ok,
>
> An update on this. It seems we are using XFS, which is available in newer
> versions of Centos. It definitely doe
1. The corrupt directory names is not something we can really handle. The
names have to work with zookeeper so if weird characters get inserted I
think the best we can do is give an error. I believe we have fixed the 0
byte file problem in 0.8.
2. The assumption for log recovery is that data that h
Yes, I think these are two separate issues.
F.
On 7/16/13 11:32 AM, "Joel Koshy" wrote:
>From a user's perspective, ConsumerRebalanceException is a bit cryptic
>-I think the other thread was to provide a more informative message
>and also be able to recover when a broker does come up (fixed in
Hi everyone,
Last week, one of our production Kafka 0.7.1 servers had a hardware failure
that resulted in an unclean restart. When the server came back up 5 minutes
later, there were two topic corruption problems that we had to handle to
get the pipeline working again.
1. The kafka log directory
Hmm
This issue continues to emerge occasionally, albeit less often than in the
past.
If I hit it after several days or months of uptime, that would be okay, but
today I have hit it twice within the first hour of 2 separate load tests.
I've cleaned up the code in my application to ensure I
>From a user's perspective, ConsumerRebalanceException is a bit cryptic
-I think the other thread was to provide a more informative message
and also be able to recover when a broker does come up (fixed in
KAFKA-969).
Thanks,
Joel
On Tue, Jul 16, 2013 at 11:04 AM, Vaibhav Puranik wrote:
> Thank
Ok,
An update on this. It seems we are using XFS, which is available in newer
versions of Centos. It definitely does pre-allocate space as a file grows,
see:
http://serverfault.com/questions/406069/why-are-my-xfs-filesystems-suddenly-consuming-more-space-and-full-of-sparse-file
Apparently it's
Thank you Joel.
In a different but related thread, somebody is asking to rename the
exception as NoBrokerAvailableExcption. But given the description above,
the exception seems to be named appropriately.
Regards,
Vaibhav
On Tue, Jul 16, 2013 at 12:05 AM, Joel Koshy wrote:
> Yes - rebalance =>
Hello All
I have been attempting to build an end-to-end example of producer with
multiple consumers using scala.
I managed to build a producer, but when I attempt to build a consumer the
following statement causes the scala REPL to block/hang
val consumerMap = consumer.createMessageStreams(topicC
Hi Jason, there are two different answers in regards to Scala version in my
opinion as there are clear distinctions between broker and
producers/consumers.
I don't know what the mix is but experience with brokers is only building
and running in production 2.8.0 and I suspect this is the same for o
Calvin,
For (1) can you clarify what you mean by "multiplied" consumer configs?
For (2) the mirror-maker actually uses the high level consumer.
Thanks,
Joel
On Sun, Jul 14, 2013 at 7:43 AM, Calvin Lei wrote:
> Hi all,
> I have two questions regarding setting up mirror maker for our cross
There has to be some live/stray consumer somewhere committing that
offset given the information shown in mtime. Can you check under
/consumers/g1/ids to see what if there are any registered consumers?
On Mon, Jul 15, 2013 at 12:27 PM, arathi maddula
wrote:
> Hi,
> I use a Java high level consumer
Yes - rebalance => consumers trying to coordinate through ZK.
Rebalances can happen when one or more of the following happen:
- a consumed topic partition appears or disappears - i.e., if a broker
comes or goes.
- a consumer instance in the group comes or goes
"goes" could also be triggered by sess
20 matches
Mail list logo
