Re: [Copycat] How will copycat serialize its metadata

Yeah, I agree that if we have the ser/de we can do anything :) I'd actually feel more comfortable if the users *have* to go through our APIs to get to the metadata (which again, is kind of internal to Copycat). If they start writing their own code that depends on this data, who knows what we may a

Re: [DISCUSS] Client-side Assignment for New Consumer

Neha, Ewen and Jason, Maybe I am over concerning and I agree that it does depend on the metadata change frequency. As Neha said, a few tests will be helpful. We can see how it goes. What worries me is that in LinkedIn we are in the progress of running Kafka as a service. That means user will have

Re: [Copycat] How will copycat serialize its metadata

Ewen, I meant we use format X to store offsets, whether you serialize your data with Y or Z and we don't expose it as something that can be configured. As far as the serialization format goes, I was suggesting just going with simple base64 encoded strings (maybe there is a reason you are saying th

Re: Review Request 33620: Patch for KAFKA-1690

> On Aug. 3, 2015, 4:50 p.m., Jun Rao wrote: > > clients/src/main/java/org/apache/kafka/common/network/SSLTransportLayer.java, > > lines 262-269 > > > > > > Could we transition from NEED_WRAP to NOT_HANDSHAKING dir

Re: Review Request 33620: Patch for KAFKA-1690

> On July 27, 2015, 1:32 p.m., Ismael Juma wrote: > > clients/src/main/java/org/apache/kafka/common/network/DefaultAuthenticator.java, > > line 34 > > > > > > Why not create the principal at this point instead of in

Re: Review Request 33620: Patch for KAFKA-1690

> On July 27, 2015, 1:32 p.m., Ismael Juma wrote: > > clients/src/main/java/org/apache/kafka/common/config/SSLConfigs.java, line > > 29 > > > > > > SSL is deprecated > > (https://ma.ttias.be/rfc-7568-ssl-3-0-is-now

Re: Review Request 33620: Patch for KAFKA-1690

> On July 29, 2015, 1:15 a.m., Dong Lin wrote: > > clients/src/test/java/org/apache/kafka/common/network/SSLFactoryTest.java, > > line 52 > > > > > > I am not sure about this. It is probably a stupid question. Do yo

Re: Review Request 33620: Patch for KAFKA-1690

> On Aug. 3, 2015, 4:50 p.m., Jun Rao wrote: > > clients/src/main/java/org/apache/kafka/common/network/SSLTransportLayer.java, > > lines 430-433 > > > > > > Is this needed? If we need to expand appReadBuffer, netRe

Re: Review Request 33620: Patch for KAFKA-1690

> On Aug. 3, 2015, 4:50 p.m., Jun Rao wrote: > > core/src/test/scala/unit/kafka/network/SocketServerTest.scala, line 207 > > > > > > Is this test needed given the tests we have on EchoServer? > > > > Also,

Re: Review Request 33620: Patch for KAFKA-1690

> On Aug. 3, 2015, 4:50 p.m., Jun Rao wrote: > > clients/src/main/java/org/apache/kafka/common/network/Selector.java, line > > 535 > > > > > > Not sure why we need to check hasSend. It's possible for a channel to

Re: Review Request 33620: Patch for KAFKA-1690

> On Aug. 3, 2015, 4:50 p.m., Jun Rao wrote: > > clients/src/main/java/org/apache/kafka/common/network/Selector.java, lines > > 250-251 > > > > > > It seems that we will need to further check whether those channels

Re: Review Request 33620: Patch for KAFKA-1690

> On Aug. 3, 2015, 4:50 p.m., Jun Rao wrote: > > clients/src/main/java/org/apache/kafka/common/network/SSLTransportLayer.java, > > line 417 > > > > > > It's still not very clear to me how renegotiation can be suppo

Re: [DISCUSS] Client-side Assignment for New Consumer

Becket, This is a clever approach for to ensure that only one thing communicates the metadata so even if it is stale, the entire group has the same view. However, the big assumption this makes is that the coordinator is that one process that has the ability to know the metadata for group members,

Re: Review Request 33620: Patch for KAFKA-1690

> On Aug. 3, 2015, 4:50 p.m., Jun Rao wrote: > > clients/src/main/java/org/apache/kafka/common/network/SSLTransportLayer.java, > > lines 306-320 > > > > > > It seems that the logic here can be simpler. In handshake

Re: [DISCUSS] Client-side Assignment for New Consumer

Agreed that talking about actual numbers would be helpful here. Only 2 things affect this metadata: list of topics and number of partitions. I don't think either of those can change *that* frequently to be relevant to our 3s default heartbeat and 30s default session timeout, or even the 5 minute m

Re: [DISCUSS] Client-side Assignment for New Consumer

Hey Jiangjie, I was thinking about the same problem. When metadata is changing frequently, the clients may not be able to ever find agreement on the current state. The server doesn't have this problem, as you say, because it can just take a snapshot and send that to the clients. Adding a dampening

[jira] [Commented] (KAFKA-2367) Add Copycat runtime data API

[ https://issues.apache.org/jira/browse/KAFKA-2367?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14698296#comment-14698296 ] Martin Kleppmann commented on KAFKA-2367: - Just looked at this in the context of h

[jira] [Created] (KAFKA-2436) log.retention.hours should be honored by LogManager

Dong Lin created KAFKA-2436: --- Summary: log.retention.hours should be honored by LogManager Key: KAFKA-2436 URL: https://issues.apache.org/jira/browse/KAFKA-2436 Project: Kafka Issue Type: Bug

[jira] [Updated] (KAFKA-2072) Add StopReplica request/response to o.a.k.common.requests and replace the usage in core module

[ https://issues.apache.org/jira/browse/KAFKA-2072?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Jacot updated KAFKA-2072: --- Status: Patch Available (was: Open) > Add StopReplica request/response to o.a.k.common.requests and r

[jira] [Commented] (KAFKA-2072) Add StopReplica request/response to o.a.k.common.requests and replace the usage in core module

[ https://issues.apache.org/jira/browse/KAFKA-2072?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14698177#comment-14698177 ] ASF GitHub Bot commented on KAFKA-2072: --- GitHub user dajac opened a pull request:

[GitHub] kafka pull request: KAFKA-2072 [WIP]: Add StopReplica request/resp...

GitHub user dajac opened a pull request: https://github.com/apache/kafka/pull/141 KAFKA-2072 [WIP]: Add StopReplica request/response to o.a.k.common.requests and replace the usage in core module Migration is done but this PR will need to be rebased on #110. I have copied some code