Looking at the SimpleConsumer, we need to leave in place:

ConsumerMetadataResponse
FetchRequest / FetchResponse
OffsetFetchRequest / OffsetFetchResponse
OffsetCommitRequest / OffsetCommitResponse
OffsetRequest / OffsetResponse
TopicMetadata
TopicMetadataRequest / TopicMetadataResponse

Specifically, TopicMetadata request and response will remain
duplicated after KAFKA-1927.
So I am no longer sure why this is a blocker for KIP-4.

Gwen



On Wed, Mar 18, 2015 at 11:11 AM, Gwen Shapira <gshap...@cloudera.com> wrote:
> Hi Jun,
>
> I was taking a slightly different approach. Let me know if it makes
> sense to you:
>
> 1. Get the bytes from network (kinda unavoidable...)
> 2. Modify RequestChannel.Request to contain header and body (instead
> of a single object)
> 3. Create the head and body from bytes as follow:
>     val header: RequestHeader = RequestHeader.parse(buffer)
>     val apiKey: Int = header.apiKey
>     val body: Struct =
> ProtoUtils.currentRequestSchema(apiKey).read(buffer).asInstanceOf[Struct]
> 4. KafkaAPIs will continue getting RequestChannel.Request, but will
> now have access to body and header separately.
>
> I agree that I need a Request/Response objects that contain only the
> body for all requests objects.
> I'm thinking of implementing them in o.a.k.Common.Requests in Java for
> consistency.
>
> When we are discussing the requests/responses used in SimpleConsumer,
> we mean everything used in javaapi, right?
>
> Gwen
>
>
>
> On Wed, Mar 18, 2015 at 9:55 AM, Jun Rao <j...@confluent.io> wrote:
>> Hi, Gwen,
>>
>> I was thinking that we will be doing the following in KAFKA-1927.
>>
>> 1. Get the bytes from network.
>> 2. Use a new generic approach to convert bytes into request objects.
>> 2.1 Read the fixed request header (using the util in client).
>> 2.2 Based on the request id in the header, deserialize the rest of the
>> bytes into a request specific object (using the new java objects).
>> 3. We will then be passing a header and an AbstractRequestResponse to
>> KafkaApis.
>>
>> In order to do that, we will need to create similar request/response
>> objects for internal requests such as StopReplica, LeaderAndIsr,
>> UpdateMetadata, ControlledShutdown. Not sure whether they should be written
>> in java or scala, but perhaps they should be only in the core project.
>>
>> Also note, there are some scala requests/responses used directly in
>> SimpleConsumer. Since that's our public api, we can't remove those scala
>> objects until the old consumer is phased out. We can remove the rest of the
>> scala request objects.
>>
>> Thanks,
>>
>> Jun
>>
>>
>> On Tue, Mar 17, 2015 at 6:08 PM, Gwen Shapira <gshap...@cloudera.com> wrote:
>>
>>> Hi,
>>>
>>> I'm starting this thread for the various questions I run into while
>>> refactoring the server to use client requests and responses.
>>>
>>> Help is appreciated :)
>>>
>>> First question: LEADER_AND_ISR request and STOP_REPLICA request are
>>> unimplemented in the client.
>>>
>>> Do we want to implement them as part of this refactoring?
>>> Or should we continue using the scala implementation for those?
>>>
>>> Gwen
>>>

Reply via email to