Hey Gwen,
Receive and Send are meant to encapsulate the idea of sending or receiving
bytes. They are supposed to represent a send or receive that is in
progress. The purpose of these interfaces is to allow sends and receives
using transferTo when we need to do that on the server side, this requires
abstracting over the sending or receiving so you can't just work with
ByteBuffers directly (we will want to be able to have an implementation of
that class that does a FileSend or something like that).
The hierarchy was
Send/Receive - Interface for sending or receiving
ByteBuffer{Send/Receive} - Simple byte array for sending or receiving
Network{Send/Receive} - Size delimited byte array
The Network* version is called that since Kafka's transport protocol is
size-delimited byte arrays and those classes encapsulate the fact that when
reading or writing you will generally have both a size and a message.
The reify method was meant to get the backing byte buffers if they existed.
The NetworkReceive has a payload method because with receives you create
the buffers so you know there are exactly two. I agree that this is not
very symmetric.
The reality is although I was trying to accommodate the future server-side
usage, I'm not sure how well I accounted for that. I suspect we will need
to improve the interfaces as we convert the server code.
For example one thing I see is that Selector.send takes a NetworkSend but I
think it should actually take a Send to enable the FetchRequest transferTo
optimization.
Here are the constraints I am aware of:
1. It would be good to support sending messages that are composed of an
array of ByteBuffers so that our ProduceRequest can directly reuse the
backing ByteBuffer for the MemoryRecords instances it has. This is not done
yet.
2. On the server side we need to be able to substitute in a special Send
implementation for the fetch request transferTo stuff.
3. Ideally the Receive and Send interfaces would be symmetric.
-Jay
On Mon, Apr 6, 2015 at 2:55 PM, Gwen Shapira <gshap...@cloudera.com> wrote:
> Hi,
>
> I started work on converting core to use common network code and I ran into
> two questions that hopefully someone can help me with:
>
> 1. BlockingChannel - Looks like the inter-broker communication is mostly
> through blocking channel. There's no client equivalent (i.e client code is
> all non-blocking). I was planning to leave this as is, but let me know if
> the intention was to make all this communication non-blocking.
>
> 2. Receive & ByteBufferReceive - It looks like ByteBufferReceive is not
> used anywhere. What was it planned for?
> NetworkReceive implements payload() method that returns the backing buffer.
> It is used all over the place (to create requests, responses, etc). Because
> ByteBufferReceive doesn't implement payload() (and its not clear how it can
> implement it), this means that payload() is not part of the Receive
> interface, although I'm pretty sure it should be there.
> Some explanation about the Receive interface and what is ByteBufferReceive
> for will help.
>
> Gwen
>
