Harsha,
I am using the code from
https://github.com/harshach/kafka/tree/KAFKA-1690-V1 with the latest commit
on 25 July which I think corresponds to the latest patch in KAFKA-1690. Is
that correct?
I have run SSLConsumerTest, SSLProducerSendTest and SSLSelectorTest several
times and the only one that fails to complete
is SSLSelectorTest.testRenegotiate(). I added some trace to the code and
the sequence in the failed run is:
1. Handshake starts
2. Handshake finishes with handshakeStatus=FINISHED
appReadBuffer=java.nio.DirectByteBuffer[pos=0 lim=16916 cap=16916] *(there
is no data in appReadBuffer)*
3. Several reads and writes occur
4. read() results in handshake() due to renegotiation.
appReadBuffer=java.nio.DirectByteBuffer[pos=0 lim=16916 cap=16916] *(no
data in appReadBuffer)*
5. Handshake status changes from NEED_TASK to NEED_WRAP to NEED_UNWRAP
6. Unwrap call during handshake results in data in appReadBuffer:
handshakeStatus=NEED_UNWRAP
appReadBuffer=java.nio.DirectByteBuffer[pos=100 lim=16916 cap=16916] *(data
is now in appReadBuffer)*
7. Handshake status changes to NEED_UNWRAP followed by FINISHED. Final
call to handshake() returns with handshakeStatus=NOT_HANDSHAKING
appReadBuffer=java.nio.DirectByteBuffer[pos=100 lim=16916 cap=16916] *(data
is still in appReadBuffer)*
Test continues to call selector.poll() forever. But no more data arrives on
the channel since all the data in the test has already arrived, and hence
SSLTransportLayer.read() is never invoked on the channel and the data that
was unwrapped earlier simply remains in appReadBuffer. This is not a
scenario that occurs in the other tests where some data always arrives on
the network after the handshake() resulting in SSLTransportLayer.read()
being invoked.
I can recreate this problem quite easily with IBM JDK (it hangs around one
in ten runs), so if you require more trace, please let me know.
Thank you,
Rajini
On Mon, Aug 10, 2015 at 5:29 PM, Sriharsha Chintalapani <
harsh...@fastmail.fm> wrote:
> Thanks for testing out Rajini. I did ran that test in a loop and it never
> hanged for me. I am hoping you are using the latest patch since the data
> left over issue is addressed in latest patch.
> Also if thats an issue SSLConsumerTest and SSLProducerTest will hang too.
> Did you notice those are having any issues?
>
> I am addressing left over reviews will be sending a new patch in a day or
> two.
>
> Thanks,
> Harsha
>
>
> On August 10, 2015 at 3:35:34 AM, Rajini Sivaram (
> rajinisiva...@googlemail.com) wrote:
>
> I was running a Kafka cluster with the latest SSL patch over the weekend
> with IBM JRE, and it has been running fine without any issues. There was
> light load on the cluster throughout and intermittent heavy load, all
> using
> SSL clients.
>
> However I am seeing an intermittent unit test hang in
> org.apache.kafka.common.network.SSLSelectorTest.testRenegotiate().
> I am not sure if that is related to the IBM JRE I am using for the build.
> It looks like data left in appReadBuffer after a handshake may not get
> processed if no more data arrives on the network, causing the test to loop
> forever. It will be good if this can be fixed (or at least the test
> commented out) before the code is committed to avoid breaking the build.
>
> Even though there are quite a few outstanding review comments, I do agree
> that being such a big patch, it will be good to commit the code soon and
> work on minor issues afterwards.
>
>
>
>
> On Mon, Aug 10, 2015 at 12:19 AM, Jun Rao (JIRA) <j...@apache.org> wrote:
>
> >
> > [
> >
> https://issues.apache.org/jira/browse/KAFKA-1690?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14679397#comment-14679397
> > ]
> >
> > Jun Rao commented on KAFKA-1690:
> > --------------------------------
> >
> > [~rsivaram], this jira is getting pretty close to be committed. Could
> you
> > test this out on an IBM jvm to see if there is any issue especially with
> > respect to the usage of the sendfile api?
> >
> >
> > > new java producer needs ssl support as a client
> > > -----------------------------------------------
> > >
> > > Key: KAFKA-1690
> > > URL: https://issues.apache.org/jira/browse/KAFKA-1690
> > > Project: Kafka
> > > Issue Type: Sub-task
> > > Reporter: Joe Stein
> > > Assignee: Sriharsha Chintalapani
> > > Fix For: 0.8.3
> > >
> > > Attachments: KAFKA-1690.patch, KAFKA-1690.patch,
> > KAFKA-1690_2015-05-10_23:20:30.patch,
> KAFKA-1690_2015-05-10_23:31:42.patch,
> > KAFKA-1690_2015-05-11_16:09:36.patch,
> KAFKA-1690_2015-05-12_16:20:08.patch,
> > KAFKA-1690_2015-05-15_07:18:21.patch,
> KAFKA-1690_2015-05-20_14:54:35.patch,
> > KAFKA-1690_2015-05-21_10:37:08.patch,
> KAFKA-1690_2015-06-03_18:52:29.patch,
> > KAFKA-1690_2015-06-23_13:18:20.patch,
> KAFKA-1690_2015-07-20_06:10:42.patch,
> > KAFKA-1690_2015-07-20_11:59:57.patch,
> KAFKA-1690_2015-07-25_12:10:55.patch
> > >
> > >
> >
> >
> >
> >
> > --
> > This message was sent by Atlassian JIRA
> > (v6.3.4#6332)
> >
>
>
