Re: Block Transfer Service encryption support

So it appears the tests fail because of an SSLHandshakeException. Tracing the failure I see: 3,0001,Using SSLEngineImpl.\0A 3,0001,\0AIs initial handshake: true\0A 3,0001,Ignoring unsupported cipher suite: SSL_RSA_WITH_DES_CBC_SHA for TLSv1.2\0A 3,0001,No available cipher suite for TLSv1.2\0A 3,0

Re: Block Transfer Service encryption support

Nb. I did notice some test failures when I ran a quick test on the pull request ( not sure if it is related - I haven't looked in any detail at the cause ). Failed tests: SslChunkFetchIntegrationSuite>ChunkFetchIntegrationSuite.fetchBothChunks:201 expected:<[]> but was:<[0, 1]> SslChunkFetchI

Re: Block Transfer Service encryption support

I created a pull request for issue SPARK-6373 Any feedback would be appreciated... https://github.com/apache/spark/pull/9416 Jeff -- View this message in context: http://apache-spark-developers-list.1001551.n3.nabble.com/Block-Transfe

Re: Block Transfer Service encryption support

Still looking for feedback... I opened the ticket as a minor, thinking of changing it to a major? Anyone object. I did see that a related ticket (https://issues.apache.org/jira/browse/SPARK-6229) is marked as a major... Cheers! Jeff -- View this message in context: http://apache-spark-develop

Re: Block Transfer Service encryption support

Hey Aaron, That is what I do, except I add the Netty SslHandler in the TransportServer and the TransportClientFactory I do this because the Server pipeline is a bit different as I have to add a Netty ChunkedWriteHandler... Again, this is a "rough" prototype, just to get something working... Ch

Re: Block Transfer Service encryption support

Out of curiosity, why could we not use Netty's SslHandler injected into the TransportContext pipeline? On Mon, Mar 16, 2015 at 7:56 PM, turp1twin wrote: > Hey Patrick, > > Sorry for the delay, I was at Elastic{ON} last week and well, my day job > has > been keeping me busy... I went ahead and op

Re: Block Transfer Service encryption support

Hey Patrick, Sorry for the delay, I was at Elastic{ON} last week and well, my day job has been keeping me busy... I went ahead and opened a Jira feature request, https://issues.apache.org/jira/browse/SPARK-6373. In it I reference a commit I made in my fork which is a "rough" implementation, defini

Re: Block Transfer Service encryption support

Hey Patrick, Yes, I will open a Jira tomorrow... For now my implementation is a basic SSL implementation for the TransportServer and TransportClient.. I will type up the design and at the same time look at the Hadoop impl for possible improvements... Cheers! Jeff On Sun, Mar 8, 2015 at 5:51 PM,

Re: Block Transfer Service encryption support

I think that yes, longer term we want to have encryption of all communicated data. However Jeff, can you open a JIRA to discuss the design before opening a pull request (it's fine to link to a WIP branch if you'd like)? I'd like to better understand the performance and operational complexity of usi

Re: Block Transfer Service encryption support

I have already written most of the code, just finishing up the unit tests right now... Jeff On Sun, Mar 8, 2015 at 5:39 PM, Andrew Ash wrote: > I'm interested in seeing this data transfer occurring over encrypted > communication channels as well. Many customers require that all network > tran

Re: Block Transfer Service encryption support

I'm interested in seeing this data transfer occurring over encrypted communication channels as well. Many customers require that all network transfer occur encrypted to prevent the "soft underbelly" that's often found inside a corporate network. On Fri, Mar 6, 2015 at 4:20 PM, turp1twin wrote: