Re: spark support on windows

2017-01-16 Thread Steve Loughran
On 16 Jan 2017, at 11:06, Hyukjin Kwon mailto:gurwls...@gmail.com>> wrote: Hi, I just looked through Jacek's page and I believe that is the correct way. That seems to be a Hadoop library specific issue[1]. Up to my knowledge, winutils and the binaries in the private repo are built by a Hadoo

Re: spark support on windows

2017-01-16 Thread Steve Loughran
On 16 Jan 2017, at 10:35, assaf.mendelson mailto:assaf.mendel...@rsa.com>> wrote: Hi, In the documentation it says spark is supported on windows. The problem, however, is that the documentation description on windows is lacking. There are sources (such as https://jaceklaskowski.gitbooks.io/mas

Re: spark support on windows

2017-01-16 Thread Hyukjin Kwon
Hi, I just looked through Jacek's page and I believe that is the correct way. That seems to be a Hadoop library specific issue[1]. Up to my knowledge, winutils and the binaries in the private repo are built by a Hadoop PMC member on a dedicated Windows VM which I believe are pretty trustable. Th