Please don’t have Cygwin on the path while running Hadoop. Its not needed and might actually interfere in case there is some Unix functionality that is still missing from the Windows utility that is compiled for Hadoop. Only sh and tar are required for building and unit testing. Win32 versions of these are available freely.
All sh scripts for Hadoop should have cmd equivalents. Please file a jira if you find one that’s missing. Usually porting them is simple and patches are welcome! You should be able to follow the same flow to install and bring up the cluster as you do with Unix. You will find that a winutils.exe file is built in your hadoop build and this is required to be able to run Hadoop on Windows. It translates Unix like file permissions and provides other Win32 native functionality for the Java code. IIRC this exe needs to be available in the %HADOOP_HOME%/bin directory. Bikas -----Original Message----- From: Andrey Klochkov [mailto:akloch...@griddynamics.com] Sent: Wednesday, September 25, 2013 11:30 AM To: common-dev Subject: Re: windows support in trunk? I don't have problems with building. I have issues with running it. Usually I do this with the following steps: 1. build using "dist" maven profile to generate the distro under hadoop-dist/target 2. set HADOOP_HOME pointing to hadoop-dist/target/hadoop-x.x.x-SNAPSHOT 3. fix configs under $HADOOP_HOME/etc 4. use start-dfs.sh, start-yarh.sh and mr-jobhistory-daemon.sh to start the daemons The "dist" profile doesn't work on Windows without Cygwin as it depends on shell and tar utilities. How can I overcome this? Actually I installed Cygwin and managed to do step #1. Then I succeed with steps 2-4 and started all daemons using *cmd scripts. The only issue I have is with the jobhistory daemon, as there's no mr-jobhistory-daemon.cmd script. Is there an alternative way to start it? On Wed, Sep 25, 2013 at 11:12 AM, Arpit Agarwal <aagar...@hortonworks.com>wrote: > Andrey, thanks for testing your patch on Windows. > > cygwin is no longer required. Instead, please install the free Windows > SDK and build from an SDK command prompt. This is documented in > BUILDING.txt. > Please feel free to email the dev list if you get stuck at any step. > > > On Tue, Sep 24, 2013 at 10:31 PM, Bikas Saha <bi...@hortonworks.com> > wrote: > > > BUILDING.txt in the source code has a section for building on Windows. > > > > Bikas > > > > -----Original Message----- > > From: Andrey Klochkov [mailto:akloch...@griddynamics.com] > > Sent: Tuesday, September 24, 2013 5:11 PM > > To: common-dev > > Subject: Re: windows support in trunk? > > > > Great, thanks! > > > > One more question on Windows support: how do I create a distribution > > and run it? It's for dev purposes - testing a patch to work on > > Windows, so I > can't > > really use HDP. On Linux/OSX I just do a build with -Pdist and then > > just use startXXX scripts from hadoop-project-dist/target to run all > > the daemons. > > Are > > there any docs on how to do it on Windows? E.g. do I need cygwin? I > > can build sources without cygwin, but hadoop-project-dist requires > > shell executable to build the tarball. > > > > > > On Tue, Sep 24, 2013 at 3:31 PM, Arun C Murthy <a...@hortonworks.com> > > wrote: > > > > > > > > On Sep 24, 2013, at 3:24 PM, Andrey Klochkov > > > <akloch...@griddynamics.com> > > > wrote: > > > > > > > Is Windows support in trunk currently? Or should I still use > > > > trunk-win to experiment with Hadoop on Windows? I've seen number > > > > of windows related patches going into trunk that's why I'm asking. > Thanks! > > > > > > > > I know I just need to ask Chris Nauroth, but sending here just > > > > in case > > > it's > > > > useful for others. > > > > > > *smile* > > > > > > Yes, FTR, Windows support is in trunk, branch-2 and branch-2.1. > > > > > > Arun > > > > > > > > > > > > > > -- > > > > Andrey Klochkov > > > > > > -- > > > Arun C. Murthy > > > Hortonworks Inc. > > > http://hortonworks.com/ > > > > > > > > > > > > -- > > > CONFIDENTIALITY NOTICE > > > NOTICE: This message is intended for the use of the individual or > > > entity to which it is addressed and may contain information that > > > is confidential, privileged and exempt from disclosure under > > > applicable law. If the reader of this message is not the intended > > > recipient, you are hereby notified that any printing, copying, > > > dissemination, distribution, disclosure or forwarding of this > > > communication is strictly prohibited. If you have received this > > > communication in error, please contact the sender immediately and > > > delete it from your system. > > > Thank You. > > > > > > > > > > > -- > > Andrey Klochkov > > Grid Dynamics > > Skype: aklochkov_gd > > www.griddynamics.com > > akloch...@griddynamics.com > > > > -- > > CONFIDENTIALITY NOTICE > > NOTICE: This message is intended for the use of the individual or > > entity > to > > which it is addressed and may contain information that is > > confidential, privileged and exempt from disclosure under applicable > > law. If the reader of this message is not the intended recipient, > > you are hereby notified > that > > any printing, copying, dissemination, distribution, disclosure or > > forwarding of this communication is strictly prohibited. If you have > > received this communication in error, please contact the sender > immediately > > and delete it from your system. Thank You. > > > > -- > CONFIDENTIALITY NOTICE > NOTICE: This message is intended for the use of the individual or > entity to which it is addressed and may contain information that is > confidential, privileged and exempt from disclosure under applicable > law. If the reader of this message is not the intended recipient, you > are hereby notified that any printing, copying, dissemination, > distribution, disclosure or forwarding of this communication is > strictly prohibited. If you have received this communication in error, > please contact the sender immediately and delete it from your system. > Thank You. > -- Andrey Klochkov Grid Dynamics Skype: aklochkov_gd www.griddynamics.com akloch...@griddynamics.com -- CONFIDENTIALITY NOTICE NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You.
