also change the permissions of these directories to 777. Regards, Mohammad Tariq
On Wed, Jun 6, 2012 at 11:54 PM, Mohammad Tariq <donta...@gmail.com> wrote: > create a directory "/home/username/hdfs" (or at some place of your > choice)..inside this hdfs directory create three sub directories - > name, data, and temp, then follow these steps : > > add following properties in your core-site.xml - > > <property> > <name>fs.default.name</name> > <value>hdfs://localhost:9000/</value> > </property> > > <property> > <name>hadoop.tmp.dir</name> > <value>/home/mohammad/hdfs/temp</value> > </property> > > then add following two properties in your hdfs-site.xml - > > <property> > <name>dfs.replication</name> > <value>1</value> > </property> > > <property> > <name>dfs.name.dir</name> > <value>/home/mohammad/hdfs/name</value> > </property> > > <property> > <name>dfs.data.dir</name> > <value>/home/mohammad/hdfs/data</value> > </property> > > finally add this property in your mapred-site.xml - > > <property> > <name>mapred.job.tracker</name> > <value>hdfs://localhost:9001</value> > </property> > > NOTE: you can give any name to these directories of your choice, just > keep in mind you have to give same names as values of > above specified properties in your configuration files. > (give full path of these directories, not just the name of the > directory) > > After this follow the steps provided in the previous reply. > > Regards, > Mohammad Tariq > > > On Wed, Jun 6, 2012 at 11:42 PM, Babak Bastan <babak...@gmail.com> wrote: >> thank's Mohammad >> >> with this command: >> >> babak@ubuntu:~/Downloads/hadoop/bin$ hadoop namenode -format >> >> this is my output: >> >> 12/06/06 20:05:20 INFO namenode.NameNode: STARTUP_MSG: >> /************************************************************ >> STARTUP_MSG: Starting NameNode >> STARTUP_MSG: host = ubuntu/127.0.1.1 >> STARTUP_MSG: args = [-format] >> STARTUP_MSG: version = 0.20.2 >> STARTUP_MSG: build = >> https://svn.apache.org/repos/asf/hadoop/common/branches/branch-0.20 -r >> 911707; compiled by 'chrisdo' on Fri Feb 19 08:07:34 UTC 2010 >> ************************************************************/ >> 12/06/06 20:05:20 INFO namenode.FSNamesystem: >> fsOwner=babak,babak,adm,dialout,cdrom,plugdev,lpadmin,admin,sambashare >> 12/06/06 20:05:20 INFO namenode.FSNamesystem: supergroup=supergroup >> 12/06/06 20:05:20 INFO namenode.FSNamesystem: isPermissionEnabled=true >> 12/06/06 20:05:20 INFO common.Storage: Image file of size 95 saved in 0 >> seconds. >> 12/06/06 20:05:20 INFO common.Storage: Storage directory >> /tmp/hadoop-babak/dfs/name has been successfully formatted. >> 12/06/06 20:05:20 INFO namenode.NameNode: SHUTDOWN_MSG: >> /************************************************************ >> SHUTDOWN_MSG: Shutting down NameNode at ubuntu/127.0.1.1 >> ************************************************************/ >> >> by this command: >> >> babak@ubuntu:~/Downloads/hadoop/bin$ start-dfs.sh >> >> this is the out put >> >> mkdir: kann Verzeichnis „/home/babak/Downloads/hadoop/bin/../logs“ nicht >> anlegen: Keine Berechtigung >> >> this out put(it's in german and it means no right to make this folder) >> >> >> On Wed, Jun 6, 2012 at 7:59 PM, Mohammad Tariq <donta...@gmail.com> wrote: >>> >>> once we are done with the configuration, we need to format the file >>> system..use this command to do that- >>> bin/hadoop namenode -format >>> >>> after this, hadoop daemon processes should be started using following >>> commands - >>> bin/start-dfs.sh (it'll start NN & DN) >>> bin/start-mapred.sh (it'll start JT & TT) >>> >>> after this use jps to check if everything is alright or point your >>> browser to localhost:50070..if you further find any problem provide us >>> with the error logs..:) >>> >>> Regards, >>> Mohammad Tariq >>> >>> >>> On Wed, Jun 6, 2012 at 11:22 PM, Babak Bastan <babak...@gmail.com> wrote: >>> > were you able to format hdfs properly??? >>> > I did'nt get your question,Do you mean HADOOP_HOME? or where did I >>> > install >>> > Hadoop? >>> > >>> > On Wed, Jun 6, 2012 at 7:49 PM, Mohammad Tariq <donta...@gmail.com> >>> > wrote: >>> >> >>> >> if you are getting only this, it means your hadoop is not >>> >> running..were you able to format hdfs properly??? >>> >> >>> >> Regards, >>> >> Mohammad Tariq >>> >> >>> >> >>> >> On Wed, Jun 6, 2012 at 11:17 PM, Babak Bastan <babak...@gmail.com> >>> >> wrote: >>> >> > Hi MohammadmI irun jps in my shel I can see this result: >>> >> > 2213 Jps >>> >> > >>> >> > >>> >> > On Wed, Jun 6, 2012 at 7:44 PM, Mohammad Tariq <donta...@gmail.com> >>> >> > wrote: >>> >> >> >>> >> >> you can also use "jps" command at your shell to see whether Hadoop >>> >> >> processes are running or not. >>> >> >> >>> >> >> Regards, >>> >> >> Mohammad Tariq >>> >> >> >>> >> >> >>> >> >> On Wed, Jun 6, 2012 at 11:12 PM, Mohammad Tariq <donta...@gmail.com> >>> >> >> wrote: >>> >> >> > Hi Babak, >>> >> >> > >>> >> >> > You have to type it in you web browser..Hadoop provides us a web >>> >> >> > GUI >>> >> >> > that not only allows us to browse through the file system, but to >>> >> >> > download the files as well..Apart from that it also provides a web >>> >> >> > GUI >>> >> >> > that can be used to see the status of Jobtracker and >>> >> >> > Tasktracker..When >>> >> >> > you run a Hive or Pig job or a Mapreduce job, you can point your >>> >> >> > browser to http://localhost:50030 to see the status and logs of >>> >> >> > your >>> >> >> > job. >>> >> >> > >>> >> >> > Regards, >>> >> >> > Mohammad Tariq >>> >> >> > >>> >> >> > >>> >> >> > On Wed, Jun 6, 2012 at 8:28 PM, Babak Bastan <babak...@gmail.com> >>> >> >> > wrote: >>> >> >> >> Thank you shashwat for the answer, >>> >> >> >> where should I type http://localhost:50070? >>> >> >> >> I typed here: hive>http://localhost:50070 but nothing as result >>> >> >> >> >>> >> >> >> >>> >> >> >> On Wed, Jun 6, 2012 at 3:32 PM, shashwat shriparv >>> >> >> >> <dwivedishash...@gmail.com> wrote: >>> >> >> >>> >>> >> >> >>> first type http://localhost:50070 whether this is opening or not >>> >> >> >>> and >>> >> >> >>> check >>> >> >> >>> how many nodes are available, check some of the hadoop shell >>> >> >> >>> commands >>> >> >> >>> >>> >> >> >>> from http://hadoop.apache.org/common/docs/r0.18.3/hdfs_shell.html >>> >> >> >>> run >>> >> >> >>> example mapreduce task on hadoop take example from here >>> >> >> >>> >>> >> >> >>> >>> >> >> >>> >>> >> >> >>> : http://www.michael-noll.com/blog/2011/04/09/benchmarking-and-stress-testing-an-hadoop-cluster-with-terasort-testdfsio-nnbench-mrbench/ >>> >> >> >>> >>> >> >> >>> if all the above you can do sucessfully means hadoop is >>> >> >> >>> configured >>> >> >> >>> correctly >>> >> >> >>> >>> >> >> >>> Regards >>> >> >> >>> Shashwat >>> >> >> >>> >>> >> >> >>> >>> >> >> >>> On Wed, Jun 6, 2012 at 1:30 AM, Babak Bastan >>> >> >> >>> <babak...@gmail.com> >>> >> >> >>> wrote: >>> >> >> >>>> >>> >> >> >>>> no I'm not working on CDH.Is there a way to test if my Hadoop >>> >> >> >>>> works >>> >> >> >>>> fine >>> >> >> >>>> or not? >>> >> >> >>>> >>> >> >> >>>> >>> >> >> >>>> On Tue, Jun 5, 2012 at 9:55 PM, Bejoy KS <bejoy...@yahoo.com> >>> >> >> >>>> wrote: >>> >> >> >>>>> >>> >> >> >>>>> Hi Babak >>> >> >> >>>>> >>> >> >> >>>>> You gotta follow those instructions in the apace site to set >>> >> >> >>>>> up >>> >> >> >>>>> hadoop >>> >> >> >>>>> from scratch and ensure that hdfs is working first. You should >>> >> >> >>>>> be >>> >> >> >>>>> able to >>> >> >> >>>>> read and write files to hdfs before you do your next steps. >>> >> >> >>>>> >>> >> >> >>>>> Are you on CDH or apache distribution of hadoop? If it is CDH >>> >> >> >>>>> there >>> >> >> >>>>> are >>> >> >> >>>>> detailed instructions on Cloudera web site. >>> >> >> >>>>> >>> >> >> >>>>> Regards >>> >> >> >>>>> Bejoy KS >>> >> >> >>>>> >>> >> >> >>>>> Sent from handheld, please excuse typos. >>> >> >> >>>>> ________________________________ >>> >> >> >>>>> From: Babak Bastan <babak...@gmail.com> >>> >> >> >>>>> Date: Tue, 5 Jun 2012 21:30:22 +0200 >>> >> >> >>>>> To: <user@hive.apache.org> >>> >> >> >>>>> ReplyTo: user@hive.apache.org >>> >> >> >>>>> Subject: Re: Error while Creating Table in Hive >>> >> >> >>>>> >>> >> >> >>>>> @Bejoy: I set the fs.default.name in the core-site.xml and I >>> >> >> >>>>> did >>> >> >> >>>>> all >>> >> >> >>>>> of >>> >> >> >>>>> thing that was mentioned in the reference but no effect >>> >> >> >>>>> >>> >> >> >>>>> On Tue, Jun 5, 2012 at 8:43 PM, Babak Bastan >>> >> >> >>>>> <babak...@gmail.com> >>> >> >> >>>>> wrote: >>> >> >> >>>>>> >>> >> >> >>>>>> Ok sorry but that was my Mistake .I thought it works but no. >>> >> >> >>>>>> I wrote the command without ; and then I think It works but >>> >> >> >>>>>> with >>> >> >> >>>>>> ; >>> >> >> >>>>>> at >>> >> >> >>>>>> the end of command >>> >> >> >>>>>> >>> >> >> >>>>>> CREATE TABLE pokes (foo INT, bar STRING); >>> >> >> >>>>>> >>> >> >> >>>>>> does'nt work >>> >> >> >>>>>> >>> >> >> >>>>>> >>> >> >> >>>>>> On Tue, Jun 5, 2012 at 8:34 PM, shashwat shriparv >>> >> >> >>>>>> <dwivedishash...@gmail.com> wrote: >>> >> >> >>>>>>> >>> >> >> >>>>>>> inside configuration. all properties will be inside the >>> >> >> >>>>>>> configuration >>> >> >> >>>>>>> tags >>> >> >> >>>>>>> >>> >> >> >>>>>>> >>> >> >> >>>>>>> On Tue, Jun 5, 2012 at 11:53 PM, Babak Bastan >>> >> >> >>>>>>> <babak...@gmail.com> >>> >> >> >>>>>>> wrote: >>> >> >> >>>>>>>> >>> >> >> >>>>>>>> Thank you so much my friend your idee works fine(no error) >>> >> >> >>>>>>>> you >>> >> >> >>>>>>>> are >>> >> >> >>>>>>>> the best :) >>> >> >> >>>>>>>> >>> >> >> >>>>>>>> >>> >> >> >>>>>>>> On Tue, Jun 5, 2012 at 8:20 PM, Babak Bastan >>> >> >> >>>>>>>> <babak...@gmail.com> >>> >> >> >>>>>>>> wrote: >>> >> >> >>>>>>>>> >>> >> >> >>>>>>>>> It must be inside the <configuration></configuration> or >>> >> >> >>>>>>>>> outside >>> >> >> >>>>>>>>> this? >>> >> >> >>>>>>>>> >>> >> >> >>>>>>>>> >>> >> >> >>>>>>>>> On Tue, Jun 5, 2012 at 8:15 PM, shashwat shriparv >>> >> >> >>>>>>>>> <dwivedishash...@gmail.com> wrote: >>> >> >> >>>>>>>>>> >>> >> >> >>>>>>>>>> It will be inside hive/conf >>> >> >> >>>>>>>>>> >>> >> >> >>>>>>>>>> >>> >> >> >>>>>>>>>> On Tue, Jun 5, 2012 at 11:43 PM, Babak Bastan >>> >> >> >>>>>>>>>> <babak...@gmail.com> >>> >> >> >>>>>>>>>> wrote: >>> >> >> >>>>>>>>>>> >>> >> >> >>>>>>>>>>> Thanks sShashwat, and where is this hive-site.xml >>> >> >> >>>>>>>>>>> >>> >> >> >>>>>>>>>>> On Tue, Jun 5, 2012 at 8:02 PM, shashwat shriparv >>> >> >> >>>>>>>>>>> <dwivedishash...@gmail.com> wrote: >>> >> >> >>>>>>>>>>>> >>> >> >> >>>>>>>>>>>> set >>> >> >> >>>>>>>>>>>> >>> >> >> >>>>>>>>>>>> hive.metastore.warehouse.dir in hive-site.xml >>> >> >> >>>>>>>>>>>> >>> >> >> >>>>>>>>>>>> <property> >>> >> >> >>>>>>>>>>>> <name>hive.metastore.local</name> >>> >> >> >>>>>>>>>>>> <value>true</value> >>> >> >> >>>>>>>>>>>> </property> >>> >> >> >>>>>>>>>>>> >>> >> >> >>>>>>>>>>>> <name>hive.metastore.warehouse.dir</name> >>> >> >> >>>>>>>>>>>> <value>/home/<your >>> >> >> >>>>>>>>>>>> username>/hivefolder</value> >>> >> >> >>>>>>>>>>>> <description>location of default >>> >> >> >>>>>>>>>>>> database >>> >> >> >>>>>>>>>>>> for >>> >> >> >>>>>>>>>>>> the >>> >> >> >>>>>>>>>>>> warehouse</description> >>> >> >> >>>>>>>>>>>> </property> >>> >> >> >>>>>>>>>>>> >>> >> >> >>>>>>>>>>>> >>> >> >> >>>>>>>>>>>> On Tue, Jun 5, 2012 at 10:43 PM, Babak Bastan >>> >> >> >>>>>>>>>>>> <babak...@gmail.com> wrote: >>> >> >> >>>>>>>>>>>>> >>> >> >> >>>>>>>>>>>>> Hello Experts , >>> >> >> >>>>>>>>>>>>> >>> >> >> >>>>>>>>>>>>> I'm new in Hive .When try to create a test Table in >>> >> >> >>>>>>>>>>>>> Hive >>> >> >> >>>>>>>>>>>>> I >>> >> >> >>>>>>>>>>>>> get >>> >> >> >>>>>>>>>>>>> an error.I want to run this command: >>> >> >> >>>>>>>>>>>>> CREATE TABLE Test (DateT STRING, Url STRING, Content >>> >> >> >>>>>>>>>>>>> STRING); >>> >> >> >>>>>>>>>>>>> but this error occured: >>> >> >> >>>>>>>>>>>>> FAILED: Error in metadata: MetaException(message:Got >>> >> >> >>>>>>>>>>>>> exception: >>> >> >> >>>>>>>>>>>>> java.io.FileNotFoundException File >>> >> >> >>>>>>>>>>>>> file:/user/hive/warehouse/test does not >>> >> >> >>>>>>>>>>>>> exist.) >>> >> >> >>>>>>>>>>>>> FAILED: Execution Error, return code 1 from >>> >> >> >>>>>>>>>>>>> org.apache.hadoop.hive.ql.exec.DDLTask >>> >> >> >>>>>>>>>>>>> How can I solve this Problem? >>> >> >> >>>>>>>>>>>>> Thank you so much >>> >> >> >>>>>>>>>>>> >>> >> >> >>>>>>>>>>>> >>> >> >> >>>>>>>>>>>> >>> >> >> >>>>>>>>>>>> >>> >> >> >>>>>>>>>>>> -- >>> >> >> >>>>>>>>>>>> >>> >> >> >>>>>>>>>>>> >>> >> >> >>>>>>>>>>>> ∞ >>> >> >> >>>>>>>>>>>> >>> >> >> >>>>>>>>>>>> Shashwat Shriparv >>> >> >> >>>>>>>>>>>> >>> >> >> >>>>>>>>>>>> >>> >> >> >>>>>>>>>>> >>> >> >> >>>>>>>>>> >>> >> >> >>>>>>>>>> >>> >> >> >>>>>>>>>> >>> >> >> >>>>>>>>>> -- >>> >> >> >>>>>>>>>> >>> >> >> >>>>>>>>>> >>> >> >> >>>>>>>>>> ∞ >>> >> >> >>>>>>>>>> >>> >> >> >>>>>>>>>> Shashwat Shriparv >>> >> >> >>>>>>>>>> >>> >> >> >>>>>>>>>> >>> >> >> >>>>>>>>> >>> >> >> >>>>>>>> >>> >> >> >>>>>>> >>> >> >> >>>>>>> >>> >> >> >>>>>>> >>> >> >> >>>>>>> -- >>> >> >> >>>>>>> >>> >> >> >>>>>>> >>> >> >> >>>>>>> ∞ >>> >> >> >>>>>>> >>> >> >> >>>>>>> Shashwat Shriparv >>> >> >> >>>>>>> >>> >> >> >>>>>>> >>> >> >> >>>>>> >>> >> >> >>>>> >>> >> >> >>>> >>> >> >> >>> >>> >> >> >>> >>> >> >> >>> >>> >> >> >>> -- >>> >> >> >>> >>> >> >> >>> >>> >> >> >>> ∞ >>> >> >> >>> >>> >> >> >>> Shashwat Shriparv >>> >> >> >>> >>> >> >> >>> >>> >> >> >> >>> >> > >>> >> > >>> > >>> > >> >>
