Re: Extract and Load to Hadoop via Pipes

I had trouble with Sqoop, so here's what I do (Perl): $cmd = qq#echo "select * from $tableName where $dateColumn >= '$dayStart 00:00:00' and $dateColumn < '$dayEnd 00:00:00'" \\ | mysql -h $dwIP --quick -B --skip-column-names --user=$USER --password=$PASS $databaseName \\ | ssh hdfs\@$

Re: Extract and Load to Hadoop via Pipes

Thanks Zoltan. I will check it out. 2011/5/6 Zoltan Prekopcsak > Hi, > > I think this is what Sqoop is for: > http://www.cloudera.com/blog/2009/06/introducing-sqoop/ > http://www.cloudera.com/downloads/sqoop/ > > Best, Zoltan > > > 5/6/11 7:45 PM keltezéssel, bichonfrise74 írta: > > >> Hi, >> >

Re: Extract and Load to Hadoop via Pipes

Hi, I think this is what Sqoop is for: http://www.cloudera.com/blog/2009/06/introducing-sqoop/ http://www.cloudera.com/downloads/sqoop/ Best, Zoltan 5/6/11 7:45 PM keltezéssel, bichonfrise74 írta: Hi, I want to be able to extract data from Mysql to Hadoop without writing the data to disk.

Extract and Load to Hadoop via Pipes

Hi, I want to be able to extract data from Mysql to Hadoop without writing the data to disk. I was thinking in the line of piping the extract and loading it to Hadoop. Something like this: mysql | hive -e 'load data into table ... Has anyone done this before?