OK I am getting a little confused now.
Consider that I am working on a scenario where there is no limit with
memory available.
In such scenario, is there any advantage of storing data in HDFS in
compressed format. Any advantage, like, if node 1 has data available and it
is executing a particular t
Out of curiosity, why not bz2 which is splittable? Definitely will try out
snappy in the meantime. Thanks!
@dennylee | http://about.me/dennylee
On Jun 8, 2012, at 8:42 PM, Raja Thiruvathuru wrote:
> Agree with Mark.
>
> On Fri, Jun 8, 2012 at 5:08 PM, Mark Grover
> wrote:
> Hi Sreenath,
>
Any idea about lzo or bzip2...any of these splittable??
Agree with Mark.
On Fri, Jun 8, 2012 at 5:08 PM, Mark Grover wrote:
> Hi Sreenath,
> All the points made on this thread are very valid. However, I wanted to
> add that you should keep in mind that Gzip compression is not splittable.
> This is because of the very nature of the codec. So, if your i
Compression will make processing faster almost all the time. Gzip
compression can shrink a text file to 40 percent its original size. Snappy
maybe about 60 percent. On average.
Then your dealing with say 1tb of data 60 percent savings is 600 gb. If you
think about the disk and network savings tha
If you are interested, you can also look at Apache hama which provides an
MPI like interface on top of hadoop map-reduce.
http://incubator.apache.org/hama/
On Jun 8, 2012 4:55 PM, "Mark Grover" wrote:
> Hi Jason,
> Hive does expose a JDBC interface which can by tools and applications. You
> woul
Hi Sreenath,
All the points made on this thread are very valid. However, I wanted to add
that you should keep in mind that Gzip compression is not splittable. This
is because of the very nature of the codec. So, if your input data contains
files of size greater than HDFS block size in Gzip format,
Hi Jason,
Hive does expose a JDBC interface which can by tools and applications. You
would check out individual tools to see if they support Hadoop (I use the
word Hadoop and not Hive since an application doesn't need Hive to run Map
Reduce jobs on data in HDFS).
Apache Mahout, as Sreenath, mentio
also visit this link on official wiki page -
https://cwiki.apache.org/Hive/developerguide.html#DeveloperGuide-Unittestsanddebugging
Regards,
Mohammad Tariq
On Sat, Jun 9, 2012 at 2:32 AM, Mohammad Tariq wrote:
> Kanna, you can visit the link given below..it talks about Hive Unit
> Testing i
Kanna, you can visit the link given below..it talks about Hive Unit
Testing in detail.
http://dev.bizo.com/2011/04/hive-unit-testing.html
Regards,
Mohammad Tariq
On Sat, Jun 9, 2012 at 1:04 AM, Kanna Karanam wrote:
> Thanks Mohammad - I downloaded the Hadoop & Hive source from apache and ma
Thanks Mohammad - I downloaded the Hadoop & Hive source from apache and made
some changes in Hadoop internally. Now I am trying to run hive unit tests
against my changes (internal Hadoop). Are there any configuration settings in
hive (unit tests) to configure and run against local Hadoop jars.
Hello Kanna,
If you are facing problem with your build, you can download
Hadoop and Hive directly from Apache and use them.
Regards,
Mohammad Tariq
On Fri, Jun 8, 2012 at 9:06 PM, Kanna Karanam wrote:
> Hi Gurus,
>
>
>
> It would be a great help if anyone can help me run hive with my
Hi Gurus,
It would be a great help if anyone can help me run hive with my local hadoop
build. I followed the wiki instructions but no luck. I am sure that I am
missing something here. It would be a great help if anyone can point me in the
right direction.
>From wiki:
* One can also compi
Hi Mohammad,
I read the tutorials and I think that will solve my problem.
I appreciate the help!
now i want to know abt the export in sqoop from hdfs ;
this is my hadoop fs
hduser@XPS-L501X:~$ hadoop fs -ls
Found 11 items
drwxr-xr-x - hduser supergroup 0 2012-06-07 15:15
/user/hduser/_sqoop
drwxr-xr-x - hduser supergroup 0 2012-06-06 15:16
/user/hduser/infor
drwxr-xr-x
Check this link and follow :
http://www.mazsoft.com/blog/post/2010/02/01/Setting-up-HadoopHive-to-use-MySQL-as-metastore.aspx
Regards
∞
Shashwat Shriparv
On Fri, Jun 8, 2012 at 1:34 PM, soham sardar wrote:
> Thanks Nanda and Bejoy btw i would be greatful if u could give me the
> connection ur
Thanks Nanda and Bejoy btw i would be greatful if u could give me the
connection url for mysql as my rdbs is mysql and not derby
Thanks in advance :)
On Fri, Jun 8, 2012 at 12:57 AM, Bejoy Ks wrote:
> Hi Soham
>
> The error looks like your meta store doesn't have the required tables .
> Try enab
Hi, Screenath
all right, I will check it out. thank you~
2012/6/8 Sreenath Menon
> Kindly check out Apache Mahout and whether it satisfies your needs.
>
--
YANG, Lin
18 matches
Mail list logo
