Re: Question about pig & HDFS

2011-08-26 Thread Daniel Dai
Pig by default use plain text file as input/output, unless you write a custom LoadFunc/StoreFunc. There is no specific Pig storage format. You can copy the file to local using copyToLocal. If you want to export directly to SQL table, you need to write a StoreFunc. Pig work on tuple rather than K,V

Question about pig & HDFS

2011-08-23 Thread Keren Ouaknine
Hello, Pig generates data to HDFS and I would like to find a way to convert it to a general format by either: 1. flatening the data (would copyToLocal work here?!) 2. export the data to SQL tables (or any other non specific Hadoop format) 3. generate K,V pairs of data (since Pig code is converted