The following script works with the latest 0.9 snapshot:
#!/usr/bin/python
# Name - embed_pig.py
# need to explicitly import the Pig class
from org.apache.pig.scripting import Pig
p = Pig.compile("""
records = LOAD 'path/to/data' AS (input_line:chararray);
DESCRIBE records;
""")
for i in range(0,2):
r = p.bind()
results = r.runSingle()
If you just want to use the command DESCRIBE, this script works better:
#!/usr/bin/python
# Name - embed_pig.py
# need to explicitly import the Pig class
from org.apache.pig.scripting import Pig
p = Pig.compile("""
records = LOAD 'path/to/data' AS (input_line:chararray);
""")
for i in range(0,2):
r = p.bind()
r. describe('records')
On 1/11/11 1:03 AM, "[email protected]" <[email protected]> wrote:
Hi,
I downloaded Pig-0.9.0-SNAPSHOT.tar.gz and set it up.
I am trying to run this:
--------
#!/usr/bin/python
# Name - embed_pig.py
p = Pig.compile("""
records = LOAD 'path/to/data' AS (input_line:chararray);
DESCRIBE records;
""")
for i in range(1,2):
r = p.bind()
results = r.run()
if results.getStatus("records") `= "FAILED":
raise "Pig job failed"
-------------
Command to Run:
$pig -x local embed_pig.py
Error I got:
Error 1000 - Parsing Error.
The purpose of this script to call the pig scripts twice, iteratively.
What is the correct way to run a code like this? Any other special environment
variables that I need to set?
Thanks,
Deepak
-----Original Message-----
From: Olga Natkovich [mailto:[email protected]]
Sent: Monday, January 10, 2011 11:24 PM
To: [email protected]
Subject: RE: Iterative MapReduce with PIG
The initial implementation has been checked into the trunk last Friday. If you
feel adventurous, you can give it a try :).
Olga
-----Original Message-----
From: Alan Gates [mailto:[email protected]]
Sent: Monday, January 10, 2011 8:19 AM
To: [email protected]
Subject: Re: Iterative MapReduce with PIG
This is one of our major initiatives for 0.9. See
http://wiki.apache.org/pig/TuringCompletePig
and https://issues.apache.org/jira/browse/PIG-1479. But until that's ready
you'll have to use Java or piglet as recommended by Dmitriy.
Alan.
On Jan 10, 2011, at 3:09 AM, [email protected] wrote:
> Hi,
>
> I need to implement an application that is iterative in nature. At the
> end of each iteration, I need to take the result and provide it as an
> input for the next iteration.
>
> Embedding PIG statements in a Java Program looks like one way to do
> it.
>
> But I prefer using Python for programming. How can I do this?
>
> Thanks,
> Deepak
Please do not print this email unless it is absolutely necessary.
The information contained in this electronic message and any attachments to
this message are intended for the exclusive use of the addressee(s) and may
contain proprietary, confidential or privileged information. If you are not the
intended recipient, you should not disseminate, distribute or copy this e-mail.
Please notify the sender immediately and destroy all copies of this message and
any attachments.
WARNING: Computer viruses can be transmitted via email. The recipient should
check this email and any attachments for the presence of viruses. The company
accepts no liability for any damage caused by any virus transmitted by this
email.
www.wipro.com
