Hi, SUM is not a macro but a built-in UDF. Please try:
log = LOAD '/tmp/data' AS (value:long); total = FOREACH (GROUP log ALL) GENERATE SUM(log.value); DUMP total; Please also see: http://pig.apache.org/docs/r0.11.0/func.html#sum Thanks, Cheolsoo On Sat, Apr 6, 2013 at 1:08 PM, raanan nitzan <[email protected]>wrote: > Hi > > I'm trying to run pig script with builtin macros like count/sum/avg ... > (org.apache.pig.builtin) > But on every method I trued I get the same ERROR > > ERROR org.apache.pig.tools.grunt.Grunt - ERROR 1200: <file /tmp/test.pig, > line 3> Cannot expand macro 'SUM'. Reason: Macro must be defined before > expansion > > I understood that macro cant be used from grunt ( > http://www.mail-archive.com/[email protected]/msg04926.html) > So how can I use the org.apache.pig.builtin macro's > I tried to run script using following methods > > - Command line : pig [script file path] > - PigRunner : java MyPigRunner.java [script file path] > - PigServer : java MyPigServer.java [script file path] > > > But for all I get the expension error, what is the way to run pig script > with builtin macros > > I'm using followng pig pacakge : pig-0.10.0+61-1.cdh4.1.3.p0.23.el6.noarch > > *Pig script:* > *log = LOAD '/tmp/data' AS (value:long) ;* > *total = SUM(log);* > *DUMP log;* > * > * > > PigRunner.java: > > *public class MyPigRunner {* > * * > * > * > *static public void main(String [] args) {* > * * > * org.apache.pig.PigRunner.run(args, null);* > * * > *}* > * > * > *}* > > PigServer.java: > *public class MyPigServer {* > * * > * > * > *static public void main (String[] args) throws IOException{* > * * > * String filename = args[0];* > * PigServer pig = new PigServer(ExecType.LOCAL); * > * pig.registerScript(filename);* > * pig.store("Result", "/tmp/pig.result");* > * * > *}* > * > * > *}* > > > *Details at logfile: /usr/lib/pig/bin/pig_1365300225744.log > 0_31/lib/tools.jar:/usr/lib/hbase/bin/..:/usr/lib/hbase/ > bin/../hbase-0.92.1-cdh4.1.3-security.jar:/usr/lib/hbase/ > bin/../hbase-0.92.1-cdh4.1.3-security-tests.jar:/usr/lib/ > hbase/bin/../hbase.jar:/usr/lib/hbase/bin/../lib/ > activation-1.1.jar:/usr/lib/hbase/bin/../lib/aopalliance- > 1.0.jar:/usr/lib/hbase/bin/../lib/asm-3.2.jar:/usr/lib/ > hbase/bin/../lib/avro-1.7.1.cloudera.2.jar:/usr/lib/hbase/ > bin/../lib/commons-beanutils-1.7.0.jar:/usr/lib/hbase/bin/. > ./lib/commons-beanutils-core-1.8.0.jar:/usr/lib/hbase/bin/. > ./lib/commons-cli-1.2.jar:/usr/lib/hbase/bin/../lib/ > commons-codec-1.4.jar:/usr/lib/hbase/bin/../lib/commons- > collections-3.2.1.jar:/usr/lib/hbase/bin/../lib/commons- > configuration-1.6.jar:/usr/lib/hbase/bin/../lib/commons- > daemon-1.0.3.jar:/usr/lib/hbase/bin/../lib/commons- > digester-1.8.jar:/usr/lib/hbase/bin/../lib/commons-el-1. > 0.jar:/usr/lib/hbase/bin/../lib/commons-httpclient-3.1. > jar:/usr/lib/hbase/bin/../lib/commons-io-2.1.jar:/usr/lib/ > hbase/bin/../lib/commons-lang-2.5.jar:/usr/lib/hbase/bin/../ > lib/commons-logging-1.1.1.jar:/usr/lib/hbase/bin/../lib/ > commons-net-3.1.jar:/usr/lib/hbase/bin/../lib/core-3.1.1. > jar:/usr/lib/hbase/bin/../lib/gmbal-api-only-3.0.0-b023.jar: > /usr/lib/hbase/bin/../lib/grizzly-framework-2.1.1.jar:/ > usr/lib/hbase/bin/../lib/grizzly-framework-2.1.1-tests. > jar:/usr/lib/hbase/bin/../lib/grizzly-http-2.1.1.jar:/usr/ > lib/hbase/bin/../lib/grizzly-http-server-2.1.1.jar:/usr/ > lib/hbase/bin/../lib/grizzly-http-servlet-2.1.1.jar:/usr/ > lib/hbase/bin/../lib/grizzly-rcm-2.1.1.jar:/usr/lib/hbase/ > bin/../lib/guava-11.0.2.jar:/usr/lib/hbase/bin/../lib/ > guice-3.0.jar:/usr/lib/hbase/bin/../lib/guice-servlet-3.0. > jar:/usr/lib/hbase/bin/../lib/high-scale-lib-1.1.1.jar:/usr/ > lib/hbase/bin/../lib/httpclient-4.0.1.jar:/usr/lib/ > hbase/bin/../lib/httpcore-4.0.1.jar:/usr/lib/hbase/bin/../ > lib/jackson-core-asl-1.8.8.jar:/usr/lib/hbase/bin/../lib/ > jackson-jaxrs-1.8.8.jar:/usr/lib/hbase/bin/../lib/jackson- > mapper-asl-1.8.8.jar:/usr/lib/hbase/bin/../lib/jackson-xc-1. > 8.8.jar:/usr/lib/hbase/bin/../lib/jamon-runtime-2.3.1.jar:/ > usr/lib/hbase/bin/../lib/jasper-compiler-5.5.23.jar:/ > usr/lib/hbase/bin/../lib/jasper-runtime-5.5.23.jar:/ > > usr/lib/hbase/bin/../lib/javax.inject-1.jar:/usr/lib/hbase/bin/../lib/javax. > servlet-3.0.jar:/usr/lib/hbase/bin/../lib/jaxb-api-2.1. > jar:/usr/lib/hbase/bin/../lib/jaxb-impl-2.2.3-1.jar:/usr/ > lib/hbase/bin/../lib/jersey-client-1.8.jar:/usr/lib/hbase/ > bin/../lib/jersey-core-1.8.jar:/usr/lib/hbase/bin/../lib/ > jersey-grizzly2-1.8.jar:/usr/lib/hbase/bin/../lib/jersey- > guice-1.8.jar:/usr/lib/hbase/bin/../lib/jersey-json-1.8. > jar:/usr/lib/hbase/bin/../lib/jersey-server-1.8.jar:/usr/ > lib/hbase/bin/../lib/jersey-test-framework-core-1.8.jar:/ > usr/lib/hbase/bin/../lib/jersey-test-framework-grizzly2-1.8.jar:/usr/lib/ > hbase/bin/../lib/jets3t-0.6.1.jar:/usr/lib/hbase/bin/../lib/ > jettison-1.1.jar:/usr/lib/hbase/bin/../lib/jetty-6.1.26. > cloudera.2.jar:/usr/lib/hbase/bin/../lib/jetty-util-6.1.26. > cloudera.2.jar:/usr/lib/hbase/bin/../lib/jruby-complete-1.6. > 5.jar:/usr/lib/hbase/bin/../lib/jsch-0.1.42.jar:/usr/lib/ > hbase/bin/../lib/jsp-2.1-6.1.14.jar:/usr/lib/hbase/bin/../ > lib/jsp-api-2.1-6.1.14.jar:/usr/lib/hbase/bin/../lib/jsp- > api-2.1.jar:/usr/lib/hbase/bin/../lib/jsr305-1.3.9.jar:/ > usr/lib/hbase/bin/../lib/kfs-0.3.jar:/usr/lib/hbase/bin/../ > lib/libthrift-0.7.0.jar:/usr/lib/hbase/bin/../lib/log4j-1. > 2.17.jar:/usr/lib/hbase/bin/../lib/management-api-3.0.0- > b012.jar:/usr/lib/hbase/bin/../lib/metrics-core-2.1.2.jar:/ > usr/lib/hbase/bin/../lib/netty-3.2.4.Final.jar:/usr/lib/hbase/bin/../lib/ > paranamer-2.3.jar:/usr/lib/hbase/bin/../lib/protobuf- > java-2.4.0a.jar:/usr/lib/hbase/bin/../lib/servlet-api- > 2.5-6.1.14.jar:/usr/lib/hbase/bin/../lib/servlet-api-2.5. > jar:/usr/lib/hbase/bin/../lib/slf4j-api-1.6.1.jar:/usr/lib/ > hbase/bin/../lib/snappy-java-1.0.4.1.jar:/usr/lib/hbase/ > bin/../lib/stax-api-1.0.1.jar:/usr/lib/hbase/bin/../lib/ > xmlenc-0.52.jar:/usr/lib/hbase/bin/../lib/zookeeper. > jar:/etc/hadoop/conf:/*:/lib/*:/usr/lib/zookeeper/*:/usr/ > lib/zookeeper/lib/*::/etc/hadoop/conf:/usr/lib/hadoop/ > lib/*:/usr/lib/hadoop/.//*:/usr/lib/hadoop-hdfs/./:/usr/ > lib/hadoop-hdfs/lib/*:/usr/lib/hadoop-hdfs/.//*:/usr/lib/ > hadoop-yarn/lib/*:/usr/lib/hadoop-yarn/.//*:/usr/lib/ > hadoop-mapreduce/lib/*:/usr/lib/hadoop-mapreduce/.//*:/ > usr/lib/pig/bin/../lib/automaton.jar:/usr/lib/pig/bin/../lib/avro-1.7.1. > cloudera.2.jar:/usr/lib/pig/bin/../lib/jackson-core-asl-1. > 8.8.jar:/usr/lib/pig/bin/../lib/jackson-mapper-asl-1.8.8. > jar:/usr/lib/pig/bin/../lib/json-simple-1.1.jar:/usr/lib/ > pig/bin/../lib/jython-2.5.0.jar:/usr/lib/pig/bin/../lib/ > > snappy-java-1.0.4.1.jar:/usr/lib/pig/bin/../pig.jar:/usr/lib/pig/bin/MyPigServer.jar' > org.apache.pig.Main /tmp/test.pig -x mapreduce* > *2013-04-06 22:03:46,009 [main] INFO org.apache.pig.Main - Apache Pig > version 0.10.0-cdh4.1.3 (rexported) compiled Jan 26 2013, 17:36:42 > 2013-04-06 22:03:46,013 [main] INFO org.apache.pig.Main - Logging error > messages to: /usr/lib/pig/bin/pig_1365300225744.log > 2013-04-06 22:03:49,344 [main] INFO org.apache.pig.backend.hadoop. > executionengine.HExecutionEngine - Connecting to hadoop file system at: > hdfs://127.0.0.1:8020 > > 2013-04-06 22:03:53,110 [main] DEBUG org.apache.pig.PigServer - Create a > new graph. > > 2013-04-06 22:03:54,036 [main] DEBUG > org.apache.pig.parser.QueryParserDriver > - Original macro AST: > (QUERY (STATEMENT log (LOAD '/tmp/data' (AS (FIELD_DEF value))))) > > 2013-04-06 22:03:54,036 [main] DEBUG > org.apache.pig.parser.QueryParserDriver > - macro AST after import: > (QUERY (STATEMENT log (LOAD '/tmp/data' (AS (FIELD_DEF value))))) > > 2013-04-06 22:03:54,036 [main] DEBUG > org.apache.pig.parser.QueryParserDriver > - Resulting macro AST: > (QUERY (STATEMENT log (LOAD '/tmp/data' (AS (FIELD_DEF value))))) > > 2013-04-06 22:03:54,867 [main] DEBUG org.apache.pig.impl. > logicalLayer.schema.Schema$FieldSchema - t: 50 Bag: 120 tuple: 110 > 2013-04-06 22:03:54,882 [main] DEBUG > org.apache.pig.parser.QueryParserDriver > - Original macro AST: > (QUERY (STATEMENT log (LOAD '/tmp/data' (AS (FIELD_DEF value)))) > (MACRO_INLINE SUM (RETURN_VAL total) (PARAMS log))) > > 2013-04-06 22:03:54,882 [main] DEBUG > org.apache.pig.parser.QueryParserDriver > - macro AST after import: > (QUERY (STATEMENT log (LOAD '/tmp/data' (AS (FIELD_DEF value)))) > (MACRO_INLINE SUM (RETURN_VAL total) (PARAMS log))) > > 2013-04-06 22:03:54,889 [main] ERROR org.apache.pig.tools.grunt.Grunt - > ERROR 1200: <file /tmp/test.pig, line 3> Cannot expand macro 'SUM'. Reason: > Macro must be defined before expansion. > Details at logfile: /usr/lib/pig/bin/pig_1365300225744.log* >
