Suggestion : you can create a join for each column individually ..and then union the result.. ??
http://pig.apache.org/docs/r0.7.0/piglatin_ref2.html#UNION *Cheers !!* Arvind On Tue, Jul 28, 2015 at 1:30 AM, paul green <[email protected]> wrote: > HelloI use Pig at home (currently version 0.13.0) regularly on data sets > that vary between 10's Megabytes and 10's Gigabytes. I wanted to be able to > join two data sets together (ideally filtering). The main problem I am > having and have not found an easily solution is:I want to join data set 1 > to data set 2 like below.data1.txtid, name, job0001,john, manager0002,phil, > deputydata2.txtid1, id2, id3, > label0001,0002,0001,useful0005,0001,0001,useful0000,0010,0009,not > usefulCode ProposaldatasetA = LOAD 'data1.txt' USING PigStorage(',') AS > (fieldA1, fieldA2, fieldA3);datasetB = LOAD 'data2.txt' USING > PigStorage(',') AS (fieldB1, fieldB2, fieldB3, fieldB4);joined = JOIN > datasetA BY fieldA1, datasetB BY (fieldB1 OR fieldB2 > OR fieldB3);DUMP joined;So essentially I want to join 1 column to n columns > in the second data set where they are equal. I am not after a partial join > but an exact join. Is there a feature already in the language to do this, > if not, would it be possible to request such a feature?Thanks. >
