I’ve not worked on Piglet after the initial spike. But the point of that work was to create something that someone else could pick up, as you have done.
If the Grunt parser is working better for you, we could consider using that instead of my hand-rolled parser. (But hopefully we can keep the VALUES operator; it’s a shame Pig latin doesn’t have this.) Yes, there are a number of issues outstanding. There is actually a test case for nested FOREACH in PigletTest.java, disabled. I figured we can work on these when people start using Piglet for real work. Log JIRA cases for the missing features and we can discuss how they could be implemented. Are you aware of the work Eli Levine is doing on the Pig Adapter[1]? (It’s the opposite of Piglet — Pig on the bottom, rather than on the top — but it proves there is interest relating to Calcite-Pig integration.) Julian [1] https://issues.apache.org/jira/browse/CALCITE-1598 <https://issues.apache.org/jira/browse/CALCITE-1598> > On Feb 13, 2017, at 12:16 PM, Khai Tran <[email protected]> wrote: > > Hi all, > > I just want to check if anyone is working on Piglet. I've not seen any > commits on the source for awhile. > > I had ~3K LOC for converting Pig scripts into Calcite logical plans. My > approach is different from the one in Piglet. I used Grunt Parser from Pig > to parse Pig scripts and then write the code to convert Pig logical plans > into Calcite logical plans. So technically, we can translating any Pig > scripts into Calcite plans (and then into SQL). > > The interesting points about the conversion are about handling Pig UDFs, > Pig flexible schemas. and converting Pig nested foreach operators > > The code will be tested in LinkedIn production. Just want to check if there > are interests in the community so that I can ask for permission to public > the code. > > Thanks, > Khai
