Thanks Julian. @Chris, I'll update the patch with Calcite version change.
On Thu, Feb 12, 2015 at 1:56 PM, Julian Hyde <julianh...@gmail.com> wrote: > +1 > > I've pushed a snapshot, versioned "1.1-chi-incubating-SNAPSHOT", and based > on > https://github.com/julianhyde/incubator-calcite/commit/b8a91c31859b6901976de8dcad43afd4877cad36 > . > > See > https://repository.apache.org/content/repositories/snapshots/org/apache/calcite/calcite/ > > > > On Feb 12, 2015, at 9:26 AM, Jakob Homan <jgho...@gmail.com> wrote: > > > > +1, assuming we're keeping RTC on the branch (, Jakob said pedantically). > > > > On 12 February 2015 at 08:25, Milinda Pathirage <mpath...@umail.iu.edu> > wrote: > >> I am +1 for this approach. > >> > >> Milinda > >> > >> On Thu, Feb 12, 2015 at 11:22 AM, Chris Riccomini < > criccom...@apache.org> > >> wrote: > >> > >>> Hey Julian, > >>> > >>>> The rule of thumb I’ve learned integrating Calcite into Hive is that > only > >>> a branch should depend on snapshot versions of another component > >>> > >>> This is very practical. I hadn't considered the Calcite dependency > issue > >>> when I pushed for merging into master rather than a branch. > >>> > >>> Based on this, it seems like the best thing to do is: > >>> > >>> 1) Work off of Calcite SNAPSHOT Maven publication. > >>> 2) Create a samza-sql branch. > >>> 3) Migrate all existing samza-sql commits from master into the > samza-sql > >>> branch. > >>> > >>> Sorry for the churn on this all. I hadn't considered the binary > dependency > >>> issue. Is everyone OK with this? > >>> > >>> Cheers, > >>> Chris > >>> > >>> On Wed, Feb 11, 2015 at 10:51 AM, Julian Hyde <jul...@hydromatic.net> > >>> wrote: > >>> > >>>> This seems more like a branch than a classifier. Calcite is > developing in > >>>> a branch, and would produce snapshots from that branch. The rule of > thumb > >>>> I’ve learned integrating Calcite into Hive is that only a branch > should > >>>> depend on snapshot versions of another component. (Hive broke this > rule > >>> and > >>>> got into a sticky situation where their trunk depended on a Calcite > >>>> snapshot and they couldn’t release because Calcite was stuck in > >>> pre-release > >>>> vote limbo.) > >>>> > >>>> I’m going to try to produce maven artifacts with {groupId: > >>>> “org.apache.calcite”, version: "1.1.0-stream-incubating-SNAPSHOT”, > >>>> artifactId: “calcite-core”, “calcite-avatica”, etc.} and push them to > >>>> Apache nexus. I do recommend that you use it on a branch. > >>>> > >>>> However, I don’t think there would be a problem getting the > functionality > >>>> into calcite’s master branch in time for Calcite’s next release, in > >>> about 4 > >>>> weeks. (We commit to releasing approximately monthly.) The > functionality > >>>> and APIs would be marked “experimental, subject to change” but at > least > >>> you > >>>> would be able to merge your work to master at that point. Doubtless > you > >>>> would need new features & bug fixes in Calcite after that, and so work > >>> that > >>>> depended on that would have to stay in a branch until Calcite was > >>> released. > >>>> > >>>> Julian > >>>> > >>>> On Feb 11, 2015, at 10:34 AM, Felix GV <fville...@linkedin.com.INVALID > > > >>>> wrote: > >>>> > >>>>> I guess one concern with Calcite SNAPSHOT releases is that they imply > >>>> they'll be included in the next release, which may not be the case if > >>>> they're still considered very experimental. > >>>>> > >>>>> Alternatively, perhaps Julian can create a new version name with a > >>>> classifier for streaming. > >>>>> > >>>>> > >>>> > >>> > http://maven.apache.org/plugins/maven-deploy-plugin/examples/deploying-with-classifiers.html > >>>>> > >>>>> Then he could release SNAPSHOT artifacts for both the main/master > code > >>>> base as well as the streaming branch, no matter how experimental it > is... > >>>>> > >>>>> -- > >>>>> > >>>>> Felix GV > >>>>> Data Infrastructure Engineer > >>>>> Distributed Data Systems > >>>>> LinkedIn > >>>>> > >>>>> f...@linkedin.com > >>>>> linkedin.com/in/felixgv > >>>>> > >>>>> ________________________________________ > >>>>> From: Chris Riccomini [criccom...@apache.org] > >>>>> Sent: Wednesday, February 11, 2015 10:23 AM > >>>>> To: Chris Riccomini > >>>>> Cc: dev@samza.apache.org > >>>>> Subject: Re: [DISCUSS] SQL workflow > >>>>> > >>>>> Hey Julian, > >>>>> > >>>>> It looks like Calcite is at least setup to publish to Maven: > >>>>> > >>>>> https://repository.apache.org/#nexus-search;quick~calcite > >>>>> > >>>>> Julian, can you publish SNAPSHOT releases with your streaming changes > >>> to > >>>> it? > >>>>> > >>>>> Thanks! > >>>>> Chris > >>>>> > >>>>> On Wed, Feb 11, 2015 at 10:08 AM, Chris Riccomini < > >>> criccom...@apache.org > >>>>> > >>>>> wrote: > >>>>> > >>>>>> Hey Milinda, > >>>>>> > >>>>>> I've just committed SAMZA-484, so you should be able to re-base and > >>> get > >>>>>> all the latest code. > >>>>>> > >>>>>>> But we need to add Calcite as a dependency and Calcite streaming > >>>>>> support is only in Julian's branch > >>>>>> > >>>>>> We'll have to have Julian publish a release to Apache. This can be a > >>>>>> SNAPSHOT to Apache's SNAPSHOT repo, or a real Calcite release. We > >>> can't > >>>>>> check in binaries to the repo (licensing, release complexity, and > just > >>>>>> generally a bad idea). We'll have to coordinate with him. > >>>>>> > >>>>>> Cheers, > >>>>>> Chris > >>>>>> > >>>>>> On Wed, Feb 11, 2015 at 9:43 AM, Milinda Pathirage < > >>>> mpath...@umail.iu.edu> > >>>>>> wrote: > >>>>>> > >>>>>>> No, SAMZA-483 doesn't depend on SAMZA-484. But we need to add > Calcite > >>>> as a > >>>>>>> dependency and Calcite streaming support is only in Julian's > branch ( > >>>>>>> https://github.com/julianhyde/incubator-calcite/tree/chi). What > can > >>>> we do > >>>>>>> about that? > >>>>>>> > >>>>>>> Milinda > >>>>>>> > >>>>>>> On Wed, Feb 11, 2015 at 12:30 PM, Chris Riccomini < > >>>> criccom...@apache.org> > >>>>>>> wrote: > >>>>>>> > >>>>>>>> Hey Milinda, > >>>>>>>> > >>>>>>>> That'd be great. Do you have any dependency on SAMZA-484? I > haven't > >>>> had > >>>>>>> a > >>>>>>>> chance to commit that one yet. > >>>>>>>> > >>>>>>>> Cheers, > >>>>>>>> Chris > >>>>>>>> > >>>>>>>> On Wed, Feb 11, 2015 at 9:23 AM, Milinda Pathirage < > >>>>>>> mpath...@umail.iu.edu> > >>>>>>>> wrote: > >>>>>>>> > >>>>>>>>> /need/want > >>>>>>>>> > >>>>>>>>> Milinda > >>>>>>>>> > >>>>>>>>> On Wed, Feb 11, 2015 at 12:22 PM, Milinda Pathirage < > >>>>>>>> mpath...@umail.iu.edu > >>>>>>>>>> > >>>>>>>>> wrote: > >>>>>>>>> > >>>>>>>>>> Hi Chris, > >>>>>>>>>> > >>>>>>>>>> Do you need me to create the SAMZA-483 patch against latest > master > >>>>>>> with > >>>>>>>>>> SAMZA-482 patch? I think that will make it easier to review the > >>>>>>> patch? > >>>>>>>>>> > >>>>>>>>>> Thanks > >>>>>>>>>> Milinda > >>>>>>>>>> > >>>>>>>>>> On Mon, Feb 9, 2015 at 11:39 AM, Chris Riccomini < > >>>>>>>> criccom...@apache.org> > >>>>>>>>>> wrote: > >>>>>>>>>> > >>>>>>>>>>> Cool. Looks like we've got consensus. I'll move forward with > RTC > >>> on > >>>>>>>> some > >>>>>>>>>>> of > >>>>>>>>>>> the early SAMZA-390 tickets. (SAMZA-482, SAMZA-483, SAMZA-484) > >>>>>>>>>>> > >>>>>>>>>>> On Mon, Feb 9, 2015 at 8:35 AM, Yan Fang <yanfang...@gmail.com > > > >>>>>>>> wrote: > >>>>>>>>>>> > >>>>>>>>>>>> +1 on this. > >>>>>>>>>>>> > >>>>>>>>>>>> Fang, Yan > >>>>>>>>>>>> yanfang...@gmail.com > >>>>>>>>>>>> +1 (206) 849-4108 > >>>>>>>>>>>> > >>>>>>>>>>>> On Fri, Feb 6, 2015 at 4:38 PM, Chris Riccomini < > >>>>>>>>> criccom...@apache.org> > >>>>>>>>>>>> wrote: > >>>>>>>>>>>> > >>>>>>>>>>>>> Hey all, > >>>>>>>>>>>>> > >>>>>>>>>>>>> Are we +1 on this? I think Jakob was the only one who was > >>>>>>> curious > >>>>>>>>>>> about > >>>>>>>>>>>> it. > >>>>>>>>>>>>> > >>>>>>>>>>>>> Cheers, > >>>>>>>>>>>>> Chris > >>>>>>>>>>>>> > >>>>>>>>>>>>> On Thu, Feb 5, 2015 at 1:22 PM, Yi Pan <nickpa...@gmail.com> > >>>>>>>> wrote: > >>>>>>>>>>>>> > >>>>>>>>>>>>>> Hi, Jakob, > >>>>>>>>>>>>>> > >>>>>>>>>>>>>> > >>>>>>>>>>>>>>>> Eh? Not sure what this means... > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> I mean SAMZA-484 depends on SAMZA-482, and neither are > >>>>>>>>> committed. > >>>>>>>>>>> So > >>>>>>>>>>>>>> Navina > >>>>>>>>>>>>>>> is having to post Yi's patch, as well as her own, on the > >>>>>>> JIRA. > >>>>>>>>> It > >>>>>>>>>>>> makes > >>>>>>>>>>>>>> it > >>>>>>>>>>>>>>> really hard to do code reviews because you can't tell > >>>>>>> whether > >>>>>>>> Yi > >>>>>>>>>>> made > >>>>>>>>>>>>> the > >>>>>>>>>>>>>>> changes or Navina did. > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>> Just to add to the point. It is also difficult to always see > >>>>>>> a > >>>>>>>>> long > >>>>>>>>>>>> list > >>>>>>>>>>>>> of > >>>>>>>>>>>>>> changed files if the RB request is always based on the > >>>>>>> master. > >>>>>>>> It > >>>>>>>>> is > >>>>>>>>>>>>>> possible to have RB request based on another RB request (I > >>>>>>> have > >>>>>>>>>>> tried > >>>>>>>>>>>> it > >>>>>>>>>>>>>> before). But what happens if the base RB request is > >>>>>>>>>>>> cancelled/discarded? > >>>>>>>>>>>>> RB > >>>>>>>>>>>>>> is not designed to track the revision changes in a > dependency > >>>>>>>>> chain. > >>>>>>>>>>>>>> > >>>>>>>>>>>>>> > >>>>>>>>>>>>>> > >>>>>>>>>>>>>>> Cheers, > >>>>>>>>>>>>>>> Chris > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> On Thu, Feb 5, 2015 at 12:16 PM, Jakob Homan < > >>>>>>>> jgho...@gmail.com > >>>>>>>>>> > >>>>>>>>>>>>> wrote: > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>> I want to avoid branches, > >>>>>>>>>>>>>>>> Just curious, any reason for this? > >>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>> and I also want to avoid revision control over JIRA > >>>>>>>>>>>>>>>> Eh? Not sure what this means... > >>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>> Thanks, > >>>>>>>>>>>>>>>> jg > >>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>> On 4 February 2015 at 17:11, Chris Riccomini < > >>>>>>>>>>>> criccom...@apache.org> > >>>>>>>>>>>>>>>> wrote: > >>>>>>>>>>>>>>>>> Hey all, > >>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>> @Jakob, yeah I was thinking we'll follow our normal > >>>>>>> flow. > >>>>>>>>>>> RTC. I > >>>>>>>>>>>>> just > >>>>>>>>>>>>>>>>> wanted to set expectation that the code committed > >>>>>>> might be > >>>>>>>>>>> not up > >>>>>>>>>>>>> to > >>>>>>>>>>>>>>> our > >>>>>>>>>>>>>>>>> normal quality initially (missing docs, no tests, etc). > >>>>>>>>> Until > >>>>>>>>>>> the > >>>>>>>>>>>>>>> quality > >>>>>>>>>>>>>>>>> is raised, we should think of this module as > >>>>>>> experimental. > >>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>> @Milinda, awesome! Thanks. :) > >>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>> Cheers, > >>>>>>>>>>>>>>>>> Chris > >>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>> On Wed, Feb 4, 2015 at 11:57 AM, Milinda Pathirage < > >>>>>>>>>>>>>>>> mpath...@umail.iu.edu> > >>>>>>>>>>>>>>>>> wrote: > >>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>> Hi Chris, > >>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>> Hope we no longer need the SQL API. I'll create a RB > >>>>>>> for > >>>>>>>>>>> Calcite > >>>>>>>>>>>>>>>>>> integration. > >>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>> Thanks > >>>>>>>>>>>>>>>>>> Milinda > >>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>> On Wed, Feb 4, 2015 at 1:31 PM, Chris Riccomini < > >>>>>>>>>>>>>>> criccom...@apache.org> > >>>>>>>>>>>>>>>>>> wrote: > >>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>> I think so. There was some RB downtime, but it just > >>>>>>> got > >>>>>>>>>>> fixed. > >>>>>>>>>>>>> Yi, > >>>>>>>>>>>>>>>>>> Navina, > >>>>>>>>>>>>>>>>>>> Milinda, can you make sure your JIRAs have up to > >>>>>>> date > >>>>>>>>> RBs? > >>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>> On Wed, Feb 4, 2015 at 10:24 AM, sriram < > >>>>>>>>>>> sriram....@gmail.com > >>>>>>>>>>>>> > >>>>>>>>>>>>>>> wrote: > >>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>> Can we have updated RBs for all the three sub > >>>>>>> tasks > >>>>>>>>>>> before > >>>>>>>>>>>> we > >>>>>>>>>>>>>>>> commit? > >>>>>>>>>>>>>>>>>>> This > >>>>>>>>>>>>>>>>>>>> would help us to review even after we commit. > >>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>> On Wed, Feb 4, 2015 at 10:15 AM, Chris Riccomini < > >>>>>>>>>>>>>>>>>> criccom...@apache.org> > >>>>>>>>>>>>>>>>>>>> wrote: > >>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>> Hey all, > >>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>> Yi, Navina, and Milinda have been working on > >>>>>>>>> SAMZA-390 > >>>>>>>>>>>>>>> sub-tickets > >>>>>>>>>>>>>>>>>>>> related > >>>>>>>>>>>>>>>>>>>>> to SQL operators. We're getting to the point > >>>>>>> where > >>>>>>>>> the > >>>>>>>>>>>>> amount > >>>>>>>>>>>>>> of > >>>>>>>>>>>>>>>> work > >>>>>>>>>>>>>>>>>>>>> floating around is quite large, and some tickets > >>>>>>>>> build > >>>>>>>>>>> off > >>>>>>>>>>>>> of > >>>>>>>>>>>>>>>> others. > >>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>> I'm proposing that we commit this work into a > >>>>>>>>> samza-sql > >>>>>>>>>>>>>>> submodule > >>>>>>>>>>>>>>>> on > >>>>>>>>>>>>>>>>>>>>> master, and treat this module as experimental. I > >>>>>>>> want > >>>>>>>>>>> to > >>>>>>>>>>>>> avoid > >>>>>>>>>>>>>>>>>>> branches, > >>>>>>>>>>>>>>>>>>>>> and I also want to avoid revision control over > >>>>>>>> JIRA. > >>>>>>>>>>> This > >>>>>>>>>>>>>> means > >>>>>>>>>>>>>>>> that > >>>>>>>>>>>>>>>>>>>> there > >>>>>>>>>>>>>>>>>>>>> will probably be a fair amount of commits/JIRAs > >>>>>>> on > >>>>>>>>> this > >>>>>>>>>>>>> module > >>>>>>>>>>>>>>> as > >>>>>>>>>>>>>>>> we > >>>>>>>>>>>>>>>>>>>>> iterate, but I think that's OK. > >>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>> Does this sound good to everyone? > >>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>> Cheers, > >>>>>>>>>>>>>>>>>>>>> Chris > >>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>> -- > >>>>>>>>>>>>>>>>>> Milinda Pathirage > >>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>> PhD Student | Research Assistant > >>>>>>>>>>>>>>>>>> School of Informatics and Computing | Data to Insight > >>>>>>>>> Center > >>>>>>>>>>>>>>>>>> Indiana University > >>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>> twitter: milindalakmal > >>>>>>>>>>>>>>>>>> skype: milinda.pathirage > >>>>>>>>>>>>>>>>>> blog: http://milinda.pathirage.org > >>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>> > >>>>>>>>>>>>> > >>>>>>>>>>>> > >>>>>>>>>>> > >>>>>>>>>> > >>>>>>>>>> > >>>>>>>>>> > >>>>>>>>>> -- > >>>>>>>>>> Milinda Pathirage > >>>>>>>>>> > >>>>>>>>>> PhD Student | Research Assistant > >>>>>>>>>> School of Informatics and Computing | Data to Insight Center > >>>>>>>>>> Indiana University > >>>>>>>>>> > >>>>>>>>>> twitter: milindalakmal > >>>>>>>>>> skype: milinda.pathirage > >>>>>>>>>> blog: http://milinda.pathirage.org > >>>>>>>>>> > >>>>>>>>> > >>>>>>>>> > >>>>>>>>> > >>>>>>>>> -- > >>>>>>>>> Milinda Pathirage > >>>>>>>>> > >>>>>>>>> PhD Student | Research Assistant > >>>>>>>>> School of Informatics and Computing | Data to Insight Center > >>>>>>>>> Indiana University > >>>>>>>>> > >>>>>>>>> twitter: milindalakmal > >>>>>>>>> skype: milinda.pathirage > >>>>>>>>> blog: http://milinda.pathirage.org > >>>>>>>>> > >>>>>>>> > >>>>>>> > >>>>>>> > >>>>>>> > >>>>>>> -- > >>>>>>> Milinda Pathirage > >>>>>>> > >>>>>>> PhD Student | Research Assistant > >>>>>>> School of Informatics and Computing | Data to Insight Center > >>>>>>> Indiana University > >>>>>>> > >>>>>>> twitter: milindalakmal > >>>>>>> skype: milinda.pathirage > >>>>>>> blog: http://milinda.pathirage.org > >>>>>>> > >>>>>> > >>>>>> > >>>> > >>>> > >>> > >> > >> > >> > >> -- > >> Milinda Pathirage > >> > >> PhD Student | Research Assistant > >> School of Informatics and Computing | Data to Insight Center > >> Indiana University > >> > >> twitter: milindalakmal > >> skype: milinda.pathirage > >> blog: http://milinda.pathirage.org > > -- Milinda Pathirage PhD Student | Research Assistant School of Informatics and Computing | Data to Insight Center Indiana University twitter: milindalakmal skype: milinda.pathirage blog: http://milinda.pathirage.org
