Summary of IRC Meeting in #aurora at Mon Oct 20 18:02:38 2014: Attendees: davmclau, dnorris, teodimoff, jfarrell, jcohen, wfarner, mchucarroll, Yasumoto, kts, mkhutornenko, dlester
- Preface - 0.6.0 release - Action: remove v2 client tickets from AURORA-711 blockers list - Python tooling - Doc day recap - Mesos egg publication IRC log follows: ## Preface ## [Mon Oct 20 18:02:51 2014] <wfarner>: roll call? [Mon Oct 20 18:02:52 2014] <wfarner>: here [Mon Oct 20 18:02:54 2014] <mchucarroll>: here [Mon Oct 20 18:02:55 2014] <kts>: here [Mon Oct 20 18:02:56 2014] <mkhutornenko>: here [Mon Oct 20 18:03:02 2014] <davmclau>: here [Mon Oct 20 18:03:06 2014] <jcohen>: here [Mon Oct 20 18:03:23 2014] <jfarrell>: here ## 0.6.0 release ## [Mon Oct 20 18:04:03 2014] <wfarner>: AURORA-711 [Mon Oct 20 18:04:27 2014] <wfarner>: unfortunately we have not made much progress over the last week in getting closer to the release [Mon Oct 20 18:04:41 2014] <kts>: added a new blocker [Mon Oct 20 18:04:52 2014] <kts>: AURORA-863 [Mon Oct 20 18:05:32 2014] <wfarner>: i would like to propose that we defer phasing out the v1 client until the next release [Mon Oct 20 18:05:52 2014] <wfarner>: that cuts out a lot of what has crept in to 0.6.0 at the last minute [Mon Oct 20 18:06:19 2014] <wfarner>: +/-1? [Mon Oct 20 18:06:39 2014] <kts>: +1, more important to get on a release cadence than to ship any particular feature [Mon Oct 20 18:07:06 2014] <davmclau>: + 1 [Mon Oct 20 18:07:22 2014] <mkhutornenko>: +1 [Mon Oct 20 18:07:29 2014] <jcohen>: +1, but would be nice to follow up with 0.6.1, or 0.7.0 ASAP [Mon Oct 20 18:07:37 2014] <jfarrell>: agree, we can always do a release just focused on the switch over to v2 [Mon Oct 20 18:09:00 2014] <wfarner>: #action remove v2 client tickets from AURORA-711 blockers list [Mon Oct 20 18:09:27 2014] <wfarner>: (whoops, that's an action for me) ## Python tooling ## [Mon Oct 20 18:10:12 2014] <kts>: I started a thread on the mailing list http://mail-archives.apache.org/mod_mbox/incubator-aurora-dev/201410.mbox/%3CCAAATh-ZwSmatUzsDKnD5ChdbN1NP_2ORoDnKh0ueRr8DYY3XFw%40mail.gmail.com%3E [Mon Oct 20 18:10:13 2014] <wfarner>: relevant thread: http://mail-archives.apache.org/mod_mbox/incubator-aurora-dev/201410.mbox/%3CCAAATh-ZwSmatUzsDKnD5ChdbN1NP_2ORoDnKh0ueRr8DYY3XFw%40mail.gmail.com%3E [Mon Oct 20 18:10:23 2014] <wfarner>: ah, kts you won the race, floor is yours [Mon Oct 20 18:10:35 2014] <kts>: thanks wfarner [Mon Oct 20 18:11:03 2014] <kts>: I want to gauge support for switching up the python build tooling to something more standard [Mon Oct 20 18:11:55 2014] <kts>: the main feature that there's no suitable replacement for is automatic thrift code generation [Mon Oct 20 18:12:20 2014] <davmclau>: I think saying anything is "standard" in Python build/package/distribution is misleading :) [Mon Oct 20 18:12:34 2014] <jfarrell>: why does there need to be automatic thrift code generation? [Mon Oct 20 18:13:06 2014] <jfarrell>: why not check the generated code in, its always generated the same for the given version in use [Mon Oct 20 18:13:23 2014] <teodimoff>: i like pants a lot .. [Mon Oct 20 18:14:06 2014] <jfarrell>: there is no really necessary need to have the thrift compiler unless you are changing something in the idl, which we already check the md5s of [Mon Oct 20 18:14:28 2014] <Yasumoto>: I'm a pretty staunch pants-supporter, but lately I've found we've needed to spend a lot of time (on the python-side) either wrangling with the build tool or fixing bugs in pants [Mon Oct 20 18:15:00 2014] <davmclau>: I'd like to see a more thorough list of where time is being spent/wasted in pants and how alternative tools solve the problem before having a vote [Mon Oct 20 18:15:03 2014] <mchucarroll>: and you think thatâs all going to disappear if we start uisng setuptools? [Mon Oct 20 18:15:19 2014] <davmclau>: right now I can only see "these other things are more standard" as a reason to switch [Mon Oct 20 18:15:23 2014] <davmclau>: so I'd be -1 [Mon Oct 20 18:15:35 2014] <Yasumoto>: Although pants recently had a few blog posts written, the community is holding back on further publication- which means that pants stays a hidden gem [Mon Oct 20 18:15:50 2014] <mchucarroll>: I think that dependency management in python is universally a mess. the problems weâve had have been dependency management issues that manifested in pants, because thatâs what we use. [Mon Oct 20 18:15:52 2014] <teodimoff>: thats ture BUILD.ini and config is in flux but i think changes are getting slowdown and concrete path is taken [Mon Oct 20 18:16:21 2014] <jcohen>: Part of the argument for moving away from pants is that newcomers to Aurora generally have nowhere to go for pants support but us, whereas more âstandardâ tools, they can go to the python community (e.g. stack overflow) for troubleshooting general problems. [Mon Oct 20 18:16:21 2014] <Yasumoto>: http://packaging.python.org lists the standards, and although pex is being highlighted there, the pushback from pants adding itself there concerns me [Mon Oct 20 18:16:48 2014] <jcohen>: (Iâm a fan of pants fwiw) [Mon Oct 20 18:17:12 2014] <wfarner>: i think it's safe to say all the aurora developers like pants, but we don't want to be the pants support team [Mon Oct 20 18:17:28 2014] <Yasumoto>: python is really trying to use the above link (packaging.python.org) to _create_ the standard. Companies are actively paying people to work together to improve these tools, and unless pants turns its direction around to work _with_ the python community, we become essentially the only OSS project using it [Mon Oct 20 18:18:12 2014] <Yasumoto>: I'm also concerned by this thread: https://github.com/pantsbuild/pants/issues/690 [Mon Oct 20 18:18:43 2014] <teodimoff>: i am not python guy but when i read the doc it gives pretty good startup [Mon Oct 20 18:18:56 2014] <wfarner>: kts: do you have a rough idea of how much time it would take to switch out this tooling? [Mon Oct 20 18:19:23 2014] <Yasumoto>: I'm perhaps being a bit over-reactionary, but I'd like the people writing our build tool to have an understanding + appreciation for rigor, and I find the "I think probably different versions are possibly ABI-compatible maybe" response to mean _we_ will need to do the work to figure it out and make pants work on 10.10 correctly. [Mon Oct 20 18:19:34 2014] <davmclau>: I'd like to see how these other tools solve the problems [Mon Oct 20 18:19:44 2014] <davmclau>: or even just more concrete examples of the actual problems [Mon Oct 20 18:20:19 2014] <kts>: wfarner: probably the first step is moving 3rdparty/ to the pants requirements.txt format [Mon Oct 20 18:20:37 2014] <kts>: then we can start configuring new tools in parallel [Mon Oct 20 18:20:54 2014] <kts>: I don't think there's a strong argument without a proof-of-concept [Mon Oct 20 18:21:43 2014] <jfarrell>: lets create a ticket for investigation/poc on this then [Mon Oct 20 18:21:53 2014] <wfarner>: davmclau: you can skim this jira search for a sampling of problems: https://issues.apache.org/jira/issues/?jql=project%20%3D%20AURORA%20and%20text~pants [Mon Oct 20 18:22:40 2014] <wfarner>: which is a stark contrast to the equivalent search for gradle, which is more about "we should use this feature" as opposed to "gradle is broken and flaky" [Mon Oct 20 18:22:53 2014] <jfarrell>: once we have a sample patch or more of an idea on effort we can vote on the dev@ list [Mon Oct 20 18:22:56 2014] <mkhutornenko>: kts: + 1 on a proof-of-concept, that'll definitely make a stronger argument (provided it works :) [Mon Oct 20 18:23:03 2014] <wfarner>: also +1 [Mon Oct 20 18:23:32 2014] <Yasumoto>: I think one of davmclau's points (correct me if I'm wrong) is there isn't a gradle-equivalent in python: setuptools, pip, etc also have a reputation for being flaky [Mon Oct 20 18:23:32 2014] <wfarner>: and i agree with jfarrell that we could check in the generated thrift in the POC, but would not want to live with that for long [Mon Oct 20 18:24:21 2014] <kts>: im +0 on checking in codegen output [Mon Oct 20 18:24:44 2014] <kts>: the ideal is definitely to have only the pristine sources (and let the build tool deal with the intermediate outputs) [Mon Oct 20 18:25:11 2014] <wfarner>: no argument there, i just also believe that perfect products never ship :-) [Mon Oct 20 18:25:14 2014] <dnorris>: Iâd be interested in helping out with a PoC [Mon Oct 20 18:25:21 2014] <wfarner>: dnorris: awesome [Mon Oct 20 18:25:29 2014] <kts>: looks like the first ticket is [Mon Oct 20 18:25:31 2014] <kts>: AURORA-617 [Mon Oct 20 18:25:42 2014] <kts>: that'll let us use a single source of dependencies from both sets of tools [Mon Oct 20 18:25:54 2014] <kts>: dnorris: wanna take that one? [Mon Oct 20 18:25:59 2014] <dnorris>: kts: sure [Mon Oct 20 18:26:30 2014] <kts>: wfarner: yeah, and we do have precedent with javascript (checking in bower output) [Mon Oct 20 18:26:31 2014] <jfarrell>: thanks dnorris, lets table this then until 617 has a review up [Mon Oct 20 18:26:58 2014] <dlester>: kts is AURORA-617 meant to be fix version 0.6.0? [Mon Oct 20 18:26:59 2014] <Yasumoto>: +1, sounds like a good path forward [Mon Oct 20 18:27:13 2014] <kts>: dlester: no I don't think so [Mon Oct 20 18:27:24 2014] <jfarrell>: dlester: lets not make that a blocker for this release [Mon Oct 20 18:27:38 2014] <wfarner>: topic seems to have settled, moving on to next topic unless anyone wants to continue [Mon Oct 20 18:27:49 2014] <dlester>: +1, just checking ## Doc day recap ## [Mon Oct 20 18:28:17 2014] <wfarner>: here's the collection of tickets we focused on: https://issues.apache.org/jira/issues/?jql=project%20%3D%20AURORA%20AND%20component%3Ddocumentation%20and%20labels%3Daurora-docday [Mon Oct 20 18:28:51 2014] <wfarner>: we resolved 6 doc tickets (some much larger in scope than others), and have 2 still in review [Mon Oct 20 18:29:25 2014] <wfarner>: thanks again to everyone who filed doc tickets, and chimed in with doc requests [Mon Oct 20 18:30:12 2014] <wfarner>: we also came out with some good ideas w.r.t. doc publishing to aurora.apache.org, which will be more important as we develop a more regular release cadence [Mon Oct 20 18:31:06 2014] <dlester>: AURORA-850 [Mon Oct 20 18:31:23 2014] <wfarner>: that's it from me, anybody else care to open a topic? [Mon Oct 20 18:31:38 2014] <jcohen>: do we want to talk about the mesos egg problems? [Mon Oct 20 18:31:39 2014] <dlester>: Just wanted to call attn to AURORA-850; I have a patch attached to the ticket which we could use for tagging docs on the website ## Mesos egg publication ## [Mon Oct 20 18:32:40 2014] <jfarrell>: i opened a ticket for this in mesos a while ago to make it available via pypi [Mon Oct 20 18:32:56 2014] <jfarrell>: i need to find some time and just go fix it [Mon Oct 20 18:33:12 2014] <kts>: AURORA-863 [Mon Oct 20 18:33:13 2014] <jcohen>: Iâve got nothing here, other than there seem to be problems getting this egg published properly which is a obviously blocker for people trying to set up Aurora. [Mon Oct 20 18:33:14 2014] <jfarrell>: until then i would say that we should just build one and put it in our svn [Mon Oct 20 18:33:44 2014] <kts>: im currently attempting to build a mesos egg to host in svn for our e2e test [Mon Oct 20 18:33:51 2014] <kts>: and running into [Mon Oct 20 18:33:52 2014] <kts>: MESOS-1010 [Mon Oct 20 18:34:19 2014] <jcohen>: https://issues.apache.org/jira/browse/MESOS-1010 Python extension build is broken if gflags-dev is installed [Mon Oct 20 18:35:32 2014] <kts>: the impact is that people can't build our executor without building mesos in just the right way [Mon Oct 20 18:35:41 2014] <kts>: and then teaching pants about it [Mon Oct 20 18:35:56 2014] <kts>: which requires us to be support across 3 build systems [Mon Oct 20 18:36:04 2014] <kts>: and takes hours [Mon Oct 20 18:36:12 2014] <kts>: which is not good for our adoption [Mon Oct 20 18:37:12 2014] <kts>: anyway I'm looking into it, follow that ticket for updates [Mon Oct 20 18:37:24 2014] <wfarner>: thanks kts [Mon Oct 20 18:37:33 2014] <jcohen>: +1, thanks kts [Mon Oct 20 18:37:36 2014] <wfarner>: last call for another topic [Mon Oct 20 18:38:31 2014] <wfarner>: ASFBot702: meeting stop Meeting ended at Mon Oct 20 18:38:31 2014
