Re: [VOTE] Release 2.9.0, release candidate #1

+1 Turns out we broke DOUBLE on purpose. Updated the demo to use DECIMAL and it doesn't hard fail. This is a docs bug. On Wed, Dec 12, 2018 at 3:55 PM Scott Wegner wrote: > +1 > > I verified the Java examples succeed on DirectRunner. > > On Wed, Dec 12, 2018 at 3:30 PM Chamikara Jayalath > wro

Re: [VOTE] Release 2.9.0, release candidate #1

+1 I verified the Java examples succeed on DirectRunner. On Wed, Dec 12, 2018 at 3:30 PM Chamikara Jayalath wrote: > Thanks Andrew. Please make this a blocker and -1 the thread if you think > we need a new RC. > > - Cham > > On Wed, Dec 12, 2018 at 3:27 PM Andrew Pilloud > wrote: > >> I was ju

Re: [VOTE] Release 2.9.0, release candidate #1

Thanks Andrew. Please make this a blocker and -1 the thread if you think we need a new RC. - Cham On Wed, Dec 12, 2018 at 3:27 PM Andrew Pilloud wrote: > I was just running the Beam SQL demo. I found one query fails with "the > keyCoder of a GroupByKey must be deterministic" and another just ha

Re: [VOTE] Release 2.9.0, release candidate #1

I was just running the Beam SQL demo. I found one query fails with "the keyCoder of a GroupByKey must be deterministic" and another just hangs. I opened an issue: https://issues.apache.org/jira/browse/BEAM-6224 Not sure if this calls for canceling the release or just a release note (SQL is still ex

Re: Issue with publishing maven artefacts locally

Not exactly sure if this is the reason but I noticed that Ismaël's command above result in a beam-sdks-java-core-2.10.0-20181212.232426-1.jar instead of a beam-sdks-java-core-2.10.0-SNAPSHOT.jar. - Cham On Wed, Dec 12, 2018 at 1:16 PM Ismaël Mejía wrote: > Thanks Garrett for the quick fix. I ju

Re: [DISCUSS] Structuring Java based DSLs

I'll send an update on schemas soon. But the tl;dr is that by the end of this month, I expect it to be generally usable across a variety of input formats. Reuven On Wed, Dec 12, 2018 at 9:38 AM Xinyu Liu wrote: > Agree with Kenn on this. From our SamzaRunner point of view, we would like > Beam

Re: Issue with publishing maven artefacts locally

Thanks Garrett for the quick fix. I just tested and it is working now. I found a second issue (not related to Garrett's PR, it was the reason why I detected that local artifacts were not updated in our jenkins (in the other thread). To validate that our daily snapshots don't break existing code w

Re: ULR Tests on commit?

Woohoo! Thanks for the info. On Wed, Dec 12, 2018, 2:26 PM Daniel Oliveira wrote: > Yeah, this is in-progress. The tests should get in for various languages > throughout the next two weeks and I'll add them to the PR template as I add > them. > > I do have a JIRA for tracking (BEAM-5449 >

Re: Issue with publishing maven artefacts locally

I'm inclined to undo a particular modification I made in my PR and re-duplicate the repositories declaration between the Gradle plugin and the new BOM module. Scott, what do you think? On Wed, Dec 12, 2018 at 11:00 AM Scott Wegner wrote: > Thanks for pointing this out Alexy. This seems like we

Re: Issue with publishing maven artefacts locally

Nevermind, I found a much easier fix (delete two characters): https://github.com/apache/beam/pull/7265 On Wed, Dec 12, 2018 at 11:03 AM Garrett Jones wrote: > I'm inclined to undo a particular modification I made in my PR and > re-duplicate the repositories declaration between the Gradle plugin

Re: Issue with publishing maven artefacts locally

On Wed, Dec 12, 2018 at 11:00 AM Scott Wegner wrote: > Thanks for pointing this out Alexy. This seems like we unintentionally > broke something in PR#7197 [1] > > +Garrett Jones , who authored the change. > Garrett can you help investigate? > > I went to check to see if we have any existing Jenki

Re: OOO

Thanks for all the work you've been doing on Beam, Luke! Hope you have some good bonding time and that it's not too hectic. On Wed, Dec 12, 2018 at 10:10 AM Kenneth Knowles wrote: > Congrats & have a super time! > > Kenn > > On Wed, Dec 12, 2018 at 10:09 AM Robert Burke wrote: > >> Have a great

Re: ULR Tests on commit?

Yeah, this is in-progress. The tests should get in for various languages throughout the next two weeks and I'll add them to the PR template as I add them. I do have a JIRA for tracking (BEAM-5449 ) but I haven't been updating it regularly.

Re: ULR Tests on commit?

+Daniel Oliveira who has been working on the ULR. I believe this is in-progress. Dan, do you have a JIRA for tracking? On Wed, Dec 12, 2018 at 10:08 AM Robert Burke wrote: > In our auto populated github PR template, we have a variety of SDK > languages to runner combos, but the Universal Local

Re: Issue with publishing maven artefacts locally

Thanks for pointing this out Alexy. This seems like we unintentionally broke something in PR#7197 [1] +Garrett Jones , who authored the change. Garrett can you help investigate? I went to check to see if we have any existing Jenkins jobs that would've caught this break. It seems the beam_Release_

Java performance tests dashboard

Hi Lukasz, I was looking for statistics on I/O performance for writes of many files (~10k) on GCS. I found this dashboard and I have some questions. 1. The tests that are "local filesystem" seem to be running on Dataflow

Re: OOO

Congrats & have a super time! Kenn On Wed, Dec 12, 2018 at 10:09 AM Robert Burke wrote: > Have a great bonding time! I'd say "break" but I expect you'll be quite > busy. > > On Wed, Dec 12, 2018, 9:57 AM Etienne Chauchot > wrote: > >> Enjoy your family time and take care of the little one >> >

Re: [Call for items] ❄️ December Beam Newsletter

Thanks Rose! Updated the list. On 12.12.18 18:41, Alexey Romanenko wrote: I've just added several things that were finished. Thanks for driving this! On 7 Dec 2018, at 23:39, Rose Nguyen > wrote: Hi folks: * * Time for the last newsletter of the year! * * *Add to

Re: OOO

Have a great bonding time! I'd say "break" but I expect you'll be quite busy. On Wed, Dec 12, 2018, 9:57 AM Etienne Chauchot wrote: > Enjoy your family time and take care of the little one > > Etienne > > Le mardi 11 décembre 2018 à 12:26 +0100, Maximilian Michels a écrit : > > Thank you for you

ULR Tests on commit?

In our auto populated github PR template, we have a variety of SDK languages to runner combos, but the Universal Local Runner (ULR) is absent. Do we currently run tests on the ULR as pre-commit or post commit? If not, why not? If so, can we add a ULR column to the PR template? Mostly curious. Th

Re: [Call for items] ❄️ December Beam Newsletter

I've just added several things that were finished. Thanks for driving this! > On 7 Dec 2018, at 23:39, Rose Nguyen wrote: > > Hi folks: > > Time for the last newsletter of the year! > > Add to [1] the highlights from November to now (or planned events and talks) > that you want to share by 1

Re: [DISCUSS] Structuring Java based DSLs

Agree with Kenn on this. From our SamzaRunner point of view, we would like Beam SQL to be self-contained and flexible enough for our users to use it in different scenarios, e.g. pure SQL and embeded in different SDKs. We are also extremely interested in the DataFrame-like API mentioned above. To di

Re: [DISCUSS] Structuring Java based DSLs

Hi all, after letting this sink for a while, I'd like to summarize the feedback and emphasize some questions that appeared:  a) there were several 'it makes sense' opinions  b) there was one 'not right now' - which makes sense, but the purpose of this discussion was to try to first answer th

Re: Beam snapshots broken

Beam9 is offline right now. But, the job also failed on beam4 and 13 with "Could not determine the dependencies of task ':beam-sdks-python:test.". Seems like the task dependency did not setup properly. On Wed, Dec 12, 2018 at 2:03 AM Ismaël Mejía wrote: > You are right it seems that it was rel

Re: OOO

Enjoy your family time and take care of the little one Etienne Le mardi 11 décembre 2018 à 12:26 +0100, Maximilian Michels a écrit : > Thank you for your amazing work on Beam. > Enjoy the time with your kid! > -Max > On 11.12.18 00:55, Pablo Estrada wrote: > See ya in three months! Take it easy! >

Re: Report to the Board, December 2018

Hi Kenn, LGTM also EtienneLe mercredi 12 décembre 2018 à 10:13 +0100, Robert Bradshaw a écrit : > Mostly looks good to me. I would probably omit > the{issues,builds}@beam.apache.org stats as "nothing significant in > thefigures" but note that the dev list excludes the previous automatedemails > (

Re: Report to the Board, December 2018

Hi Kenn,LGTM alsoLe mercredi 12 décembre 2018 à 10:13 +0100, Robert Bradshaw a écrit : > Mostly looks good to me. I would probably omit > the{issues,builds}@beam.apache.org stats as "nothing significant in > thefigures" but note that the dev list excludes the previous automatedemails > (or at le

Issue with publishing maven artefacts locally

Hi all, I used to publish maven artefacts into local repository using this kind of command for example: ./gradlew -Ppublishing --no-parallel -PdistMgmtSnapshotsUrl=file:///path/to/.m2/repository/ -p sdks/java/io/kafka/ publish It worked fine till today. Seems like (according to "git bisect”)

Re: contributor in the Beam

Hi, I have another pull request on MongoIO: https://github.com/apache/beam/pull/7256 thanks chaim On Mon, Dec 3, 2018 at 11:36 AM Jean-Baptiste Onofré wrote: > > Can you please fix the conflict in the PR ? > > Thanks > Regards > JB > > On 03/12/2018 08:52, Chaim Turkel wrote: > > it looks like t

Re: Beam snapshots broken

You are right it seems that it was related to beam9 (wondering if it was bad luck that it was always assigned to beam9 or we can improve that poor balancing error). However it failed again today against beam13 maybe this time is just a build issue but seems related to python too. On Tue, Dec 11, 2

Re: Report to the Board, December 2018

Mostly looks good to me. I would probably omit the {issues,builds}@beam.apache.org stats as "nothing significant in the figures" but note that the dev list excludes the previous automated emails (or at least a reference to where you say this above). Is it worth noting StackOverflow activity here in

Re: GSOC - Summer of Code, on Beam?

Oh I had not seen that the announce was official, so time to get ready. https://opensource.googleblog.com/2018/11/google-summer-of-code-15-years-strong.html Mentors should have proposals ready around January 15, 2019. Remember timeline matters. https://developers.google.com/open-source/gsoc/timeli