Re: Beam Summits!

I think for EU there is a proposal to have it next to Berlin Buzzwords in June. That would provide better spacing and avoid conflict with ApacheCon. Thomas On Wed, Dec 19, 2018 at 3:09 PM Suneel Marthi wrote: > How about Beam Summit in Berlin on Sep 6 immediately following Flink > Forward Berl

Re: ElementCount PR + Proper package locations for file visibility in my element count PR.

High level point: the only SDK implementation that any of these could be relevant for is Java, so "visible to SDK implementations" is just "visible to the Java SDK Harness" (sdks/java/core must not depend on any of this) Also sdks/java/harness is not a library, but a (containerized) service. So it

ElementCount PR + Proper package locations for file visibility in my element count PR.

Hello Robert + beam community, I have added the element count metrics to the Java SDK in this PR. In doing so, I enhanced the Metrics.counter call to have a LabeledMetrics.counter() call which allows constructing a metric for a MonitoringInfo urn and set of labels, so they can be extracted properl

Re: [VOTE] Release Vendored gRPC 1.13.1 v0.2, release candidate #1

+1 - sigs good - `jar tf` looks good On Wed, Dec 19, 2018 at 7:54 PM Scott Wegner wrote: > Please review and vote on the release candidate #1 for the vendored > artifact gRPC 1.13.1 v0.2 > [ ] +1, Approve the release > [ ] -1, Do not approve the release (please provide specific comments) > >

[VOTE] Release Vendored gRPC 1.13.1 v0.2, release candidate #1

Please review and vote on the release candidate #1 for the vendored artifact gRPC 1.13.1 v0.2 [ ] +1, Approve the release [ ] -1, Do not approve the release (please provide specific comments) This is a follow-up to the previous thread about vendoring updates [1] The complete staging area is avail

Re: Beam Summits!

How about Beam Summit in Berlin on Sep 6 immediately following Flink Forward Berlin on the previous 2 days. Same may be for Asia also following Flink Forward Asia where and whenever it happens. On Wed, Dec 19, 2018 at 6:06 PM Austin Bennett wrote: > Hi All, > > I really enjoyed Beam Summit in L

Beam Summits!

Hi All, I really enjoyed Beam Summit in London (Thanks Matthias!), and there was much enthusiasm for continuations. We had selected that location in a large part due to the growing community there, and we have users in a variety of locations. In our 2019 calendar, https://docs.google.com/spreads

Re: (fyi) PR#7324 updates how projects consume vendored guava

> Can testing of the artifact prior to release not be done against the staging repo? I suspect it could be. The process for making these updates has not yet been defined, and I agree that having this intermediate state on every update is not ideal. (also, I mistakenly wrote Guava in the above ema

Re: excessive java precommit logging

The gradle scan doesn't pinpoint the error message, and it doesn't contain all the lines: https://scans.gradle.com/s/ckhjrjdexpuzm/console-log The logs might be useful, but usually not from passing tests. Doesn't gradle log output from failed tests by default? On Wed, Dec 19, 2018 at 1:22 PM Thom

Re: (fyi) PR#7324 updates how projects consume vendored guava

Hi Scott, Can testing of the artifact prior to release not be done against the staging repo? See here for an example: https://github.com/apache/beam/pull/7322/files It is unfortunate that we have to change the imports on every patch. Is it not sufficient to just update the artifact version? Tho

(fyi) PR#7324 updates how projects consume vendored guava

Just a heads up: PR#7324 [1] just got merged which fixes an issue with our vendored guava artifact and JNI: see BEAM-6056 [2] for details. Important changes for you: a. IntelliJ doesn't understand how our vendored guava is built and will give red squigglies. This is annoying but temporary while we

Re: excessive java precommit logging

I usually follow the download procedure outlined by Scott to look at the logs. These logs are big, but when there is a problem it is sometimes essential to have the extra output, especially for less frequent flakes. Reducing logs would then require the author to add extra logging to the PR (and a

Re: excessive java precommit logging

I'm not sure what we lose by dropping the --info flag, but I generally worry about reducing log output since logs are the main resource for diagnosing Jenkins build errors. It seems the issue is that Chrome doesn't scale well to large log files. A few alternative solutions: 1. Use the produced Bu

excessive java precommit logging

Hi all, I'd like to reduce precommit log sizes on Jenkins. For example: https://builds.apache.org/job/beam_PreCommit_Java_Commit/3181/consoleFull is 79M, which makes Chrome sluggish to use on it (tab is constantly using a whole cpu core). I know this might be controversial, but I'd like to propose

Re: Beam Contribution

Hi Theo, and welcome! I have added you to the "Contributors" role on Beam Jira, so you should be able to assign tickets to yourself if you want. Kenn On Wed, Dec 19, 2018 at 12:00 PM Theodore Siu wrote: > To whom it may concern, > > My name is Theodore Siu. I am a customer facing data engineer

Beam Contribution

To whom it may concern, My name is Theodore Siu. I am a customer facing data engineer at Google. I would like to be added as a contributor to Beam. My Jira user name is tsiu. Thank you for your time. -Theo

Re: [Fwd: [Apache Beam] Custom DataSourceV2 instanciation: parameters passing and Encoders]

Yes, this is thanks to these spark community meetings that I got the name of Ryan. And, indeed, when I saw the design sync meetings, I realized how recent the DataSourceV2 API is. I think you are right, I should wait for it to be finished and in the meantime use V1. Etienne Le mercredi 19 décembr

Re: [Fwd: [Apache Beam] Custom DataSourceV2 instanciation: parameters passing and Encoders]

The Spark community has been holding a weekly sync meeting on DataSourceV2 and sharing notes back to their dev list  https://lists.apache.org/list.html?d...@spark.apache.org:lte=3M:DataSourceV2%20sync. At this time, there are still some moving pieces at Spark’s side. Is it too early to target Da

Re: [RFC] I made a new tabbed Beam view in Jenkins

Thanks Kenn. Very useful. +1 for making it the default view! -Max On 18.12.18 23:12, Mark Liu wrote: That looks great. Thank you! Mark On Tue, Dec 18, 2018 at 1:53 PM Jason Kuster > wrote: Oh, a good fact! Thanks for the info. :) On Tue, Dec 18, 2018

Re: [Fwd: [Apache Beam] Custom DataSourceV2 instanciation: parameters passing and Encoders]

Thanks Kenn for taking the time to take a look Le mardi 18 décembre 2018 à 11:39 -0500, Kenneth Knowles a écrit : > I don't know DataSourceV2 well, but I am reading around to try to help. I see > the problem with the SparkSession API. > Is there no other way to instantiate a DataSourceV2 and read