Hi team,
Just my first very small contribution:
i. By using the SDK 2.5 python dev distribution JB shared, I have just
managed to successfully launch my own custom streaming job on Dataflow.
It's working fine.
ii. this custom streaming reads data from pub/sub, conducts some simple
transformations, then injects data into BigQuery.
FYI.
Cheers.
Jasper
On Sat, Jun 9, 2018 at 9:37 AM, Boyuan Zhang <[email protected]> wrote:
> Hey JB,
>
> We just updated the release guide(https://github.com/
> apache/beam-site/commit/dac24e6bee4b3202fbd33543f9d66e80e8b7cd32). Please
> add your updates to release guide if you want to.
>
> Thanks so much for your help!
>
> Boyuan
>
> On Tue, Jun 5, 2018 at 10:16 AM Jean-Baptiste Onofré <[email protected]>
> wrote:
>
>> Thanks, yeah I saw that but I'm planning to add some additional notes.
>>
>> Regards
>> JB
>> Le 5 juin 2018, à 19:02, Boyuan Zhang <[email protected]> a écrit:
>>>
>>> Hey JB,
>>>
>>> We have some updates in : https://github.com/pabloem/beam-site/blob/
>>> 372c93ecbafbf3a1440492df1e12050dfe939e91/src/contribute/release-guide.md
>>> and https://github.com/apache/beam-site/pull/424/files
>>> <https://www.google.com/url?q=https://github.com/apache/beam-site/pull/424/files&sa=D&usg=AFQjCNHm8O1DwRKZ1828EQxzmx881O5aWA>.
>>> They may be helpful.
>>>
>>> Boyuan
>>>
>>> On Tue, Jun 5, 2018 at 9:50 AM Jean-Baptiste Onofré < [email protected]>
>>> wrote:
>>>
>>>> Was checking if there was something like the release branch of the
>>>> maven release plugin, but there's not with the gradle one.
>>>>
>>>> I'm creating the branch by hand and I'm updating the release guide in
>>>> the mean time.
>>>>
>>>> Regards
>>>> JB
>>>> Le 5 juin 2018, à 18:44, "Jean-Baptiste Onofré" < [email protected]> a
>>>> écrit:
>>>>>
>>>>> On the way
>>>>> Le 5 juin 2018, à 18:23, Kenneth Knowles < [email protected]> a écrit:
>>>>>>
>>>>>> Have you cut the release branch? It is much easier to stabilize a cut
>>>>>> branch that is separated from continued development on master. I think we
>>>>>> have to cut it before continuing.
>>>>>>
>>>>>> Kenn
>>>>>>
>>>>>> On Tue, Jun 5, 2018 at 1:14 AM Jean-Baptiste Onofré < [email protected]>
>>>>>> wrote:
>>>>>>
>>>>>>> Sorry for the noise: this build error was due to a corrupted file in
>>>>>>> my
>>>>>>> .m2/repository.
>>>>>>>
>>>>>>> The HDFS extension build is now OK. I'm launching a full build.
>>>>>>>
>>>>>>> Regards
>>>>>>> JB
>>>>>>>
>>>>>>> On 05/06/2018 07:43, Jean-Baptiste Onofré wrote:
>>>>>>> > New failure on the build:
>>>>>>> >
>>>>>>> > FAILURE: Build failed with an exception.
>>>>>>> >
>>>>>>> > * What went wrong:
>>>>>>> > Could not resolve all files for configuration
>>>>>>> > ':beam-sdks-java-io-hadoop-file-system:testCompileClasspath'.
>>>>>>> >> Could not find zookeeper-tests.jar
>>>>>>> >> (org.apache.zookeeper:zookeeper:3.4.6).
>>>>>>>
>>>>>>> > Searched in the following locations:
>>>>>>> >
>>>>>>> > file:/home/jbonofre/.m2/repository/org/apache/
>>>>>>> zookeeper/zookeeper/3.4.6/zookeeper-3.4.6-tests.jar
>>>>>>> >
>>>>>>> > I'm fixing the HDFS extension.
>>>>>>> >
>>>>>>> > Regards
>>>>>>> > JB
>>>>>>> >
>>>>>>> > On 05/06/2018 07:18, Jean-Baptiste Onofré wrote:
>>>>>>> >> Hi,
>>>>>>> >>
>>>>>>> >> yes, it's release blocker: the build is not fully stable. I'm
>>>>>>> trying to
>>>>>>> >> build the release for one week and it fails with different
>>>>>>> errors.
>>>>>>> >>
>>>>>>> >> I have a new build in progress. I hope it will be good. I keep
>>>>>>> you posted.
>>>>>>> >>
>>>>>>> >> Regards
>>>>>>> >> JB
>>>>>>> >>
>>>>>>> >> On 05/06/2018 01:38, Scott Wegner wrote:
>>>>>>> >>> Hey JB, you mentioned some build issues on Slack [1]. Is this
>>>>>>> blocking
>>>>>>> >>> the release? Let me know if there's anything I can help with.
>>>>>>> >>>
>>>>>>> >>> [1] https://the-asf.slack.com/archives/C9H0YNP3P/
>>>>>>> p1528133545000136
>>>>>>> >>>
>>>>>>> >>> On Sun, Jun 3, 2018 at 10:58 PM Jean-Baptiste Onofré <
>>>>>>> [email protected]
>>>>>>> >>> <mailto: [email protected]>> wrote:
>>>>>>> >>>
>>>>>>> >>> Hi guys,
>>>>>>> >>>
>>>>>>> >>> just to let you know that the build is now OK. I'm
>>>>>>> completing the Jira
>>>>>>> >>> triage this morning (my time) and cut the release branch
>>>>>>> (starting the
>>>>>>> >>> release process). I will validate the release guide in the
>>>>>>> mean time.
>>>>>>> >>>
>>>>>>> >>> Thanks,
>>>>>>> >>> Regards
>>>>>>> >>> JB
>>>>>>> >>>
>>>>>>> >>> On 06/04/2018 10:48, Jean-Baptiste Onofré wrote:
>>>>>>> >>> > Hi guys,
>>>>>>> >>> >
>>>>>>> >>> > Apache Beam 2.4.0 has been released on March 20th.
>>>>>>> >>> >
>>>>>>> >>> > According to our cycle of release (roughly 6 weeks), we
>>>>>>> should
>>>>>>> >>> think about 2.5.0.
>>>>>>> >>> >
>>>>>>> >>> > I'm volunteer to tackle this release.
>>>>>>> >>> >
>>>>>>> >>> > I'm proposing the following items:
>>>>>>> >>> >
>>>>>>> >>> > 1. We start the Jira triage now, up to Tuesday
>>>>>>> >>> > 2. I would like to cut the release on Tuesday night
>>>>>>> (Europe time)
>>>>>>> >>> > 2bis. I think it's wiser to still use Maven for this
>>>>>>> release. Do
>>>>>>> >>> you think we
>>>>>>> >>> > will be ready to try a release with Gradle ?
>>>>>>> >>> >
>>>>>>> >>> > After this release, I would like a discussion about:
>>>>>>> >>> > 1. Gradle release (if we release 2.5.0 with Maven)
>>>>>>> >>> > 2. Isolate release cycle per Beam part. I think it would
>>>>>>> be
>>>>>>> >>> interesting to have
>>>>>>> >>> > different release cycle: SDKs, DSLs, Runners, IOs. That's
>>>>>>> another
>>>>>>> >>> discussion, I
>>>>>>> >>> > will start a thread about that.
>>>>>>> >>> >
>>>>>>> >>> > Thoughts ?
>>>>>>> >>> >
>>>>>>> >>> > Regards
>>>>>>> >>> > JB
>>>>>>> >>> >
>>>>>>> >>>
>>>>>>> >>> --
>>>>>>> >>> Jean-Baptiste Onofré
>>>>>>> >>> [email protected] <mailto: [email protected]>
>>>>>>> >>> http://blog.nanthrax.net
>>>>>>> >>> Talend - http://www.talend.com
>>>>>>> >>>
>>>>>>> >>
>>>>>>> >
>>>>>>>
>>>>>>> --
>>>>>>> Jean-Baptiste Onofré
>>>>>>> [email protected]
>>>>>>> http://blog.nanthrax.net
>>>>>>> Talend - http://www.talend.com
>>>>>>>
>>>>>>
