Re: [ANNOUNCE] New committer announcement: Mark Liu

2019-03-24 Thread Pablo Estrada
Yeaah Mark! : ) Congrats : D On Sun, Mar 24, 2019 at 10:32 PM Yifan Zou wrote: > Congratulations Mark! > > On Sun, Mar 24, 2019 at 10:25 PM Connell O'Callaghan > wrote: > >> Well done congratulations Mark!!! >> >> On Sun, Mar 24, 2019 at 10:17 PM Robert Burke wrote: >> >>> Congratulations Mar

Re: Beam Dependency Check Report (2019-03-23)

2019-03-24 Thread Yifan Zou
The job failed because of PyYAML load(file) is deprecated. I have PR-8123 to fix that. The job passed with the PR and the report was sent to myself just for verification purpose. You can see the latest report in here: https://builds.apache.org/view/A-D/view/Beam/view/All/job/beam_Dependency_Check/

Re: [ANNOUNCE] New committer announcement: Mark Liu

2019-03-24 Thread Yifan Zou
Congratulations Mark! On Sun, Mar 24, 2019 at 10:25 PM Connell O'Callaghan wrote: > Well done congratulations Mark!!! > > On Sun, Mar 24, 2019 at 10:17 PM Robert Burke wrote: > >> Congratulations Mark! 🎉 >> >> On Sun, Mar 24, 2019, 10:08 PM Valentyn Tymofieiev >> wrote: >> >>> Congratulations,

Re: [ANNOUNCE] New committer announcement: Mark Liu

2019-03-24 Thread Connell O'Callaghan
Well done congratulations Mark!!! On Sun, Mar 24, 2019 at 10:17 PM Robert Burke wrote: > Congratulations Mark! 🎉 > > On Sun, Mar 24, 2019, 10:08 PM Valentyn Tymofieiev > wrote: > >> Congratulations, Mark! >> >> Thanks for your contributions, in particular for your efforts to >> parallelize test

Re: [ANNOUNCE] New committer announcement: Mark Liu

2019-03-24 Thread Robert Burke
Congratulations Mark! 🎉 On Sun, Mar 24, 2019, 10:08 PM Valentyn Tymofieiev wrote: > Congratulations, Mark! > > Thanks for your contributions, in particular for your efforts to > parallelize test execution for Python SDK and increase the speed of Python > precommit checks. > > On Sun, Mar 24, 201

Re: Writing bytes to BigQuery with beam

2019-03-24 Thread Valentyn Tymofieiev
We received feedback on https://issuetracker.google.com/issues/129006689 - BQ developers say that schema identification is done and they discourage to use schema autodetection in tables using BYTES. In light of this, I think may be fair to recommend Beam users to specify BQ schemas as well when the

Re: [ANNOUNCE] New committer announcement: Mark Liu

2019-03-24 Thread Valentyn Tymofieiev
Congratulations, Mark! Thanks for your contributions, in particular for your efforts to parallelize test execution for Python SDK and increase the speed of Python precommit checks. On Sun, Mar 24, 2019 at 9:40 PM Kenneth Knowles wrote: > Hi all, > > Please join me and the rest of the Beam PMC i

[ANNOUNCE] New committer announcement: Mark Liu

2019-03-24 Thread Kenneth Knowles
Hi all, Please join me and the rest of the Beam PMC in welcoming a new committer: Mark Liu. Mark has been contributing to Beam since late 2016! He has proposed 100+ pull requests. Mark was instrumental in expanding test and infrastructure coverage, especially for Python. In consideration of Mark'

Re: docs: java-dependencies

2019-03-24 Thread Kenneth Knowles
I had forgotten about that page. I think it is a good idea to include it in the release process. I would rephrase the page a little bit to make it clear that dependency conflicts are normal and expected in Java so this is an FYI page about the versions we test with. Users may have to pin to other v

docs: java-dependencies

2019-03-24 Thread David Morávek
Hello, I've run into java-dependencies page , while upgrading joda-time. This page is supposed to track dependencies across releases, but the last entry is for 2.9 release. Do we still want to maintain this? If so, could we somehow in

Apache Beam, Spark, Hadoop and AWS cross accounts

2019-03-24 Thread Carlos Baeza
Hi Guys, I’m new in Apache Beam. In my current project we have follow scenario: - We run transformations via Apache Beam Pipeline into Amazon AWS (using Cluster by Spark, Hadoop). We can in future produce big Files. - The generated a AVRO file that should be stored from AWS Account A to AWS Ac