+1 for single blog/news section Also I wouldn't mind quarterly cadence, to provide more focus for folks to contribute.
On Mon, Mar 11, 2019 at 6:35 PM Kenneth Knowles <[email protected]> wrote: > Could the newsletter be a blog entry? If you check out > https://blogs.apache.org/ many of the posts are "Apache News Round-up". > We could rename the "Blog" section to "News" if you ask me. > > Kenn > > On Mon, Mar 11, 2019 at 5:13 PM Aizhamal Nurmamat kyzy < > [email protected]> wrote: > >> Hello everyone, >> >> I had a chat with Rose on how I can support the effort and keep sending >> the newsletters on a monthly basis. The new workflow would look as follows: >> >> 1. Send out the same [CALL FOR ITEMS] where you can contribute to the >> Google doc with deadlines >> 2. Edit the doc after the deadline >> 3. Convert the file into Markdown >> 4. Send in PR to add the file to Beam repo in Newsletter directory >> 5. Have people send their fixes/updates through PRs >> >> In this effort, I can support Rose in steps 3 & 4. >> >> We would also need: >> >> - Create a Newsletter section under the Community tab >> - Write guidelines on newsletter contributions >> - Make a note about timing e.g. if upcoming event, then add to the >> next newsletter >> >> How does that sound to you all? >> >> >> On Wed, Mar 6, 2019 at 9:19 PM Rose Nguyen <[email protected]> wrote: >> >>> Good points. With the suggested workflow I think I can support a >>> quarterly newsletter. I'm also happy to get more involvement from others to >>> do this work and we can see what cadence that allows. >>> >>> On Wed, Mar 6, 2019 at 8:22 PM Kenneth Knowles <[email protected]> wrote: >>> >>>> Good points Melissa & Austin. >>>> >>>> - archive in the repo & on the website >>>> - put missed items on the next newsletter, so anyone following sees >>>> them >>>> >>>> Kenn >>>> >>>> On Wed, Mar 6, 2019 at 3:26 PM Suneel Marthi <[email protected]> >>>> wrote: >>>> >>>>> I believe there was also a Beam workshop or working session in Warsaw >>>>> last week. >>>>> >>>>> On Wed, Mar 6, 2019 at 6:20 PM Austin Bennett < >>>>> [email protected]> wrote: >>>>> >>>>>> +1 for archive in our repo. >>>>>> >>>>>> I do follow the newsletter, but am unlikely to go back and look into >>>>>> the past for changes/updates. >>>>>> >>>>>> Would suggest that things that get missed in one newsletter (a >>>>>> concrete example, Suneel's talks not mentioned in the newsletter) would >>>>>> get >>>>>> published in the next iteration, rather than editing the past 'published' >>>>>> newsletter. Put another way, save editing the past for corrections >>>>>> (typos, >>>>>> things being incorrect). Else, I imagine that I'm unlikely to catch a >>>>>> great announcement that warranted being in the newsletter in the first >>>>>> place. This certainly works better with a regular/frequent release >>>>>> cadence, like we arrived at for version releases (then, if something >>>>>> misses >>>>>> one cut, it is not too big a deal, as the next release is coming soon). >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> On Wed, Mar 6, 2019 at 12:50 PM Melissa Pashniak < >>>>>> [email protected]> wrote: >>>>>> >>>>>>> >>>>>>> For step #2 (publishing onto the website), I think it would be good >>>>>>> to stay consistent with our existing workflows if possible. Rather than >>>>>>> using an external tool, what about: >>>>>>> >>>>>>> After a google doc newsletter draft is ready, convert it into a >>>>>>> standard markdown file and put it into our GitHub repo, perhaps in a new >>>>>>> newsletter directory in the website community directory [1]. These >>>>>>> would be >>>>>>> listed for browsing on a Newsletters page as mentioned in step #4. >>>>>>> People >>>>>>> can then just open a PR to add missing things to the pages later, and >>>>>>> the >>>>>>> newsletter will be automatically updated on the website through our >>>>>>> standard website workflow. It also avoids the potential issue of the >>>>>>> source >>>>>>> google docs disappearing in the future, as they are stored in a >>>>>>> community >>>>>>> location. >>>>>>> >>>>>>> [1] https://github.com/apache/beam/tree/master/website/src/community >>>>>>> >>>>>>> >>>>>>> On Wed, Mar 6, 2019 at 10:36 AM Rose Nguyen <[email protected]> >>>>>>> wrote: >>>>>>> >>>>>>>> I think that would be a great idea to change formats to help with >>>>>>>> distribution. I'm open to suggestions! I'm currently using a Google >>>>>>>> doc to >>>>>>>> collect and edit, then copy/paste sending the newsletter out directly, >>>>>>>> based >>>>>>>> on an interpretation of this discussion >>>>>>>> <https://lists.apache.org/thread.html/1f638eae43fe8abcb2f8752141c96d3dbdac86a583e0790044eea727@%3Cdev.beam.apache.org%3E> >>>>>>>> . >>>>>>>> >>>>>>>> How about this doc->website->Beam site workflow?: >>>>>>>> >>>>>>>> 1. The same usual newsletter [CALL FOR ITEMS] where you can >>>>>>>> contribute to the google doc, with soft deadlines for when I'll >>>>>>>> publish. >>>>>>>> 2. I'll publish the doc itself onto a website. >>>>>>>> 3. The newsletter is mailed out in the same way, but now with a >>>>>>>> shareable website link. >>>>>>>> 4. We'll keep an index of archived newsletter web pages on the >>>>>>>> Beam site, under the Community tab. >>>>>>>> 5. If you want to submit more content after the soft deadline, >>>>>>>> add it to the google doc and let me know to republish. I don't want >>>>>>>> to make >>>>>>>> the publication changes automatic because that leaves us open to >>>>>>>> tampering. >>>>>>>> >>>>>>>> >>>>>>>> This process is more laggy, so I'd suggest doing a 2 month vs >>>>>>>> monthly newsletter cadence. If we're happy with this idea, I'll send >>>>>>>> in a >>>>>>>> website PR for a new "Newsletter" left nav item under Community. >>>>>>>> >>>>>>>> Here's an example of a published newsletter: Apache Beam >>>>>>>> February-March 2019 >>>>>>>> <https://gdoc.pub/doc/e/2PACX-1vTQIS4WkxV-HpgX5Lb6q05g4-wuIVcYd82123Mp4Y6q9fMv6Ynwd-l7dI4TrMyCrKilyU-YsoitbnZB> >>>>>>>> >>>>>>>> >>>>>>>> - This link is permanent unless the principal google doc is >>>>>>>> deleted. >>>>>>>> - Changes to the google doc after web publication are not >>>>>>>> automatically published on the website to protect the information >>>>>>>> integrity. >>>>>>>> - Republishing is quick and easy for me if you let me know >>>>>>>> you've added more. >>>>>>>> - I'll improve the formatting later if we go with this route. >>>>>>>> >>>>>>>> Any thoughts? >>>>>>>> >>>>>>>> On Wed, Mar 6, 2019 at 6:13 AM Thomas Weise <[email protected]> wrote: >>>>>>>> >>>>>>>>> Similar to blog posts. A link that can be shared would also help >>>>>>>>> to distribute over other channels, such as Twitter. >>>>>>>>> >>>>>>>>> >>>>>>>>> On Wed, Mar 6, 2019, 6:06 AM Ismaël Mejía <[email protected]> >>>>>>>>> wrote: >>>>>>>>> >>>>>>>>>> We should have these newsletters published somewhere with a fixed >>>>>>>>>> URL so we can add missing updates, I have at least missed putting >>>>>>>>>> stuff in >>>>>>>>>> the last 3 ones just to realize when the newsletter has been already >>>>>>>>>> 'published' (and sadly interesting features like zstd have not been >>>>>>>>>> announced because of this). >>>>>>>>>> >>>>>>>>>> On Wed, Mar 6, 2019 at 2:48 PM Etienne Chauchot < >>>>>>>>>> [email protected]> wrote: >>>>>>>>>> >>>>>>>>>>> Hi, >>>>>>>>>>> I would add in what's been done: >>>>>>>>>>> >>>>>>>>>>> Work on cassandraIO (Etienne Chauchot, Mathieu Blanchard, >>>>>>>>>>> Frank Shahar) : refactorings, bugfixes, new where clause, security >>>>>>>>>>> fix >>>>>>>>>>> >>>>>>>>>>> Etienne >>>>>>>>>>> >>>>>>>>>>> Le lundi 04 mars 2019 à 18:36 +0100, Suneel Marthi a écrit : >>>>>>>>>>> >>>>>>>>>>> Is this the final draft? - we had 2 beam talks at Big Data Tech >>>>>>>>>>> Warsaw last Wednesday - I can send the updates offline. >>>>>>>>>>> >>>>>>>>>>> On Mon, Mar 4, 2019 at 6:16 PM Rose Nguyen <[email protected]> >>>>>>>>>>> wrote: >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> [image: Beam.png] >>>>>>>>>>> >>>>>>>>>>> February-March 2019 | Newsletter >>>>>>>>>>> >>>>>>>>>>> What’s been done >>>>>>>>>>> >>>>>>>>>>> ------------------------------ >>>>>>>>>>> >>>>>>>>>>> Apache Beam 2.10.0 released (by: many contributors) >>>>>>>>>>> >>>>>>>>>>> - >>>>>>>>>>> >>>>>>>>>>> Download the release here. >>>>>>>>>>> <https://beam.apache.org/get-started/downloads/> >>>>>>>>>>> - >>>>>>>>>>> >>>>>>>>>>> See the blog post >>>>>>>>>>> <https://beam.apache.org/blog/2019/02/15/beam-2.10.0.html> >>>>>>>>>>> for more details. >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> Apache Beam awarded the 2019 Technology of the Year Award! >>>>>>>>>>> >>>>>>>>>>> - >>>>>>>>>>> >>>>>>>>>>> InfoWorld just awarded Beam the 2019 Technology of the Year >>>>>>>>>>> Award. >>>>>>>>>>> - >>>>>>>>>>> >>>>>>>>>>> See this article >>>>>>>>>>> >>>>>>>>>>> <https://www.infoworld.com/article/3336072/application-development/infoworlds-2019-technology-of-the-year-award-winners.html?nsdr=true> >>>>>>>>>>> for more details. >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> Kettle Beam 0.5 released with support for flink (by: Matt >>>>>>>>>>> Casters) >>>>>>>>>>> >>>>>>>>>>> - >>>>>>>>>>> >>>>>>>>>>> Kettle now supports Apache Flink as well as Cloud Dataflow >>>>>>>>>>> and Spark. >>>>>>>>>>> - >>>>>>>>>>> >>>>>>>>>>> See Matt’s Blog >>>>>>>>>>> >>>>>>>>>>> <http://sandbox.kettle.be/wordpress/index.php/2019/02/24/kettle-beam-update-0-5-0/> >>>>>>>>>>> for more details. >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> What we’re working on... >>>>>>>>>>> >>>>>>>>>>> ------------------------------ >>>>>>>>>>> >>>>>>>>>>> Apache Beam 2.11.0 release (by: many contributors) >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> Hive Metastore Table provider for SQL (by: Anton Kedin) >>>>>>>>>>> >>>>>>>>>>> - >>>>>>>>>>> >>>>>>>>>>> Support for plugging table providers through Beam SQL API to >>>>>>>>>>> allow obtaining table schemas from external sources. >>>>>>>>>>> - >>>>>>>>>>> >>>>>>>>>>> See the PR <https://github.com/apache/beam/pull/7746> for >>>>>>>>>>> more details. >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> User Defined Coders for the Beam Go SDK (by: Robert Burke) >>>>>>>>>>> >>>>>>>>>>> - >>>>>>>>>>> >>>>>>>>>>> Working on expanding the variety of user defined types that >>>>>>>>>>> can be a member of a PCollection in the Go SDK. >>>>>>>>>>> - >>>>>>>>>>> >>>>>>>>>>> See BEAM-3306 >>>>>>>>>>> <https://issues.apache.org/jira/browse/BEAM-3306> for more >>>>>>>>>>> details. >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> Python 3 (by: Ahmet Altay, Robert Bradshaw, Charles Chen, Mark >>>>>>>>>>> Liu, Robbe Sneyders, Juta Staes, Valentyn Tymofieiev) >>>>>>>>>>> >>>>>>>>>>> - >>>>>>>>>>> >>>>>>>>>>> Beam 2.11.0 is the first release offering partial Python 3 >>>>>>>>>>> support. >>>>>>>>>>> - >>>>>>>>>>> >>>>>>>>>>> Many thanks to all contributors who helped to reach this >>>>>>>>>>> milestone. >>>>>>>>>>> - >>>>>>>>>>> >>>>>>>>>>> IO availablility on Python 3 is currently limited and only >>>>>>>>>>> Python 3.5 version has been tested extensively. >>>>>>>>>>> - >>>>>>>>>>> >>>>>>>>>>> Stay tuned on BEAM-1251 for more details. >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> Notebooks for quickstarts and custom I/O (by: David Cavazos) >>>>>>>>>>> >>>>>>>>>>> - >>>>>>>>>>> >>>>>>>>>>> Adding IPython notebooks and snippets >>>>>>>>>>> - >>>>>>>>>>> >>>>>>>>>>> See [BEAM-6557] <https://github.com/apache/beam/pull/7679> >>>>>>>>>>> for more details. >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> New members >>>>>>>>>>> ------------------------------ >>>>>>>>>>> >>>>>>>>>>> New PMC member! >>>>>>>>>>> >>>>>>>>>>> - >>>>>>>>>>> >>>>>>>>>>> Etienne Chauchot, Nantes, France >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> New Committers! >>>>>>>>>>> >>>>>>>>>>> - >>>>>>>>>>> >>>>>>>>>>> Gleb Kanterov, Stockholm, Sweden >>>>>>>>>>> - >>>>>>>>>>> >>>>>>>>>>> Michael Luckey >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> New Contributors! >>>>>>>>>>> >>>>>>>>>>> - >>>>>>>>>>> >>>>>>>>>>> Kyle Weaver, San Francisco, CA >>>>>>>>>>> - >>>>>>>>>>> >>>>>>>>>>> Would like to help begin implementing portability support >>>>>>>>>>> for the Spark runner >>>>>>>>>>> - >>>>>>>>>>> >>>>>>>>>>> Tanay Tummapalli, Delhi, India >>>>>>>>>>> - >>>>>>>>>>> >>>>>>>>>>> Would like to contribute to Open Source this summer as >>>>>>>>>>> part of Google Summer of Code >>>>>>>>>>> - >>>>>>>>>>> >>>>>>>>>>> Brian Hulette, Seattle, WA >>>>>>>>>>> - >>>>>>>>>>> >>>>>>>>>>> Contributing to Beam Portability >>>>>>>>>>> - >>>>>>>>>>> >>>>>>>>>>> Michał Walenia, Warsaw, Poland >>>>>>>>>>> - >>>>>>>>>>> >>>>>>>>>>> Working on integration and load testing >>>>>>>>>>> - >>>>>>>>>>> >>>>>>>>>>> Daniel Chen, San Francisco, CA >>>>>>>>>>> - >>>>>>>>>>> >>>>>>>>>>> Working on Beam Samza runner >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> Talks & meetups >>>>>>>>>>> ------------------------------ >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> Plugin Machine Intelligence and Apache Beam with Pentaho - Feb 7 >>>>>>>>>>> @ London >>>>>>>>>>> >>>>>>>>>>> - >>>>>>>>>>> >>>>>>>>>>> Watch the How to Run Kettle on Apache Beam video here >>>>>>>>>>> >>>>>>>>>>> <https://skillsmatter.com/skillscasts/13405-how-to-run-kettle-on-apache-beam#video>. >>>>>>>>>>> >>>>>>>>>>> - >>>>>>>>>>> >>>>>>>>>>> See event details here >>>>>>>>>>> >>>>>>>>>>> <https://www.meetup.com/Pentaho-London-User-Group/events/256773962/> >>>>>>>>>>> .. >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> Beam @Lyft / Streaming, TensorFlow and use-cases - Feb 7 @ San >>>>>>>>>>> Francisco, CA >>>>>>>>>>> >>>>>>>>>>> - >>>>>>>>>>> >>>>>>>>>>> Organized by Thomas Weise and Austin Bennet, with speakers >>>>>>>>>>> Tyler Akidau, Robert Crowe, Thomas Weise and Amar Pai >>>>>>>>>>> - >>>>>>>>>>> >>>>>>>>>>> See event details here >>>>>>>>>>> >>>>>>>>>>> <https://www.meetup.com/San-Francisco-Apache-Beam/events/257482350/> >>>>>>>>>>> and the slides for these presentation: Overview of Apache >>>>>>>>>>> Beam and TensorFlow Transform (TFX) with Apache Beam >>>>>>>>>>> <http://s.apache.org/beam-intro-feb-2019>, Python Streaming >>>>>>>>>>> Pipelines with Beam on Flink >>>>>>>>>>> <http://go.lyft.com/python-flink-beam-meetup-2019>, Dynamic >>>>>>>>>>> pricing of Lyft rides using streaming >>>>>>>>>>> >>>>>>>>>>> <https://www.slideshare.net/AmarPai2/dynamic-pricing-of-lyft-rides-using-streaming> >>>>>>>>>>> >>>>>>>>>>> . >>>>>>>>>>> >>>>>>>>>>> Flink meetup - Feb 21@ Seattle, WA >>>>>>>>>>> >>>>>>>>>>> - >>>>>>>>>>> >>>>>>>>>>> Speakers from Alibaba, Google, and Uber gave talks about >>>>>>>>>>> Apache Flink with Hive, Tensorflow, Beam, and AthenaX. >>>>>>>>>>> - >>>>>>>>>>> >>>>>>>>>>> See event details here >>>>>>>>>>> <https://www.meetup.com/seattle-flink/events/258723322/> and >>>>>>>>>>> presentations here >>>>>>>>>>> <https://www.slideshare.net/BowenLi9/presentations>. >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> Beam Summit Europe 2019 - June 19-20 @ Berlin >>>>>>>>>>> >>>>>>>>>>> - >>>>>>>>>>> >>>>>>>>>>> Beam Summit Europe 2019 will take place in Berlin on June >>>>>>>>>>> 19-20. >>>>>>>>>>> - >>>>>>>>>>> >>>>>>>>>>> Speaker CfP and other details to follow soon! >>>>>>>>>>> - >>>>>>>>>>> >>>>>>>>>>> Twitter announcement! >>>>>>>>>>> <https://twitter.com/matthiasbaetens/status/1098854758893273088> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> Resources >>>>>>>>>>> ------------------------------ >>>>>>>>>>> >>>>>>>>>>> Apache Jira Beginner’s Guide (by: Daniel Oliveira) >>>>>>>>>>> >>>>>>>>>>> - >>>>>>>>>>> >>>>>>>>>>> A guide >>>>>>>>>>> >>>>>>>>>>> <https://cwiki.apache.org/confluence/display/BEAM/Beam+Jira+Beginner%27s+Guide> >>>>>>>>>>> to introduce Beam contributors to the basics of using the Apache >>>>>>>>>>> Jira for >>>>>>>>>>> Beam development. Feedback welcomed! >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> An approach to community building from Apache Beam (by: Kenn >>>>>>>>>>> Knowles) >>>>>>>>>>> >>>>>>>>>>> - >>>>>>>>>>> >>>>>>>>>>> The Apache Software Foundation has published committer >>>>>>>>>>> guidelines to help Beam's community building work. >>>>>>>>>>> - >>>>>>>>>>> >>>>>>>>>>> See the post <https://blogs.apache.org/comdev/date/20190222> >>>>>>>>>>> on the ASF blog. >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> Exploring Beam SQL on Google Cloud Platform (by: Graham Polley) >>>>>>>>>>> >>>>>>>>>>> - >>>>>>>>>>> >>>>>>>>>>> “In this article, I’ll dive into this new feature of Beam, >>>>>>>>>>> and see how it works by using a pipeline to read a data file >>>>>>>>>>> from GCS, >>>>>>>>>>> transform it, and then perform a basic calculation on the values >>>>>>>>>>> contained >>>>>>>>>>> in the file”. >>>>>>>>>>> - >>>>>>>>>>> >>>>>>>>>>> See article >>>>>>>>>>> >>>>>>>>>>> <https://medium.com/weareservian/exploring-beam-sql-on-google-cloud-platform-b6c77f9b4af4> >>>>>>>>>>> and full source code >>>>>>>>>>> >>>>>>>>>>> <https://github.com/polleyg/gcp-batch-ingestion-bigquery/blob/beam_sql/src/main/java/org/polleyg/BeamSQLMagic.java> >>>>>>>>>>> . >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> *Until Next Time!* >>>>>>>>>>> -- >>>>>>>>>>> Rose Thị Nguyễn >>>>>>>>>>> >>>>>>>>>>> >>>>>>>> >>>>>>>> -- >>>>>>>> Rose Thị Nguyễn >>>>>>>> >>>>>>> >>> >>> -- >>> Rose Thị Nguyễn >>> >> >> >> -- >> >> *Aizhamal Nurmamat kyzy* >> >> Open Source Program Manager >> >> 646-355-9740 Mobile >> >> 601 North 34th Street, Seattle, WA 98103 >> >> >>
