Re: Beam Summits!

2019-01-23 Thread Austin Bennett
Hi All, PMC approval still pending for Summit in SF (so things may change), but wanted to get a preliminary CfP out there to start to get sense of interest -- giving the targeted dates are approaching. Much of this delay/uncertainty my fault and I should have done more before the holidays and my

Re: ParquetIO write of CSV document data

2019-01-23 Thread Sridevi Nookala
Hi Alex, Thanks for the suggestion. I tried like in the github example by infering AVRO schema, PCollection input = pipeline.apply(TextIO.read().from("/tmp/beam/input.csv")); input .apply("Produce Avro records", ParDo.of(new DeterministicallyConstructAvroRecordsFn(

Re: ParquetIO write of CSV document data

2019-01-23 Thread Sridevi Nookala
Hi Alex, Thanks for the suggestion. I tried like in the github example by infering AVRO schema, PCollection input = pipeline.apply(TextIO.read().from("/tmp/beam/input.csv")); input .apply("Produce Avro records", ParDo.of(new DeterministicallyConstructAvroRecordsFn(

Re: Global Window and cumulative trigger

2019-01-23 Thread Kenneth Knowles
Hi Parviz, Your code looks good. You are correct about the meaning of accumulatingFiredPanes(). You should always see the whole sum so far for each key. Since your code is so clear, I would immediately move to porting this example to a test case. Can you provide inputs and outputs so we can reprod