RE: Enforcer Rule- JDK1.7 for Beam

2017-03-06 Thread Dipti Kulkarni
ng.) It seems to me like we it would be preferable to change enforcer's rules in the cassandra module instead of disabling all compiler warnings (again, this is on the assumption that that is possible.) S On Mon, Feb 27, 2017 at 9:18 PM, Dipti Kulkarni < dipti_dkulka...@pe

RE: Enforcer Rule- JDK1.7 for Beam

2017-02-27 Thread Dipti Kulkarni
n-Baptiste Onofré wrote: > Hi Dipti, > > I had the same "issue" on CassandraIO. > > If you take a look on: > > https://github.com/apache/beam/pull/592/files > > you will see that I use to > remove the default -Werror causing issue with Java8 dependencies. &g

Enforcer Rule- JDK1.7 for Beam

2017-02-27 Thread Dipti Kulkarni
Hi all, While working on HadoopInputFormatIO, when I rebased my code to fetch the latest from Beam repo, I see an Enforcer rule added to the parent POM to ensure that all dependencies used are compliant with JDK 1.7 I have written tests for Cassandra and Elasticsearch to test if HIFIO works ok

RE: IT issues faced for HadoopInputFormat IO on Spark and Dataflow runners

2017-02-24 Thread Dipti Kulkarni
gt; Are the tests part of the PR (in order for me to take a look) ? > > Regards > JB > > On 02/19/2017 06:14 PM, Dipti Kulkarni wrote: >> Hi folks, >> >> I am currently working on running HadoopInputFormatIO's CassandraIT >> and Elasticsearch IT on Spark

IT issues faced for HadoopInputFormat IO on Spark and Dataflow runners

2017-02-19 Thread Dipti Kulkarni
Hi folks, I am currently working on running HadoopInputFormatIO's CassandraIT and Elasticsearch IT on Spark and Dataflow runners. However I am facing issues in both ITs for specific Classes not being found at run time only when I use the Dataflow runner profile or Spark runner profile. I see i

RE: Merge HadoopInputFormatIO and HDFSIO in a single module

2017-02-17 Thread Dipti Kulkarni
Thank you all for your inputs! -Original Message- From: Dan Halperin [mailto:dhalp...@google.com.INVALID] Sent: Friday, February 17, 2017 12:17 PM To: dev@beam.apache.org Subject: Re: Merge HadoopInputFormatIO and HDFSIO in a single module Raghu, Amit -- +1 to your expertise :) On Th

Merge HadoopInputFormatIO and HDFSIO in a single module

2017-02-15 Thread Dipti Kulkarni
Hello there! I am working on writing a Read IO for Hadoop InputFormat. This will enable reading from any datasource which supports Hadoop InputFormat, i.e. provides source to read from InputFormat for integration with Hadoop. It makes sense for the HadoopInputFormatIO to share some code with the