[VOTE] Spark 2.1.2 (RC2)

2017-09-26 Thread Holden Karau
Please vote on releasing the following candidate as Apache Spark version 2 .1.2. The vote is open until Wednesday October 4th at 23:59 PST and passes if a majority of at least 3 +1 PMC votes are cast. [ ] +1 Release this package as Apache Spark 2.1.2 [ ] -1 Do not release this package because ...

Re: Should Flume integration be behind a profile?

2017-09-26 Thread Mridul Muralidharan
Sounds good to me. +1 Regards, Mridul On Tue, Sep 26, 2017 at 2:36 AM, Sean Owen wrote: > Not a big deal, but I'm wondering whether Flume integration should at least > be opt-in and behind a profile? it still sees some use (at least on our end) > but not applicable to the majority of users. Mo

Re: [discuss] Data Source V2 write path

2017-09-26 Thread Wenchen Fan
I'm trying to give a summary: Ideally data source API should only deal with data, not metadata. But one key problem is, Spark still need to support data sources without metastore, e.g. file format data sources. For this kind of data sources, users have to pass the metadata information like partit

Re: Should Flume integration be behind a profile?

2017-09-26 Thread Ryan Blue
+1 for a Flume profile. On Tue, Sep 26, 2017 at 2:36 AM, Sean Owen wrote: > Not a big deal, but I'm wondering whether Flume integration should at > least be opt-in and behind a profile? it still sees some use (at least on > our end) but not applicable to the majority of users. Most other > third

Re: [Spark Core] Custom Catalog. Integration between Apache Ignite and Apache Spark

2017-09-26 Thread Николай Ижиков
Hello, Xin. Thank you for an answer. Is there any plans to make catalog API public? Any specific release versions or dates? 2017-09-25 20:54 GMT+03:00 Reynold Xin : > It's probably just an indication of lack of interest (or at least there > isn't a substantial overlap between Ignite users and

Should Flume integration be behind a profile?

2017-09-26 Thread Sean Owen
Not a big deal, but I'm wondering whether Flume integration should at least be opt-in and behind a profile? it still sees some use (at least on our end) but not applicable to the majority of users. Most other third-party framework integrations are behind a profile, like YARN, Mesos, Kinesis, Kafka