[no subject]

Unsubscribe With Best Regards, Dipayan Dev Author of *Deep Learning with Hadoop <https://www.amazon.com/Deep-Learning-Hadoop-Dipayan-Dev/dp/1787124762>* M.Tech (AI), IISc, Bangalore

Unsubscribe

Unsubscribe >>

unscribe

> 2024年10月23日 09:41，Manu Zhang 写道： > > Hi community, > > I've opened a PR[1] to support custom driver metrics in writing to the v2 > table. > Please help review and leave your comments. Appreciate it! > > [1] https://github.com/apache/spark/pull/48573 >

Re: Elasticsearch support for Spark 3.x

@Alfie Davidson : Awesome, it worked with "“org.elasticsearch.spark.sql”" But as soon as I switched to *elasticsearch-spark-20_2.12, *"es" also worked. On Fri, Sep 8, 2023 at 12:45 PM Dipayan Dev wrote: > > Let me try that and get back. Just wondering, if there a ch

Re: Elasticsearch support for Spark 3.x

t; > Sent from my iPhone > > On 8 Sep 2023, at 03:10, Dipayan Dev wrote: > >  > > ++ Dev > > On Thu, 7 Sep 2023 at 10:22 PM, Dipayan Dev > wrote: > >> Hi, >> >> Can you please elaborate your last response? I don’t have any external >> depende

Re: Elasticsearch support for Spark 3.x

++ Dev On Thu, 7 Sep 2023 at 10:22 PM, Dipayan Dev wrote: > Hi, > > Can you please elaborate your last response? I don’t have any external > dependencies added, and just updated the Spark version as mentioned below. > > Can someone help me with this? > > On Fri, 1 Se

Feature to restart Spark job from previous failure point

Spark job till that point, and on a retry, resume the operation from that point? With Best Regards, Dipayan Dev

Re: [VOTE] Release Apache Spark 3.5.0 (RC3)

s v3.5.0-rc3 (commit >> 9f137aa4dc43398aafa0c3e035ed3174182d7d6c): >> >> https://github.com/apache/spark/tree/v3.5.0-rc3 >> >> The release files, including signatures, digests, etc. can be found at: >> >> https://dist.apache.org/repos/dist/dev/spark/v3.5.0-rc

Probable Bug in Spark 3.3.0

Hi Dev Team, https://issues.apache.org/jira/browse/SPARK-44884 We have recently upgraded to Spark 3.3.0 in our Production Dataproc. We have a lot of downstream application that relies on the SUCCESS file. Please let me know if this is a bug or I need to any additional configuration to fix this

Re: Starting sparkthrift server

folder (instead of /tmp/spark-events), you could use --hiveconf to assign > those to another folder. > > > On Mon, Mar 23, 2015 at 8:39 AM Neil Dev wrote: > >> Hi, >> >> I am having issues starting spark-thriftserver. I'm running spark 1.3.o >> with Hadoop

Spark-thriftserver Issue

Hi, I am having issue starting spark-thriftserver. I'm running spark 1.3.with Hadoop 2.4.0. I would like to be able to change its port too so, I can hive hive-thriftserver as well as spark-thriftserver running at the same time. Starting sparkthrift server:- sudo ./start-thriftserver.sh --master s

Starting sparkthrift server

Hi, I am having issues starting spark-thriftserver. I'm running spark 1.3.o with Hadoop 2.4.0. I would like to be able to change its port too so, I can hive hive-thriftserver as well as spark-thriftserver running at the same time. Starting sparkthrift server:- sudo ./start-thriftserver.sh --maste

Re: [spark-streaming] Is this LocalInputDStream useful to someone?

Hi guys, Nobody saw/answered my question so I resend it ;) Thanks Pascal On Thu, Mar 20, 2014 at 9:17 AM, Pascal Voitot Dev < pascal.voitot@gmail.com> wrote: > Hi guys, > > In my recent blog post (http://mandubian.com/2014/03/08/zpark-ml-nio-1/), > I needed to have an I

Re: Making RDDs Covariant

rmation like that and stop too soon :) You know that I've been struggling for some time playing with spark & scalaz-stream and I'm curious ;) So without covariance I cannot for now define a generic noop Sink. > > My0.02c > Andy > > Sent from Tab, sorry for the typos... &

Re: Making RDDs Covariant

I think we are going to be okay here since, as Koert > said, the current serializers use runtime type information. We could also > keep at ClassTag around for the original type when the RDD was created. > Good things to be aware of though. > > Michael > > On Sat, Mar 22, 201

Re: Making RDDs Covariant

On Sat, Mar 22, 2014 at 8:38 PM, David Hall wrote: > On Sat, Mar 22, 2014 at 8:59 AM, Pascal Voitot Dev < > pascal.voitot@gmail.com> wrote: > > > The problem I was talking about is when you try to use typeclass > converters > > and make them contravari

Re: Making RDDs Covariant

On Sat, Mar 22, 2014 at 3:45 PM, Michael Armbrust wrote: > > > > From my experience, covariance often becomes a pain when dealing with > > serialization/deserialization (I've experienced a few cases while > > developing play-json & datomisca). > > Moreover, if you have implicits, variance often be

Re: Making RDDs Covariant

Hi, Covariance always seems like a good idea at first but you must be really careful as it always has unexpected consequences... >From my experience, covariance often becomes a pain when dealing with serialization/deserialization (I've experienced a few cases while developing play-json & datomisca

[spark-streaming] Is this LocalInputDStream useful to someone?

Hi guys, In my recent blog post (http://mandubian.com/2014/03/08/zpark-ml-nio-1/), I needed to have an InputDStream helper looking like NetworkInputDStream to be able to push my data into DStream in an async way. But I didn't want the remoting aspect as my data source runs locally and nowhere else

Re: [re-cont] map and flatMap

On Wed, Mar 12, 2014 at 3:06 PM, andy petrella wrote: > Folks, > > I want just to pint something out... > I didn't had time yet to sort it out and to think enough to give valuable > strict explanation of -- event though, intuitively I feel they are a lot > ===> need spark people or time to move fo