Hi Everyone,
I am running a spark application where I have applied 2 left joins. 1st
join in Broadcast and another one is normal.
Out of 200 tasks , last 1 task is stuck . It is running at "ANY" Locality
level. It seems data skewness issue.
It is doing too much spill and shuffle write is too much.
Hi,
One word : SKEW
It seems the classic skew problem, you would have to apply skew techniques
to repartition your data properly or if you are in spark 3.0+ try the
skewJoin optimization.
On Tue, 26 Jan 2021 at 11:20, rajat kumar
wrote:
> Hi Everyone,
>
> I am running a spark application where
Hi ,
Yes I understand its skew based problem but how can it be avoided . Could
you please suggest?
I am in Spark2.4
Thanks
Rajat
On Tue, Jan 26, 2021 at 3:58 PM German Schiavon
wrote:
> Hi,
>
> One word : SKEW
>
> It seems the classic skew problem, you would have to apply skew techniques
> to
Well if your data is skewed I don't think it can be avoided but mitigated
using skew techniques.
I'd recommend you to take a look at "salted join" maybe.
On Tue, 26 Jan 2021 at 11:29, rajat kumar
wrote:
> Hi ,
>
> Yes I understand its skew based problem but how can it be avoided . Could
> you
Hello!
We plan to use Apache Spark software in our organization, can I purchase paid
technical support for this software?
С уважением,
Андрей Синий
Руководитель направления
Центр управления программным обеспечением
Филиал ПАО «МТС» в Нижегородской области
Публичное акционерное общество «Мобиль
Hi Andrey,
It looks like you may contact Databricks for that.
Also it would be easier for non-russian spaekers to respond you if your
name would be written in English.
On 21/01/26 12:41PM, Синий Андрей wrote:
> Hello!
>
> We plan to use Apache Spark software in our organization, can I purchase p
Hi,
why do you want to buy paid SPARK?
Regards,
Gourav
On Tue, Jan 26, 2021 at 1:22 PM Pasha Finkelshteyn <
pavel.finkelsht...@gmail.com> wrote:
> Hi Andrey,
>
> It looks like you may contact Databricks for that.
> Also it would be easier for non-russian spaekers to respond you if your
> name w
To clarify: Apache projects and the ASF do not provide paid support.
However there are many vendors who provide distributions of Apache Spark
who will provide technical support - not nearly just Databricks but
Cloudera, etc. There are also plenty of consultancies and individuals who
can provide pro
Hello Spark Community,
I have a Spark-SQL problem where I am receiving a NoClassDefFoundError error
for org.apache.spark.sql.catalyst.util.RebaseDateTime$ . This happens for any
query with a filter on a Timestamp column when the query is first run
programmatically but not when the query is f
Hello Andrey,
you can try to reach Beeline beeline.ru, they use Databricks as far as I
know.
вт, 26 янв. 2021 г. в 15:01, Sean Owen :
> To clarify: Apache projects and the ASF do not provide paid support.
> However there are many vendors who provide distributions of Apache Spark
> who will provi
All of the major cloud vendors have some sort of Spark offering. They provide
support if you build in their cloud.
From: Синий Андрей
Date: Tuesday, January 26, 2021 at 7:52 AM
To: "user@spark.apache.org"
Subject: [EXTERNAL] Apache Spark
CAUTION: This email originated from outside of the orga
Hi Андрей,
You can write to https://databricks.com/company/contact . Probably, we can
offer something to you. For instance, Databricks has OEM program which
might be interesting to you:
https://partners.databricks.com/prm/English/c/Overview
Maxim Gekk
Software Engineer
Databricks, Inc.
On Tue
Unsubscribe
Thank you for the information. If necessary, I will write.
С уважением,
Андрей Синий
Руководитель направления
Центр управления программным обеспечением
Филиал ПАО «МТС» в Нижегородской области
Публичное акционерное общество «Мобильные ТелеСистемы»
__
IP: 90096
mob: +79
15 matches
Mail list logo
