Hi Reynold,
I had version 2.6.1 in my project which was provided by the fine folks
from spring-boot-dependencies.
Now have overridden it to 2.7.8 :)
Sjoerd
2015-11-01 8:22 GMT+01:00 Reynold Xin :
> Thanks for reporting it, Sjoerd. You might have a different version of
> Janino brought in from
Thanks for reporting it, Sjoerd. You might have a different version of
Janino brought in from somewhere else.
This should fix your problem: https://github.com/apache/spark/pull/9372
Can you give it a try?
On Tue, Oct 27, 2015 at 9:12 PM, Sjoerd Mulder
wrote:
> No the job actually doesn't fai
No the job actually doesn't fail, but since our tests is generating all
these stacktraces i have disabled the tungsten mode just to be sure (and
don't have gazilion stacktraces in production).
2015-10-27 20:59 GMT+01:00 Josh Rosen :
> Hi Sjoerd,
>
> Did your job actually *fail* or did it just gen
Hi Sjoerd,
Did your job actually *fail* or did it just generate many spurious
exceptions? While the stacktrace that you posted does indicate a bug, I
don't think that it should have stopped query execution because Spark
should have fallen back to an interpreted code path (note the "Failed to
gener
I have disabled it because of it started generating ERROR's when upgrading
from Spark 1.4 to 1.5.1
2015-10-27T20:50:11.574+0100 ERROR TungstenSort.newOrdering() - Failed to
generate ordering, fallback to interpreted
java.util.concurrent.ExecutionException: java.lang.Exception: failed to
compile: o
cPartitionWriterContainer.writeRows(WriterContainer.scala:339)
>>
>>
>> On Tue, Oct 20, 2015 at 9:10 PM, Reynold Xin wrote:
>>
>>> With Jerry's permission, sending this back to the dev list to close the
>>> loop.
>>>
>>>
>>>
27;s permission, sending this back to the dev list to close the
>> loop.
>>
>>
>> -- Forwarded message --
>> From: Jerry Lam
>> Date: Tue, Oct 20, 2015 at 3:54 PM
>> Subject: Re: If you use Spark 1.5 and disabled Tungsten mode ...
>>
wrote:
> With Jerry's permission, sending this back to the dev list to close the
> loop.
>
>
> -- Forwarded message --
> From: Jerry Lam
> Date: Tue, Oct 20, 2015 at 3:54 PM
> Subject: Re: If you use Spark 1.5 and disabled Tungsten mode ...
> To
Hi Reynold,
Yes, I'm using 1.5.1. I see them quite often. Sometimes it recovers but
sometimes it does not. For one particular job, it failed all the time with
the acquire-memory issue. I'm using spark on mesos with fine grained mode.
Does it make a difference?
Best Regards,
Jerry
On Tue, Oct 20
Jerry - I think that's been fixed in 1.5.1. Do you still see it?
On Tue, Oct 20, 2015 at 2:11 PM, Jerry Lam wrote:
> I disabled it because of the "Could not acquire 65536 bytes of memory". It
> happens to fail the job. So for now, I'm not touching it.
>
> On Tue, Oct 20, 2015 at 4:48 PM, charmee
I disabled it because of the "Could not acquire 65536 bytes of memory". It
happens to fail the job. So for now, I'm not touching it.
On Tue, Oct 20, 2015 at 4:48 PM, charmee wrote:
> We had disabled tungsten after we found few performance issues, but had to
> enable it back because we found that
We had disabled tungsten after we found few performance issues, but had to
enable it back because we found that when we had large number of group by
fields, if tungsten is disabled the shuffle keeps failing.
Here is an excerpt from one of our engineers with his analysis.
With Tungsten Enabled (
My apologies for mixing up what was being referred to in that case! :)
Mark.
--
View this message in context:
http://apache-spark-developers-list.1001551.n3.nabble.com/If-you-use-Spark-1-5-and-disabled-Tungsten-mode-tp14604p14629.html
Sent from the Apache Spark Developers List mailing list a
To clarify, we're asking about the *spark.sql.tungsten.enabled* flag, which
was introduced in Spark 1.5 and enables Project Tungsten optimizations in
Spark SQL. This option is set to *true* by default in Spark 1.5+ and exists
primarily to allow users to disable the new code paths if they encounter
Are you referring to spark.shuffle.manager=tungsten-sort? If so, we saw the
default value as still being as the regular sort, and since it was only
first introduced in 1.5, were actually waiting a bit to see if anyone
ENABLED it as opposed to DISABLING it since - it's disabled by default! :)
I rec
15 matches
Mail list logo
