e workaround
>
> On Mon, Mar 14, 2022, 8:59 AM Bulldog20630405
> wrote:
>
>>
>> using tag v3.2.1 with java 8 getting a stackoverflow when building the
>> distribution:
>>
>> > alias mvn
>> alias mvn='mvn --errors --fail-at-end -DskipTests '
using tag v3.2.1 with java 8 getting a stackoverflow when building the
distribution:
> alias mvn
alias mvn='mvn --errors --fail-at-end -DskipTests '
> dev/make-distribution.sh --name 'hadoop-3.2' --pip --tgz -Phive
-Phive-thriftserver -Pmesos -Pyarn -Pkubernetes
[INFO]
---
coming from a yarn background; log files can be found after job finishes...
with spark master/workers how to configure to get logs after job finishes?
we have setup our spark history server and spark-defaults include:
spark.eventLog.enabled true
spark.eventLog.dir file:///sha
yes; a hard refresh did it... was a caching issue...thanx
On Mon, Mar 7, 2022 at 11:54 AM Sean Owen wrote:
> Hm, 3.2.1 shows up for me, it's the default. Try refreshing the page?
> sometimes people have an old cached copy.
>
> On Mon, Mar 7, 2022 at 10:30 AM Bulldog
from website spark 3.2.1 has been release in january 2020; however not
available for download from => https://spark.apache.org/downloads.html
(only 3.2.0)
when will spark binary 3.2.1 be available for download?
we have spark 2.4.x clusters running fine; however when running spark 3.1.1
the spark history server fails to boot
note: we build spark 3.1.1 from source for hadoop 3.2.1
we use supervisord to start services so our start command is:
$SPARK_HOME/bin/spark-class org.apache.spark.deploy.history.Hist
we have a spark cluster running on with multiple users...
when running with the user owning the cluster jobs run fine... however,
when trying to run pyspark with a different user it fails because the
.sparkStaging/application_* is written with 700 so the user cannot write to
that directory
how
0, 2021 at 1:54 PM Bulldog20630405
> wrote:
>
>>
>> what is the expected ballpark release date of spark 3.1.1 ?
>> will it be built with scala 2.13 ?
>>
>
what is the expected ballpark release date of spark 3.1.1 ?
will it be built with scala 2.13 ?
when running spark jobs we find when running the following command:
top -H -i -p
showed that a single thread labeled "map-output-disp" was running at 99.7%
for a majority of the delay period. this delay gets progressively worse
with the increase in partition count.
it seems the delay comes from t
after blowing away my m2 repo cache; i was able to build just fine... i
dont know why; but now it works :-)
On Sun, May 19, 2019 at 10:22 PM Bulldog20630405
wrote:
> i am trying to build spark 2.4.3 with the following env:
>
>- fedora 29
>- 1.8.0_202
>- spark 2.4
i am trying to build spark 2.4.3 with the following env:
- fedora 29
- 1.8.0_202
- spark 2.4.3
- scala 2.11.12
- maven 3.5.4
- hadoop 2.6.5
according to the documentation this can be done with the following commands:
*export TERM=xterm-color*
*./build/mvn -Pyarn -DskipTests clea
When running spark from spark-shell, when each defined variable created the
shell prints out the type signature of that variable along with the
toString of the instance.
how can i programmatically generated the same signature without using the
shell (for debugging purposes) from a spark script or
13 matches
Mail list logo
