i was reading this blog post from last year about structured streaming
run-once trigger:
https://databricks.com/blog/2017/05/22/running-streaming-jobs-day-10x-cost-savings.html
its a nice idea to replace a batch job with structured streaming because it
does the bookkeeping (whats new, failure reco
Now its uploaded! Thanks :)
--
Sent from: http://apache-spark-user-list.1001560.n3.nabble.com/
-
To unsubscribe e-mail: user-unsubscr...@spark.apache.org
I don't seem to find the log.
Can you double check ?
Thanks
Original message From: alexzautke
Date: 10/27/18 8:54 AM (GMT-08:00) To:
user@spark.apache.org Subject: Re: SIGBUS (0xa) when using
DataFrameWriter.insertInto
Please also find attached a complete error log.
--
Se
Hi Jorn,
Thanks for your kind reply. I do accept that there might be something in the
code. Any help would be appreciated.
To give you some insights, I checked the source of the message in kafka if
it has been repeated twice. But, I could only find it once. Also, it would
have been convincing if
Please also find attached a complete error log.
--
Sent from: http://apache-spark-user-list.1001560.n3.nabble.com/
-
To unsubscribe e-mail: user-unsubscr...@spark.apache.org
Hi everyone!
I am currently running into the issue that a call to
DataFrameWriter.insertInto is reproducibly crashing the JVM.
#
# A fatal error has been detected by the Java Runtime Environment:
#
# SIGBUS (0xa) at pc=3D0x0001194a3520, pid=3D16154,
tid=3D0x8417
#
# JRE version:
Hi,
There is an unmerged PR which can be used against spark 2.4(if you are
interested) or master branch(3.0). Spark 2.3 K8S lacks lot of features. I
suggest you upgrade to 2.4 which will be released in few days from now.
https://github.com/apache/spark/pull/22722
Regards
Surya
From: Matt Cheah
Do you have some code that you can share?
Maybe it is something in your code that unintentionally duplicates it?
Maybe your source (eg the application putting it on Kafka?)duplicates them
already?
Once and only once processing needs to be done end to end.
> Am 27.10.2018 um 02:10 schrieb ravids
