Retry.
From: Anil Dasari
Date: Tuesday, July 12, 2022 at 3:42 PM
To: user@spark.apache.org
Subject: Spark streaming pending mircobatches queue max length
Hello,
Spark is adding entry to pending microbatches queue at periodic batch interval.
Is there config to set the max size for pending micro
Hello,
As of now Spark Continous Processing does not support logical relation
operations like "dataframe.join()". Are there any plans to make it happen
in future relases?
Thanks in advance for your work.
Mikołaj
Hi,
I think that this is a pure example of over engineering.
Ayan's advice is the best. Please use SPARK SQL function called as
input_file_name() to join the tables. People do not think in terms of RDD
anymore unless absolutely required.
Also if you have different JSON schemas, just use the SPAR
