for your attention.
>
>
> 2024年9月9日(月) 12:40 Prabodh Agarwal :
>
>> My 2 cents regarding my experience with using spark connect in cluster
>> mode.
>>
>> 1. Create a spark cluster of 2 or more nodes. Make 1 node as master &
>> other nodes as workers. Dep
; Elango
>
>
> On Tue, 6 Aug 2024 at 6:08 PM, Prabodh Agarwal
> wrote:
>
>> Do you get some error on passing the master option to your spark connect
>> command?
>>
>> On Tue, 6 Aug, 2024, 15:36 Ilango, wrote:
>>
>>>
>>>
>>>
&g
>
>
>
> This is my spark connect connection
>
>
>
> spark = SparkSession.builder\
>
> .remote("sc://:15002")\
>
> .getOrCreate()
>
>
> Thanks,
> Elango
>
>
> On Tue, 6 Aug 2024 at 5:45 PM, Prabodh Agarwal
> wrote:
>
>
There is an executors tab on spark connect. It's contents are generally
similar to the workers section of the spark master ui.
You might need to specify --master option in your spark connect command if
you haven't done so yet.
On Tue, 6 Aug, 2024, 14:19 Ilango, wrote:
>
> Hi all,
>
> I am evalu
lp.
>
>
>
> Thanks,
> Elango
>
>
> On Tue, 30 Jul 2024 at 11:05 AM, Prabodh Agarwal
> wrote:
>
>> Yeah. I understand the problem. One of the ways is to actually place the
>> spark connect jar in the $SPARK_HOME/jars folder. That is how we run spark
>>
.deploy.SparkSubmit.doRunMain$1(SparkSubmit.scala:194)
>
> at
> org.apache.spark.deploy.SparkSubmit.submit(SparkSubmit.scala:217)
>
> at
> org.apache.spark.deploy.SparkSubmit.doSubmit(SparkSubmit.scala:91)
>
> at
> org.apache.spark.deploy.SparkSubmi
The spark connect startup prints the log location. Is that not feasible for
you?
For me log comes to $SPARK_HOME/logs
On Mon, 29 Jul, 2024, 15:30 Ilango, wrote:
>
> Hi all,
>
>
> I am facing issues with a Spark Connect application running on a Spark
> standalone cluster (without YARN and HDFS).
I have configured a spark standalone cluster as follows:
```
# start spark master
$SPARK_HOME/sbin/start-master.sh
# start 2 spark workers
SPARK_WORKER_INSTANCES=2 $SPARK_HOME/sbin/start-worker.sh
spark://localhost:7077
# start spark connect
$SPARK_HOME/sbin/start-connect-server.sh --properties-
