Memory are not used according to setting

[William Li]

Hi All - I have a four worker node cluster, each with 8GB memory. When I submit 
a job, the driver node takes 1gb memory, each worker node only allocates one 
executor, also just take 1gb memory. The setting of the job has:


sparkConf
  .setExecutorEnv("spark.driver.memory", "6g")
  .setExecutorEnv("spark.dynamicAllocation.enabled", "true")
  .setExecutorEnv("spark.executor.cores","8")
  .setExecutorEnv("spark.executor.memory", "6g")


Any one knows how to make the worker or executor use more memory?


Thanks,


William.