I use java 17 to build this. Are there any reasons why you have to build spark yourself? Can't you start from spark 3.3.1 tar file and build a docker image from there?
fre. 16. des. 2022 kl. 18:26 skrev Gnana Kumar <gnana.kumar...@gmail.com>: > I have opened the pom file under cd ~/spark/sql/catalyst$ vi pom.xml and > increases from "-Xss4m" to "-Xss4g" but still no luck..It is the same > stackoverflow error. > > On Fri, Dec 16, 2022 at 9:42 PM Sean Owen <sro...@gmail.com> wrote: > >> OK that's good. Hm, I seem to recall the build needs more mem in Java 11 >> and/or some envs. As a quick check, try replacing all "-Xss4m" with >> "-Xss16m" or something larger, in the project build files. Just search and >> replace. >> >> On Fri, Dec 16, 2022 at 9:53 AM Gnana Kumar <gnana.kumar...@gmail.com> >> wrote: >> >>> I have been following below steps. >>> >>> git clone --branch branch-3.3 https://github.com/apache/spark.git >>> cd spark >>> ./dev/make-distribution.sh --tgz --name with-volcano >>> -Pkubernetes,volcano,hadoop-3 >>> >>> How to increase stack size ? Please let me know. >>> >>> Thanks >>> Gnana >>> >>> On Fri, Dec 16, 2022 at 8:45 PM Sean Owen <sro...@gmail.com> wrote: >>> >>>> You need to increase the stack size during compilation. The included >>>> mvn wrapper in build does this. Are you using it? >>>> >>>> On Fri, Dec 16, 2022 at 9:13 AM Gnana Kumar <gnana.kumar...@gmail.com> >>>> wrote: >>>> >>>>> This is my latest error and fails to build SPARK CATALYST >>>>> >>>>> Exception in thread "main" java.lang.StackOverflowError >>>>> at scala.reflect.internal.Trees.itransform(Trees.scala:1402) >>>>> at scala.reflect.internal.Trees.itransform$(Trees.scala:1400) >>>>> at >>>>> scala.reflect.internal.SymbolTable.itransform(SymbolTable.scala:28) >>>>> at >>>>> scala.reflect.internal.SymbolTable.itransform(SymbolTable.scala:28) >>>>> at >>>>> scala.reflect.api.Trees$Transformer.transform(Trees.scala:2563) >>>>> at >>>>> scala.tools.nsc.transform.TypingTransformers$TypingTransformer.transform(TypingTransformers.scala:57) >>>>> at >>>>> scala.tools.nsc.transform.ExtensionMethods$Extender.transform(ExtensionMethods.scala:275) >>>>> at >>>>> scala.tools.nsc.transform.ExtensionMethods$Extender.transform(ExtensionMethods.scala:133) >>>>> at scala.reflect.internal.Trees.itransform(Trees.scala:1436) >>>>> at scala.reflect.internal.Trees.itransform$(Trees.scala:1400) >>>>> at >>>>> scala.reflect.internal.SymbolTable.itransform(SymbolTable.scala:28) >>>>> at >>>>> scala.reflect.internal.SymbolTable.itransform(SymbolTable.scala:28) >>>>> at >>>>> scala.reflect.api.Trees$Transformer.transform(Trees.scala:2563) >>>>> >>>>> On Fri, Dec 16, 2022 at 7:16 PM Gnana Kumar <gnana.kumar...@gmail.com> >>>>> wrote: >>>>> >>>>>> Any updates on this please ? >>>>>> >>>>>> On Sat, Dec 10, 2022 at 1:36 PM Gnana Kumar <gnana.kumar...@gmail.com> >>>>>> wrote: >>>>>> >>>>>>> Thanks Bjorn. >>>>>>> >>>>>>> I have created Google Cloud Computre Engine and tried multiple times >>>>>>> but getting below error. >>>>>>> >>>>>>> I have installed JAVA 11 in my VM machine and is there any other >>>>>>> dependency which needs to be installed in VM Machine ? Please let me >>>>>>> know. >>>>>>> >>>>>>> at >>>>>>> scala.tools.nsc.transform.TypingTransformers$TypingTransformer.transform(TypingTransformers.scala:57) >>>>>>> at >>>>>>> scala.tools.nsc.transform.ExtensionMethods$Extender.transform(ExtensionMethods.scala:275) >>>>>>> at >>>>>>> scala.tools.nsc.transform.ExtensionMethods$Extender.transform(ExtensionMethods.scala:133) >>>>>>> at scala.reflect.internal.Trees.itransform(Trees.scala:1436) >>>>>>> at scala.reflect.internal.Trees.itransform$(Trees.scala:1400) >>>>>>> at >>>>>>> scala.reflect.internal.SymbolTable.itransform(SymbolTable.scala:28) >>>>>>> at >>>>>>> scala.reflect.internal.SymbolTable.itransform(SymbolTable.scala:28) >>>>>>> at >>>>>>> scala.reflect.api.Trees$Transformer.transform(Trees.scala:2563) >>>>>>> at >>>>>>> scala.tools.nsc.transform.TypingTransformers$TypingTransformer.transform(TypingTransformers.scala:57) >>>>>>> at >>>>>>> scala.tools.nsc.transform.ExtensionMethods$Extender.transform(ExtensionMethods.scala:275) >>>>>>> at >>>>>>> scala.tools.nsc.transform.ExtensionMethods$Extender.transform(ExtensionMethods.scala:133) >>>>>>> at scala.reflect.internal.Trees.itransform(Trees.scala:1411) >>>>>>> at scala.reflect.internal.Trees.itransform$(Trees.scala:1400) >>>>>>> at >>>>>>> scala.reflect.internal.SymbolTable.itransform(SymbolTable.scala:28) >>>>>>> at >>>>>>> scala.reflect.internal.SymbolTable.itransform(SymbolTable.scala:28) >>>>>>> at >>>>>>> scala.reflect.api.Trees$Transformer.transform(Trees.scala:2563) >>>>>>> at >>>>>>> scala.tools.nsc.transform.TypingTransformers$TypingTransformer.transform(TypingTransformers.scala:57) >>>>>>> at >>>>>>> scala.tools.nsc.transform.ExtensionMethods$Extender.transform(ExtensionMethods.scala:275) >>>>>>> at >>>>>>> scala.tools.nsc.transform.ExtensionMethods$Extender.transform(ExtensionMethods.scala:133) >>>>>>> at scala.reflect.internal.Trees.itransform(Trees.scala:1430) >>>>>>> at scala.reflect.internal.Trees.itransform$(Trees.scala:1400) >>>>>>> at >>>>>>> scala.reflect.internal.SymbolTable.itransform(SymbolTable.scala:28) >>>>>>> at >>>>>>> scala.reflect.internal.SymbolTable.itransform(SymbolTable.scala:28) >>>>>>> at >>>>>>> scala.reflect.api.Trees$Transformer.transform(Trees.scala:2563) >>>>>>> at >>>>>>> scala.tools.nsc.transform.TypingTransformers$TypingTransformer.transform(TypingTransformers.scala:57) >>>>>>> at >>>>>>> scala.tools.nsc.transform.ExtensionMethods$Extender.transform(ExtensionMethods.scala:275) >>>>>>> at >>>>>>> scala.tools.nsc.transform.ExtensionMethods$Extender.transform(ExtensionMethods.scala:133) >>>>>>> at scala.reflect.internal.Trees.itransform(Trees.scala:1409) >>>>>>> at scala.reflect.internal.Trees.itransform$(Trees.scala:1400) >>>>>>> at >>>>>>> scala.reflect.internal.SymbolTable.itransform(SymbolTable.scala:28) >>>>>>> at >>>>>>> scala.reflect.internal.SymbolTable.itransform(SymbolTable.scala:28) >>>>>>> at >>>>>>> scala.reflect.api.Trees$Transformer.transform(Trees.scala:2563) >>>>>>> at >>>>>>> scala.tools.nsc.transform.TypingTransformers$TypingTransformer.transform(TypingTransformers.scala:57) >>>>>>> at >>>>>>> scala.tools.nsc.transform.ExtensionMethods$Extender.transform(ExtensionMethods.scala:275) >>>>>>> at >>>>>>> scala.tools.nsc.transform.ExtensionMethods$Extender.transform(ExtensionMethods.scala:133) >>>>>>> at scala.reflect.internal.Trees.itransform(Trees.scala:1436) >>>>>>> at scala.reflect.internal.Trees.itransform$(Trees.scala:1400) >>>>>>> at >>>>>>> scala.reflect.internal.SymbolTable.itransform(SymbolTable.scala:28) >>>>>>> at >>>>>>> scala.reflect.internal.SymbolTable.itransform(SymbolTable.scala:28) >>>>>>> at >>>>>>> scala.reflect.api.Trees$Transformer.transform(Trees.scala:2563) >>>>>>> at >>>>>>> scala.tools.nsc.transform.TypingTransformers$TypingTransformer.transform(TypingTransformers.scala:57) >>>>>>> at >>>>>>> scala.tools.nsc.transform.ExtensionMethods$Extender.transform(ExtensionMethods.scala:275) >>>>>>> at >>>>>>> scala.tools.nsc.transform.ExtensionMethods$Extender.transform(ExtensionMethods.scala:133) >>>>>>> at scala.reflect.internal.Trees.itransform(Trees.scala:1411) >>>>>>> >>>>>>> >>> >>> -- >>> Thanks >>> Gnana >>> >> > > -- > Thanks > Gnana > -- Bjørn Jørgensen Vestre Aspehaug 4, 6010 Ålesund Norge +47 480 94 297
