Hi Marco, Many thanks for such a quick response. With that, I'll direct my curiosity into a different direction. Thanks!
Pozdrawiam, Jacek Laskowski ---- https://about.me/JacekLaskowski Mastering Spark SQL https://bit.ly/mastering-spark-sql Spark Structured Streaming https://bit.ly/spark-structured-streaming Mastering Kafka Streams https://bit.ly/mastering-kafka-streams Follow me at https://twitter.com/jaceklaskowski On Fri, Nov 16, 2018 at 1:44 PM Marco Gaido <marcogaid...@gmail.com> wrote: > Hi Jacek, > > I do believe it is correct. Please check the method you mentioned > (CodeGeneratorWithInterpretedFallback.createObject): the value is relevant > only if Utils.isTesting. > > Thanks, > Marco > > Il giorno ven 16 nov 2018 alle ore 13:28 Jacek Laskowski <ja...@japila.pl> > ha scritto: > >> Hi, >> >> While reviewing the changes in 2.4 I stumbled >> upon spark.sql.codegen.factoryMode internal configuration property [1]. The >> doc says: >> >> > Note that this config works only for tests. >> >> Is that correct? I've got some doubts. >> >> I found that it's used in UnsafeProjection.create [2] (through >> CodeGeneratorWithInterpretedFallback.createObject) which is used outside >> the tests and so made me think if "this config works only for tests" part >> is correct. >> >> Are my doubts correct? If not, what am I missing? Thanks. >> >> [1] >> https://github.com/apache/spark/blob/v2.4.0/sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala#L758-L767 >> >> [2] >> https://github.com/apache/spark/blob/v2.4.0/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/Projection.scala#L159 >> >> Pozdrawiam, >> Jacek Laskowski >> ---- >> https://about.me/JacekLaskowski >> Mastering Spark SQL https://bit.ly/mastering-spark-sql >> Spark Structured Streaming https://bit.ly/spark-structured-streaming >> Mastering Kafka Streams https://bit.ly/mastering-kafka-streams >> Follow me at https://twitter.com/jaceklaskowski >> >
