Hi,

While reviewing the changes in 2.4 I stumbled
upon spark.sql.codegen.factoryMode internal configuration property [1]. The
doc says:

> Note that this config works only for tests.

Is that correct? I've got some doubts.

I found that it's used in UnsafeProjection.create [2] (through
CodeGeneratorWithInterpretedFallback.createObject) which is used outside
the tests and so made me think if "this config works only for tests" part
is correct.

Are my doubts correct? If not, what am I missing? Thanks.

[1]
https://github.com/apache/spark/blob/v2.4.0/sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala#L758-L767

[2]
https://github.com/apache/spark/blob/v2.4.0/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/Projection.scala#L159

Pozdrawiam,
Jacek Laskowski
----
https://about.me/JacekLaskowski
Mastering Spark SQL https://bit.ly/mastering-spark-sql
Spark Structured Streaming https://bit.ly/spark-structured-streaming
Mastering Kafka Streams https://bit.ly/mastering-kafka-streams
Follow me at https://twitter.com/jaceklaskowski

Reply via email to