zhengruifeng commented on code in PR #50530:
URL: https://github.com/apache/spark/pull/50530#discussion_r2032371968
##########
sql/connect/server/src/main/scala/org/apache/spark/sql/connect/config/Connect.scala:
##########
@@ -333,4 +333,23 @@ object Connect {
Option(System.getenv.get(CONNECT_AUTHENTICATE_TOKEN_ENV))
}
}
+
+ val CONNECT_SESSION_CONNECT_ML_CACHE_MAX_SIZE =
+ buildConf("spark.connect.session.connectML.mlCache.maxSize")
+ .doc("Maximum size of the MLCache per session. The cache will evict the
least recently" +
+ "used models if the size exceeds this limit. The size is in bytes.")
+ .version("4.0.0")
Review Comment:
```suggestion
.version("4.1.0")
```
I think this PR doesn't need to be included in 4.0.0
##########
sql/connect/server/src/main/scala/org/apache/spark/sql/connect/config/Connect.scala:
##########
@@ -333,4 +333,23 @@ object Connect {
Option(System.getenv.get(CONNECT_AUTHENTICATE_TOKEN_ENV))
}
}
+
+ val CONNECT_SESSION_CONNECT_ML_CACHE_MAX_SIZE =
+ buildConf("spark.connect.session.connectML.mlCache.maxSize")
+ .doc("Maximum size of the MLCache per session. The cache will evict the
least recently" +
+ "used models if the size exceeds this limit. The size is in bytes.")
+ .version("4.0.0")
+ .internal()
+ .bytesConf(ByteUnit.BYTE)
+ // By default, 1/3 of total designated memory (the configured -Xmx).
+ .createWithDefault(Runtime.getRuntime.maxMemory() / 3)
+
+ val CONNECT_SESSION_CONNECT_ML_CACHE_TIMEOUT =
+ buildConf("spark.connect.session.connectML.mlCache.timeout")
+ .doc("Timeout of models in MLCache. Models will be evicted from the
cache if they are not " +
+ "used for this amount of time. The timeout is in minutes.")
+ .version("4.0.0")
Review Comment:
```suggestion
.version("4.1.0")
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
