HeartSaVioR commented on code in PR #49488:
URL: https://github.com/apache/spark/pull/49488#discussion_r1948009666
##########
sql/connect/common/src/main/protobuf/spark/connect/relations.proto:
##########
@@ -1031,6 +1033,30 @@ message GroupMap {
// (Optional) The schema for the grouped state.
optional DataType state_schema = 10;
+
+ // Below fields are only used by TransformWithState
+ // (Optional) TransformWithState related parameters.
+ optional TransformWithStateInfo transformWithStateInfo = 11;
+}
+
+// Event time rule needed for chaining of operator of TransformWithState
Review Comment:
Good point. It is really not needed to make the logical representation of
Spark Connect and the logical representation of Catalyst to be same. We can
just include event time column into protocal and defer dealing with it to the
catalyst. Ideally the protocol should reflect the users' input and trying to
have another (small) analyzer in client side is never great.
@jingz-db Shall we follow @hvanhovell 's suggestion?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
