Jose Torres created SPARK-21988:
-----------------------------------
Summary: Add default stats to StreamingExecutionRelation
Key: SPARK-21988
URL: https://issues.apache.org/jira/browse/SPARK-21988
Project: Spark
Issue Type: Improvement
Components: Structured Streaming
Affects Versions: 2.2.0
Reporter: Jose Torres
StreamingExecutionRelation currently doesn't implement stats.
This makes some sense, but unfortunately the LeafNode contract requires that
nodes which survive analysis implement stats, and StreamingExecutionRelation
can indeed survive analysis when running explain() on a streaming dataframe.
This value won't ever be used during execution, because
StreamingExecutionRelation does *not* survive analysis on the execution path;
it's replaced with each batch.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
