Hyukjin Kwon created SPARK-61185:
------------------------------------
Summary: Remove each registration of a streaming query listener
exactly once, also of a listener added twice or removed through another
session, in the Python client as PySpark does
Key: SPARK-61185
URL: https://issues.apache.org/jira/browse/SPARK-61185
Project: Spark
Issue Type: Bug
Components: Connect
Affects Versions: connect-rust-4.3.0
Reporter: Hyukjin Kwon
The Python client keeps one registration id per listener object, so
removeListener removes only one entry of the listener bus of the session and
forgets the id even when that bus never had the listener.
Repro: add the same listener twice, then remove it twice; the bus keeps one
entry and the listener keeps getting query events.
Also: addListener on spark.streams, then removeListener on spark2.streams
(another session) drops the id, so the later spark.streams.removeListener is a
no-op.
PySpark removes one entry per removeListener from the bus of the session, so
both cases end with an empty bus and no events.
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
