gjxdxh commented on code in PR #50780:
URL: https://github.com/apache/spark/pull/50780#discussion_r2078668841
##########
python/pyspark/sql/streaming/listener.py:
##########
@@ -178,29 +178,44 @@ class QueryStartedEvent:
"""
def __init__(
- self, id: uuid.UUID, runId: uuid.UUID, name: Optional[str], timestamp:
str
+ self,
+ id: uuid.UUID,
+ runId: uuid.UUID,
+ name: Optional[str],
+ timestamp: str,
+ jobTags: Set[str] = set(),
) -> None:
self._id: uuid.UUID = id
self._runId: uuid.UUID = runId
self._name: Optional[str] = name
self._timestamp: str = timestamp
+ self._jobTags: Set[str] = jobTags
@classmethod
def fromJObject(cls, jevent: "JavaObject") -> "QueryStartedEvent":
+ job_tags = set()
+ java_iterator = jevent.jobTags().iterator()
Review Comment:
I tried, but it didn't works. See the comment I put above
https://github.com/apache/spark/pull/50780#discussion_r2072222250. I don't know
why though, this is the tests result I got when running locally
```
lingkai.kong@K9WHXLR93K spark % python/run-tests --testnames
'pyspark.sql.tests.streaming.test_streaming_listener
StreamingListenerTests.test_listener_events'
Running PySpark tests. Output is in
/Users/lingkai.kong/spark/python/unit-tests.log
Will test against the following Python executables: ['python3.9']
Will test the following Python tests:
['pyspark.sql.tests.streaming.test_streaming_listener
StreamingListenerTests.test_listener_events']
python3.9 python_implementation is CPython
python3.9 version is: Python 3.9.6
Starting test(python3.9):
pyspark.sql.tests.streaming.test_streaming_listener
StreamingListenerTests.test_listener_events (temp output:
/Users/lingkai.kong/spark/python/target/ed1f4b6e-6661-4815-84fd-00bf6cedd0ab/python3.9__pyspark.sql.tests.streaming.test_streaming_listener_StreamingListenerTests.test_listener_events__ck29eqqs.log)
WARNING: Using incubator modules: jdk.incubator.vector
Setting default log level to "WARN".
To adjust logging level use sc.setLogLevel(newLevel). For SparkR, use
setLogLevel(newLevel).
test_listener_events
(pyspark.sql.tests.streaming.test_streaming_listener.StreamingListenerTests)
... FAIL
======================================================================
FAIL: test_listener_events
(pyspark.sql.tests.streaming.test_streaming_listener.StreamingListenerTests)
----------------------------------------------------------------------
Traceback (most recent call last):
File
"/Users/lingkai.kong/spark/python/pyspark/sql/tests/streaming/test_streaming_listener.py",
line 413, in test_listener_events
verify(TestListenerV1())
File
"/Users/lingkai.kong/spark/python/pyspark/sql/tests/streaming/test_streaming_listener.py",
line 396, in verify
self.check_start_event(start_event)
File
"/Users/lingkai.kong/spark/python/pyspark/sql/tests/streaming/test_streaming_listener.py",
line 40, in check_start_event
self.assertTrue(isinstance(event, QueryStartedEvent))
AssertionError: False is not true
----------------------------------------------------------------------
Ran 1 test in 5.903s
FAILED (failures=1)
Had test failures in pyspark.sql.tests.streaming.test_streaming_listener
StreamingListenerTests.test_listener_events with python3.9; see logs.
```
Could this be a issue with regarding to package version etc?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
