Kent Yao created SPARK-55832:
--------------------------------
Summary: Reduce periodic GC interval in ThriftServerQueryTestSuite
to mitigate OOM
Key: SPARK-55832
URL: https://issues.apache.org/jira/browse/SPARK-55832
Project: Spark
Issue Type: Improvement
Components: SQL, Tests
Affects Versions: 4.2.0
Reporter: Kent Yao
ThriftServerQueryTestSuite runs 800+ SQL golden file tests sequentially in a
single SparkSession. The default spark.cleaner.periodicGC.interval is 30
minutes, which is too infrequent for the memory pressure in this long-running
suite (~50 min). This causes intermittent OOM (java.lang.OutOfMemoryError: Java
heap space) during scalar subquery tests, with 100+ failed page allocations
(64MB each).
Reduce the periodic GC interval to 60 seconds for this test suite to allow more
frequent garbage collection and prevent memory buildup.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
