andygrove commented on code in PR #5543:
URL: https://github.com/apache/datafusion-comet/pull/5543#discussion_r4018240605
##########
spark/src/test/scala/org/apache/comet/exec/CometInMemoryCacheSuite.scala:
##########
@@ -1584,52 +1838,64 @@ class CometInMemoryCacheSuite extends CometTestBase {
.cachedRepresentation
try {
- f(relation, relation.cacheBuilder.cachedColumnBuffers.collect())
+ f(relation)
} finally {
spark.catalog.clearCache()
}
}
}
- test("Comet in-memory cache broadcasts a batch whose columns have separate
dictionaries") {
- // A broadcast of a cache scan re-serializes each decoded batch as one
stream covering every
- // column, and the writer resolves all of their dictionary IDs against the
single provider it
- // is handed. The columns were decoded from separate streams, so they
arrive carrying separate
- // providers: passing any one of them cannot resolve the others.
- withDictionaryCache { (relation, batches) =>
+ test("Comet in-memory cache releases its vectors when a column fails after a
partial decode") {
Review Comment:
Merged — one test, two corruption points, one fixture and one allocator
snapshot per point. The commentary about what each point catches moved into the
test's own comment.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
