ErikBPF commented on issue #7455: URL: https://github.com/apache/datafusion-comet/issues/7455#issuecomment-6749864920
@dwsmith1983 Thanks for outlining the check. I ran it on Apollo with Spark 4.2.0, Hadoop 3.5.0, Delta 4.4.1 and Comet 1.2.0, using the matching Delta contrib artifact. Both stock Spark and Comet passed ten checks each: - Six malformed-context Delta reads failed with matching `IllegalArgumentException` messages and Hadoop failure frames, including `S3AFileSystem.initialize`. No Spark jobs started in any of the test groups. - Four valid-context controls initialized a real `S3AFileSystem` successfully. The malformed cases cover an unset encryption algorithm, AES256, SSE-KMS, DSSE-KMS, and short/long bucket-scoped context keys. A separate successful local Parquet control confirmed Comet native scan, aggregation and shuffle operators were active. This supports the driver-setup behavior you proposed. The reads intentionally fail during Delta metadata resolution, so this does not establish successful native Delta/S3 reads or independently justify the exemption. No admission rules or SSE-C/CSE declines were changed. Delta 4.4.0 was not rerun for this check. I have the standalone reproduction script and run receipts locally and can contribute an equivalent regression test to the pairing PR. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
