mateczagany commented on PR #25509: URL: https://github.com/apache/flink/pull/25509#issuecomment-2418969804
I have reproduced this issue easily, but I could not reproduce it if snapshot compression is turned on. IIUC when using snapshot compression, `SnappyFramedInputStream#available()` will return the number of bytes uncompressed in the buffer, and we should skip those. When not using snapshot compression, we call `BufferedInputStream#available()` which will be the number of buffered bytes, and in my tests the buffer size was 4096, so if we read e.g. 50 bytes of data during the last deserialization, this was 4046. Skipping this many bytes resulted in the next seek having to seek backwards, and that will cause the S3 client to close and re-open the stream. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
