Re: test flakiness with SocketException of broken pipe in HiveMetaStoreClient

2021-01-07 Thread OpenInx
> I was able to almost 100% reproduce the HiveMetaStoreClient aborted connection problem locally with Flink tests after adding another DeleteReadTests for the new FLIP-27 source impl in my dev branch I think I found the cause why it's easy to fail. The TestFlinkInputFormatReaderDeletes will crea

About proposal for including Partition stats for Iceberg tables

2021-01-07 Thread Piyush Vinay Hurpade
Hi Team, Need some help regarding our proposal for a partition-stats file within each snapshot. With each snapshot we are proposing a partition-stats avro file that contains information about all partitions in the table. So the schema we decide to have is *(partition_spec_id(int), partition(Partiti

Re: test flakiness with SocketException of broken pipe in HiveMetaStoreClient

2021-01-07 Thread Steven Wu
Ryan/OpenInx, thanks a lot for the pointers. I was able to almost 100% reproduce the HiveMetaStoreClient aborted connection problem locally with Flink tests after adding another DeleteReadTests for the new FLIP-27 source impl in my dev branch. I don't see the problem anymore after switching the Fl