Re: [PR] Remove redundant statistics from FileScanConfig [datafusion]

2025-04-03 Thread via GitHub
alamb commented on PR #14955: URL: https://github.com/apache/datafusion/pull/14955#issuecomment-2775468539 Thanks again @Standing-Man and @xudong963 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

Re: [PR] Remove redundant statistics from FileScanConfig [datafusion]

2025-04-03 Thread via GitHub
alamb merged PR #14955: URL: https://github.com/apache/datafusion/pull/14955 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@datafusi

Re: [PR] Remove redundant statistics from FileScanConfig [datafusion]

2025-03-30 Thread via GitHub
alamb commented on PR #14955: URL: https://github.com/apache/datafusion/pull/14955#issuecomment-2764527928 > It seems the `statistics` generated in line 884 will be lost. > > https://github.com/apache/datafusion/blob/main/datafusion/core/src/datasource/listing/table.rs#L884-L960

Re: [PR] Remove redundant statistics from FileScanConfig [datafusion]

2025-03-30 Thread via GitHub
xudong963 commented on PR #14955: URL: https://github.com/apache/datafusion/pull/14955#issuecomment-2764553128 > > It seems the `statistics` generated in line 884 will be lost. > > > > > > https://github.com/apache/datafusion/blob/main/datafusion/core/src/datasource/listing/ta

Re: [PR] Remove redundant statistics from FileScanConfig [datafusion]

2025-03-29 Thread via GitHub
Standing-Man commented on PR #14955: URL: https://github.com/apache/datafusion/pull/14955#issuecomment-2764329379 Thanks for your valuable contributions! I’ll continue working on fixing this issue. -- This is an automated message from the Apache Git Service. To respond to the message, ple

Re: [PR] Remove redundant statistics from FileScanConfig [datafusion]

2025-03-28 Thread via GitHub
blaginin commented on PR #14955: URL: https://github.com/apache/datafusion/pull/14955#issuecomment-2762094667 hey @Standing-Man https://github.com/apache/datafusion/pull/15352 just got merged so this pr may be easier to finish fyi 🌻 -- This is an automated message from the Apache Git Serv

Re: [PR] Remove redundant statistics from FileScanConfig [datafusion]

2025-03-03 Thread via GitHub
Standing-Man commented on PR #14955: URL: https://github.com/apache/datafusion/pull/14955#issuecomment-2694274105 > Looks like there are some CI issues to address > > Note that @blaginin fixed some issues recently, so if you merge up from main it might be better now Hi @alamb, i wi

Re: [PR] Remove redundant statistics from FileScanConfig [datafusion]

2025-03-03 Thread via GitHub
blaginin commented on PR #14955: URL: https://github.com/apache/datafusion/pull/14955#issuecomment-2694459822 i feel it may be easier if we fix https://github.com/apache/datafusion/issues/14936 first. I was planning to do it this week, but feel free to take over (just take the issue then)

Re: [PR] Remove redundant statistics from FileScanConfig [datafusion]

2025-03-03 Thread via GitHub
Standing-Man commented on PR #14955: URL: https://github.com/apache/datafusion/pull/14955#issuecomment-2694451044 Hi @alamb and @blaginin, I found that four tests failed due to the statistics `num_rows` and `total_byte_size`. I'm confused about how to proceed with fixing this issue, and I n

Re: [PR] Remove redundant statistics from FileScanConfig [datafusion]

2025-03-03 Thread via GitHub
alamb commented on PR #14955: URL: https://github.com/apache/datafusion/pull/14955#issuecomment-2694196626 Looks like there are some CI issues to address Note that @blaginin fixed some issues recently, so if you merge up from main it might be better now -- This is an automated mess