Re: [PR] Made imdb download (data_imdb) function atomic [datafusion]

2025-01-23 Thread via GitHub
alamb commented on PR #14225: URL: https://github.com/apache/datafusion/pull/14225#issuecomment-2611135717 > Oh thanks! Sorry for the oversight. I will be more careful from now on. Thanks again! You've been extremely patient!! No worries -- I am sorry we have had so many back and fort

Re: [PR] Made imdb download (data_imdb) function atomic [datafusion]

2025-01-23 Thread via GitHub
alamb merged PR #14225: URL: https://github.com/apache/datafusion/pull/14225 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@datafusi

Re: [PR] Made imdb download (data_imdb) function atomic [datafusion]

2025-01-23 Thread via GitHub
Spaarsh commented on PR #14225: URL: https://github.com/apache/datafusion/pull/14225#issuecomment-2609946746 > Thank you @Spaarsh > > I tried this out locally and found that now the conversion code was not triggered. > > I pushed a fix to your branch locally (moving where the `

Re: [PR] Made imdb download (data_imdb) function atomic [datafusion]

2025-01-22 Thread via GitHub
Spaarsh commented on code in PR #14225: URL: https://github.com/apache/datafusion/pull/14225#discussion_r1926476600 ## benchmarks/bench.sh: ## @@ -536,23 +536,52 @@ data_imdb() { done if [ "$convert_needed" = true ]; then -if [ ! -f "${imdb_dir}/imdb.tgz" ];

Re: [PR] Made imdb download (data_imdb) function atomic [datafusion]

2025-01-21 Thread via GitHub
alamb commented on code in PR #14225: URL: https://github.com/apache/datafusion/pull/14225#discussion_r1924394309 ## benchmarks/bench.sh: ## @@ -536,23 +536,52 @@ data_imdb() { done if [ "$convert_needed" = true ]; then -if [ ! -f "${imdb_dir}/imdb.tgz" ]; th

[PR] Made imdb download (data_imdb) function atomic [datafusion]

2025-01-21 Thread via GitHub
Spaarsh opened a new pull request, #14225: URL: https://github.com/apache/datafusion/pull/14225 ## Which issue does this PR close? Closes #14128 ## Rationale for this change Due to non-atomic downloads, the user would need to manually remove files/folders created by