parthchandra commented on PR #2031:
URL:
https://github.com/apache/datafusion-comet/pull/2031#issuecomment-3085617517
Merged. Thank you for the reviews @andygrove @comphead @Kontinuation!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on
parthchandra merged PR #2031:
URL: https://github.com/apache/datafusion-comet/pull/2031
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscr.
parthchandra commented on PR #2031:
URL:
https://github.com/apache/datafusion-comet/pull/2031#issuecomment-3085613885
@comphead @Kontinuation @drexler-sky Opened a tracking issue for fs-hdfs
issues: https://github.com/apache/datafusion-comet/issues/2034
--
This is an automated message fr
parthchandra commented on PR #2031:
URL:
https://github.com/apache/datafusion-comet/pull/2031#issuecomment-3084912625
> > reminds me another problem with fs-hdfs.
> > The `HdfsErr` returned by `fs-hdfs` read functions does not contain JVM
stack traces. If there's a read failure caused by
parthchandra commented on code in PR #2031:
URL: https://github.com/apache/datafusion-comet/pull/2031#discussion_r2213926681
##
native/hdfs/src/object_store/hdfs.rs:
##
@@ -88,19 +88,33 @@ impl HadoopFileSystem {
fn read_range(range: &Range, file: &HdfsFile) -> Result {
comphead commented on PR #2031:
URL:
https://github.com/apache/datafusion-comet/pull/2031#issuecomment-3084571582
> This patch looks good to me, and it reminds me another problem with
fs-hdfs.
>
> The `HdfsErr` returned by `fs-hdfs` read functions does not contain JVM
stack traces.
comphead commented on code in PR #2031:
URL: https://github.com/apache/datafusion-comet/pull/2031#discussion_r2212009586
##
native/hdfs/src/object_store/hdfs.rs:
##
@@ -88,19 +88,33 @@ impl HadoopFileSystem {
fn read_range(range: &Range, file: &HdfsFile) -> Result {
parthchandra commented on PR #2031:
URL:
https://github.com/apache/datafusion-comet/pull/2031#issuecomment-3081817670
@Kontinuation @andygrove @comphead, updated based on review comments
--
This is an automated message from the Apache Git Service.
To respond to the message, please log
parthchandra commented on code in PR #2031:
URL: https://github.com/apache/datafusion-comet/pull/2031#discussion_r2210968794
##
native/hdfs/src/object_store/hdfs.rs:
##
@@ -88,19 +88,18 @@ impl HadoopFileSystem {
fn read_range(range: &Range, file: &HdfsFile) -> Result {
Kontinuation commented on PR #2031:
URL:
https://github.com/apache/datafusion-comet/pull/2031#issuecomment-3079205816
> Sorry @Kontinuation if I check your references
https://github.com/datafusion-contrib/fs-hdfs/blob/8c03c5ef0942b75abc79ed673931355fa9552131/c_src/libhdfs/hdfs.c#L1564C15-L1
Kontinuation commented on code in PR #2031:
URL: https://github.com/apache/datafusion-comet/pull/2031#discussion_r2210729874
##
native/hdfs/src/object_store/hdfs.rs:
##
@@ -141,13 +140,15 @@ impl ObjectStore for HadoopFileSystem {
let file_status = file.get_file_sta
Kontinuation commented on code in PR #2031:
URL: https://github.com/apache/datafusion-comet/pull/2031#discussion_r2210716251
##
native/hdfs/src/object_store/hdfs.rs:
##
@@ -88,19 +88,18 @@ impl HadoopFileSystem {
fn read_range(range: &Range, file: &HdfsFile) -> Result {
comphead commented on code in PR #2031:
URL: https://github.com/apache/datafusion-comet/pull/2031#discussion_r2210658711
##
native/hdfs/src/object_store/hdfs.rs:
##
@@ -88,19 +88,18 @@ impl HadoopFileSystem {
fn read_range(range: &Range, file: &HdfsFile) -> Result {
andygrove commented on code in PR #2031:
URL: https://github.com/apache/datafusion-comet/pull/2031#discussion_r2209180081
##
native/hdfs/src/object_store/hdfs.rs:
##
@@ -88,19 +88,18 @@ impl HadoopFileSystem {
fn read_range(range: &Range, file: &HdfsFile) -> Result {
codecov-commenter commented on PR #2031:
URL:
https://github.com/apache/datafusion-comet/pull/2031#issuecomment-3076413162
##
[Codecov](https://app.codecov.io/gh/apache/datafusion-comet/pull/2031?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_ca
parthchandra commented on PR #2031:
URL:
https://github.com/apache/datafusion-comet/pull/2031#issuecomment-3076378047
@Kontinuation, fyi
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the speci
parthchandra opened a new pull request, #2031:
URL: https://github.com/apache/datafusion-comet/pull/2031
The `get` and `read_range` methods in hdfs object store implementation do
not always read the data requested because the underlying hdfs call may return
fewer bytes than requested.
ht
17 matches
Mail list logo
