[jira] [Commented] (HUDI-3242) Checkpoint 0 is ignored -Partial parquet file discovery after the first commit

2022-01-20 Thread Harsha Teja Kanna (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3242?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17479240#comment-17479240 ] Harsha Teja Kanna commented on HUDI-3242: - So what I found from further debugging

[jira] [Comment Edited] (HUDI-3242) Checkpoint 0 is ignored -Partial parquet file discovery after the first commit

2022-01-20 Thread Harsha Teja Kanna (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3242?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17479240#comment-17479240 ] Harsha Teja Kanna edited comment on HUDI-3242 at 1/20/22, 10:23 AM:

[jira] [Commented] (HUDI-3242) Checkpoint 0 is ignored -Partial parquet file discovery after the first commit

2022-01-20 Thread Harsha Teja Kanna (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3242?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17479631#comment-17479631 ] Harsha Teja Kanna commented on HUDI-3242: - Hi, Initially I saw the behavior of no

[jira] [Updated] (HUDI-3242) Checkpoint 0 is ignored -Partial parquet file discovery after the first commit

2022-01-20 Thread Harsha Teja Kanna (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3242?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Harsha Teja Kanna updated HUDI-3242: Attachment: Screen Shot 2022-01-20 at 1.36.48 PM.png > Checkpoint 0 is ignored -Partial parq

[jira] [Commented] (HUDI-3242) Checkpoint 0 is ignored -Partial parquet file discovery after the first commit

2022-01-20 Thread Harsha Teja Kanna (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3242?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17479633#comment-17479633 ] Harsha Teja Kanna commented on HUDI-3242: - 5) checkpoint from the last commit befo

[jira] [Commented] (HUDI-3242) Checkpoint 0 is ignored -Partial parquet file discovery after the first commit

2022-01-20 Thread Harsha Teja Kanna (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3242?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17479644#comment-17479644 ] Harsha Teja Kanna commented on HUDI-3242: - ./20220114204813242.commit:    "deltast

[jira] [Commented] (HUDI-3242) Checkpoint 0 is ignored -Partial parquet file discovery after the first commit

2022-01-20 Thread Harsha Teja Kanna (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3242?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17479650#comment-17479650 ] Harsha Teja Kanna commented on HUDI-3242: - ./20220114204813242.commit:    "deltast

[jira] [Commented] (HUDI-3242) Checkpoint 0 is ignored -Partial parquet file discovery after the first commit

2022-01-20 Thread Harsha Teja Kanna (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3242?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17479685#comment-17479685 ] Harsha Teja Kanna commented on HUDI-3242: - Let me try that. Yes, I am aware that

[jira] [Comment Edited] (HUDI-3242) Checkpoint 0 is ignored -Partial parquet file discovery after the first commit

2022-01-20 Thread Harsha Teja Kanna (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3242?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17479696#comment-17479696 ] Harsha Teja Kanna edited comment on HUDI-3242 at 1/20/22, 9:41 PM: -

[jira] [Commented] (HUDI-3242) Checkpoint 0 is ignored -Partial parquet file discovery after the first commit

2022-01-20 Thread Harsha Teja Kanna (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3242?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17479696#comment-17479696 ] Harsha Teja Kanna commented on HUDI-3242: - I have a dataset partitioned but have v

[jira] [Comment Edited] (HUDI-3242) Checkpoint 0 is ignored -Partial parquet file discovery after the first commit

2022-01-20 Thread Harsha Teja Kanna (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3242?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17479696#comment-17479696 ] Harsha Teja Kanna edited comment on HUDI-3242 at 1/20/22, 9:42 PM: -

[jira] [Commented] (HUDI-3242) Checkpoint 0 is ignored -Partial parquet file discovery after the first commit

2022-01-20 Thread Harsha Teja Kanna (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3242?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17479835#comment-17479835 ] Harsha Teja Kanna commented on HUDI-3242: - Input: monthly partitions partition=20

[jira] [Commented] (HUDI-3216) Support timestamp with microseconds precision

2022-01-22 Thread Harsha Teja Kanna (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3216?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17480496#comment-17480496 ] Harsha Teja Kanna commented on HUDI-3216: - Hi team, Need confirmation if this wil

[jira] [Created] (HUDI-3335) Loading Hudi table fails with NullPointerException

2022-01-26 Thread Harsha Teja Kanna (Jira)
Harsha Teja Kanna created HUDI-3335: --- Summary: Loading Hudi table fails with NullPointerException Key: HUDI-3335 URL: https://issues.apache.org/jira/browse/HUDI-3335 Project: Apache Hudi Is

[jira] [Updated] (HUDI-3335) Loading Hudi table fails with NullPointerException

2022-01-26 Thread Harsha Teja Kanna (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3335?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Harsha Teja Kanna updated HUDI-3335: Description: Environment Spark 3.1.2 Hudi 0.10.1 Query import org.apache.hudi.DataSource

[jira] [Updated] (HUDI-3335) Loading Hudi table fails with NullPointerException

2022-01-26 Thread Harsha Teja Kanna (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3335?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Harsha Teja Kanna updated HUDI-3335: Description: Have a COW table with metadata enabled. Loading from Spark query fails with ja

[jira] [Updated] (HUDI-3335) Loading Hudi table fails with NullPointerException

2022-01-26 Thread Harsha Teja Kanna (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3335?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Harsha Teja Kanna updated HUDI-3335: Description: Have a COW table with metadata enabled. Loading from Spark query fails with ja

[jira] [Updated] (HUDI-3335) Loading Hudi table fails with NullPointerException

2022-01-27 Thread Harsha Teja Kanna (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3335?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Harsha Teja Kanna updated HUDI-3335: Description: Have a COW table with metadata enabled. Loading from Spark query fails with ja

[jira] [Updated] (HUDI-3335) Loading Hudi table fails with NullPointerException

2022-01-27 Thread Harsha Teja Kanna (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3335?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Harsha Teja Kanna updated HUDI-3335: Description: Have a COW table with metadata enabled. Loading from Spark query fails with ja

[jira] [Updated] (HUDI-3335) Loading Hudi table fails with NullPointerException

2022-01-27 Thread Harsha Teja Kanna (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3335?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Harsha Teja Kanna updated HUDI-3335: Description: Have a COW table with metadata enabled. Loading from Spark query fails with ja

[jira] [Commented] (HUDI-3335) Loading Hudi table fails with NullPointerException

2022-01-27 Thread Harsha Teja Kanna (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3335?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17483603#comment-17483603 ] Harsha Teja Kanna commented on HUDI-3335: - Hi, 1) Yes partitions are of type 'yyy

[jira] [Comment Edited] (HUDI-3335) Loading Hudi table fails with NullPointerException

2022-01-27 Thread Harsha Teja Kanna (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3335?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17483603#comment-17483603 ] Harsha Teja Kanna edited comment on HUDI-3335 at 1/28/22, 7:25 AM: -

[jira] [Comment Edited] (HUDI-3335) Loading Hudi table fails with NullPointerException

2022-01-27 Thread Harsha Teja Kanna (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3335?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17483603#comment-17483603 ] Harsha Teja Kanna edited comment on HUDI-3335 at 1/28/22, 7:27 AM: -

[jira] [Comment Edited] (HUDI-3335) Loading Hudi table fails with NullPointerException

2022-01-27 Thread Harsha Teja Kanna (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3335?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17483603#comment-17483603 ] Harsha Teja Kanna edited comment on HUDI-3335 at 1/28/22, 7:30 AM: -

[jira] [Commented] (HUDI-3335) Loading Hudi table fails with NullPointerException

2022-01-27 Thread Harsha Teja Kanna (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3335?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17483610#comment-17483610 ] Harsha Teja Kanna commented on HUDI-3335: - Log 22/01/28 01:29:34 INFO Executor:

[jira] [Comment Edited] (HUDI-3335) Loading Hudi table fails with NullPointerException

2022-01-28 Thread Harsha Teja Kanna (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3335?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17483603#comment-17483603 ] Harsha Teja Kanna edited comment on HUDI-3335 at 1/28/22, 8:34 AM: -

[jira] [Commented] (HUDI-3335) Loading Hudi table fails with NullPointerException

2022-01-28 Thread Harsha Teja Kanna (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3335?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17483970#comment-17483970 ] Harsha Teja Kanna commented on HUDI-3335: - 22/01/28 15:14:14 INFO HoodieFileIndex:

[jira] [Commented] (HUDI-3335) Loading Hudi table fails with NullPointerException

2022-01-30 Thread Harsha Teja Kanna (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3335?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17484420#comment-17484420 ] Harsha Teja Kanna commented on HUDI-3335: - Hi, I cannot do that immediately(I will

[jira] [Comment Edited] (HUDI-3335) Loading Hudi table fails with NullPointerException

2022-01-30 Thread Harsha Teja Kanna (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3335?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17484420#comment-17484420 ] Harsha Teja Kanna edited comment on HUDI-3335 at 1/30/22, 8:17 PM: -

[jira] [Commented] (HUDI-3335) Loading Hudi table fails with NullPointerException

2022-01-30 Thread Harsha Teja Kanna (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3335?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17484469#comment-17484469 ] Harsha Teja Kanna commented on HUDI-3335: - Hi, Thanks. by deleting the metadata an

[jira] [Commented] (HUDI-3335) Loading Hudi table fails with NullPointerException

2022-02-04 Thread Harsha Teja Kanna (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3335?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17487008#comment-17487008 ] Harsha Teja Kanna commented on HUDI-3335: - This is happening again on the same tab

[jira] [Updated] (HUDI-3066) Very slow file listing after enabling metadata for existing tables in 0.10.0 release

2021-12-21 Thread Harsha Teja Kanna (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3066?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Harsha Teja Kanna updated HUDI-3066: Attachment: writer_log.txt > Very slow file listing after enabling metadata for existing tab

[jira] [Updated] (HUDI-3066) Very slow file listing after enabling metadata for existing tables in 0.10.0 release

2021-12-21 Thread Harsha Teja Kanna (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3066?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Harsha Teja Kanna updated HUDI-3066: Attachment: (was: writer_log.txt) > Very slow file listing after enabling metadata for e

[jira] [Updated] (HUDI-3066) Very slow file listing after enabling metadata for existing tables in 0.10.0 release

2021-12-21 Thread Harsha Teja Kanna (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3066?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Harsha Teja Kanna updated HUDI-3066: Attachment: writer_log.txt > Very slow file listing after enabling metadata for existing tab

[jira] [Commented] (HUDI-3066) Very slow file listing after enabling metadata for existing tables in 0.10.0 release

2021-12-21 Thread Harsha Teja Kanna (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3066?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17463385#comment-17463385 ] Harsha Teja Kanna commented on HUDI-3066: - Hi [~manojg]  It is single writer, inl

[jira] [Comment Edited] (HUDI-3066) Very slow file listing after enabling metadata for existing tables in 0.10.0 release

2021-12-21 Thread Harsha Teja Kanna (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3066?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17463385#comment-17463385 ] Harsha Teja Kanna edited comment on HUDI-3066 at 12/21/21, 5:58 PM:

[jira] [Commented] (HUDI-3066) Very slow file listing after enabling metadata for existing tables in 0.10.0 release

2021-12-21 Thread Harsha Teja Kanna (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3066?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17463442#comment-17463442 ] Harsha Teja Kanna commented on HUDI-3066: - There is no separate cleaner job. It i

[jira] [Comment Edited] (HUDI-3066) Very slow file listing after enabling metadata for existing tables in 0.10.0 release

2021-12-21 Thread Harsha Teja Kanna (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3066?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17463442#comment-17463442 ] Harsha Teja Kanna edited comment on HUDI-3066 at 12/21/21, 7:52 PM:

[jira] [Commented] (HUDI-3066) Very slow file listing after enabling metadata for existing tables in 0.10.0 release

2021-12-21 Thread Harsha Teja Kanna (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3066?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17463444#comment-17463444 ] Harsha Teja Kanna commented on HUDI-3066: - metadata files [^metadata_files.txt]

[jira] [Updated] (HUDI-3066) Very slow file listing after enabling metadata for existing tables in 0.10.0 release

2021-12-21 Thread Harsha Teja Kanna (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3066?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Harsha Teja Kanna updated HUDI-3066: Attachment: metadata_files.txt > Very slow file listing after enabling metadata for existing

[jira] [Commented] (HUDI-3066) Very slow file listing after enabling metadata for existing tables in 0.10.0 release

2021-12-21 Thread Harsha Teja Kanna (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3066?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17463448#comment-17463448 ] Harsha Teja Kanna commented on HUDI-3066: - yes this is for testing the clustering

[jira] [Commented] (HUDI-3066) Very slow file listing after enabling metadata for existing tables in 0.10.0 release

2021-12-21 Thread Harsha Teja Kanna (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3066?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17463453#comment-17463453 ] Harsha Teja Kanna commented on HUDI-3066: - rollback.requested file { "instantToRo

[jira] [Updated] (HUDI-3066) Very slow file listing after enabling metadata for existing tables in 0.10.0 release

2021-12-21 Thread Harsha Teja Kanna (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3066?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Harsha Teja Kanna updated HUDI-3066: Attachment: metadata_files_compacted.txt metadata_timeline_compacted.txt > V

[jira] [Commented] (HUDI-3066) Very slow file listing after enabling metadata for existing tables in 0.10.0 release

2021-12-21 Thread Harsha Teja Kanna (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3066?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17463528#comment-17463528 ] Harsha Teja Kanna commented on HUDI-3066: - I made changes to the sync 1) Clusteri

[jira] [Comment Edited] (HUDI-3066) Very slow file listing after enabling metadata for existing tables in 0.10.0 release

2021-12-21 Thread Harsha Teja Kanna (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3066?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17463442#comment-17463442 ] Harsha Teja Kanna edited comment on HUDI-3066 at 12/22/21, 1:24 AM:

[jira] [Commented] (HUDI-3066) Very slow file listing after enabling metadata for existing tables in 0.10.0 release

2021-12-21 Thread Harsha Teja Kanna (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3066?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17463539#comment-17463539 ] Harsha Teja Kanna commented on HUDI-3066: - Yes it is non-partitioned table. I will

[jira] [Commented] (HUDI-3066) Very slow file listing after enabling metadata for existing tables in 0.10.0 release

2021-12-21 Thread Harsha Teja Kanna (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3066?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17463549#comment-17463549 ] Harsha Teja Kanna commented on HUDI-3066: - Seems like disabling the metadata in th

[jira] [Comment Edited] (HUDI-3066) Very slow file listing after enabling metadata for existing tables in 0.10.0 release

2021-12-21 Thread Harsha Teja Kanna (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3066?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17463549#comment-17463549 ] Harsha Teja Kanna edited comment on HUDI-3066 at 12/22/21, 2:55 AM:

[jira] [Updated] (HUDI-3066) Very slow file listing after enabling metadata for existing tables in 0.10.0 release

2021-12-21 Thread Harsha Teja Kanna (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3066?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Harsha Teja Kanna updated HUDI-3066: Attachment: Screen Shot 2021-12-21 at 10.24.12 PM.png > Very slow file listing after enablin

[jira] [Updated] (HUDI-3066) Very slow file listing after enabling metadata for existing tables in 0.10.0 release

2021-12-21 Thread Harsha Teja Kanna (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3066?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Harsha Teja Kanna updated HUDI-3066: Attachment: Screen Shot 2021-12-21 at 10.22.54 PM.png > Very slow file listing after enablin

[jira] [Commented] (HUDI-3066) Very slow file listing after enabling metadata for existing tables in 0.10.0 release

2021-12-21 Thread Harsha Teja Kanna (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3066?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17463577#comment-17463577 ] Harsha Teja Kanna commented on HUDI-3066: - After few runs of the query, I see even

[jira] [Commented] (HUDI-3066) Very slow file listing after enabling metadata for existing tables in 0.10.0 release

2022-01-07 Thread Harsha Teja Kanna (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3066?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17470802#comment-17470802 ] Harsha Teja Kanna commented on HUDI-3066: - Hi I am working on re-testing this. Se

[jira] [Commented] (HUDI-2909) Partition field parsing fails due to KeyGenerator giving inconsistent value for logical timestamp type

2022-01-08 Thread Harsha Teja Kanna (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2909?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17471264#comment-17471264 ] Harsha Teja Kanna commented on HUDI-2909: - I am not able to determine if I fall un

[jira] [Comment Edited] (HUDI-2909) Partition field parsing fails due to KeyGenerator giving inconsistent value for logical timestamp type

2022-01-08 Thread Harsha Teja Kanna (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2909?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17471264#comment-17471264 ] Harsha Teja Kanna edited comment on HUDI-2909 at 1/9/22, 2:05 AM: --

[jira] [Comment Edited] (HUDI-2909) Partition field parsing fails due to KeyGenerator giving inconsistent value for logical timestamp type

2022-01-08 Thread Harsha Teja Kanna (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2909?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17471264#comment-17471264 ] Harsha Teja Kanna edited comment on HUDI-2909 at 1/9/22, 2:05 AM: --

[jira] [Commented] (HUDI-3066) Very slow file listing after enabling metadata for existing tables in 0.10.0 release

2022-01-08 Thread Harsha Teja Kanna (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3066?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17471280#comment-17471280 ] Harsha Teja Kanna commented on HUDI-3066: - Hi [~shivnarayan] Basic question I am

[jira] [Comment Edited] (HUDI-3066) Very slow file listing after enabling metadata for existing tables in 0.10.0 release

2022-01-08 Thread Harsha Teja Kanna (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3066?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17471280#comment-17471280 ] Harsha Teja Kanna edited comment on HUDI-3066 at 1/9/22, 4:15 AM: --

[jira] [Comment Edited] (HUDI-3066) Very slow file listing after enabling metadata for existing tables in 0.10.0 release

2022-01-08 Thread Harsha Teja Kanna (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3066?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17471280#comment-17471280 ] Harsha Teja Kanna edited comment on HUDI-3066 at 1/9/22, 6:11 AM: --

[jira] [Commented] (HUDI-2943) Deltastreamer fails to continue with pending clustering after restart in 0.10.0 and inline clustering

2022-01-10 Thread Harsha Teja Kanna (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2943?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17472393#comment-17472393 ] Harsha Teja Kanna commented on HUDI-2943: - Hi, Any chance this can be fixed in 0.1

[jira] [Comment Edited] (HUDI-2943) Deltastreamer fails to continue with pending clustering after restart in 0.10.0 and inline clustering

2022-01-10 Thread Harsha Teja Kanna (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2943?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17472393#comment-17472393 ] Harsha Teja Kanna edited comment on HUDI-2943 at 1/11/22, 1:38 AM: -

[jira] [Commented] (HUDI-2909) Partition field parsing fails due to KeyGenerator giving inconsistent value for logical timestamp type

2022-01-10 Thread Harsha Teja Kanna (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2909?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17472466#comment-17472466 ] Harsha Teja Kanna commented on HUDI-2909: - Hi, Thanks, I will recreate the table.

[jira] [Created] (HUDI-3242) Partial parquet file discovery after the first commit

2022-01-13 Thread Harsha Teja Kanna (Jira)
Harsha Teja Kanna created HUDI-3242: --- Summary: Partial parquet file discovery after the first commit Key: HUDI-3242 URL: https://issues.apache.org/jira/browse/HUDI-3242 Project: Apache Hudi

[jira] [Updated] (HUDI-3242) Partial parquet file discovery after the first commit

2022-01-13 Thread Harsha Teja Kanna (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3242?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Harsha Teja Kanna updated HUDI-3242: Description: Hi, I am testing release branch 0.10.1 as I needed few bug fixes from it. Howe

[jira] [Updated] (HUDI-3242) Partial parquet file discovery after the first commit

2022-01-13 Thread Harsha Teja Kanna (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3242?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Harsha Teja Kanna updated HUDI-3242: Description: Hi, I am testing release branch 0.10.1 as I needed few bug fixes from it. Howe

[jira] [Updated] (HUDI-3242) Partial parquet file discovery after the first commit

2022-01-13 Thread Harsha Teja Kanna (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3242?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Harsha Teja Kanna updated HUDI-3242: Attachment: Screen Shot 2022-01-13 at 2.40.55 AM.png > Partial parquet file discovery after

[jira] [Updated] (HUDI-3242) Partial parquet file discovery after the first commit

2022-01-13 Thread Harsha Teja Kanna (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3242?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Harsha Teja Kanna updated HUDI-3242: Attachment: (was: Screen Shot 2022-01-13 at 2.55.35 AM.png) > Partial parquet file disco

[jira] [Updated] (HUDI-3242) Partial parquet file discovery after the first commit

2022-01-13 Thread Harsha Teja Kanna (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3242?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Harsha Teja Kanna updated HUDI-3242: Attachment: Screen Shot 2022-01-13 at 2.55.35 AM.png > Partial parquet file discovery after

[jira] [Updated] (HUDI-3242) Partial parquet file discovery after the first commit

2022-01-13 Thread Harsha Teja Kanna (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3242?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Harsha Teja Kanna updated HUDI-3242: Attachment: (was: Screen Shot 2022-01-13 at 2.40.55 AM-2.png) > Partial parquet file dis

[jira] [Updated] (HUDI-3242) Partial parquet file discovery after the first commit

2022-01-13 Thread Harsha Teja Kanna (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3242?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Harsha Teja Kanna updated HUDI-3242: Attachment: (was: Screen Shot 2022-01-13 at 2.40.55 AM.png) > Partial parquet file disco

[jira] [Updated] (HUDI-3242) Partial parquet file discovery after the first commit

2022-01-13 Thread Harsha Teja Kanna (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3242?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Harsha Teja Kanna updated HUDI-3242: Attachment: Screen Shot 2022-01-13 at 2.40.55 AM.png Screen Shot 2022-01-13 a

[jira] [Updated] (HUDI-3242) Partial parquet file discovery after the first commit

2022-01-13 Thread Harsha Teja Kanna (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3242?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Harsha Teja Kanna updated HUDI-3242: Description: Hi, I am testing release branch 0.10.1 as I needed few bug fixes from it. Howe

[jira] [Updated] (HUDI-3242) Checkpoint 0 is ignored -Partial parquet file discovery after the first commit

2022-01-13 Thread Harsha Teja Kanna (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3242?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Harsha Teja Kanna updated HUDI-3242: Summary: Checkpoint 0 is ignored -Partial parquet file discovery after the first commit (wa

[jira] [Updated] (HUDI-3242) Checkpoint 0 is ignored -Partial parquet file discovery after the first commit

2022-01-13 Thread Harsha Teja Kanna (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3242?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Harsha Teja Kanna updated HUDI-3242: Priority: Blocker (was: Critical) > Checkpoint 0 is ignored -Partial parquet file discovery

[jira] [Updated] (HUDI-3242) Checkpoint 0 is ignored -Partial parquet file discovery after the first commit

2022-01-13 Thread Harsha Teja Kanna (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3242?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Harsha Teja Kanna updated HUDI-3242: Description: Hi, I am testing release branch 0.10.1 as I needed few bug fixes from it. Howe

[jira] [Comment Edited] (HUDI-2947) HoodieDeltaStreamer/DeltaSync can improperly pick up the checkpoint config from CLI in continuous mode

2022-01-13 Thread Harsha Teja Kanna (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2947?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17475869#comment-17475869 ] Harsha Teja Kanna edited comment on HUDI-2947 at 1/14/22, 12:17 AM:

[jira] [Commented] (HUDI-2947) HoodieDeltaStreamer/DeltaSync can improperly pick up the checkpoint config from CLI in continuous mode

2022-01-13 Thread Harsha Teja Kanna (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2947?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17475869#comment-17475869 ] Harsha Teja Kanna commented on HUDI-2947: - I think this problem still exists  htt

[jira] [Commented] (HUDI-3066) Very slow file listing after enabling metadata for existing tables in 0.10.0 release

2022-01-13 Thread Harsha Teja Kanna (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3066?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17475935#comment-17475935 ] Harsha Teja Kanna commented on HUDI-3066: - With only base path in the load. File l

[jira] [Comment Edited] (HUDI-3066) Very slow file listing after enabling metadata for existing tables in 0.10.0 release

2022-01-13 Thread Harsha Teja Kanna (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3066?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17475935#comment-17475935 ] Harsha Teja Kanna edited comment on HUDI-3066 at 1/14/22, 5:04 AM: -

[jira] [Updated] (HUDI-3242) Checkpoint 0 is ignored -Partial parquet file discovery after the first commit

2022-01-14 Thread Harsha Teja Kanna (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3242?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Harsha Teja Kanna updated HUDI-3242: Priority: Critical (was: Blocker) > Checkpoint 0 is ignored -Partial parquet file discovery

[jira] [Updated] (HUDI-3242) Checkpoint 0 is ignored -Partial parquet file discovery after the first commit

2022-01-16 Thread Harsha Teja Kanna (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3242?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Harsha Teja Kanna updated HUDI-3242: Priority: Blocker (was: Critical) > Checkpoint 0 is ignored -Partial parquet file discovery

[jira] [Commented] (HUDI-3242) Checkpoint 0 is ignored -Partial parquet file discovery after the first commit

2022-01-16 Thread Harsha Teja Kanna (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3242?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17476923#comment-17476923 ] Harsha Teja Kanna commented on HUDI-3242: - I am now seeing this for all the tables

[jira] [Comment Edited] (HUDI-3242) Checkpoint 0 is ignored -Partial parquet file discovery after the first commit

2022-01-16 Thread Harsha Teja Kanna (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3242?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17476923#comment-17476923 ] Harsha Teja Kanna edited comment on HUDI-3242 at 1/17/22, 2:30 AM: -

[jira] [Created] (HUDI-2909) KeyGenerator is broken in 0.10.0

2021-12-01 Thread Harsha Teja Kanna (Jira)
Harsha Teja Kanna created HUDI-2909: --- Summary: KeyGenerator is broken in 0.10.0 Key: HUDI-2909 URL: https://issues.apache.org/jira/browse/HUDI-2909 Project: Apache Hudi Issue Type: Bug

[jira] [Updated] (HUDI-2909) KeyGenerator is broken in 0.10.0

2021-12-01 Thread Harsha Teja Kanna (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2909?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Harsha Teja Kanna updated HUDI-2909: Priority: Blocker (was: Major) > KeyGenerator is broken in 0.10.0 > ---

[jira] [Created] (HUDI-2943) Deltastreamer fails to continue with pending clustering after restart in 0.10.0

2021-12-06 Thread Harsha Teja Kanna (Jira)
Harsha Teja Kanna created HUDI-2943: --- Summary: Deltastreamer fails to continue with pending clustering after restart in 0.10.0 Key: HUDI-2943 URL: https://issues.apache.org/jira/browse/HUDI-2943 Pro

[jira] [Commented] (HUDI-2943) Deltastreamer fails to continue with pending clustering after restart in 0.10.0

2021-12-06 Thread Harsha Teja Kanna (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2943?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17454385#comment-17454385 ] Harsha Teja Kanna commented on HUDI-2943: - Yes. I am running latest master. Actua

[jira] [Comment Edited] (HUDI-2943) Deltastreamer fails to continue with pending clustering after restart in 0.10.0

2021-12-06 Thread Harsha Teja Kanna (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2943?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17454385#comment-17454385 ] Harsha Teja Kanna edited comment on HUDI-2943 at 12/7/21, 5:53 AM: -

[jira] [Commented] (HUDI-2943) Deltastreamer fails to continue with pending clustering after restart in 0.10.0

2021-12-08 Thread Harsha Teja Kanna (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2943?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17455984#comment-17455984 ] Harsha Teja Kanna commented on HUDI-2943: - I see this issue even after multiple re

[jira] [Comment Edited] (HUDI-2943) Deltastreamer fails to continue with pending clustering after restart in 0.10.0

2021-12-08 Thread Harsha Teja Kanna (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2943?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17455984#comment-17455984 ] Harsha Teja Kanna edited comment on HUDI-2943 at 12/8/21, 8:50 PM: -

[jira] [Comment Edited] (HUDI-2943) Deltastreamer fails to continue with pending clustering after restart in 0.10.0

2021-12-08 Thread Harsha Teja Kanna (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2943?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17455984#comment-17455984 ] Harsha Teja Kanna edited comment on HUDI-2943 at 12/8/21, 9:10 PM: -

[jira] [Updated] (HUDI-2943) Deltastreamer fails to continue with pending clustering after restart in 0.10.0

2021-12-08 Thread Harsha Teja Kanna (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2943?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Harsha Teja Kanna updated HUDI-2943: Description: Deltastreamer fails to restart when there is a pending clustering commit from a

[jira] [Commented] (HUDI-2943) Deltastreamer fails to continue with pending clustering after restart in 0.10.0

2021-12-09 Thread Harsha Teja Kanna (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2943?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17456770#comment-17456770 ] Harsha Teja Kanna commented on HUDI-2943: - Yes async clustering does work, that is

[jira] [Commented] (HUDI-2943) Deltastreamer fails to continue with pending clustering after restart in 0.10.0

2021-12-09 Thread Harsha Teja Kanna (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2943?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17456775#comment-17456775 ] Harsha Teja Kanna commented on HUDI-2943: - I am using HoodieClusteringJob util wit

[jira] [Created] (HUDI-3066) Very slow file listing after enabling metadata for existing tables in 0.10.0 release

2021-12-18 Thread Harsha Teja Kanna (Jira)
Harsha Teja Kanna created HUDI-3066: --- Summary: Very slow file listing after enabling metadata for existing tables in 0.10.0 release Key: HUDI-3066 URL: https://issues.apache.org/jira/browse/HUDI-3066

[jira] [Updated] (HUDI-3066) Very slow file listing after enabling metadata for existing tables in 0.10.0 release

2021-12-18 Thread Harsha Teja Kanna (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3066?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Harsha Teja Kanna updated HUDI-3066: Attachment: Screen Shot 2021-12-18 at 6.16.29 PM.png > Very slow file listing after enabling

[jira] [Updated] (HUDI-3066) Very slow file listing after enabling metadata for existing tables in 0.10.0 release

2021-12-18 Thread Harsha Teja Kanna (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3066?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Harsha Teja Kanna updated HUDI-3066: Attachment: Screen Shot 2021-12-18 at 6.16.29 PM.png > Very slow file listing after enabling

[jira] [Updated] (HUDI-3066) Very slow file listing after enabling metadata for existing tables in 0.10.0 release

2021-12-18 Thread Harsha Teja Kanna (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3066?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Harsha Teja Kanna updated HUDI-3066: Attachment: (was: Screen Shot 2021-12-18 at 6.16.29 PM.png) > Very slow file listing aft

[jira] [Updated] (HUDI-3066) Very slow file listing after enabling metadata for existing tables in 0.10.0 release

2021-12-18 Thread Harsha Teja Kanna (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3066?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Harsha Teja Kanna updated HUDI-3066: Attachment: (was: Screen Shot 2021-12-18 at 6.16.29 PM.png) > Very slow file listing aft

[jira] [Updated] (HUDI-3066) Very slow file listing after enabling metadata for existing tables in 0.10.0 release

2021-12-18 Thread Harsha Teja Kanna (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3066?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Harsha Teja Kanna updated HUDI-3066: Description: After 'metadata table' is enabled, File listing takes long time. Existing tabl

[jira] [Updated] (HUDI-3066) Very slow file listing after enabling metadata for existing tables in 0.10.0 release

2021-12-18 Thread Harsha Teja Kanna (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3066?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Harsha Teja Kanna updated HUDI-3066: Attachment: Screen Shot 2021-12-18 at 6.16.29 PM.png > Very slow file listing after enabling

  1   2   >