[jira] [Created] (KYLIN-5745) The historical garbage cleanup task was not completed, causing the subsequent scheduled garbage cleanup task cannot be executed normally

2023-12-11 Thread zhong.zhu (Jira)
zhong.zhu created KYLIN-5745: Summary: The historical garbage cleanup task was not completed, causing the subsequent scheduled garbage cleanup task cannot be executed normally Key: KYLIN-5745 URL: https://issues.apac

[jira] [Created] (KYLIN-5746) On the page, select online model operation offline, click the model online again, and put the model online button into ash.

2023-12-11 Thread zhong.zhu (Jira)
zhong.zhu created KYLIN-5746: Summary: On the page, select online model operation offline, click the model online again, and put the model online button into ash. Key: KYLIN-5746 URL: https://issues.apache.org/jira/br

[jira] [Created] (KYLIN-5747) Calcite constant folding, adding strings to numbers, results not as expected when multiple plus signs are used together

2023-12-11 Thread zhong.zhu (Jira)
zhong.zhu created KYLIN-5747: Summary: Calcite constant folding, adding strings to numbers, results not as expected when multiple plus signs are used together Key: KYLIN-5747 URL: https://issues.apache.org/jira/browse

ACID with Hive/Kylin

2023-12-11 Thread Nam Đỗ Duy
Dear Xiaoxiang, Sirs/Madams I face an issue with deleting data of user according to GPDR-like policy which means when user send request to delete their personal data, we need to delete it from all system, that means to delete data: 1- from Kylin index (cube) 2- from Hive 3- from HDFS Have you ha

Re: ACID with Hive/Kylin

2023-12-11 Thread ShaoFeng Shi
Hi Nam, As Kylin is used to store the aggregated data, there should be no PII information. (if you use Kylin to manage person level data, that is not a good case). If you do need to delete certain personal data, refresh the whole index or some partitions is what we can do. Best regards, Shaofen

Re: ACID with Hive/Kylin

2023-12-11 Thread Xiaoxiang Yu
I don't know GDPR very well. Here is my understanding. For hive and hdfs, you can consider using these techniques which support ACID in Spark and Hive(I recommend first one): 1) Delta Lake, https://docs.databricks.com/en/security/privacy/gdpr-delta.html 2) Hive ACID table, here is a link, https://