Re: ACID with Hive/Kylin

2023-12-12 Thread Nam Đỗ Duy
Thank you both of you for your valuable information. I will test and revert soon. Best regards On Tue, Dec 12, 2023 at 2:39 PM Xiaoxiang Yu wrote: > I don't know GDPR very well. Here is my understanding. > > For hive and hdfs, you can consider using these techniques which support > ACID in Spar

Re: ACID with Hive/Kylin

2023-12-11 Thread Xiaoxiang Yu
I don't know GDPR very well. Here is my understanding. For hive and hdfs, you can consider using these techniques which support ACID in Spark and Hive(I recommend first one): 1) Delta Lake, https://docs.databricks.com/en/security/privacy/gdpr-delta.html 2) Hive ACID table, here is a link, https://

Re: ACID with Hive/Kylin

2023-12-11 Thread ShaoFeng Shi
Hi Nam, As Kylin is used to store the aggregated data, there should be no PII information. (if you use Kylin to manage person level data, that is not a good case). If you do need to delete certain personal data, refresh the whole index or some partitions is what we can do. Best regards, Shaofen