[GitHub] [hudi] developerwxl opened a new issue, #5938: Why Hudi publish data size much more than the input file size when publish to hive

[GitBox]

developerwxl opened a new issue, #5938:
URL: https://github.com/apache/hudi/issues/5938

   I use the insert operation to publish data to the hive. 
   I insert a 541 partitions table.
   And the input data size is 153GB .But there are  2.6TB in the 
UpsertPartitioner job.
   Here is the code
   <img width="1138" alt="image" 
src="https://user-images.githubusercontent.com/4186869/175003745-65c0767a-eb05-45e2-9314-5fd3b8e36ddf.png";>
   
   
   Here is the spark ui
   <img width="1635" alt="image" 
src="https://user-images.githubusercontent.com/4186869/175002927-0c7221d8-2072-4dd1-b246-bd9b851d88a3.png";>
   
   <img width="1638" alt="image" 
src="https://user-images.githubusercontent.com/4186869/175003033-3ca82130-78da-4955-a99d-9cc9f9efe283.png";>
   
   <img width="1787" alt="image" 
src="https://user-images.githubusercontent.com/4186869/175003079-348f558c-7c64-40e0-aa58-4423ff1551ba.png";>
   
   
   
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org