hehuiyuan opened a new pull request, #23945:
URL: https://github.com/apache/flink/pull/23945

   
   ## What is the purpose of the change
   
   HiveCatalog.getHiveTable method wastes a lot of time when generate graph, 
because the number of calls  is relatively high.
   
   
   I have an sql task with over 2000 rows,  the HiveCatalog.getHiveTable  
method is called 4879 times , but only six hive tables were used. 
   
   
![image](https://github.com/apache/flink/assets/18002496/d5f0daf3-f80a-4790-ae21-4e75dff9cfd7)
   
   The client.getTable method costs a lot of time.  One call takes 
approximately 50 milliseconds,
   
   4879 * 50 =243950ms  = 243.95s  = 4min
   
   ## Brief change log
   
   Reduce getTable call for hive client by using cache.
   
   
   ## Verifying this change
   
   
   
   ## Does this pull request potentially affect one of the following parts:
   
     - Dependencies (does it add or upgrade a dependency): (yes / no)
     - The public API, i.e., is any changed class annotated with 
`@Public(Evolving)`: (yes / no)
     - The serializers: (yes / no / don't know)
     - The runtime per-record code paths (performance sensitive): (yes / no / 
don't know)
     - Anything that affects deployment or recovery: JobManager (and its 
components), Checkpointing, Kubernetes/Yarn, ZooKeeper: (yes / no / don't know)
     - The S3 file system connector: (yes / no / don't know)
   
   ## Documentation
   
     - Does this pull request introduce a new feature? (yes / no)
     - If yes, how is the feature documented? (not applicable / docs / JavaDocs 
/ not documented)
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

Reply via email to