woofyzhao opened a new pull request, #6010:
URL: https://github.com/apache/inlong/pull/6010

   - Fixes #6003
   
   ### Motivation
   
   In the beginning only **lightweight mode** supports the ETL transform nodes. 
Clients configure their extract/transform/load nodes  at will and what they 
submit is what they see and so every one is happy.
   
   This ceases to be the case when **standard mode** tries to support the 
transform as well. In standard mode multiple upstream sources are collected to 
MQ first and are not the real extract nodes anymore. Instead the MQ cache node 
becomes **the real extract node**.  
   
   This is achieved by manager through replacing the original upstream source 
nodes (file、binlog...) with the MQ cache node when constructing the extract 
node in standard mode.
   
   However the clients are ignorant of any of this.  From their point of view 
nothing is changed and the MQ node is invisible to them. In their submitted 
**node relation configuration** the input node of a transform node might still 
be some upstream source node (file、binlog...) rather than the MQ cache node. 
This results in the incorrect dataflow generation:
   
   
![image](https://user-images.githubusercontent.com/941634/191928549-3e486be1-925e-44fd-a9a3-fbc773fcd268.png)
   
   
   ### Changes
   - change the transform fields's original node id to be MQ cache node
   -  update the user submitted node relation to use MQ cache node as the real 
extract node


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@inlong.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

Reply via email to