Re: [I] 【2.3.9】【又来BUG了】mysql到starrocks跑批，source加上过滤条件，结果读不到数据，导致写SR不生效 [seatunnel]

[via GitHub]

FrommyMind commented on issue #8567:
URL: https://github.com/apache/seatunnel/issues/8567#issuecomment-2644815326

   Verified with Seatunnel 2.3.9, and it works perfectly.
   
   MySQL data:
   
   
![Image](https://github.com/user-attachments/assets/b7697da9-8a50-4d21-850f-061c44c75716)
   
   Config file:
   
   ```
   #定义一些作业的运行参数,具体可以参考 
https://seatunnel.apache.org/docs/2.3.1/concept/JobEnvConfig
   env {
     job.mode="BATCH"  #作业的运行模式，BATCH=离线批同步，STREAMING=实时同步
     job.name="SeaTunnel_market_new_58_163_sr_Job"
     checkpoint.interval=10000 #每10000ms进行一次checkpoint，后面会详细介绍checkpoint对JDBC 
Source和StarRocks Sink这两个连接器的影响
     read_limit.bytes_per_second=7000000
     read_limit.rows_per_second=400
   }
   source {
     Jdbc {
       plugin_output="Table9210050164000"
       connection_check_timeout_sec = 1000
       parallelism=5 # 并行度，这里是启动5个Source Task来并行的读取数据
       partition_column="id" # 
使用id字段来进行split的拆分，目前只支持数字类型的主键列，而且该列的值最好是离线的，自增id最佳
       partition_num="10" # 拆分成20个split，这20个split会被分配给5个Source Task来处理
       query="SELECT `id`,username, `email` FROM `test_db`.`users` where `id` > 
2 "
       password=""
       driver="com.mysql.cj.jdbc.Driver"
       user=test_user
       
url="jdbc:mysql://10.191.80.xxx:23306/test_db?useUnicode=true&characterEncoding=utf-8&autoReconnect=true&zeroDateTimeBehavior=convertToNull&allowPublicKeyRetrieval=true&useSSL=false"
     }
   }
   transform {
     # 在本次示例中我们不需要做任务的Transform操作，所以这里为空，也可以将transform整个元素删除
   }
   sink {
     StarRocks {
       data_save_mode="APPEND_DATA"
       plugin_input="Table9210050164000"
       nodeUrls = ["10.191.80.xxx:8040"]
       base-url = "jdbc:mysql://10.191.80.xxx:9030/"
       username = test_user
       password = ""
       database = "test_db"
       table = "users"
       batch_max_rows = 50000
       starrocks.config = {
         format = "JSON"
         strip_outer_array = true
       }
     }
   }
   ```
   Execution log:
   
   
![Image](https://github.com/user-attachments/assets/81139f26-ce0d-4c2e-9cd1-d975cebfea98)
   
   
   Starrocks  result:
   
   
![Image](https://github.com/user-attachments/assets/2df5f4ae-04de-46f4-8bd2-c760fb2a78a8)
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@seatunnel.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org