yuelongsun opened a new issue, #9382: URL: https://github.com/apache/seatunnel/issues/9382
### Search before asking - [x] I had searched in the [issues](https://github.com/apache/seatunnel/issues?q=is%3Aissue+label%3A%22bug%22) and found no similar issues. ### What happened 有如下hive表: CREATE TABLE IF NOT EXISTS default.seatunnel_test( `stat_dt` Date COMMENT '统计日期', `id` STRING COMMENT 'id', `name` STRING COMMENT '名称', `event_dt` Date ) COMMENT 'seatunnel test'PARTITIONED BY (`dt` string) STORED AS orc tblproperties ('orc.compress' = 'SNAPPY'); 分区dt=2025-05-24中没有数据，从此分区中读数据时，同时通过fillter插件把分区字段dt过滤掉, 但是报错了: ErrorDescription:[The input fields 'dt' of 'Filter' transform not found in upstream schema]， 若是分区中有数据， seatunnel是可以正常运行的。 ### SeaTunnel Version seatunnel-2.3.10 ### SeaTunnel Config ```conf env { parallelism = 2 job.mode = "BATCH" checkpoint.interval = 10000 } source { Hive { table_name = "default.seatunnel_test" metastore_uri = "thrift://xxx.xxx.xxx.xxx:9083" hive.hadoop.conf-path = "/root/apache-seatunnel-2.3.10/config" hive_site_path = "/root/apache-seatunnel-2.3.10/config/hive-site.xml" read_columns = [ "stat_dt", "id", "name", "event_dt" ] read_partitions = ["dt=2025-05-24"] plugin_output = "hive_data" } } transform { Filter { plugin_input = "hive_data" plugin_output = "hive_data_trans" exclude_fields = [dt] } } sink { console { } } ``` ### Running Command ```shell ./bin/start-seatunnel-flink-15-connector-v2.sh --config ./config/demo/seatunnel_test.console.config ``` ### Error Exception ```log org.apache.flink.client.program.ProgramInvocationException: The main method caused an error: Flink job executed failed at org.apache.flink.client.program.PackagedProgram.callMainMethod(PackagedProgram.java:372) at org.apache.flink.client.program.PackagedProgram.invokeInteractiveModeForExecution(PackagedProgram.java:222) at org.apache.flink.client.ClientUtils.executeProgram(ClientUtils.java:105) at org.apache.flink.client.cli.CliFrontend.executeProgram(CliFrontend.java:851) at org.apache.flink.client.cli.CliFrontend.run(CliFrontend.java:245) at org.apache.flink.client.cli.CliFrontend.parseAndRun(CliFrontend.java:1095) at org.apache.flink.client.cli.CliFrontend.lambda$mainInternal$9(CliFrontend.java:1189) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:422) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1729) at org.apache.flink.runtime.security.contexts.HadoopSecurityContext.runSecured(HadoopSecurityContext.java:41) at org.apache.flink.client.cli.CliFrontend.mainInternal(CliFrontend.java:1189) at org.apache.flink.client.cli.CliFrontend.main(CliFrontend.java:1157) Caused by: org.apache.seatunnel.core.starter.exception.CommandExecuteException: Flink job executed failed at org.apache.seatunnel.core.starter.flink.command.FlinkTaskExecuteCommand.execute(FlinkTaskExecuteCommand.java:63) at org.apache.seatunnel.core.starter.SeaTunnel.run(SeaTunnel.java:40) at org.apache.seatunnel.core.starter.flink.SeaTunnelFlink.main(SeaTunnelFlink.java:34) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.flink.client.program.PackagedProgram.callMainMethod(PackagedProgram.java:355) ... 12 more Caused by: org.apache.seatunnel.core.starter.exception.TaskExecuteException: SeaTunnel transform task: Filter execute error at org.apache.seatunnel.core.starter.flink.execution.TransformExecuteProcessor.execute(TransformExecuteProcessor.java:148) at org.apache.seatunnel.core.starter.flink.execution.FlinkExecution.execute(FlinkExecution.java:116) at org.apache.seatunnel.core.starter.flink.command.FlinkTaskExecuteCommand.execute(FlinkTaskExecuteCommand.java:61) ... 19 more Caused by: org.apache.seatunnel.transform.exception.TransformException: ErrorCode:[TRANSFORM_COMMON-02], ErrorDescription:[The input fields 'dt' of 'Filter' transform not found in upstream schema] at org.apache.seatunnel.transform.exception.TransformCommonError.cannotFindInputFieldsError(TransformCommonError.java:46) at org.apache.seatunnel.transform.filter.FilterFieldTransform.<init>(FilterFieldTransform.java:79) at org.apache.seatunnel.transform.filter.FieldFieldMultiCatalogTransform.buildTransform(FieldFieldMultiCatalogTransform.java:43) at org.apache.seatunnel.transform.common.AbstractMultiCatalogTransform.lambda$new$2(AbstractMultiCatalogTransform.java:74) at java.util.ArrayList.forEach(ArrayList.java:1257) at org.apache.seatunnel.transform.common.AbstractMultiCatalogTransform.<init>(AbstractMultiCatalogTransform.java:62) at org.apache.seatunnel.transform.common.AbstractMultiCatalogMapTransform.<init>(AbstractMultiCatalogMapTransform.java:33) at org.apache.seatunnel.transform.filter.FieldFieldMultiCatalogTransform.<init>(FieldFieldMultiCatalogTransform.java:32) at org.apache.seatunnel.transform.filter.FilterFieldTransformFactory.lambda$createTransform$0(FilterFieldTransformFactory.java:53) at org.apache.seatunnel.core.starter.flink.execution.TransformExecuteProcessor.execute(TransformExecuteProcessor.java:132) ... 21 more ``` ### Zeta or Flink or Spark Version flink-1.18.1 ### Java or Scala Version openjdk version "1.8.0_222" ### Screenshots _No response_ ### Are you willing to submit PR? - [ ] Yes I am willing to submit a PR! ### Code of Conduct - [x] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
