[I] [Bug] [Sftp Reader] If there are wildcard file names in the sftp directory and there happen to be other matching file names, a uniqueness error will occur [seatunnel]

[via GitHub]

e-mhui opened a new issue, #8500:
URL: https://github.com/apache/seatunnel/issues/8500

   ### Search before asking
   
   - [X] I had searched in the 
[issues](https://github.com/apache/seatunnel/issues?q=is%3Aissue+label%3A%22bug%22)
 and found no similar issues.
   
   
   ### What happened
   
   If there are wildcard file names in the sftp directory and there happen to 
be other matching file names, a uniqueness error will occur.
   For example, there are two files, `a * b.txt` and `abc.txt`, in the `/data` 
directory. When configuring/data/a * b.txt, a uniqueness error occurs.
   <img width="1421" alt="Clipboard_Screenshot_1736600413" 
src="https://github.com/user-attachments/assets/8a4acb91-9c46-4e76-ac47-82cb7af4b197";
 />
   
   
   ### SeaTunnel Version
   
   dev
   
   ### SeaTunnel Config
   
   ```conf
   env {
     parallelism = 1
     job.mode = "BATCH"
   }
   
   source {
     SftpFile {
       host = ""
       port = xxx
       user = xxx
       password = xxx
       path = "/data/a*b.txt"
       file_format_type = "text"
       field_delimiter = ","
       schema = {
         fields {
           c_int = int
           c_string = string
         }
       }
     }
   }
   
   transform {
   }
   
   sink {
     console {
     }
   }
   ```
   
   
   ### Running Command
   
   ```shell
   `org.apache.seatunnel.example.engine.SeaTunnelEngineLocalExample`
   
   
   public static void main(String[] args)
               throws FileNotFoundException, URISyntaxException, 
CommandException {
           String configurePath = args.length > 0 ? args[0] : 
"/examples/sftp_to_console.conf";
           String configFile = getTestConfigFile(configurePath);
           ClientCommandArgs clientCommandArgs = new ClientCommandArgs();
           clientCommandArgs.setConfigFile(configFile);
           clientCommandArgs.setCheckConfig(false);
           
clientCommandArgs.setJobName(Paths.get(configFile).getFileName().toString());
           // Change Execution Mode to CLUSTER to use client mode, before do 
this, you should start
           // SeaTunnelEngineClusterServerExample
           clientCommandArgs.setMasterType(MasterType.LOCAL);
           SeaTunnel.run(clientCommandArgs.buildCommand());
       }
   ```
   
   
   ### Error Exception
   
   ```log
   Caused by: 
org.apache.seatunnel.engine.common.exception.SeaTunnelEngineException: 
org.apache.seatunnel.connectors.seatunnel.file.exception.FileConnectorException:
 ErrorCode:[FILE-08], ErrorDescription:[File read failed] - Read data from this 
file [default.default.default_sftp://xxxx/data/aa*bbb.txt] failed
       at 
org.apache.seatunnel.connectors.seatunnel.file.source.reader.MultipleTableFileSourceReader.pollNext(MultipleTableFileSourceReader.java:85)
       at 
org.apache.seatunnel.engine.server.task.flow.SourceFlowLifeCycle.collect(SourceFlowLifeCycle.java:159)
       at 
org.apache.seatunnel.engine.server.task.SourceSeaTunnelTask.collect(SourceSeaTunnelTask.java:127)
       at 
org.apache.seatunnel.engine.server.task.SeaTunnelTask.stateProcess(SeaTunnelTask.java:169)
       at 
org.apache.seatunnel.engine.server.task.SourceSeaTunnelTask.call(SourceSeaTunnelTask.java:132)
       at 
org.apache.seatunnel.engine.server.TaskExecutionService$BlockingWorker.run(TaskExecutionService.java:694)
       at 
org.apache.seatunnel.engine.server.TaskExecutionService$NamedTaskWrapper.run(TaskExecutionService.java:1019)
       at org.apache.seatunnel.api.tracing.MDCRunnable.run(MDCRunnable.java:43)
       at 
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
       at java.util.concurrent.FutureTask.run(FutureTask.java:266)
       at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
       at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
       at java.lang.Thread.run(Thread.java:748)
   Caused by: java.io.IOException: 4: /data/aa*bbb.txt is not unique: 
[/data/aa123bbb.txt, /data/aa*bbb.txt]
       at 
org.apache.seatunnel.connectors.seatunnel.file.sftp.system.SFTPFileSystem.open(SFTPFileSystem.java:495)
       at org.apache.hadoop.fs.FileSystem.open(FileSystem.java:899)
       at 
org.apache.seatunnel.connectors.seatunnel.file.hadoop.HadoopFileSystemProxy.lambda$getInputStream$11(HadoopFileSystemProxy.java:195)
       at 
org.apache.seatunnel.connectors.seatunnel.file.hadoop.HadoopFileSystemProxy.execute(HadoopFileSystemProxy.java:327)
       at 
org.apache.seatunnel.connectors.seatunnel.file.hadoop.HadoopFileSystemProxy.getInputStream(HadoopFileSystemProxy.java:195)
       at 
org.apache.seatunnel.connectors.seatunnel.file.source.reader.AbstractReadStrategy.resolveArchiveCompressedInputStream(AbstractReadStrategy.java:272)
       at 
org.apache.seatunnel.connectors.seatunnel.file.source.reader.TextReadStrategy.read(TextReadStrategy.java:71)
       at 
org.apache.seatunnel.connectors.seatunnel.file.source.reader.MultipleTableFileSourceReader.pollNext(MultipleTableFileSourceReader.java:81)
       ... 12 more
   ```
   
   
   ### Zeta or Flink or Spark Version
   
   _No response_
   
   ### Java or Scala Version
   
   _No response_
   
   ### Screenshots
   
   _No response_
   
   ### Are you willing to submit PR?
   
   - [X] Yes I am willing to submit a PR!
   
   ### Code of Conduct
   
   - [X] I agree to follow this project's [Code of 
Conduct](https://www.apache.org/foundation/policies/conduct)
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@seatunnel.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org