Hi, The problem is not that concurrent FTP components are accessing the same files. The problem is that the one FTP component which handles the files from the FTP account is not doing it in separate connections. And this is a bad thing to do in a very delicate folder based environment like a FTP account.
As you can see from the FTP server logs, there is only one FTP consumer connection which handles all the files it finds. The problem is that handling the files concurrently on a single FTP connection makes the output unpredictable. I can send my test. Where should I put it? What endpoints should I use? Do you have a public access ftp account to test it? On Mon, May 23, 2011 at 7:13 PM, Taariq Levack <taar...@gmail.com> wrote: > Hi > > I don't know if it's a bug or not, but I would expect it to work. > There's an old ticket to make FTP threadsafe[1], but I only see a > concurrent > producer test. > > Can you post your test? > > [1]https://issues.apache.org/jira/browse/CAMEL-1641 > > > On Mon, May 23, 2011 at 11:12 AM, Laurentiu Trica < > laurentiu.tr...@finalfolder.biz> wrote: > > > Hello again, > > > > Anyone had this issue before? Is it a bug in Camel FTP or it's just the > way > > it is supposed to work? > > > > On Fri, May 20, 2011 at 6:00 PM, Laurentiu Trica < > > laurentiu.tr...@finalfolder.biz> wrote: > > > > > Hello, > > > > > > I've been struggling to solve a situation in which a FTP poller was not > > > working from time to time. > > > I've got to the point where I think I know what the problem is. > > > > > > I'm using Camel 2.6 in servicemix. > > > > > > *I've put up a test for this:* > > > - three FTP file producers at different repeating period of time: 35s, > > > 5min, 1h. Note that when starting servicemix they all put the first > file > > at > > > once. > > > - the files ar put on the FTP in /in folder > > > - one FTP (ftps) consumer which deletes the files after consuming them > > > - the route is simply moving the file from input (ftps) to output (a > > local > > > folder). > > > > > > *The problem*: > > > - only the first produced file is consumed but is processed over and > over > > > again > > > - the rest of the files (the rest of the first 3 files + the others > that > > > are added every 35s) are just ignored > > > > > > Possible problem: > > > The FTP component is getting the list of the first 3 files from the FTP > > in > > > a threaded manner which depends on the state of the current directory > on > > the > > > FTP server > > > - it changes the current directory to /in > > > - retrieves the file > > > - it changes the current directory to / > > > - the second (probably) thread changes the directory to /in > > > - retreives the second file > > > - the first (probably) thread sends a delete command but it sends it as > > if > > > it were in the root folder but it is actually in the /in folder > > > - the FTP gives an error that the file doesn't exist > > > - somehow the FTP Component blocks itself on the first file... > > > > > > You can see the partial (useful) log below: > > > > > > (002386)5/20/2011 17:07:03 PM - sftp-test (192.168.0.194)> 257 "/" is > > > current directory. > > > (002386)5/20/2011 17:07:03 PM - sftp-test (192.168.0.194)> CWD in > > > (002386)5/20/2011 17:07:03 PM - sftp-test (192.168.0.194)> 250 CWD > > > successful. "/in" is current directory. > > > (002386)5/20/2011 17:07:03 PM - sftp-test (192.168.0.194)> PASV > > > (002386)5/20/2011 17:07:03 PM - sftp-test (192.168.0.194)> 227 Entering > > > Passive Mode (192,168,0,25,228,206) > > > (002386)5/20/2011 17:07:03 PM - sftp-test (192.168.0.194)> RETR > > > WS_1001_1.xml > > > (002386)5/20/2011 17:07:03 PM - sftp-test (192.168.0.194)> 150 > Connection > > > accepted > > > (002386)5/20/2011 17:07:03 PM - sftp-test (192.168.0.194)> SSL > connection > > > for data connection established > > > (002386)5/20/2011 17:07:03 PM - sftp-test (192.168.0.194)> 226 Transfer > > OK > > > (002386)5/20/2011 17:07:03 PM - sftp-test (192.168.0.194)> CWD / > > > (002386)5/20/2011 17:07:03 PM - sftp-test (192.168.0.194)> 250 CWD > > > successful. "/" is current directory. > > > (002386)5/20/2011 17:07:03 PM - sftp-test (192.168.0.194)> PWD > > > (002386)5/20/2011 17:07:03 PM - sftp-test (192.168.0.194)> 257 "/" is > > > current directory. > > > (002386)5/20/2011 17:07:03 PM - sftp-test (192.168.0.194)> CWD in > > > (002386)5/20/2011 17:07:03 PM - sftp-test (192.168.0.194)> 250 CWD > > > successful. "/in" is current directory. > > > (002386)5/20/2011 17:07:03 PM - sftp-test (192.168.0.194)> PASV > > > (002386)5/20/2011 17:07:03 PM - sftp-test (192.168.0.194)> 227 Entering > > > Passive Mode (192,168,0,25,228,207) > > > (002386)5/20/2011 17:07:03 PM - sftp-test (192.168.0.194)> RETR > > > WS_1001_2.xml > > > (002386)5/20/2011 17:07:03 PM - sftp-test (192.168.0.194)> 150 > Connection > > > accepted > > > (002386)5/20/2011 17:07:03 PM - sftp-test (192.168.0.194)> DELE > > > in/WS_1001_1.xml > > > (002386)5/20/2011 17:07:03 PM - sftp-test (192.168.0.194)> 550 File not > > > found > > > (002386)5/20/2011 17:07:03 PM - sftp-test (192.168.0.194)> PASV > > > (002386)5/20/2011 17:07:03 PM - sftp-test (192.168.0.194)> 227 Entering > > > Passive Mode (192,168,0,25,228,208) > > > (002386)5/20/2011 17:07:03 PM - sftp-test (192.168.0.194)> NLST in > > > (002386)5/20/2011 17:07:03 PM - sftp-test (192.168.0.194)> 550 > Directory > > > not found > > > (002386)5/20/2011 17:07:03 PM - sftp-test (192.168.0.194)> PWD > > > (002386)5/20/2011 17:07:03 PM - sftp-test (192.168.0.194)> 257 "/in" is > > > current directory. > > > (002386)5/20/2011 17:07:03 PM - sftp-test (192.168.0.194)> CWD in > > > (002386)5/20/2011 17:07:03 PM - sftp-test (192.168.0.194)> 550 CWD > > failed. > > > "/in/in": directory not found. > > > > > > For the whole logs go here: > > > http://pastie.org/private/kknnkxpbyzqmai7gsm8z5g > > > For the Servicemix log: > http://pastie.org/private/eulookxiepan3nkjyu41a > > > > > > A workaround that seems to work is to use the maxMessagesPerPoll=1 > > > option... > > > > > > Anyone has the same problem? Can this be fixed? > > > > > > Thank you for your time. Enjoy the weekend! > > > > > > Laurentiu Trica > > > > > > > > > > > -- > > Laurentiu Trica > > Software Developer Mobile: (+40) 722 329318 > > S.C MoreDevs S.R.L. Email: laurentiu.tr...@finalfolder.biz > > > > This message can contain privileged or confidential information and it is > > intended only for addressee. Any unauthorized disclosure is strictly > > prohibited. > > > -- Laurentiu Trica Software Developer Mobile: (+40) 722 329318 S.C MoreDevs S.R.L. Email: laurentiu.tr...@finalfolder.biz This message can contain privileged or confidential information and it is intended only for addressee. Any unauthorized disclosure is strictly prohibited.
