Re: [lftp] Status for incomplete files

2014-09-19 Thread akshay gupta
ant "_tmp_" prefixed to temporary files. ~ Akshay On Fri, Sep 19, 2014 at 12:09 PM, Alexander V. Lukyanov wrote: > On Fri, Sep 19, 2014 at 01:36:18AM +0530, akshay gupta wrote: > > I am looking for a way to know the file is still getting transferred and > is > > incomple

[lftp] Status for incomplete files

2014-09-19 Thread akshay gupta
Hi All, I am looking for a way to know the file is still getting transferred and is incomplete. Some file downloaders append some token such as _tmp or something else to the file name until its getting transferred when the file transfer completes its renamed to its original name. I know about pg

Re: [lftp] Status for incomplete files

2014-09-19 Thread akshay gupta
919.tar.gz, it > introduces another setting xfer:temp-file-name. You can thus change the > temporary file name; an asterisk is replaced by the original file name. So > you would do "set xfer:temp-file-name _tmp_*" to achieve wanted behaviour. > > 2014-09-19 11:11 GMT+04:00 aksh

Re: [lftp] Status for incomplete files

2014-09-19 Thread akshay gupta
Thats just great, cheers!! On Fri, Sep 19, 2014 at 5:11 PM, Alexander Lukyanov wrote: > It will be released in one or two weeks. > > 2014-09-19 15:38 GMT+04:00 akshay gupta : > >> Thanks a lot Alexander, when will these features come out in release >> version? I want

[lftp] Ways to know number of new files after mirror

2014-09-23 Thread akshay gupta
Hi All, I want to know how many new files are added after mirror command has completed. I am going to run the lftp command through java code using runtime exec() call. Does lftp command provide some special exit status code when mirror succeeds but no new files are added. if not can you suggest so

[lftp] possible bug in lftp mirror using xfer:use-temp-file option

2014-09-30 Thread akshay gupta
Hi, I am trying to mirror files on Fuse mounted hdfs file system ( https://wiki.apache.org/hadoop/MountableHDFS). Lftp is creating some 0 byte files. Observations: 1. Files of size larger then 64k are getting transferred properly. Smaller files have 0 byte file size. 2. Debug logs shows that lftp

Re: [lftp] possible bug in lftp mirror using xfer:use-temp-file option

2014-09-30 Thread akshay gupta
; I'll see if it is possible to reverse the order. > > 2014-09-30 12:18 GMT+04:00 akshay gupta : > >> Hi, >> >> I am trying to mirror files on Fuse mounted hdfs file system ( >> https://wiki.apache.org/hadoop/MountableHDFS). Lftp is creating some 0 >> b

Re: [lftp] possible bug in lftp mirror using xfer:use-temp-file option

2014-10-01 Thread akshay gupta
This option (xfer:use-temp-file) is available in development snapshot http://lftp.yar.ru/ftp/devel/lftp-4.5.5.20140919.tar.gz. On Tue, Sep 30, 2014 at 10:35 PM, akshay gupta wrote: > Its causing unpredictable behaviour in case of fuse mounted file system. > Please look into it if it

Re: [lftp] possible bug in lftp mirror using xfer:use-temp-file option

2014-10-03 Thread akshay gupta
I am using this feature to know whether a file is completely transferred or lftp is still downloading this file. On Wed, Oct 1, 2014 at 12:46 PM, akshay gupta wrote: > This option (xfer:use-temp-file) is available in development snapshot > http://lftp.yar.ru/ftp/devel/lftp-4.5.5.20140919.

Re: [lftp] possible bug in lftp mirror using xfer:use-temp-file option

2014-10-13 Thread akshay gupta
Sorry for the delayed response, It works!! I'll let you know if i find any more bugs :) ~ Akshay On Fri, Oct 3, 2014 at 7:42 PM, Alexander V. Lukyanov wrote: > On Tue, Sep 30, 2014 at 10:35:37PM +0530, akshay gupta wrote: > > Its causing unpredictable behaviour in case of fus

[lftp] providing key file for ssh in fish protocol

2014-10-14 Thread akshay gupta
Hi All, I want provide a key file for authentication in fish protocol. Fish internally uses ssh for connection and ssh have a support for providing key files for password less authentication, how can I provide key file using lftp for FISH protocol. Thanks, Akshay _

[lftp] Bug in lftp mirror when using xfer:use-temp-file option

2014-10-27 Thread akshay gupta
Hi, I am trying to mirror files using lftp, I have tested a use case where a file at the source machine gets modified but the file name remains the same. After mirroring, lftp is unable to replace the old file and also leaves a temporary file. For example. If source have file a.txt and its mirror

Re: [lftp] providing key file for ssh in fish protocol

2014-10-27 Thread akshay gupta
What if I also want to specify the ssh user for the transfer. On Wed, Oct 15, 2014 at 10:51 AM, Alexander V. Lukyanov wrote: > On Tue, Oct 14, 2014 at 04:12:50PM +0530, akshay gupta wrote: > > I want provide a key file for authentication in fish protocol. Fish > > internal

Re: [lftp] Bug in lftp mirror when using xfer:use-temp-file option

2014-10-27 Thread akshay gupta
- <--- Data traffic for this session was 172301 bytes in 7 files. <--- Total traffic for this session was 181339 bytes in 13 transfers. <--- 221 Thank you for using the FTP service on 192.168.128.124. Closing control socket ~Akshay On Mon, Oct 27, 2014 at 1:18 PM, Alexander Lu

Re: [lftp] Bug in lftp mirror when using xfer:use-temp-file option

2014-10-27 Thread akshay gupta
no, I am running as a root user. There is some issue with lftp, when file with same name already exist its not able to rename the temp file to actual file. On Mon, Oct 27, 2014 at 1:44 PM, Alexander V. Lukyanov wrote: > On Mon, Oct 27, 2014 at 01:30:42PM +0530, akshay gupta wrote: > >

Re: [lftp] Bug in lftp mirror when using xfer:use-temp-file option

2014-10-27 Thread akshay gupta
B/s) To be removed: 0 directories, 7 files, 0 symlinks 7 errors detected On Mon, Oct 27, 2014 at 5:01 PM, Alexander Lukyanov wrote: > This change should fix the problem. > > https://github.com/lavv17/lftp/commit/b1164d695393cf8ec8429bb2b5caa9f05cfc4d39 > > 2014-10-27 14:20 GMT+03:

Re: [lftp] Bug in lftp mirror when using xfer:use-temp-file option

2014-10-27 Thread akshay gupta
. Lukyanov wrote: > On Mon, Oct 27, 2014 at 05:53:19PM +0530, akshay gupta wrote: > > now its producing 0 byte files and i am also getting input/output error > > here are the debug logs: > > > > copy: renaming `/mount/user/tom/test.txt_tmp_' to `test.txt

Re: [lftp] Bug in lftp mirror when using xfer:use-temp-file option

2014-10-28 Thread akshay gupta
11:50 PM, akshay gupta wrote: > I am using fuse mounted hadoop distributed file system(HDFS). Can you > suggest what changes need to be done to make it work as this is a very > critical requirement for my project. Will deleting file and then renaming > temp file help? how to achieve

Re: [lftp] Bug in lftp mirror when using xfer:use-temp-file option

2014-10-28 Thread akshay gupta
//groups.google.com/a/cloudera.org/forum/#!msg/cdh-user/R1GR2Asj_lc/f_uuIjG5R1UJ >> >> Please try this workaround. >> >> 2014-10-28 10:29 GMT+03:00 akshay gupta : >> >>> will giving --delete-first option in lftp mirror help in my case? I >>> think its

[lftp] Possible Bug in lftp mirror using mirror:sort-by option

2014-10-29 Thread akshay gupta
Hi, I am trying to mirror files using mirror:sort-by option set to 'date-desc'. My file structure is like this $ ls -l -R total 3191264 drwxr-xr-x 3 vikasgoyal biadmin 102 Oct 22 12:35 1 -rw-r--r-- 1 vikasgoyal biadmin 0 Aug 1 01:00 tmp.txt -rw-r--r-- 1 vikasgoyal biadmi

Re: [lftp] Possible Bug in lftp mirror using mirror:sort-by option

2014-10-29 Thread akshay gupta
: > Mirror in lftp works directory by directory. It does not build full list > of files first, so it cannot sort the whole list. Maybe the mirror > algorithm should be redesigned to build the whole recursive list first. > What do you think? > > 2014-10-29 10:07 GMT+03:00 aksh

Re: [lftp] Possible Bug in lftp mirror using mirror:sort-by option

2014-10-30 Thread akshay gupta
lftp have options like --depth-first and --no-recursion I think it will make sense to have an option like " --iterative" for iterative file transfer. This option will fetch the full file list and then fetch them iteratively. On Wed, Oct 29, 2014 at 2:29 PM, akshay gupta wrote: >

Re: [lftp] Possible Bug in lftp mirror using mirror:sort-by option

2014-11-11 Thread akshay gupta
Hi Alexander, Are there any plans for redesigning LFTP Mirror algorithm to build the whole recursive list first. How much change would that require if we go that path? Thanks, Akshay On Thu, Oct 30, 2014 at 4:44 PM, akshay gupta wrote: > lftp have options like --depth-first and --no-recurs

Re: [lftp] Possible Bug in lftp mirror using mirror:sort-by option

2014-11-17 Thread akshay gupta
wrote: > I think it is possible and the amount of programming work is moderate. The > recursive model of mirror will stay but the file transfers will be deferred > and done on the top level when recursive directory traversal is finished. > > 2014-11-12 8:32 GMT+03:00 akshay gupta : &g

[lftp] Compressing data at destination machine

2014-11-25 Thread akshay gupta
Hi, I want to store the data that is being pulled by lftp mirror job in a compressed format. Is it possible that mirror function can check file download status from inside of that tar, zip etc. Is it possible to implement such a functionality in lftp? Thanks, Akshay __

Re: [lftp] Compressing data at destination machine

2014-11-27 Thread akshay gupta
I dont want lftp to extract data from archives, I want it to compress it into archive so that it occupies less space at the destination side. Is this feasible to do in lftp? On Wed, Nov 26, 2014 at 11:33 AM, Alexander V. Lukyanov wrote: > On Tue, Nov 25, 2014 at 06:44:32PM +0530, akshay gu

Re: [lftp] Compressing data at destination machine

2014-12-02 Thread akshay gupta
support writes. Thanks, Akshay On Fri, Nov 28, 2014 at 1:06 PM, Alexander V. Lukyanov wrote: > On Fri, Nov 28, 2014 at 11:55:07AM +0530, akshay gupta wrote: > > I dont want lftp to extract data from archives, I want it to compress it > > into archive so that it occupies le

Re: [lftp] Possible Bug in lftp mirror using mirror:sort-by option

2014-12-02 Thread akshay gupta
Are these feature present in LFTP implementation timeline? By when will they be implemented? ___ lftp mailing list lftp@uniyar.ac.ru http://univ.uniyar.ac.ru/mailman/listinfo/lftp

[lftp] Handling for small files

2015-02-24 Thread akshay gupta
Hi, I am trying to transfer about 80GB of data using mirror option. File sizes are very small about 0.5 KB to 3 KB per file. Transfer is taking a lot of time which I am attributing to small file size. Is there any option in lftp which I can use to increase the transfer speed. May be some sort of c

Re: [lftp] Handling for small files

2015-02-24 Thread akshay gupta
I am using fish protocol. On Tue, Feb 24, 2015, 4:23 PM Alexander Lukyanov wrote: > Which protocol do you use? > > вт, 24 февр. 2015, 13:48, akshay gupta : > > Hi, >> >> I am trying to transfer about 80GB of data using mirror option. File >> sizes are very sm