On 12/11/2015 06:55 PM, Shlomi Fish wrote:
Hi all,
On Fri, 11 Dec 2015 11:07:54 -0500
Shawn H Corey <shawnhco...@gmail.com> wrote:
On Fri, 11 Dec 2015 16:28:39 +0100
Ori Raz <fcb...@gmail.com> wrote:
Hi,
Did anyone encounter the scenario where scp_put is failing (too many
arguments) when the directory contains too many files?
We have 36K files in the directory and it is failing... (with lower
amount it works fine)
This is how we use it:
$dr_node->scp_put( { recursive => 1,
glob => 1,
copy_user_attrs => 1 } ,
"$ib_backup_path/*",$ib_backup_path );
And we get the error:
Can't exec "rsync": Argument list too long at
/perl/lib/perl5/site_perl/5.16.0/Net/OpenSSH.pm line 1433
Appreciate any advise :)
Thanks.
rsync(1) is a UNIX utility http://linux.die.net/man/1/rsync Because of
this, the only solution I can think of is to do one file at a time. Set
up a loop to read their names and send them on at a time.
To increase performance one can either:
1. Segment the list of files into $N-size
chunks (where $N is a relatively large integer) - one can use natatime (= "N at
a time") from List::MoreUtils ( https://metacpan.org/pod/List::MoreUtils ) for
that.
2. Copy an entire directory along with all of its contents.
I wonder why Net::OpenSSH is using rsync, though.
Yes, it should be calling 'scp'.
There is a rsync_put method too that works as scp_put but using 'rsync'
instead of 'scp' under the hood. Maybe Ori has tried both methods and
posted here the code for calling 'scp_put' mixed with the output from
calling 'rsync_put'.
Anyway, Ori, instead of SCP/rsync, you can copy the files using SFTP, a
proper protocol for file transfer that doesn't have that limitation,
check Net::SFTP::Foreign.
Otherwise, you could also try the development version of Net::SSH::Any
(https://github.com/salva/p5-Net-SSH-Any). It has a pure-Perl
implementation of SCP that doesn't rely on an external programs and so
is not affected by a limited command line size.
--
To unsubscribe, e-mail: beginners-unsubscr...@perl.org
For additional commands, e-mail: beginners-h...@perl.org
http://learn.perl.org/
