RE: [gentoo-user] network transfer speed

2021-01-15 Thread Raffaele BELARDI 
> -Original Message-
> From: the...@sys-concept.com 
> Sent: Friday, January 15, 2021 07:57
> To: Gentoo mailing list 
> Subject: [gentoo-user] network transfer speed
> 
> 
> On both of my systems the network card speed is showing 1000
> cat /sys/class/net/enp4s0/speed   1000
> 
> but when I do rsync larage file I only see about: 20 to 22MB/s On my home
> network I get about 110MB/s between PC's
> 
> Both PC's have SSD  and the swith is Gigabit (I think).
> How to find a the bottleneck?

If the PCs attached to the switch show 1000 then the switch _is_ gigabit.

On my 1Gb home network I have an FTP transfer speed between Gentoo PCs A and B 
of almost 900Mbps, the other way round is almost half of that. One difference 
between the two systems is the disk, A uses SATA-2 disk while B has SATA-3.

Does the 'B' in 110MB/s stand for byte? If so you have 880Mbps which is not 
bad, the problem probably lies somewhere else. Otherwise you could check the 
switch error count (if you have a managed switch) or the network card error 
count, just to ensure you don't have a cabling/connector problem.

Have you tried other transfer methods just for comparison? I think FTP is still 
the fastest way to transfer files, though insecure or inconvenient as it might 
be. I have no experience with rsync.

raffaele

Re: [gentoo-user] [OT] Differences between wget and browser file retrieval?

2021-01-15 Thread Walter Dnes 
On Thu, Jan 14, 2021 at 11:00:38PM +0100, David Haller wrote

> So, try:
> 
> wget -S --no-check-certificate -U 'Mozilla/5.0 ...' \
> https://files.ontario.ca/moh-covid-19-report-en-2021-01-14.pdf

  No luck.  For DNS, I use my ISP's servers (Teksavvy) with fallback to
Google 8.8.8.8.


[i3][waltdnes][/dev/shm]  wget -S --no-check-certificate -U 'Mozilla/5.0 
(Windows NT 10.0; Win64; x64; rv:83.0) Gecko/20100101 Firefox/83.0' 
https://files.ontario.ca/moh-covid-19-report-en-2021-01-14.pdf
--2021-01-15 02:15:30--  
https://files.ontario.ca/moh-covid-19-report-en-2021-01-14.pdf
Resolving files.ontario.ca... 13.33.160.117, 13.33.160.123, 13.33.160.45, ...
Connecting to files.ontario.ca|13.33.160.117|:443... connected.
HTTP request sent, awaiting response... 
  HTTP/1.1 200 OK
  Content-Type: application/pdf
  Content-Length: 0
  Connection: keep-alive
  Date: Thu, 14 Jan 2021 15:15:50 GMT
  Last-Modified: Thu, 14 Jan 2021 15:15:50 GMT
  ETag: "d41d8cd98f00b204e9800998ecf8427e"
  x-amz-meta-ctime: 1610637349
  x-amz-meta-mode: 33188
  x-amz-meta-gid: 500
  x-amz-meta-uid: 500
  x-amz-meta-mtime: 1610637349
  Accept-Ranges: bytes
  Server: AmazonS3
  X-Cache: Hit from cloudfront
  Via: 1.1 47dbad48e25df8c5ccf2822e46c2aaa6.cloudfront.net (CloudFront)
  X-Amz-Cf-Pop: YTO50-C3
  X-Amz-Cf-Id: ARgHfF6QMVfUtkxqkr0AL5ljxIfE7Yd5xPmA4eDMx46NdPXOwIftnQ==
  Age: 57573
Length: 0 [application/pdf]
Saving to: 'moh-covid-19-report-en-2021-01-14.pdf'

moh-covid-19-report [ <=>]   0  --.-KB/sin 0s  

2021-01-15 02:15:30 (0.00 B/s) - 'moh-covid-19-report-en-2021-01-14.pdf' saved 
[0/0]



> BTW: you know that you can let date format that URL? e.g.:
> 
> wget -S --no-check-certificate -U 'Mozilla/5.0 ...' \
>   "$(date 
> '+https://files.ontario.ca/moh-covid-19-report-en-%Y-%m-%d.pdf')"

  Nice, but civil servants get stat holidays off.  I downloaded Dec 25th
and 26th PDFs on the 26th.  Monday Dec 28th was a lieu day for Boxing
day, so I downloaded the 28th and 29th PDFs on the 29th.  And of course
Jan 1st and 2nd PDFs on Jan 2nd.  That's why I can't automate the date.
I have a script "getone"...

[i3][waltdnes][~/covid] cat getone 
#!/bin/bash
wget https://files.ontario.ca/moh-covid-19-report-en-2021-01-${1}.pdf

  On the 14th it was invoked as "../getone 14" (called from the working
directory, one level below the main "covid" directory).  I tweak the
script once a month to match year+month.  In a worst-case scenario. I
can go to
https://covid-19.ontario.ca/covid-19-epidemiologic-summaries-public-health-ontario#daily
to manually retrieve a daily PDF.  Note that on this page, they list
the date that the report is up to.  The report issued 10:15 AM on the
14th shows up in the listing as "COVID-19 in Ontario: January 13, 2021".
That's because it contains data up to the 13th.

-- 
Walter Dnes 
I don't run "desktop environments"; I run useful applications

RE: [gentoo-user] network transfer speed

2021-01-15 Thread Raffaele BELARDI 
> -Original Message-
> From: bobwxc 
> Sent: Friday, January 15, 2021 08:57
> To: gentoo-user@lists.gentoo.org
> Subject: Re: [gentoo-user] network transfer speed
> 
> 在 2021/1/15 下午2:56, the...@sys-concept.com 写道:
> > On both of my systems the network card speed is showing 1000
> > cat /sys/class/net/enp4s0/speed   1000
> >
> > but when I do rsync larage file I only see about: 20 to 22MB/s On my
> > home network I get about 110MB/s between PC's
> >
> > Both PC's have SSD  and the swith is Gigabit (I think).
> > How to find a the bottleneck?
> 1000Mbps network card's maximum theoretical speed is about 125MiB/s.
> It only works in short distances.

Correct but that's the line speed that you'll never reach, when you take into 
account Ethernet frame overhead, IP (and possibly TCP) header overhead and 
application ( rsync, FTP, SMB, NFS) overhead you get lower figures. In my 
experience 900Mbps (110MiBps) on a 1000Mbps line is more realistic for 'normal' 
transfers.

raffaele

RE: [gentoo-user] network transfer speed

2021-01-15 Thread Raffaele BELARDI 


ST Restricted

> -Original Message-
> From: Hogren 
> Sent: Friday, January 15, 2021 08:50
> To: gentoo-user@lists.gentoo.org
> Subject: Re: [gentoo-user] network transfer speed
> 
> 
> On 15/01/2021 07:56, the...@sys-concept.com wrote:
> Hello
> > On both of my systems the network card speed is showing 1000
> > cat /sys/class/net/enp4s0/speed   1000
> >
> > but when I do rsync larage file I only see about: 20 to 22MB/s On my
> > home network I get about 110MB/s between PC's
> >
> > Both PC's have SSD  and the swith is Gigabit (I think).
> > How to find a the bottleneck?
> >
> 20MB = 80Mb so it sounds like your network is a 100Mb network. What is the
> perfs of your switch(s) between your systems ?

I disagree, /sys/class/net/enp4s0/speed shows the speed negotiated by the 
network card with the switch, it cannot be 1000 if the switch is a only a 
10/100. I think we can safely assume the network is a gigabit one.

raffaele

Re: [gentoo-user] network transfer speed

2021-01-15 Thread bobwxc 

在 2021/1/15 下午4:27, Raffaele BELARDI 写道:

-Original Message-
From: bobwxc 
Sent: Friday, January 15, 2021 08:57
To: gentoo-user@lists.gentoo.org
Subject: Re: [gentoo-user] network transfer speed

在 2021/1/15 下午2:56, the...@sys-concept.com 写道:

On both of my systems the network card speed is showing 1000
cat /sys/class/net/enp4s0/speed   1000

but when I do rsync larage file I only see about: 20 to 22MB/s On my
home network I get about 110MB/s between PC's

Both PC's have SSD  and the swith is Gigabit (I think).
How to find a the bottleneck?

1000Mbps network card's maximum theoretical speed is about 125MiB/s.
It only works in short distances.

Correct but that's the line speed that you'll never reach, when you take into 
account Ethernet frame overhead, IP (and possibly TCP) header overhead and 
application ( rsync, FTP, SMB, NFS) overhead you get lower figures. In my 
experience 900Mbps (110MiBps) on a 1000Mbps line is more realistic for 'normal' 
transfers.


Yes, you are right. So it is just *theoretical* speed :-)

I don't know where does the file he sync from.
If you sync a file from a server in other city, for a 20 to 22MB/s speed 
is very normal. But if in home, that is not good.


And for ftp and rsync.
    ftp is better for transferring a single large file once.
    rsync is better for a long-term, incremental synchronization. The 
file verification of rsync may take a lot of time for first sync.

[gentoo-user] syslogd hibernating

2021-01-15 Thread n952162 

Hello,

can anyone explain this?

I noticed today (15. January) that the /var/log/{messages,kern.log,etc.}
files on a box were last touched on 22. November.

sysklogd was in the rc-open /started/ state and was running.

The configuration file, /etc/syslog.conf matches completely that file on
another machine of mine, where the logs are properly updated.

/etc/syslog.d/ was empty on both machines.

I sent the process a HUP signal, as follows

sudo kill -HUP $(cat /var/run/syslogd.pid )

and all the log files were immediately updated and reporting.

I rebooted my machine, and the log files are untouched, it is again
hibernating.

Re: [gentoo-user] syslogd hibernating

2021-01-15 Thread Michael 
On Friday, 15 January 2021 08:47:18 GMT n952162 wrote:
> Hello,
> 
> can anyone explain this?
> 
> I noticed today (15. January) that the /var/log/{messages,kern.log,etc.}
> files on a box were last touched on 22. November.
> 
> sysklogd was in the rc-open /started/ state and was running.
> 
> The configuration file, /etc/syslog.conf matches completely that file on
> another machine of mine, where the logs are properly updated.
> 
> /etc/syslog.d/ was empty on both machines.
> 
> I sent the process a HUP signal, as follows
> 
> sudo kill -HUP $(cat /var/run/syslogd.pid )
> 
> and all the log files were immediately updated and reporting.
> 
> I rebooted my machine, and the log files are untouched, it is again
> hibernating.

Do you get something like this on your system?

$ rc-update -s -v | grep syslog
syslog-ng |  default 

and,

$ rc-service -v syslog-ng status
 * Executing: /lib/rc/sh/openrc-run.sh /lib/rc/sh/openrc-run.sh /etc/init.d/
syslog-ng status
 * status: started



signature.asc
Description: This is a digitally signed message part.

Re: [gentoo-user] network transfer speed

2021-01-15 Thread Michael 
On Friday, 15 January 2021 08:42:16 GMT bobwxc wrote:
> 在 2021/1/15 下午4:27, Raffaele BELARDI 写道:
> >> -Original Message-
> >> From: bobwxc 
> >> Sent: Friday, January 15, 2021 08:57
> >> To: gentoo-user@lists.gentoo.org
> >> Subject: Re: [gentoo-user] network transfer speed
> >> 
> >> 在 2021/1/15 下午2:56, the...@sys-concept.com 写道:
> >>> On both of my systems the network card speed is showing 1000
> >>> cat /sys/class/net/enp4s0/speed   1000
> >>> 
> >>> but when I do rsync larage file I only see about: 20 to 22MB/s On my
> >>> home network I get about 110MB/s between PC's
> >>> 
> >>> Both PC's have SSD  and the swith is Gigabit (I think).
> >>> How to find a the bottleneck?
> >> 
> >> 1000Mbps network card's maximum theoretical speed is about 125MiB/s.
> >> It only works in short distances.
> > 
> > Correct but that's the line speed that you'll never reach, when you take
> > into account Ethernet frame overhead, IP (and possibly TCP) header
> > overhead and application ( rsync, FTP, SMB, NFS) overhead you get lower
> > figures. In my experience 900Mbps (110MiBps) on a 1000Mbps line is more
> > realistic for 'normal' transfers.
> Yes, you are right. So it is just *theoretical* speed :-)
> 
> I don't know where does the file he sync from.
> If you sync a file from a server in other city, for a 20 to 22MB/s speed
> is very normal. But if in home, that is not good.
> 
> And for ftp and rsync.
>  ftp is better for transferring a single large file once.
>  rsync is better for a long-term, incremental synchronization. The
> file verification of rsync may take a lot of time for first sync.

There is a theoretical network speed as already mentioned.  There is a 
protocol speed, which may limit throughput if it has e.g. heavy encryption/
compression and the CPU is anaemic.  Finally, there is a MoBo bus (SCSI/SATA/
USB) and the media storage limit.  If using USB 1.1 or 2.0 and/or the disks 
are slow or experience write amplification, you'll find this will constrain 
the final transfer speed significantly.

signature.asc
Description: This is a digitally signed message part.

Re: [gentoo-user] network transfer speed

2021-01-15 Thread Hogren 



On 15/01/2021 09:34, Raffaele BELARDI wrote:


ST Restricted


-Original Message-
From: Hogren 
Sent: Friday, January 15, 2021 08:50
To: gentoo-user@lists.gentoo.org
Subject: Re: [gentoo-user] network transfer speed


On 15/01/2021 07:56, the...@sys-concept.com wrote:
Hello

On both of my systems the network card speed is showing 1000
cat /sys/class/net/enp4s0/speed   1000

but when I do rsync larage file I only see about: 20 to 22MB/s On my
home network I get about 110MB/s between PC's

Both PC's have SSD  and the swith is Gigabit (I think).
How to find a the bottleneck?


20MB = 80Mb so it sounds like your network is a 100Mb network. What is the
perfs of your switch(s) between your systems ?

I disagree, /sys/class/net/enp4s0/speed shows the speed negotiated by the 
network card with the switch, it cannot be 1000 if the switch is a only a 
10/100. I think we can safely assume the network is a gigabit one.

raffaele


Yes, I thought about that after. But may be he has several switchs 
between the two systems.


Hogren

Re: [gentoo-user] network transfer speed

2021-01-15 Thread Michael 
On Friday, 15 January 2021 13:26:23 GMT Hogren wrote:
> On 15/01/2021 09:34, Raffaele BELARDI wrote:
> > ST Restricted
> > 
> >> -Original Message-
> >> From: Hogren 
> >> Sent: Friday, January 15, 2021 08:50
> >> To: gentoo-user@lists.gentoo.org
> >> Subject: Re: [gentoo-user] network transfer speed
> >> 
> >> 
> >> On 15/01/2021 07:56, the...@sys-concept.com wrote:
> >> Hello
> >> 
> >>> On both of my systems the network card speed is showing 1000
> >>> cat /sys/class/net/enp4s0/speed   1000
> >>> 
> >>> but when I do rsync larage file I only see about: 20 to 22MB/s On my
> >>> home network I get about 110MB/s between PC's
> >>> 
> >>> Both PC's have SSD  and the swith is Gigabit (I think).
> >>> How to find a the bottleneck?
> >> 
> >> 20MB = 80Mb so it sounds like your network is a 100Mb network. What is
> >> the
> >> perfs of your switch(s) between your systems ?
> > 
> > I disagree, /sys/class/net/enp4s0/speed shows the speed negotiated by the
> > network card with the switch, it cannot be 1000 if the switch is a only a
> > 10/100. I think we can safely assume the network is a gigabit one.
> > 
> > raffaele
> 
> Yes, I thought about that after. But may be he has several switchs
> between the two systems.
> 
> Hogren

There's an easy way to test the speed limits of the network Vs the limits of 
the storage media.  Use netcat/telnet to send a large file across from tmpfs 
on host A to a tmpfs on host B.  As long as tmpfs is large enough to not start 
using swap, the speed will reflect what the network can achieve.

signature.asc
Description: This is a digitally signed message part.

Re: [gentoo-user] syslogd hibernating

2021-01-15 Thread n952162 

On 1/15/21 10:24 AM, Michael wrote:

On Friday, 15 January 2021 08:47:18 GMT n952162 wrote:

Hello,

can anyone explain this?

I noticed today (15. January) that the /var/log/{messages,kern.log,etc.}
files on a box were last touched on 22. November.

sysklogd was in the rc-open /started/ state and was running.

The configuration file, /etc/syslog.conf matches completely that file on
another machine of mine, where the logs are properly updated.

/etc/syslog.d/ was empty on both machines.

I sent the process a HUP signal, as follows

sudo kill -HUP $(cat /var/run/syslogd.pid )

and all the log files were immediately updated and reporting.

I rebooted my machine, and the log files are untouched, it is again
hibernating.

Do you get something like this on your system?

$ rc-update -s -v | grep syslog
 syslog-ng |  default

and,

$ rc-service -v syslog-ng status
  * Executing: /lib/rc/sh/openrc-run.sh /lib/rc/sh/openrc-run.sh /etc/init.d/
syslog-ng status
  * status: started


$ rc-service -v syslog-ng status
 * rc-service: service `syslog-ng' does not exist

$ rc-service -v syslog status
 * rc-service: service `syslog' does not exist

$ rc-service -v syslogd status
 * rc-service: service `syslogd' does not exist

$ rc-service -v sysklogd status
 * Executing: /lib/rc/sh/openrc-run.sh /lib/rc/sh/openrc-run.sh
/etc/init.d/sysklogd status
 * status: started

$ rc-update -s -v | grep syslog

$ rc-update -s -v | grep sysklog
 sysklogd |  default    sysinit


I ran that as a script on a machine where the logs are properly updated:

01~>bash -x  /tmp/test
+ rc-service -v syslog-ng status
 * rc-service: service `syslog-ng' does not exist
+ rc-service -v syslog status
 * rc-service: service `syslog' does not exist
+ rc-service -v syslogd status
 * rc-service: service `syslogd' does not exist
+ rc-service -v sysklogd status
 * Executing: /lib/rc/sh/openrc-run.sh /lib/rc/sh/openrc-run.sh
/etc/init.d/sysklogd status
 * status: started
+ rc-update -s -v
+ grep syslog
+ rc-update -s -v
+ grep sysklog
 sysklogd |  default

Re: [gentoo-user] [OT] Differences between wget and browser file retrieval?

2021-01-15 Thread Walter Dnes 
On Fri, Jan 15, 2021 at 02:40:51AM -0500, Philip Webb wrote
>  
> Here in Toronto, I get the same result as Walter via his URL
> & similar results from the  2  longer versions above,
> except that the escaped version give "ERROR 403: Forbidden".

  I get "ERROR 403: Forbidden" when downloading a non-existant file,
e.g. when I make a typo, or when the government site is late updating
and they haven't posted the file by the time I request it.

-- 
Walter Dnes 
I don't run "desktop environments"; I run useful applications

Re: [gentoo-user] syslogd hibernating

2021-01-15 Thread Peter Humphrey 
On Friday, 15 January 2021 15:08:15 GMT n952162 wrote:
> On 1/15/21 10:24 AM, Michael wrote:
> > On Friday, 15 January 2021 08:47:18 GMT n952162 wrote:
> >> Hello,
> >> 
> >> can anyone explain this?
> >> 
> >> I noticed today (15. January) that the /var/log/{messages,kern.log,etc.}
> >> files on a box were last touched on 22. November.
> >> 
> >> sysklogd was in the rc-open /started/ state and was running.
> >> 
> >> The configuration file, /etc/syslog.conf matches completely that file on
> >> another machine of mine, where the logs are properly updated.
> >> 
> >> /etc/syslog.d/ was empty on both machines.
> >> 
> >> I sent the process a HUP signal, as follows
> >> 
> >> sudo kill -HUP $(cat /var/run/syslogd.pid )
> >> 
> >> and all the log files were immediately updated and reporting.
> >> 
> >> I rebooted my machine, and the log files are untouched, it is again
> >> hibernating.
> > 
> > Do you get something like this on your system?
> > 
> > $ rc-update -s -v | grep syslog
> > 
> >  syslog-ng |  default
> > 
> > and,
> > 
> > $ rc-service -v syslog-ng status
> > 
> >   * Executing: /lib/rc/sh/openrc-run.sh /lib/rc/sh/openrc-run.sh
> >   /etc/init.d/
> > 
> > syslog-ng status
> > 
> >   * status: started
> 
> $ rc-service -v syslog-ng status
>   * rc-service: service `syslog-ng' does not exist
> 
> $ rc-service -v syslog status
>   * rc-service: service `syslog' does not exist
> 
> $ rc-service -v syslogd status
>   * rc-service: service `syslogd' does not exist
> 
> $ rc-service -v sysklogd status
>   * Executing: /lib/rc/sh/openrc-run.sh /lib/rc/sh/openrc-run.sh
> /etc/init.d/sysklogd status
>   * status: started
> 
> $ rc-update -s -v | grep syslog
> 
> $ rc-update -s -v | grep sysklog
>   sysklogd |  defaultsysinit
> 
> 
> I ran that as a script on a machine where the logs are properly updated:
> 
> 01~>bash -x  /tmp/test
> + rc-service -v syslog-ng status
>   * rc-service: service `syslog-ng' does not exist
> + rc-service -v syslog status
>   * rc-service: service `syslog' does not exist
> + rc-service -v syslogd status
>   * rc-service: service `syslogd' does not exist
> + rc-service -v sysklogd status
>   * Executing: /lib/rc/sh/openrc-run.sh /lib/rc/sh/openrc-run.sh
> /etc/init.d/sysklogd status
>   * status: started
> + rc-update -s -v
> + grep syslog
> + rc-update -s -v
> + grep sysklog
>   sysklogd |  default

Substitute your syslog program's name for syslog-ng. Michael did say 
"something like."

-- 
Regards,
Peter.

Re: [gentoo-user] syslogd hibernating

2021-01-15 Thread Michael 
On Friday, 15 January 2021 15:23:34 GMT Peter Humphrey wrote:
> On Friday, 15 January 2021 15:08:15 GMT n952162 wrote:
> > On 1/15/21 10:24 AM, Michael wrote:
> > > On Friday, 15 January 2021 08:47:18 GMT n952162 wrote:
> > >> Hello,
> > >> 
> > >> can anyone explain this?
> > >> 
> > >> I noticed today (15. January) that the
> > >> /var/log/{messages,kern.log,etc.}
> > >> files on a box were last touched on 22. November.
> > >> 
> > >> sysklogd was in the rc-open /started/ state and was running.
> > >> 
> > >> The configuration file, /etc/syslog.conf matches completely that file
> > >> on
> > >> another machine of mine, where the logs are properly updated.
> > >> 
> > >> /etc/syslog.d/ was empty on both machines.
> > >> 
> > >> I sent the process a HUP signal, as follows
> > >> 
> > >> sudo kill -HUP $(cat /var/run/syslogd.pid )
> > >> 
> > >> and all the log files were immediately updated and reporting.
> > >> 
> > >> I rebooted my machine, and the log files are untouched, it is again
> > >> hibernating.
> > > 
> > > Do you get something like this on your system?
> > > 
> > > $ rc-update -s -v | grep syslog
> > > 
> > >  syslog-ng |  default
> > > 
> > > and,
> > > 
> > > $ rc-service -v syslog-ng status
> > > 
> > >   * Executing: /lib/rc/sh/openrc-run.sh /lib/rc/sh/openrc-run.sh
> > >   /etc/init.d/
> > > 
> > > syslog-ng status
> > > 
> > >   * status: started
> > 
> > $ rc-service -v syslog-ng status
> > 
> >   * rc-service: service `syslog-ng' does not exist
> > 
> > $ rc-service -v syslog status
> > 
> >   * rc-service: service `syslog' does not exist
> > 
> > $ rc-service -v syslogd status
> > 
> >   * rc-service: service `syslogd' does not exist
> > 
> > $ rc-service -v sysklogd status
> > 
> >   * Executing: /lib/rc/sh/openrc-run.sh /lib/rc/sh/openrc-run.sh
> > 
> > /etc/init.d/sysklogd status
> > 
> >   * status: started
> > 
> > $ rc-update -s -v | grep syslog
> > 
> > $ rc-update -s -v | grep sysklog
> > 
> >   sysklogd |  defaultsysinit
> > 
> > I ran that as a script on a machine where the logs are properly updated:
> > 
> > 01~>bash -x  /tmp/test
> > + rc-service -v syslog-ng status
> > 
> >   * rc-service: service `syslog-ng' does not exist
> > 
> > + rc-service -v syslog status
> > 
> >   * rc-service: service `syslog' does not exist
> > 
> > + rc-service -v syslogd status
> > 
> >   * rc-service: service `syslogd' does not exist
> > 
> > + rc-service -v sysklogd status
> > 
> >   * Executing: /lib/rc/sh/openrc-run.sh /lib/rc/sh/openrc-run.sh
> > 
> > /etc/init.d/sysklogd status
> > 
> >   * status: started
> > 
> > + rc-update -s -v
> > + grep syslog
> > + rc-update -s -v
> > + grep sysklog
> > 
> >   sysklogd |  default
> 
> Substitute your syslog program's name for syslog-ng. Michael did say
> "something like."

Yes, quite, other log daemons are available.  :-)

It seems your PC on which sysklogd works as expected has its rc script only on 
'default' runlevel, rather than default +sysinit.  I don't know if sysinit is 
required.  If you remove sysinit and restart sysklogd from a terminal does it 
spew out any errors?

signature.asc
Description: This is a digitally signed message part.

Re: [gentoo-user] syslogd hibernating

2021-01-15 Thread Peter Humphrey 
On Friday, 15 January 2021 15:30:08 GMT Michael wrote:

> It seems your PC on which sysklogd works as expected has its rc script only
> on 'default' runlevel, rather than default +sysinit.  I don't know if
> sysinit is required.  If you remove sysinit and restart sysklogd from a
> terminal does it spew out any errors?

Syslog-ng is not in sysinit here.

-- 
Regards,
Peter.

[gentoo-user] Re: [OT SOLVED] Differences between wget and browser file retrieval?

2021-01-15 Thread Walter Dnes 
  It looks like a temporary server hiccup yesterday. wget correctly
pulled down the PDF file for the 15th today.  I checked and it also
pulled down the file for the 14th.

-- 
Walter Dnes 
I don't run "desktop environments"; I run useful applications

Re: [gentoo-user] syslogd hibernating

2021-01-15 Thread n952162 

On 1/15/21 4:30 PM, Michael wrote:

On Friday, 15 January 2021 15:23:34 GMT Peter Humphrey wrote:

On Friday, 15 January 2021 15:08:15 GMT n952162 wrote:

On 1/15/21 10:24 AM, Michael wrote:

On Friday, 15 January 2021 08:47:18 GMT n952162 wrote:

Hello,

can anyone explain this?

I noticed today (15. January) that the
/var/log/{messages,kern.log,etc.}
files on a box were last touched on 22. November.

sysklogd was in the rc-open /started/ state and was running.

The configuration file, /etc/syslog.conf matches completely that file
on
another machine of mine, where the logs are properly updated.

/etc/syslog.d/ was empty on both machines.

I sent the process a HUP signal, as follows

sudo kill -HUP $(cat /var/run/syslogd.pid )

and all the log files were immediately updated and reporting.

I rebooted my machine, and the log files are untouched, it is again
hibernating.

Do you get something like this on your system?

$ rc-update -s -v | grep syslog

  syslog-ng |  default

and,

$ rc-service -v syslog-ng status

   * Executing: /lib/rc/sh/openrc-run.sh /lib/rc/sh/openrc-run.sh
   /etc/init.d/

syslog-ng status

   * status: started

$ rc-service -v syslog-ng status

   * rc-service: service `syslog-ng' does not exist

$ rc-service -v syslog status

   * rc-service: service `syslog' does not exist

$ rc-service -v syslogd status

   * rc-service: service `syslogd' does not exist

$ rc-service -v sysklogd status

   * Executing: /lib/rc/sh/openrc-run.sh /lib/rc/sh/openrc-run.sh

/etc/init.d/sysklogd status

   * status: started

$ rc-update -s -v | grep syslog

$ rc-update -s -v | grep sysklog

   sysklogd |  defaultsysinit

I ran that as a script on a machine where the logs are properly updated:

01~>bash -x  /tmp/test
+ rc-service -v syslog-ng status

   * rc-service: service `syslog-ng' does not exist

+ rc-service -v syslog status

   * rc-service: service `syslog' does not exist

+ rc-service -v syslogd status

   * rc-service: service `syslogd' does not exist

+ rc-service -v sysklogd status

   * Executing: /lib/rc/sh/openrc-run.sh /lib/rc/sh/openrc-run.sh

/etc/init.d/sysklogd status

   * status: started

+ rc-update -s -v
+ grep syslog
+ rc-update -s -v
+ grep sysklog

   sysklogd |  default

Substitute your syslog program's name for syslog-ng. Michael did say
"something like."

Yes, quite, other log daemons are available.  :-)

It seems your PC on which sysklogd works as expected has its rc script only on
'default' runlevel, rather than default +sysinit.  I don't know if sysinit is
required.  If you remove sysinit and restart sysklogd from a terminal does it
spew out any errors?



Oh, I missed that sysinit way out there on the left ;-)

It looks like that was it, no errors after removing runlevel sysinit,
and /var/log/messages has like the current time.

Thanks, that was great!

Re: [gentoo-user] syslogd hibernating [RESOLVED]

2021-01-15 Thread n952162 

On 1/15/21 5:29 PM, n952162 wrote:

On 1/15/21 4:30 PM, Michael wrote:

On Friday, 15 January 2021 15:23:34 GMT Peter Humphrey wrote:

On Friday, 15 January 2021 15:08:15 GMT n952162 wrote:

On 1/15/21 10:24 AM, Michael wrote:

On Friday, 15 January 2021 08:47:18 GMT n952162 wrote:

Hello,

can anyone explain this?

I noticed today (15. January) that the
/var/log/{messages,kern.log,etc.}
files on a box were last touched on 22. November.

sysklogd was in the rc-open /started/ state and was running.

The configuration file, /etc/syslog.conf matches completely that
file
on
another machine of mine, where the logs are properly updated.

/etc/syslog.d/ was empty on both machines.

I sent the process a HUP signal, as follows

sudo kill -HUP $(cat /var/run/syslogd.pid )

and all the log files were immediately updated and reporting.

I rebooted my machine, and the log files are untouched, it is again
hibernating.

Do you get something like this on your system?

$ rc-update -s -v | grep syslog

  syslog-ng |  default

and,

$ rc-service -v syslog-ng status

   * Executing: /lib/rc/sh/openrc-run.sh /lib/rc/sh/openrc-run.sh
   /etc/init.d/

syslog-ng status

   * status: started

$ rc-service -v syslog-ng status

   * rc-service: service `syslog-ng' does not exist

$ rc-service -v syslog status

   * rc-service: service `syslog' does not exist

$ rc-service -v syslogd status

   * rc-service: service `syslogd' does not exist

$ rc-service -v sysklogd status

   * Executing: /lib/rc/sh/openrc-run.sh /lib/rc/sh/openrc-run.sh

/etc/init.d/sysklogd status

   * status: started

$ rc-update -s -v | grep syslog

$ rc-update -s -v | grep sysklog

   sysklogd |  default sysinit

I ran that as a script on a machine where the logs are properly
updated:

01~>bash -x  /tmp/test
+ rc-service -v syslog-ng status

   * rc-service: service `syslog-ng' does not exist

+ rc-service -v syslog status

   * rc-service: service `syslog' does not exist

+ rc-service -v syslogd status

   * rc-service: service `syslogd' does not exist

+ rc-service -v sysklogd status

   * Executing: /lib/rc/sh/openrc-run.sh /lib/rc/sh/openrc-run.sh

/etc/init.d/sysklogd status

   * status: started

+ rc-update -s -v
+ grep syslog
+ rc-update -s -v
+ grep sysklog

   sysklogd |  default

Substitute your syslog program's name for syslog-ng. Michael did say
"something like."

Yes, quite, other log daemons are available.  :-)

It seems your PC on which sysklogd works as expected has its rc
script only on
'default' runlevel, rather than default +sysinit.  I don't know if
sysinit is
required.  If you remove sysinit and restart sysklogd from a terminal
does it
spew out any errors?



Oh, I missed that sysinit way out there on the left ;-)


(correction: way out there on the right)

Re: [gentoo-user] network transfer speed

2021-01-15 Thread thelma 
On 1/15/21 6:26 AM, Hogren wrote:
> 
> On 15/01/2021 09:34, Raffaele BELARDI wrote:
>>
>> ST Restricted
>>
>>> -Original Message-
>>> From: Hogren 
>>> Sent: Friday, January 15, 2021 08:50
>>> To: gentoo-user@lists.gentoo.org
>>> Subject: Re: [gentoo-user] network transfer speed
>>>
>>>
>>> On 15/01/2021 07:56, the...@sys-concept.com wrote:
>>> Hello
 On both of my systems the network card speed is showing 1000
 cat /sys/class/net/enp4s0/speed   1000

 but when I do rsync larage file I only see about: 20 to 22MB/s On my
 home network I get about 110MB/s between PC's

 Both PC's have SSD  and the swith is Gigabit (I think).
 How to find a the bottleneck?

>>> 20MB = 80Mb so it sounds like your network is a 100Mb network. What is the
>>> perfs of your switch(s) between your systems ?
>> I disagree, /sys/class/net/enp4s0/speed shows the speed negotiated by the 
>> network card with the switch, it cannot be 1000 if the switch is a only a 
>> 10/100. I think we can safely assume the network is a gigabit one.
>>
>> raffaele
> 
> Yes, I thought about that after. But may be he has several switchs between 
> the two systems.
> 
> Hogren

I just checked the remote location and there are two swiches:
- D-link Green Technology (I think it is DSG-1005D
- Trident Gigabit Switch  - TEG-S80g
 
They are both Gigabit switches.

Re: [gentoo-user] syslogd hibernating

2021-01-15 Thread n952162 

On 1/15/21 4:30 PM, Michael wrote:

On Friday, 15 January 2021 15:23:34 GMT Peter Humphrey wrote:

On Friday, 15 January 2021 15:08:15 GMT n952162 wrote:

On 1/15/21 10:24 AM, Michael wrote:

On Friday, 15 January 2021 08:47:18 GMT n952162 wrote:

Hello,

can anyone explain this?

I noticed today (15. January) that the
/var/log/{messages,kern.log,etc.}
files on a box were last touched on 22. November.

sysklogd was in the rc-open /started/ state and was running.

The configuration file, /etc/syslog.conf matches completely that file
on
another machine of mine, where the logs are properly updated.

/etc/syslog.d/ was empty on both machines.

I sent the process a HUP signal, as follows

sudo kill -HUP $(cat /var/run/syslogd.pid )

and all the log files were immediately updated and reporting.

I rebooted my machine, and the log files are untouched, it is again
hibernating.

Do you get something like this on your system?

$ rc-update -s -v | grep syslog

syslog-ng | default

and,

$ rc-service -v syslog-ng status

* Executing: /lib/rc/sh/openrc-run.sh /lib/rc/sh/openrc-run.sh
/etc/init.d/

syslog-ng status

* status: started

$ rc-service -v syslog-ng status

* rc-service: service `syslog-ng' does not exist

$ rc-service -v syslog status

* rc-service: service `syslog' does not exist

$ rc-service -v syslogd status

* rc-service: service `syslogd' does not exist

$ rc-service -v sysklogd status

* Executing: /lib/rc/sh/openrc-run.sh /lib/rc/sh/openrc-run.sh

/etc/init.d/sysklogd status

* status: started

$ rc-update -s -v | grep syslog

$ rc-update -s -v | grep sysklog

sysklogd | default sysinit

I ran that as a script on a machine where the logs are properly updated:

01~>bash -x /tmp/test
+ rc-service -v syslog-ng status

* rc-service: service `syslog-ng' does not exist

+ rc-service -v syslog status

* rc-service: service `syslog' does not exist

+ rc-service -v syslogd status

* rc-service: service `syslogd' does not exist

+ rc-service -v sysklogd status

* Executing: /lib/rc/sh/openrc-run.sh /lib/rc/sh/openrc-run.sh

/etc/init.d/sysklogd status

* status: started

+ rc-update -s -v
+ grep syslog
+ rc-update -s -v
+ grep sysklog

sysklogd | default

Substitute your syslog program's name for syslog-ng. Michael did say
"something like."

Yes, quite, other log daemons are available. :-)

It seems your PC on which sysklogd works as expected has its rc script
only on
'default' runlevel, rather than default +sysinit. I don't know if
sysinit is
required. If you remove sysinit and restart sysklogd from a terminal
does it
spew out any errors?



Oh, I missed that sysinit way out there on the left ;-)

It looks like that was it, no errors after removing runlevel sysinit,
and /var/log/messages has like the current time.

Thanks, that was great!

Re: [gentoo-user] network transfer speed

2021-01-15 Thread thelma 
On 1/15/21 2:58 AM, Michael wrote:
> On Friday, 15 January 2021 08:42:16 GMT bobwxc wrote:
>> 在 2021/1/15 下午4:27, Raffaele BELARDI 写道:
 -Original Message-
 From: bobwxc 
 Sent: Friday, January 15, 2021 08:57
 To: gentoo-user@lists.gentoo.org
 Subject: Re: [gentoo-user] network transfer speed

 在 2021/1/15 下午2:56, the...@sys-concept.com 写道:
> On both of my systems the network card speed is showing 1000
> cat /sys/class/net/enp4s0/speed   1000
>
> but when I do rsync larage file I only see about: 20 to 22MB/s On my
> home network I get about 110MB/s between PC's
>
> Both PC's have SSD  and the swith is Gigabit (I think).
> How to find a the bottleneck?

 1000Mbps network card's maximum theoretical speed is about 125MiB/s.
 It only works in short distances.
>>>
>>> Correct but that's the line speed that you'll never reach, when you take
>>> into account Ethernet frame overhead, IP (and possibly TCP) header
>>> overhead and application ( rsync, FTP, SMB, NFS) overhead you get lower
>>> figures. In my experience 900Mbps (110MiBps) on a 1000Mbps line is more
>>> realistic for 'normal' transfers.
>> Yes, you are right. So it is just *theoretical* speed :-)
>>
>> I don't know where does the file he sync from.
>> If you sync a file from a server in other city, for a 20 to 22MB/s speed
>> is very normal. But if in home, that is not good.
>>
>> And for ftp and rsync.
>>  ftp is better for transferring a single large file once.
>>  rsync is better for a long-term, incremental synchronization. The
>> file verification of rsync may take a lot of time for first sync.
> 
> There is a theoretical network speed as already mentioned.  There is a 
> protocol speed, which may limit throughput if it has e.g. heavy encryption/
> compression and the CPU is anaemic.  Finally, there is a MoBo bus (SCSI/SATA/
> USB) and the media storage limit.  If using USB 1.1 or 2.0 and/or the disks 
> are slow or experience write amplification, you'll find this will constrain 
> the final transfer speed significantly.

The computers on this network are 2-meters apart and they both use SSD Drive 
(so USB limitation doesn't come under consideration).
Like I said, on my home network when I transfer the 24GB file I get about 
110MiBps transfer, so I was expecting the same in remote location).
Some units are connected to a router Ausus RT-AC66U B1 but these ports are 
gigabit too.

Re: [gentoo-user] network transfer speed

2021-01-15 Thread Jack 

On 1/15/21 11:51 AM, the...@sys-concept.com wrote:

On 1/15/21 2:58 AM, Michael wrote:

On Friday, 15 January 2021 08:42:16 GMT bobwxc wrote:

在 2021/1/15 下午4:27, Raffaele BELARDI 写道:

-Original Message-
From: bobwxc 
Sent: Friday, January 15, 2021 08:57
To: gentoo-user@lists.gentoo.org
Subject: Re: [gentoo-user] network transfer speed

在 2021/1/15 下午2:56, the...@sys-concept.com 写道:

On both of my systems the network card speed is showing 1000
cat /sys/class/net/enp4s0/speed   1000

but when I do rsync larage file I only see about: 20 to 22MB/s On my
home network I get about 110MB/s between PC's

Both PC's have SSD  and the swith is Gigabit (I think).
How to find a the bottleneck?

1000Mbps network card's maximum theoretical speed is about 125MiB/s.
It only works in short distances.

Correct but that's the line speed that you'll never reach, when you take
into account Ethernet frame overhead, IP (and possibly TCP) header
overhead and application ( rsync, FTP, SMB, NFS) overhead you get lower
figures. In my experience 900Mbps (110MiBps) on a 1000Mbps line is more
realistic for 'normal' transfers.

Yes, you are right. So it is just *theoretical* speed :-)

I don't know where does the file he sync from.
If you sync a file from a server in other city, for a 20 to 22MB/s speed
is very normal. But if in home, that is not good.

And for ftp and rsync.
  ftp is better for transferring a single large file once.
  rsync is better for a long-term, incremental synchronization. The
file verification of rsync may take a lot of time for first sync.

There is a theoretical network speed as already mentioned.  There is a
protocol speed, which may limit throughput if it has e.g. heavy encryption/
compression and the CPU is anaemic.  Finally, there is a MoBo bus (SCSI/SATA/
USB) and the media storage limit.  If using USB 1.1 or 2.0 and/or the disks
are slow or experience write amplification, you'll find this will constrain
the final transfer speed significantly.

The computers on this network are 2-meters apart and they both use SSD Drive 
(so USB limitation doesn't come under consideration).
Like I said, on my home network when I transfer the 24GB file I get about 
110MiBps transfer, so I was expecting the same in remote location).
Some units are connected to a router Ausus RT-AC66U B1 but these ports are 
gigabit too.
When you say the computers are remote, is it possible the file is 
passing through your local computer on the way between the two remote 
machines?  Where are you actually running the rsync command?

Re: [gentoo-user] network transfer speed

2021-01-15 Thread thelma 
On 1/15/21 9:55 AM, Jack wrote:
> On 1/15/21 11:51 AM, the...@sys-concept.com wrote:
>> On 1/15/21 2:58 AM, Michael wrote:
>>> On Friday, 15 January 2021 08:42:16 GMT bobwxc wrote:
 在 2021/1/15 下午4:27, Raffaele BELARDI 写道:
>> -Original Message-
>> From: bobwxc 
>> Sent: Friday, January 15, 2021 08:57
>> To: gentoo-user@lists.gentoo.org
>> Subject: Re: [gentoo-user] network transfer speed
>>
>> 在 2021/1/15 下午2:56, the...@sys-concept.com 写道:
>>> On both of my systems the network card speed is showing 1000
>>> cat /sys/class/net/enp4s0/speed   1000
>>>
>>> but when I do rsync larage file I only see about: 20 to 22MB/s On my
>>> home network I get about 110MB/s between PC's
>>>
>>> Both PC's have SSD  and the swith is Gigabit (I think).
>>> How to find a the bottleneck?
>> 1000Mbps network card's maximum theoretical speed is about 125MiB/s.
>> It only works in short distances.
> Correct but that's the line speed that you'll never reach, when you take
> into account Ethernet frame overhead, IP (and possibly TCP) header
> overhead and application ( rsync, FTP, SMB, NFS) overhead you get lower
> figures. In my experience 900Mbps (110MiBps) on a 1000Mbps line is more
> realistic for 'normal' transfers.
 Yes, you are right. So it is just *theoretical* speed :-)

 I don't know where does the file he sync from.
 If you sync a file from a server in other city, for a 20 to 22MB/s speed
 is very normal. But if in home, that is not good.

 And for ftp and rsync.
   ftp is better for transferring a single large file once.
   rsync is better for a long-term, incremental synchronization. The
 file verification of rsync may take a lot of time for first sync.
>>> There is a theoretical network speed as already mentioned.  There is a
>>> protocol speed, which may limit throughput if it has e.g. heavy encryption/
>>> compression and the CPU is anaemic.  Finally, there is a MoBo bus 
>>> (SCSI/SATA/
>>> USB) and the media storage limit.  If using USB 1.1 or 2.0 and/or the disks
>>> are slow or experience write amplification, you'll find this will constrain
>>> the final transfer speed significantly.
>> The computers on this network are 2-meters apart and they both use SSD Drive 
>> (so USB limitation doesn't come under consideration).
>> Like I said, on my home network when I transfer the 24GB file I get about 
>> 110MiBps transfer, so I was expecting the same in remote location).
>> Some units are connected to a router Ausus RT-AC66U B1 but these ports are 
>> gigabit too.
> When you say the computers are remote, is it possible the file is passing 
> through your local computer on the way between the two remote machines?  
> Where are you actually running the rsync command?

I ssh over VPN to remote computers and run "rsync" there.  Will it effect the 
speed?

Re: [gentoo-user] network transfer speed

2021-01-15 Thread thelma 
On 1/15/21 9:55 AM, Jack wrote:
[snip]

 I don't know where does the file he sync from.
 If you sync a file from a server in other city, for a 20 to 22MB/s speed
 is very normal. But if in home, that is not good.

 And for ftp and rsync.
   ftp is better for transferring a single large file once.
   rsync is better for a long-term, incremental synchronization. The
 file verification of rsync may take a lot of time for first sync.
>>> There is a theoretical network speed as already mentioned.  There is a
>>> protocol speed, which may limit throughput if it has e.g. heavy encryption/
>>> compression and the CPU is anaemic.  Finally, there is a MoBo bus 
>>> (SCSI/SATA/
>>> USB) and the media storage limit.  If using USB 1.1 or 2.0 and/or the disks
>>> are slow or experience write amplification, you'll find this will constrain
>>> the final transfer speed significantly.
>> The computers on this network are 2-meters apart and they both use SSD Drive 
>> (so USB limitation doesn't come under consideration).
>> Like I said, on my home network when I transfer the 24GB file I get about 
>> 110MiBps transfer, so I was expecting the same in remote location).
>> Some units are connected to a router Ausus RT-AC66U B1 but these ports are 
>> gigabit too.
> When you say the computers are remote, is it possible the file is passing 
> through your local computer on the way between the two remote machines?  
> Where are you actually running the rsync command?

The fact that I'm logged via ssh over VPN to a remote network should not have 
any influence over network speed.
I just made a loop:
Network A ==> Internet ==> Network B  
ssh back to Network A over internet and run "rsync" I got same speed (as if I 
run the command locally) on Network A 112MB/s  

So the limiting factor is somewhere else.

Re: [gentoo-user] network transfer speed

2021-01-15 Thread thelma 
On 1/15/21 1:11 AM, Raffaele BELARDI wrote:
>> -Original Message-
>> From: the...@sys-concept.com 
>> Sent: Friday, January 15, 2021 07:57
>> To: Gentoo mailing list 
>> Subject: [gentoo-user] network transfer speed
>>
>>
>> On both of my systems the network card speed is showing 1000
>> cat /sys/class/net/enp4s0/speed   1000
>>
>> but when I do rsync larage file I only see about: 20 to 22MB/s On my home
>> network I get about 110MB/s between PC's
>>
>> Both PC's have SSD  and the swith is Gigabit (I think).
>> How to find a the bottleneck?
> 
> If the PCs attached to the switch show 1000 then the switch _is_ gigabit.
> 
> On my 1Gb home network I have an FTP transfer speed between Gentoo PCs A and 
> B of almost 900Mbps, the other way round is almost half of that. One 
> difference between the two systems is the disk, A uses SATA-2 disk while B 
> has SATA-3.
> 
> Does the 'B' in 110MB/s stand for byte? If so you have 880Mbps which is not 
> bad, the problem probably lies somewhere else. Otherwise you could check the 
> switch error count (if you have a managed switch) or the network card error 
> count, just to ensure you don't have a cabling/connector problem.
> 
> Have you tried other transfer methods just for comparison? I think FTP is 
> still the fastest way to transfer files, though insecure or inconvenient as 
> it might be. I have no experience with rsync.
> 
> raffaele

On a remote network I run ethtool on both cards and I got both 1000Mb/s speed

1.)
ethtool net0
Settings for net0:
Supported ports: [ TP MII ]
Supported link modes:   10baseT/Half 10baseT/Full 
100baseT/Half 100baseT/Full 
1000baseT/Half 1000baseT/Full 
Supported pause frame use: No
Supports auto-negotiation: Yes
Advertised link modes:  10baseT/Half 10baseT/Full 
100baseT/Half 100baseT/Full 
1000baseT/Half 1000baseT/Full 
Advertised pause frame use: Symmetric Receive-only
Advertised auto-negotiation: Yes
Link partner advertised link modes:  10baseT/Half 10baseT/Full 
 100baseT/Half 100baseT/Full 
 1000baseT/Full 
Link partner advertised pause frame use: Symmetric
Link partner advertised auto-negotiation: Yes
Speed: 1000Mb/s
Duplex: Full
Port: MII
PHYAD: 0
Transceiver: internal
Auto-negotiation: on
MDI-X: on (auto)

2.) 
ethtool enp4s0
Settings for enp4s0:
Supported ports: [ TP ]
Supported link modes:   10baseT/Half 10baseT/Full 
100baseT/Half 100baseT/Full 
1000baseT/Full 
Supported pause frame use: Symmetric
Supports auto-negotiation: Yes
Advertised link modes:  10baseT/Half 10baseT/Full 
100baseT/Half 100baseT/Full 
1000baseT/Full 
Advertised pause frame use: Symmetric
Advertised auto-negotiation: Yes
Speed: 1000Mb/s
Duplex: Full
Port: Twisted Pair
PHYAD: 1
Transceiver: internal
Auto-negotiation: on
MDI-X: on (auto)

Re: [gentoo-user] network transfer speed

2021-01-15 Thread William Kenworthy 


On 16/1/21 6:56 am, the...@sys-concept.com wrote:
> On 1/15/21 1:11 AM, Raffaele BELARDI wrote:
>>> -Original Message-
>>> From: the...@sys-concept.com 
>>> Sent: Friday, January 15, 2021 07:57
>>> To: Gentoo mailing list 
>>> Subject: [gentoo-user] network transfer speed
>>>
>>>
>>> On both of my systems the network card speed is showing 1000
>>> cat /sys/class/net/enp4s0/speed   1000
>>>
>>> but when I do rsync larage file I only see about: 20 to 22MB/s On my home
>>> network I get about 110MB/s between PC's
>>>
>>> Both PC's have SSD  and the swith is Gigabit (I think).
>>> How to find a the bottleneck?
>> If the PCs attached to the switch show 1000 then the switch _is_ gigabit.
>>
>> On my 1Gb home network I have an FTP transfer speed between Gentoo PCs A and 
>> B of almost 900Mbps, the other way round is almost half of that. One 
>> difference between the two systems is the disk, A uses SATA-2 disk while B 
>> has SATA-3.
>>
>> Does the 'B' in 110MB/s stand for byte? If so you have 880Mbps which is not 
>> bad, the problem probably lies somewhere else. Otherwise you could check the 
>> switch error count (if you have a managed switch) or the network card error 
>> count, just to ensure you don't have a cabling/connector problem.
>>
>> Have you tried other transfer methods just for comparison? I think FTP is 
>> still the fastest way to transfer files, though insecure or inconvenient as 
>> it might be. I have no experience with rsync.
>>
>> raffaele
> On a remote network I run ethtool on both cards and I got both 1000Mb/s speed
>
> 1.)
> ethtool net0
> Settings for net0:
>   Supported ports: [ TP MII ]
>   Supported link modes:   10baseT/Half 10baseT/Full 
>   100baseT/Half 100baseT/Full 
>   1000baseT/Half 1000baseT/Full 
>   Supported pause frame use: No
>   Supports auto-negotiation: Yes
>   Advertised link modes:  10baseT/Half 10baseT/Full 
>   100baseT/Half 100baseT/Full 
>   1000baseT/Half 1000baseT/Full 
>   Advertised pause frame use: Symmetric Receive-only
>   Advertised auto-negotiation: Yes
>   Link partner advertised link modes:  10baseT/Half 10baseT/Full 
>100baseT/Half 100baseT/Full 
>1000baseT/Full 
>   Link partner advertised pause frame use: Symmetric
>   Link partner advertised auto-negotiation: Yes
>   Speed: 1000Mb/s
>   Duplex: Full
> Port: MII
>   PHYAD: 0
>   Transceiver: internal
>   Auto-negotiation: on
>   MDI-X: on (auto)
>
> 2.) 
> ethtool enp4s0
> Settings for enp4s0:
>   Supported ports: [ TP ]
>   Supported link modes:   10baseT/Half 10baseT/Full 
>   100baseT/Half 100baseT/Full 
>   1000baseT/Full 
>   Supported pause frame use: Symmetric
>   Supports auto-negotiation: Yes
>   Advertised link modes:  10baseT/Half 10baseT/Full 
>   100baseT/Half 100baseT/Full 
>   1000baseT/Full 
>   Advertised pause frame use: Symmetric
>   Advertised auto-negotiation: Yes
>   Speed: 1000Mb/s
>   Duplex: Full
>   Port: Twisted Pair
>   PHYAD: 1
>   Transceiver: internal
>   Auto-negotiation: on
>   MDI-X: on (auto)
>
What brand/type of interface (Realtek, int/ext USB ?)  There is an
obscure bug that drops throughput to ~1/3 in Realtek RTL8153 chip on
USB2 and the r8152 kernel driver.  Some interfaces (e.g., most raspberry
pi) connect via internal USB with its own throughput problems.  Probably
not the problem in this case but worth mentioning.

Rsync is not a good tool for measuring throughput as its optimisations
are problematic - and this can show up when comparing local to remote
tests.  If on a file system it will always transfer the whole file (and
it considers a network filesystem local) while on a network it will use
the transfer algorithm which can vary depending on commandline switches
used and where the remote file lives, and this can change between tests
depending on conditions.  Whole files have little overhead, whereas the
algorithm overhead can actually be much slower (hence why it tries to do
it this way) Try and use a largish file created with /dev/random and dd
as well as a simple netcat type transfer mechanism to measure.  Note
that encryption can severely limit transfers due to cpu limiting if on a
low powered system and rsync over ssh uses encrytion.  And if a secure
VPN is in the mix its orders of magnitude worse.  Also note that
something like an inline IDS on a link can drop throughput by 50% or
more (measured on an older enterprise Cisco device) which further
complicates local/remote via internet comparisons.


Good luck in finding the problem - its really not simple and easy.

BillK

Re: [gentoo-user] network transfer speed

2021-01-15 Thread Manuel McLure 
On Fri, Jan 15, 2021 at 3:40 PM William Kenworthy 
wrote:

>
>
> Rsync is not a good tool for measuring throughput as its optimisations
> are problematic - and this can show up when comparing local to remote
> tests.
>
>
> Yup. A better option is to use something like net-misc/iperf. That will
give you a much better idea of the raw network performance. It's also
available on Windows so you can use it to test Linux<->Windows and
Windows<->Windows speeds as well as Linux<->Linux speeds.

-- 
Manuel A. McLure WW1FA  
...for in Ulthar, according to an ancient and significant law,
no man may kill a cat.   -- H.P. Lovecraft

Re: [gentoo-user] network transfer speed

2021-01-15 Thread Adam Carter 
> On a remote network I run ethtool on both cards and I got both 1000Mb/s
> speed
>
>
As the 20 odd MB/s you're getting is above what is possible on 100M
ethernet, you can rule out any ethernet interfaces at 100M.

Can you describe the network between the two systems with the slow transfer?

If there is a fast WAN from one side of the globe to the other it could be
latency related. OpenSSH used to have a fixed internal window size that
made it slow on high bandwidth high latency links, and I notice the hpn USE
flag still exists in the openssh ebuild, which implies the issue with
openssh still exists. Rsync can use either ssh or its own protocol, so if
there's a high latency link between the two boxes and rsync is using ssh,
you could investigate rebuilding openssh with +hpn.

What does ping show the latency as?

Otherwise i'd be thinking about packet loss. First place to start for that
is on the endpoint interfaces;
ifconfig enp35s0f0 | grep err
RX errors 0  dropped 0  overruns 0  frame 0
TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0