With 10 minute samples anything that happened gets averaged enough that
even the worst event is almost impossible to see.
Sar will report the same as date ie local time. And a 12:51 event would be
in the 13:00 sample (started at about 12:50 and ended at 1300).
What I do see is that during that window your io rate was about 2x prior 10
minute windows. With the 1 minute data we would be able to see if the disk
was excessively busy. You average iops were about 10% of the disk capacity.
I have debugged issues where the badly behaving IO was maxing out
everything for 10sec on/10 sec off, in the 1 minute data there appeared to
be nothing interesting to see (50% capacity), but it was playing hell with
the interactive apps since during the 10 sec on window operations that the
user was doing that were normally taking .5 sec were taking 1-2 seconds and
so clearly slow for the users. With the sample size (60sec) close to
the event size (45sec) it should be visible on 1 minute data, but less than
clear on 10 minute data (9.25 minutes to average it out and hide/mask
it).
do "systemctl edit sysstat-collect.timer"
And add this to the file:
[Timer]
OnCalendar=*:00/1
That will change it to 1minutes.
if you do this:
#!/bin/bash
while [ true ] ; do
export hour=$(date +%H)
iostat -t -x 10 360 > filename.${hour}
done
that will give you 10 sec iostat data, and start a new file each hour, and
overwrite the hour files the next day.
On Sun, Oct 3, 2021 at 4:00 AM Terry Barnaby <ter...@beam.ltd.uk> wrote:
> 45 second event happened at: 2021-10-02T11:51:02 UTC. Not sure what sar
> time is based on (maybe local time BST rather than UTC so would be
> 2021-10-02T12:51:02
> BST.
>
> Continuing info ...
>
> sar -n NFSD on the server
> 11:00:01 24.16 0.00 24.16 0.00 24.16
> 0.00 0.00 0.35 1.48 2.07 21.08
> 11:10:01 21.13 0.00 21.13 0.00 21.13
> 0.00 0.00 0.28 0.89 1.72 19.58
> 11:20:02 17.85 0.00 17.85 0.00 17.85
> 0.00 0.00 0.27 0.69 0.82 16.65
> 11:30:02 20.66 0.00 20.66 0.00 20.66
> 0.00 0.00 0.29 0.83 1.42 19.15
> 11:40:02 39.80 0.00 39.80 0.00 39.80
> 0.00 0.00 0.89 2.05 3.67 25.51
> 11:50:02 35.40 0.00 35.40 0.00 35.40
> 0.00 0.00 0.39 0.65 1.22 18.21
> 12:00:02 41.85 0.00 41.85 0.00 41.85
> 0.00 0.00 0.84 1.14 2.08 20.50
> 12:10:02 38.54 0.00 38.54 0.00 38.54
> 0.00 0.00 0.48 0.82 1.48 19.62
> 12:20:02 39.85 0.00 39.85 0.00 39.85
> 0.00 0.00 0.37 1.50 1.29 19.44
> 12:30:02 39.84 0.00 39.84 0.00 39.84
> 0.00 0.00 0.70 1.03 2.28 19.78
> 12:40:02 38.29 0.00 38.29 0.00 38.29
> 0.00 0.00 0.46 0.81 1.26 19.37
> 12:50:02 71.38 0.00 71.38 0.00 71.37
> 0.00 0.00 1.12 2.41 8.19 34.87
> 13:00:00 77.46 0.00 77.46 0.00 77.45
> 0.00 0.00 1.43 3.30 7.36 38.31
> 13:10:00 67.62 0.00 67.63 0.00 67.62
> 0.00 0.00 3.85 2.84 4.68 29.66
>
> sar -n SOCK on the server
> 11:20:02 480 41 32 1 0 0
> 11:30:02 482 41 32 1 0 4
> 11:40:02 480 41 32 1 0 0
> 11:50:02 480 41 32 1 0 1
> 12:00:02 480 41 32 1 0 1
> 12:10:02 480 41 32 1 0 1
> 12:20:02 480 41 32 1 0 1
> 12:30:02 480 41 32 1 0 1
> 12:40:02 480 41 32 1 0 1
>
> 12:40:02 totsck tcpsck udpsck rawsck ip-frag tcp-tw
> 12:50:02 480 41 32 1 0 1
> 13:00:00 480 41 32 1 0 1
> 13:10:00 490 43 32 1 0 1
>
> sar -n NFS on the client
> 11:10:02 19.82 0.00 0.28 0.34 0.71 15.13
> 11:20:03 16.53 0.00 0.27 0.15 0.34 13.80
> 11:30:04 17.20 0.00 0.13 0.08 0.30 15.17
> 11:40:04 37.46 0.00 0.89 1.47 2.07 14.42
> 11:50:05 32.97 0.00 0.28 0.11 0.34 15.00
> 12:00:05 36.31 0.00 0.59 0.47 0.75 14.17
> 12:10:06 34.77 0.00 0.36 0.26 0.65 14.95
> 12:20:07 37.55 0.00 0.27 0.97 0.35 15.36
> 12:30:07 33.90 0.00 0.46 0.37 0.62 13.47
> 12:40:07 35.67 0.00 0.36 0.28 0.64 15.44
> 12:50:07 68.97 0.00 1.01 1.89 6.64 13.28
> 13:00:07 71.08 0.00 1.17 2.58 4.32 17.87
> 13:10:07 49.28 0.00 0.84 1.55 1.81 15.23
> 13:20:00 41.87 0.00 0.50 1.24 0.87 14.98
>
> sar -n SOCK client
> 11:20:03 1166 39 14 0 0 0
> 11:30:04 1164 35 14 0 0 2
> 11:40:04 1191 50 15 0 0 1
> 11:50:05 1182 40 14 0 0 0
> 12:00:05 1182 39 14 0 0 2
> 12:10:06 1182 39 14 0 0 3
> 12:20:07 1171 39 14 0 0 1
> 12:30:07 1179 40 15 0 0 0
> 12:40:07 1179 39 15 0 0 0
> 12:50:07 1200 45 17 0 0 1
> 13:00:07 1188 40 14 0 0 2
>
> Nothing obvious I can see there ...
>
> Terry
>
> _______________________________________________
> users mailing list -- users@lists.fedoraproject.org
> To unsubscribe send an email to users-le...@lists.fedoraproject.org
> Fedora Code of Conduct:
> https://docs.fedoraproject.org/en-US/project/code-of-conduct/
> List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
> List Archives:
> https://lists.fedoraproject.org/archives/list/users@lists.fedoraproject.org
> Do not reply to spam on the list, report it:
> https://pagure.io/fedora-infrastructure
>
_______________________________________________
users mailing list -- users@lists.fedoraproject.org
To unsubscribe send an email to users-le...@lists.fedoraproject.org
Fedora Code of Conduct:
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives:
https://lists.fedoraproject.org/archives/list/users@lists.fedoraproject.org
Do not reply to spam on the list, report it:
https://pagure.io/fedora-infrastructure
