Re: [gentoo-user] Is there a way to misconfigure USB ports in the kernel?
On Tuesday, 30 March 2021 18:11:56 BST Dr Rainer Woitok wrote:
> On Saturday, 2020-12-05 19:07:51 +0100, I myself wrote:
>
> ("> >" refers to Michael )
>
> > Michael,
> >
> > On Friday, 2020-11-27 19:07:17 +, you wrote:
> > > ...
> > > A 4k block size is recommended for ntfs-3g which is the default sector
> > > created by fdisk and friends on Linux these days. This will align your
> > > partition optimally. In addition, mkfs.ntfs will use 4096 bytes as the
> > > default cluster size, so you should be good in that respect.
> > >
> > > Another setting you may want to try is mounting the USB with
> > > 'big_writes' -
> > > check the man page. This should help particularly with large files,
> > > which
> > > will use larger blocks up to 128KB when copying data to the NTFS.
> >
> > Both, the VeraCrypt command line (--fs-options=big_writes) and the Vera-
> > Crypt GUI (under "Settings --> Preferences") allow setting this mount
> > option. But
> >
> >$ mount | grep veracrypt
> >
> > never shows it, initially causing me to erroneously believe it wasn't
> > set and to try finding on the web another way of setting it. By pure
> > chance I finally found out that
> >
> >$ ps -ef | grep veracrypt
> >
> > lists a "/usr/sbin/mount.ntfs" task which shows the options really in
> > effect. However, I haven't yet had the time to test the effect of this
> > option when writing plenty of really big files. I will report on that
> > later.
>
> Well, it's been quite a while, due to my being almost permanently con-
> fronted with more pressing tasks ... :-(
>
> To sum up my experience with my new 128 GB Philips USB 3.0 sticks: while
> the Philips sticks are significantly faster for reading operations than
> my old 64 GB Verbatim ones (probably USB 2.0), writing operations to the
> Philips sticks are unbearably slow, regardless of whether I created a
> normal unencrypted NTFS filesystem on them or an encrypted NTFS filesys-
> tem using VeraCrypt. Writing to the USB stick while at the same time
> reading from it in a different terminal window caused commands like "cd"
> or "ls" to simply stall. Thus while running
>
>$ cp --preserve=timestamps -ru $source_dir .
>
> in one terminal window, I ran
>
>$ while true
>
>> do n=$(ps -ef|g 'cp --preserve'|g -v grep)
>>
>>if [[ "$n" = "${o-}" ]]
>>then sleep 10
>>else o="$n"
>>
>> echo "$n"
>>
>>fi
>>
>> done
>
> in another, to get the wall clock times when copying a new file began.
> That way I found that copying a 30 MB file took about 40 minutes.
OK, unless you made a typo and the "minutes" were meant to say seconds, this
is ridiculously slow.
You could run some tests to see what is causing the delay. The veracrypt
algos & cipher iterations, the fuse based ntfs-3g, or the USB stick's
controller.
However if, as I understand it, all other variables are the same and the only
change was to replace your Verbatim 64G USB 2.0 sticks with Philips 128G USB
3.0 sticks, then the slow writes point to the Philips devices being the
culprit.
Some years ago I tested some USB 2.0 sticks of various sizes, from 256M up to
32G and recall the smaller the USB stick the faster the write performance, so
differences in writing speed are normal. The writing speed you're describing
however is a clear indication of something being wrong.
> So what are my options?
>
>- Stay away from Philips USB 3.0 sticks?
>
>- Stay away from Philips USB sticks in general?
Without knowing the internals, a brand may offer only an unwarranted
assumption of performance. We saw Western Digital disks being sold as CMR,
while having SMR internals. A brand could switch OEM suppliers, or
components, making benchmarking unreliable.
>- Stay away from USB 3.0 sticks in general?
USB 3.0 is faster and USB 3.2 when available will be even faster. So use
whatever the USB ports on your PC offer.
>- Stay away from Filesystem in User Space using a non-stable 5.10 or
> 5.11 kernel (currently I'm using stable 5.4.97)?
>
>- Stay away from Gentoo?
>
>- Stay away from Linux in general and go back to OTOS (aka the Only
> True Operating System aka Windoze)?
>
>- ...?
In-kernel fs drivers are measurably faster than fuse based fs for well
understood reasons. However, if needs must and the fs you require is not
available on Linux, then some compromise will be required.
> Any ideas and comments welcome ...
>
> Sincerely,
> Rainer
You may want to run some tests on the sticks you have, if only to bottom out
what their performance is on different PCs and USB ports:
dd if=/dev/zero of=/run/media///TESTFILE bs=512
count=60 oflag=direct conv=notrunc,fsync status=progress
Use a large enough file to make sure the USB controller cache gets saturated.
You could use a ramdisk/tmpfs as an input file.
If you write directly to the devic
Re: [gentoo-user] Is there a way to misconfigure USB ports in the kernel?
Michael wrote: > You may want to run some tests on the sticks you have, if only to bottom out > what their performance is on different PCs and USB ports: > > dd if=/dev/zero of=/run/media///TESTFILE bs=512 > count=60 oflag=direct conv=notrunc,fsync status=progress > > Use a large enough file to make sure the USB controller cache gets saturated. > > You could use a ramdisk/tmpfs as an input file. > > If you write directly to the device as Dale suggested it will wipe data, so > keep a backup of anything you need first. I checked the man page, I see nothing about the show progress option. You can bet I'll try that next time tho. I saw a video of someone else using it and it is a lot easier than having to switch Konsoles and type in more commands. Thanks for sharing that option, that isn't in the freaking manual I might add. This is one time where telling someone to read the manual wouldn't work. ROFL Awesome!! Dale :-) :_)
Re: [gentoo-user] Is there a way to misconfigure USB ports in the kernel?
On Wednesday, 31 March 2021 12:37:49 BST Dale wrote: > I checked the man page, I see nothing about the show progress option. Are you sure? This is what I see here on line 47: "status=LEVEL The LEVEL of information to print to stderr; 'none' suppresses everything but error messages, 'noxfer' suppresses the final transfer statistics, 'progress' shows periodic transfer statistics" I find this useful in seeing the transfer speed drop in real time as the cache gets saturated. signature.asc Description: This is a digitally signed message part.
Re: [gentoo-user] Is there a way to misconfigure USB ports in the kernel?
Michael wrote: > On Wednesday, 31 March 2021 12:37:49 BST Dale wrote: > >> I checked the man page, I see nothing about the show progress option. > Are you sure? > > This is what I see here on line 47: > > "status=LEVEL > The LEVEL of information to print to stderr; 'none' suppresses > everything but error messages, 'noxfer' suppresses the final transfer > statistics, 'progress' shows periodic transfer statistics" > > I find this useful in seeing the transfer speed drop in real time as the > cache > gets saturated. Ahhh, I didn't see the status part. It's sort of hiding in a subsection. At least I know now that the version I have installed has this option. I wish I could view man pages like I used to in Konqueror. It displays like a webpage and is much easier to search through. Thanks for pointing that out. Dale :-) :-)
Re: [gentoo-user] Is there a way to misconfigure USB ports in the kernel?
On Wednesday, 31 March 2021 13:09:03 BST Dale wrote: > Michael wrote: > > On Wednesday, 31 March 2021 12:37:49 BST Dale wrote: > >> I checked the man page, I see nothing about the show progress option. > > > > Are you sure? > > > > This is what I see here on line 47: > > > > "status=LEVEL > > > > The LEVEL of information to print to stderr; 'none' suppresses > > everything but error messages, 'noxfer' suppresses the final > > transfer > > statistics, 'progress' shows periodic transfer statistics" > > > > I find this useful in seeing the transfer speed drop in real time as the > > cache gets saturated. > > Ahhh, I didn't see the status part. It's sort of hiding in a > subsection. At least I know now that the version I have installed has > this option. > > I wish I could view man pages like I used to in Konqueror. It displays > like a webpage and is much easier to search through. > > Thanks for pointing that out. > > Dale > > :-) :-) Yes, I also liked the old Konqueror interface. Searching for keywords e.g. "progress" within man pages works if you preface the keyword with "/": /progress will find it and "n" or "Shift+n" will jump forward and backward to any other instances in the man page. signature.asc Description: This is a digitally signed message part.
Re: [gentoo-user] Is there a way to misconfigure USB ports in the kernel?
Michael wrote: > On Wednesday, 31 March 2021 13:09:03 BST Dale wrote: >> Michael wrote: >>> On Wednesday, 31 March 2021 12:37:49 BST Dale wrote: I checked the man page, I see nothing about the show progress option. >>> Are you sure? >>> >>> This is what I see here on line 47: >>> >>> "status=LEVEL >>> >>> The LEVEL of information to print to stderr; 'none' suppresses >>> everything but error messages, 'noxfer' suppresses the final >>> transfer >>> statistics, 'progress' shows periodic transfer statistics" >>> >>> I find this useful in seeing the transfer speed drop in real time as the >>> cache gets saturated. >> Ahhh, I didn't see the status part. It's sort of hiding in a >> subsection. At least I know now that the version I have installed has >> this option. >> >> I wish I could view man pages like I used to in Konqueror. It displays >> like a webpage and is much easier to search through. >> >> Thanks for pointing that out. >> >> Dale >> >> :-) :-) > Yes, I also liked the old Konqueror interface. Searching for keywords e.g. > "progress" within man pages works if you preface the keyword with "/": > > /progress > > will find it and "n" or "Shift+n" will jump forward and backward to any other > instances in the man page. That doesn't work here. I can type in /progress but it just shows up at the bottom. If I try "n" or shift+n I just get a n or N. Maybe my man page uses something different. That said, I haven't tried Konqueror in a while so I found it, it gives a error but I can type in man:dd and it shows up. It has a search tool. Last time I tried it, wouldn't even come up. That was a while ago tho. Guess it got fixed. Dale :-) :-)
Re: [gentoo-user] Is there a way to misconfigure USB ports in the kernel?
On Wednesday, 31 March 2021 13:23:27 BST Dale wrote: > Michael wrote: > > On Wednesday, 31 March 2021 13:09:03 BST Dale wrote: > >> Michael wrote: > >>> On Wednesday, 31 March 2021 12:37:49 BST Dale wrote: > I checked the man page, I see nothing about the show progress option. > >>> > >>> Are you sure? > >>> > >>> This is what I see here on line 47: > >>> > >>> "status=LEVEL > >>> > >>> The LEVEL of information to print to stderr; 'none' suppresses > >>> everything but error messages, 'noxfer' suppresses the final > >>> transfer > >>> statistics, 'progress' shows periodic transfer statistics" > >>> > >>> I find this useful in seeing the transfer speed drop in real time as the > >>> cache gets saturated. > >> > >> Ahhh, I didn't see the status part. It's sort of hiding in a > >> subsection. At least I know now that the version I have installed has > >> this option. > >> > >> I wish I could view man pages like I used to in Konqueror. It displays > >> like a webpage and is much easier to search through. > >> > >> Thanks for pointing that out. > >> > >> Dale > >> > >> :-) :-) > > > > Yes, I also liked the old Konqueror interface. Searching for keywords > > e.g. > > "progress" within man pages works if you preface the keyword with "/": > > > > /progress > > > > will find it and "n" or "Shift+n" will jump forward and backward to any > > other instances in the man page. > > That doesn't work here. I can type in /progress but it just shows up at > the bottom. Yes, it shows at the bottom until you hit enter to execute the search. Then it highlights the next instance of the searched string. Just like Vim/Vi does. Hmm ... I wonder if I have set up some special environment parameter on my systems and forgotten about it. :-/ signature.asc Description: This is a digitally signed message part.
Re: [gentoo-user] Is there a way to misconfigure USB ports in the kernel?
Michael wrote: > On Wednesday, 31 March 2021 13:23:27 BST Dale wrote: >> Michael wrote: >>> On Wednesday, 31 March 2021 13:09:03 BST Dale wrote: Michael wrote: > On Wednesday, 31 March 2021 12:37:49 BST Dale wrote: >> I checked the man page, I see nothing about the show progress option. > Are you sure? > > This is what I see here on line 47: > > "status=LEVEL > > The LEVEL of information to print to stderr; 'none' suppresses > everything but error messages, 'noxfer' suppresses the final > transfer > statistics, 'progress' shows periodic transfer statistics" > > I find this useful in seeing the transfer speed drop in real time as the > cache gets saturated. Ahhh, I didn't see the status part. It's sort of hiding in a subsection. At least I know now that the version I have installed has this option. I wish I could view man pages like I used to in Konqueror. It displays like a webpage and is much easier to search through. Thanks for pointing that out. Dale :-) :-) >>> Yes, I also liked the old Konqueror interface. Searching for keywords >>> e.g. >>> "progress" within man pages works if you preface the keyword with "/": >>> >>> /progress >>> >>> will find it and "n" or "Shift+n" will jump forward and backward to any >>> other instances in the man page. >> That doesn't work here. I can type in /progress but it just shows up at >> the bottom. > Yes, it shows at the bottom until you hit enter to execute the search. Then > it highlights the next instance of the searched string. Just like Vim/Vi > does. Hmm ... I wonder if I have set up some special environment parameter > on > my systems and forgotten about it. :-/ > > *cough cough* I didn't hit enter. Just did and it worked. Now to get that info to stick in this old dog's brain for next time. ROFL Dale :-) :-)
[gentoo-user] Gparted leaves gaps
Hello list, I use gparted often, usually from SystemRescueCD, and a common task is to move partitions to allow for one to be enlarged. I should be able to specify all the operations in a list, but whenever I do that gparted inserts 1MB gaps between partitions, so I have to do one at a time. Even the latest bootable gparted CD image does the same. Can anyone tell me what causes this? Has it anything to do with my always specifying partition size as a power of 8? (I'm of the old school, having been sent on my first computer hardware course in 1972.) -- Regards, Peter.
Re: [gentoo-user] Is there a way to misconfigure USB ports in the kernel?
On Wed, 31 Mar 2021 07:23:27 -0500, Dale wrote: > > Yes, I also liked the old Konqueror interface. Searching for > > keywords e.g. "progress" within man pages works if you preface the > > keyword with "/": > > > > /progress > > > > will find it and "n" or "Shift+n" will jump forward and backward to > > any other instances in the man page. > > That doesn't work here. I can type in /progress but it just shows up at > the bottom. If I try "n" or shift+n I just get a n or N. Maybe my man > page uses something different. It's a feature of whichever pager you use, rather than man itself. Less does it, as does most, but others may differ. -- Neil Bothwick Having children will turn you into your parents. pgpBmNLFYGPYU.pgp Description: OpenPGP digital signature
Re: [gentoo-user] Is there a way to misconfigure USB ports in the kernel?
On Wed, 31 Mar 2021 07:09:03 -0500, Dale wrote: > I wish I could view man pages like I used to in Konqueror. It displays > like a webpage and is much easier to search through. I miss that too. I use mankier.com these days, which gives similar benefits. I have a shortcut set up in chromium so typing "man dd" opens the page in mankier.com. -- Neil Bothwick Life's a cache, and then you flush... pgpXpSNRLU1Pd.pgp Description: OpenPGP digital signature
[gentoo-user] Re: Is there a way to misconfigure USB ports in the kernel?
On 2021-03-31, Neil Bothwick wrote: > On Wed, 31 Mar 2021 07:09:03 -0500, Dale wrote: > >> I wish I could view man pages like I used to in Konqueror. It displays >> like a webpage and is much easier to search through. > > I miss that too. I use mankier.com these days, which gives similar > benefits. I have a shortcut set up in chromium so typing "man dd" > opens the page in mankier.com. Many years ago, there was an X11 man page and gnu info viewer that I used to use, but I can't remember the name of it. This was probably 20+ years ago (pre GTK and Qt), so the chances that it's still around are small... -- Grant
Re: [gentoo-user] Re: Is there a way to misconfigure USB ports in the kernel?
On 2021.03.31 16:28, Grant Edwards wrote: On 2021-03-31, Neil Bothwick wrote: > On Wed, 31 Mar 2021 07:09:03 -0500, Dale wrote: > >> I wish I could view man pages like I used to in Konqueror. It displays >> like a webpage and is much easier to search through. > > I miss that too. I use mankier.com these days, which gives similar > benefits. I have a shortcut set up in chromium so typing "man dd" > opens the page in mankier.com. Many years ago, there was an X11 man page and gnu info viewer that I used to use, but I can't remember the name of it. This was probably 20+ years ago (pre GTK and Qt), so the chances that it's still around are small... Perhaps x11-apps/xman? I remember it as you do, but have not used it in years, so I'm not sure if this is it or not. The info command also seems to have access to man pages, but it doesn't seem the most obvious way to get there. Jack
[gentoo-user] Re: Is there a way to misconfigure USB ports in the kernel?
On 2021-03-31, Jack wrote: > On 2021.03.31 16:28, Grant Edwards wrote: > >> Many years ago, there was an X11 man page and gnu info viewer that I >> used to use, but I can't remember the name of it. This was probably >> 20+ years ago (pre GTK and Qt), so the chances that it's still around >> are small... > > Perhaps x11-apps/xman? I do remember using xman closer to 30 years ago, but what I was thinking of was something a bit newer and fancier than that -- I recall it also being able to navigate info pages. Or I may be conflating two different apps: one for man and one for info. It may have been tkinfo or tkman http://math-www.uni-paderborn.de/~axel/tkinfo/ https://sourceforge.net/projects/tkman/ > I remember it as you do, but have not used it in years, so I'm not > sure if this is it or not. The info command also seems to have > access to man pages, but it doesn't seem the most obvious way to get > there.
Re: [gentoo-user] Gparted leaves gaps
On 1/4/21 12:39 am, Peter Humphrey wrote: > Hello list, > > I use gparted often, usually from SystemRescueCD, and a common task is to > move partitions to allow for one to be enlarged. I should be able to specify > all the operations in a list, but whenever I do that gparted inserts 1MB gaps > between partitions, so I have to do one at a time. Even the latest bootable > gparted CD image does the same. > > Can anyone tell me what causes this? Has it anything to do with my always > specifying partition size as a power of 8? (I'm of the old school, having > been > sent on my first computer hardware course in 1972.) > Forcing alignment on megabyte boundaries? - I cant find a reason, but I think I read in the past it was for efficiency with modern file systems. BillK
[gentoo-user] Disable password required to mount removable hard disk.
Hi, I use a sata drive caddy with 2Tb hard disks for offline backups. Almost everytime (within sessions are ok?) it asks for a password before automounting. This is just annoying and has no security benefit in my environment (why just hard disks when USB keys and SD cards don't ask for one?). So, how can I disable the automounter asking for a password either in general, or just for my backup drives? BillK
