JMicron jms561 umass on arm64?

2021-04-07 Thread Peter Cornelius
G'day, folks,
 
Is there, by chance, anyone out there who has a JMicron jms561-based USB3 
'umass' kind of device up & running who can share experience or quirks, please?
 
I'm trying to get mine [2] to work under FreeBSD [3] but it does not even show 
up with usbconfig list. While, with Raspbian, I was able to make it work easily.
 
Thanks,
 
Peter.
 
---
 
[1] I believe, 
https://www.jmicron.com/file/download/1026/JMS561_Product+Brief.pdf
[2] 
https://wiki.radxa.com/Dual_Quad_SATA_HAT[https://wiki.radxa.com/Dual_Quad_SATA_HAT]
[3] Note: Later builds so far have not booted despite of current u-boot (March 
2021)
    FreeBSD rpi4 14.0-CURRENT FreeBSD 14.0-CURRENT #1: Tue Feb 23 02:30:31 UTC 
2021
    root@rpi4:/usr/obj/usr/src/arm64.aarch64/sys/GENERIC  arm64
 
___
freebsd-stable@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"


Re: Deprecating base system ftpd?

2021-04-07 Thread tech-lists

Hi, I'm a bit late to the discussion

On Mon, Apr 05, 2021 at 07:44:59AM -0700, Cy Schubert wrote:


I think this is an excellent start. My shopping list includes:

- remove ftp(1)
- remove ftpd(8)
- remove telnet(1)
- remove telnetd(8)
- remove ftp:// and http:// from libfetch. This is 2021 and we should all
use https://.
- replace DNS lookups with DoH and/or DoT. Why let your ISP see your DNS
traffic?


Very firmly against this, and this sort of thing, for the following reasons:

1. I want an OS, not a kernel. If I just want a kernel, then why not go
with linux? FreeBSD is meant to be, I think, (generally), a server OS. 
So, would you agree that it needs the ability to have server protocols 
easily configured, with a minimum of fuss, without packages?


2. a lot of infrastructure depends on ftpd. it's easy to configure
securely ftpd in base.

3. there are some networks, like internal ones, where encryption is not
a requirement, or appropriate.

4. there are some places where encryption is in fact illegal.


Personally, I'd suggest we remove the ftpd server *AND* ftp client and rely
on ports. Having worked on UNIX, Internet security, and firewalls over the
last 3/5 of my almost 50 year career, I have lamented the existence of the
FTP protocol back in 1995 and I hate the FTP protocol with greater a
passion today. Let's simply remove all vestiges of FTP from the base
system, including libfetch, sooner than later. We don't need it now that we
have HTTPS and POST; and sftp.


5. some services commonly don't use https. Lots of internet radio
stations don't. If https is enforced then the user will have to jump
through more hoops than they already do in order to, in this case,
listen to internet radio. Or face a loss of functionality.

6. not everywhere will have constant internet access. Not everyone will
want to use pkgs or have space for the ports tree.


I think we should make it our goal to remove any and all unencrypted
protocols from FreeBSD by 2025.


I think you should carefully think of the consequences of removing
functionality in the default install. It will make it less useful, not
more.
--
J.


signature.asc
Description: PGP signature


Re: Deprecating base system ftpd?

2021-04-07 Thread Chris

On 2021-04-05 07:44, Cy Schubert wrote:

In message 
, Ed Maste writes:

I propose deprecating the ftpd currently included in the base system
before FreeBSD 14, and opened review D26447
(https://reviews.freebsd.org/D26447) to add a notice to the man page.
I had originally planned to try to do this before 13.0, but it dropped
off my list. FTP is not nearly as relevant now as it once was, and it
had a security vulnerability that secteam had to address.


I think this is an excellent start. My shopping list includes:

- remove ftp(1)
- remove ftpd(8)
- remove telnet(1)
- remove telnetd(8)
- remove ftp:// and http:// from libfetch. This is 2021 and we should all
use https://.
- replace DNS lookups with DoH and/or DoT. Why let your ISP see your DNS
traffic?

You've clearly never worked on extremely large networks. Or at least not
considered them in this statement -- think LAN/intranet in large corporate
settings. ftp(1) as well as ftpd(8) are lightweight, and utilitarian. It's
because of this that gives them such great value. They require nothing to
use. They just work with no setup required. With very little setup you can
manage something a little more sophisticated. I can easily script ftp for
complex situations needing nothing more than sh(1) and ftp(1), and it's all
available right-out-of-the-box. This isn't true of the others.
In an internet public facing scenario. It's enough to utilize one specific
line in inetd(8) and 2 in hosts.allow(2). This simplicity and lack of
overhead is not available with the other options.

Because something is old and un-featured does not make it valueless.





I'm happy to make a port for it if anyone needs it. Comments?

A port would be a nice option. But it should remain an option; as in
one _should_ be allowed to get ftp || ftpd out of the box if they so
choose.



I've started working on splitting ftp and ftpd into an external git repo.
The problem I've encountered is that though only ftp and ftpd are left the
resultant repo is still 1.2 GB. If my last attempt fails, there is a choice
between a 1.2 GB repo and burning ftp forever then the choice is clear:
burn it forever.

Adding the following as an option:

Also note that the tnftp ports are the NetBSD ftp and ftpd. The FreeBSD ftp
and ftpd are simply copies of tnftp and tnfpd. Would it make more sense to
share our customizations with NetBSD and we simply reply on NetBSD for the
client and server in our ports? This last option might be simpler than
creating a port.

Personally, I'd suggest we remove the ftpd server *AND* ftp client and rely
on ports. Having worked on UNIX, Internet security, and firewalls over the
last 3/5 of my almost 50 year career, I have lamented the existence of the
FTP protocol back in 1995 and I hate the FTP protocol with greater a
passion today. Let's simply remove all vestiges of FTP from the base
system, including libfetch, sooner than later. We don't need it now that we
have HTTPS and POST; and sftp.

This assumes your willing to expend all the time and overhead to setup a web
server for a simple but absolutely mandatory one time task. When none of the
boxes you're working on are slated for or perhaps are even capable of running
as much. I (or anyone) should be able to have a FULLY functional system 
WITHOUT

the need to get additional sources to build additional functionality -- this
ain't Linux.



I think we should make it our goal to remove any and all unencrypted
protocols from FreeBSD by 2025.

Not everyone works exclusively "in the wild". Many also work within safe
environments, where such things, while nice, are unnecessary.

--Chris
___
freebsd-stable@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"


Re: Deprecating base system ftpd?

2021-04-07 Thread Chris

On 2021-04-05 11:27, Roger Leigh wrote:

On 3 Apr 2021, at 22:21, Eugene Grosbein  wrote:


04.04.2021 3:39, Ed Maste wrote:


I propose deprecating the ftpd currently included in the base system
before FreeBSD 14, and opened review D26447
(https://reviews.freebsd.org/D26447) to add a notice to the man page.
I had originally planned to try to do this before 13.0, but it dropped
off my list. FTP is not nearly as relevant now as it once was, and it
had a security vulnerability that secteam had to address.

I'm happy to make a port for it if anyone needs it. Comments?


I'm strongly against remove of stock ftpd. FTP is fastest protocol for both 
testing
and daily file transfer for trusted isolated segments, and even for WAN 
wrapped in IPSec.


Our stock ftpd has very short backlog of security issues comparing with 
other FTP server implementations,

mostly linked with libc or other libraries and not with ftpd code itself.

Please don't fix what ain't broken. Please.


How would you draw the line between something that must be part of the base 
system
vs. something that would be better off as part of the ports tree?  What bar 
should

ftpd have to meet to warrant remaining in base vs moving to ports?

Personally, I’ve never enabled it nor had any desire to.  FTP is, at this 
point in
time, thoroughly obsolescent, and I cannot imagine that it is something that 
most
people enable, if they are even aware of its existence.  Why can’t it simply 
be

installed from the ports for the occasional user who still requires it?  Why
should the base system contain obsolete stuff that few people will use?  
Surely

the ports tree serves this need better?

Can I ask, for those who do enable it, why isn’t “sftp” acceptable (or 
“scp”)?
Sure. Because it's part of a one-time task. It might be part of a server 
setup. Or
might a task that must be done on thousands of machines. It needs to be 
available
out-of-the-box, and needs no overhead for setup (key exchange, config, 
etc...).

This scenario may also be on machines w/o any external sources/packages. IOW
everything should be available out of the box, with little to no additional
setup overhead. ftp(1), and ftpd(8) provide everything required at no 
additional

cost. :-)

Both provide a similar function, securely, which also works with a basic
installation without any ports.  SSHFXP, the protocol underlying sftp is 
better
specified, less ambiguous and more fault tolerant and safe than the FTP 
protocol

ever was.  The client is better than most ftp clients, and the server
(/usr/libexec/sftp-server) is started on demand on a per-connection basis.  
What
makes FTP more desirable than a service over SSH which is (from a technical 
and

usability point of view) a better FTP than FTP ever was?

Kind regards,
Roger


--Chris
___
freebsd-stable@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"


Re: Deprecating base system ftpd?

2021-04-07 Thread Chris

On 2021-04-05 12:01, Patrick M. Hausen wrote:

Hi all,


Am 03.04.2021 um 22:39 schrieb Ed Maste :
I'm happy to make a port for it if anyone needs it. Comments?


A bit late to the party, but my take is: please just don't.

I absolutely freaked out when Apple removed the telnet and ftp clients
from Mac OS and I needed to reinstall them via MacPorts.

That route uses an inferior (opinion) version of ftp(1) && ftpd(8).
It might interest you to know that the FreeBSD versions that (currently)
already come with FreeBSD can be copied over to your Mac. I *too*
became upset when I found MacOS w/o these, and on a hunch tried it,
and worked as intended/expected. The FreeBSD version(s) are more
"featurefull" than those that originally came on the Mac, or the GNU
one that is supplied w/MacPorts.



People who manage any larger collection of networking gear *depend*
on these outdated but simple services. Client and server side alike.

TFTP is not going away, neither is FTP. I'm dead serious. Remote media
via Supermicro IPMI in 2021? SMB1. Firmware updates for my UPS? FTP.
Scanner/printer/fax all-in-one thingy? Uploads received fax transmissions
via FTP. PBX? Uploads usage reports via FTP. This stuff is here to stay.
In local networks, of course.

But still even on "the Internet", FTP is the most used method for customers
of static website hosting. You cannot teach these people what an SSH key is.
Just my experience, but backed by a load of customer interactions over more
than 20 years ...

Kind regards,
Patrick
--
punkt.de GmbH
Patrick M. Hausen
.infrastructure

Kaiserallee 13a
76133 Karlsruhe

Tel. +49 721 9109500

https://infrastructure.punkt.de
i...@punkt.de

AG Mannheim 108285
Geschäftsführer: Jürgen Egeling, Daniel Lienert, Fabian Stein

--Chris
___
freebsd-stable@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"


Re: Deprecating base system ftpd?

2021-04-07 Thread Chris

On 2021-04-05 13:43, Ed Maste wrote:

On Sat, 3 Apr 2021 at 16:39, Ed Maste  wrote:


I propose deprecating the ftpd currently included in the base system
before FreeBSD 14, and opened review D26447
(https://reviews.freebsd.org/D26447) to add a notice to the man page.


I posted this as a proposal for community feedback, and there's
clearly a strong objection to removing the base system ftpd. So, I'm
not going to pursue this any further.

I like to take the time to extend a massive _Thank You_ for your
chosen course for handling this. I'm also grateful for it's outcome. :-)

Thanks!

--Chris


___
freebsd-stable@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"

___
freebsd-stable@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"


Re: Deprecating base system ftpd?

2021-04-07 Thread Alan Somers
On Tue, Apr 6, 2021 at 1:25 PM Chris  wrote:

> On 2021-04-03 13:45, Warner Losh wrote:
> > On Sat, Apr 3, 2021 at 2:40 PM Ed Maste  wrote:
> >
> >> I propose deprecating the ftpd currently included in the base system
> >> before FreeBSD 14, and opened review D26447
> >> (https://reviews.freebsd.org/D26447) to add a notice to the man page.
> >> I had originally planned to try to do this before 13.0, but it dropped
> >> off my list. FTP is not nearly as relevant now as it once was, and it
> >> had a security vulnerability that secteam had to address.
> I *strongly* object. MacOS also did this. Which made me discover that I
> could simply copy my already built FreeBSD version over to all my MacOS
> laptops and now enjoy an even better version than had previously existed.
> This fact has made my use and need for FreeBSD' ftp even more important.
> It has also made FreeBSD more popular with the Mac folks.
> I depend upon ftp(1) && ftpd(8). I have on FreeBSD, for as many years as
> FreeBSD has existed. I find the ssh and related ports are probed and
> hammered on constantly. Whereas the ftp ports are quite rare by comparison.
> So keeping sshd(8) and friends ports closed removes overhead. I have no
> difficulty managing ftpd(8) via inet(8) && hosts.allow(5). Ftp && ftpd
> are both trivial programs and should not be considered for removal.
> If the reason for their suggested removal is "development overhead".
> Please allow me to maintain both. I will happily assume full
> responsibility for them.
>
> Thank you for listening. :-)
>

Great!  Even though they work well, they haven't seen a lot of TLC.  What
really worries me most is that ftpd has zero test coverage.  It would be
great to fix that, and not too hard.  You could start be adapting the
existing tests in  libexec/tftpd/tests/ .  Capscium support would be nice,
too.

-Alan
___
freebsd-stable@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"


Re: Deprecating base system ftpd?

2021-04-07 Thread Kurt Buff
On Wed, Apr 7, 2021 at 6:18 AM tech-lists  wrote:
>
> Hi, I'm a bit late to the discussion
>
> On Mon, Apr 05, 2021 at 07:44:59AM -0700, Cy Schubert wrote:
>
> >I think this is an excellent start. My shopping list includes:
> >
> >- remove ftp(1)
> >- remove ftpd(8)
> >- remove telnet(1)
> >- remove telnetd(8)
> >- remove ftp:// and http:// from libfetch. This is 2021 and we should all
> >use https://.
> >- replace DNS lookups with DoH and/or DoT. Why let your ISP see your DNS
> >traffic?
>
> Very firmly against this, and this sort of thing, for the following reasons:
>
> 1. I want an OS, not a kernel. If I just want a kernel, then why not go
> with linux? FreeBSD is meant to be, I think, (generally), a server OS.
> So, would you agree that it needs the ability to have server protocols
> easily configured, with a minimum of fuss, without packages?
>
> 2. a lot of infrastructure depends on ftpd. it's easy to configure
> securely ftpd in base.
>
> 3. there are some networks, like internal ones, where encryption is not
> a requirement, or appropriate.
>
> 4. there are some places where encryption is in fact illegal.
>
> >Personally, I'd suggest we remove the ftpd server *AND* ftp client and rely
> >on ports. Having worked on UNIX, Internet security, and firewalls over the
> >last 3/5 of my almost 50 year career, I have lamented the existence of the
> >FTP protocol back in 1995 and I hate the FTP protocol with greater a
> >passion today. Let's simply remove all vestiges of FTP from the base
> >system, including libfetch, sooner than later. We don't need it now that we
> >have HTTPS and POST; and sftp.
>
> 5. some services commonly don't use https. Lots of internet radio
> stations don't. If https is enforced then the user will have to jump
> through more hoops than they already do in order to, in this case,
> listen to internet radio. Or face a loss of functionality.
>
> 6. not everywhere will have constant internet access. Not everyone will
> want to use pkgs or have space for the ports tree.
>
> >I think we should make it our goal to remove any and all unencrypted
> >protocols from FreeBSD by 2025.
>
> I think you should carefully think of the consequences of removing
> functionality in the default install. It will make it less useful, not
> more.
> --
> J.

To amplify this a bit: Those who are all about secure protocols (and
I'm one of them) should realize that public cryptography (not just
public key, but public use of cryptographic protocols i general) is
not a solved problem. In particular, multi-party key management in an
open Internet is problematic.

Open or plain text protocols do have a place.

Kurt
___
freebsd-stable@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"


stable/13, vm page counts do not add up

2021-04-07 Thread Andriy Gapon


I regularly see that the top's memory line does not add up (and by a lot).
That can be seen with vm.stats as well.

For example:
$ sysctl vm.stats | fgrep count
vm.stats.vm.v_cache_count: 0
vm.stats.vm.v_user_wire_count: 3231
vm.stats.vm.v_laundry_count: 262058
vm.stats.vm.v_inactive_count: 3054178
vm.stats.vm.v_active_count: 621131
vm.stats.vm.v_wire_count: 1871176
vm.stats.vm.v_free_count: 18
vm.stats.vm.v_page_count: 8134982

$ bc
>>> 18 + 1871176 + 621131 + 3054178 + 262058
5996320
>>> 8134982 - 5996320
2138662

As you can see, it's not a small number of pages either.
Approximately 2 million pages, 8 gigabytes or 25% of the whole memory on this
system.

This is 47c00a9835926e96, 13.0-STABLE amd64.
I do not think that I saw anything like that when I used (much) older FreeBSD.

-- 
Andriy Gapon
___
freebsd-stable@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"


Re: stable/13, vm page counts do not add up

2021-04-07 Thread Mark Johnston
On Wed, Apr 07, 2021 at 10:42:57PM +0300, Andriy Gapon wrote:
> 
> I regularly see that the top's memory line does not add up (and by a lot).
> That can be seen with vm.stats as well.
> 
> For example:
> $ sysctl vm.stats | fgrep count
> vm.stats.vm.v_cache_count: 0
> vm.stats.vm.v_user_wire_count: 3231
> vm.stats.vm.v_laundry_count: 262058
> vm.stats.vm.v_inactive_count: 3054178
> vm.stats.vm.v_active_count: 621131
> vm.stats.vm.v_wire_count: 1871176
> vm.stats.vm.v_free_count: 18
> vm.stats.vm.v_page_count: 8134982
> 
> $ bc
> >>> 18 + 1871176 + 621131 + 3054178 + 262058
> 5996320
> >>> 8134982 - 5996320
> 2138662
> 
> As you can see, it's not a small number of pages either.
> Approximately 2 million pages, 8 gigabytes or 25% of the whole memory on this
> system.
> 
> This is 47c00a9835926e96, 13.0-STABLE amd64.
> I do not think that I saw anything like that when I used (much) older FreeBSD.

One relevant change is that vm_page_wire() no longer removes pages from
LRU queues, so the count of pages in the queues can include wired pages.
If the page daemon runs, it will dequeue any wired pages that are
encountered.

This was done to reduce queue lock contention, operations like
sendfile() which transiently wire pages would otherwise trigger two
queue operations per page.  Now that queue operations are batched this
might not be as important.

We could perhaps add a new flavour of vm_page_wire() which is not lazy
and would be suited for e.g., the buffer cache.  What is the primary
source of wired pages in this case?
___
freebsd-stable@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"


Re: stable/13, vm page counts do not add up

2021-04-07 Thread Andriy Gapon
On 07/04/2021 22:54, Mark Johnston wrote:
> On Wed, Apr 07, 2021 at 10:42:57PM +0300, Andriy Gapon wrote:
>>
>> I regularly see that the top's memory line does not add up (and by a lot).
>> That can be seen with vm.stats as well.
>>
>> For example:
>> $ sysctl vm.stats | fgrep count
>> vm.stats.vm.v_cache_count: 0
>> vm.stats.vm.v_user_wire_count: 3231
>> vm.stats.vm.v_laundry_count: 262058
>> vm.stats.vm.v_inactive_count: 3054178
>> vm.stats.vm.v_active_count: 621131
>> vm.stats.vm.v_wire_count: 1871176
>> vm.stats.vm.v_free_count: 18
>> vm.stats.vm.v_page_count: 8134982
>>
>> $ bc
> 18 + 1871176 + 621131 + 3054178 + 262058
>> 5996320
> 8134982 - 5996320
>> 2138662
>>
>> As you can see, it's not a small number of pages either.
>> Approximately 2 million pages, 8 gigabytes or 25% of the whole memory on this
>> system.
>>
>> This is 47c00a9835926e96, 13.0-STABLE amd64.
>> I do not think that I saw anything like that when I used (much) older 
>> FreeBSD.
> 
> One relevant change is that vm_page_wire() no longer removes pages from
> LRU queues, so the count of pages in the queues can include wired pages.
> If the page daemon runs, it will dequeue any wired pages that are
> encountered.

Maybe I misunderstand how that works, but I would expect that the sum of all
counters could be greater than v_page_count at times.  But in my case it's less.

> This was done to reduce queue lock contention, operations like
> sendfile() which transiently wire pages would otherwise trigger two
> queue operations per page.  Now that queue operations are batched this
> might not be as important.
> 
> We could perhaps add a new flavour of vm_page_wire() which is not lazy
> and would be suited for e.g., the buffer cache.  What is the primary
> source of wired pages in this case?

It should be ZFS, I guess.

-- 
Andriy Gapon
___
freebsd-stable@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"


Re: stable/13, vm page counts do not add up

2021-04-07 Thread Mark Johnston
On Wed, Apr 07, 2021 at 11:22:41PM +0300, Andriy Gapon wrote:
> On 07/04/2021 22:54, Mark Johnston wrote:
> > On Wed, Apr 07, 2021 at 10:42:57PM +0300, Andriy Gapon wrote:
> >>
> >> I regularly see that the top's memory line does not add up (and by a lot).
> >> That can be seen with vm.stats as well.
> >>
> >> For example:
> >> $ sysctl vm.stats | fgrep count
> >> vm.stats.vm.v_cache_count: 0
> >> vm.stats.vm.v_user_wire_count: 3231
> >> vm.stats.vm.v_laundry_count: 262058
> >> vm.stats.vm.v_inactive_count: 3054178
> >> vm.stats.vm.v_active_count: 621131
> >> vm.stats.vm.v_wire_count: 1871176
> >> vm.stats.vm.v_free_count: 18
> >> vm.stats.vm.v_page_count: 8134982
> >>
> >> $ bc
> > 18 + 1871176 + 621131 + 3054178 + 262058
> >> 5996320
> > 8134982 - 5996320
> >> 2138662
> >>
> >> As you can see, it's not a small number of pages either.
> >> Approximately 2 million pages, 8 gigabytes or 25% of the whole memory on 
> >> this
> >> system.
> >>
> >> This is 47c00a9835926e96, 13.0-STABLE amd64.
> >> I do not think that I saw anything like that when I used (much) older 
> >> FreeBSD.
> > 
> > One relevant change is that vm_page_wire() no longer removes pages from
> > LRU queues, so the count of pages in the queues can include wired pages.
> > If the page daemon runs, it will dequeue any wired pages that are
> > encountered.
> 
> Maybe I misunderstand how that works, but I would expect that the sum of all
> counters could be greater than v_page_count at times.  But in my case it's 
> less.

I misread, sorry.  You're right, what I described would cause double
counting.

I don't know what might be causing it then.  It could be a page leak.
The kernel allocates wired pages without adjusting the v_wire_count
counter in some cases, but the ones I know about happen at boot and
should not account for such a large disparity.  I do not see it on a few
systems that I have access to.

> > This was done to reduce queue lock contention, operations like
> > sendfile() which transiently wire pages would otherwise trigger two
> > queue operations per page.  Now that queue operations are batched this
> > might not be as important.
> > 
> > We could perhaps add a new flavour of vm_page_wire() which is not lazy
> > and would be suited for e.g., the buffer cache.  What is the primary
> > source of wired pages in this case?
> 
> It should be ZFS, I guess.
> 
> -- 
> Andriy Gapon
___
freebsd-stable@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"


Re: stable/13, vm page counts do not add up

2021-04-07 Thread Dewayne Geraghty
On 8/04/2021 6:56 am, Mark Johnston wrote:
> On Wed, Apr 07, 2021 at 11:22:41PM +0300, Andriy Gapon wrote:
>> On 07/04/2021 22:54, Mark Johnston wrote:
>>> On Wed, Apr 07, 2021 at 10:42:57PM +0300, Andriy Gapon wrote:

 I regularly see that the top's memory line does not add up (and by a lot).
 That can be seen with vm.stats as well.

 For example:
 $ sysctl vm.stats | fgrep count
 vm.stats.vm.v_cache_count: 0
 vm.stats.vm.v_user_wire_count: 3231
 vm.stats.vm.v_laundry_count: 262058
 vm.stats.vm.v_inactive_count: 3054178
 vm.stats.vm.v_active_count: 621131
 vm.stats.vm.v_wire_count: 1871176
 vm.stats.vm.v_free_count: 18
 vm.stats.vm.v_page_count: 8134982

 $ bc
>>> 18 + 1871176 + 621131 + 3054178 + 262058
 5996320
>>> 8134982 - 5996320
 2138662

 As you can see, it's not a small number of pages either.
 Approximately 2 million pages, 8 gigabytes or 25% of the whole memory on 
 this
 system.

 This is 47c00a9835926e96, 13.0-STABLE amd64.
 I do not think that I saw anything like that when I used (much) older 
 FreeBSD.

For reference, I think that a smaller error has been around for awhile.
 On a UFS only system,
FreeBSD 12.2-STABLE #0 r369523M: Sat Mar 27 00:27:03 AEDT 2021
I have:

# sysctl vm.stats | fgrep count; top -b
vm.stats.vm.v_cache_count: 0
vm.stats.vm.v_user_wire_count: 0
vm.stats.vm.v_laundry_count: 0
vm.stats.vm.v_inactive_count: 423959
vm.stats.vm.v_active_count: 82623
vm.stats.vm.v_wire_count: 256273
vm.stats.vm.v_free_count: 5457329
vm.stats.vm.v_page_count: 6112118
last pid: 83881;  load averages:  0.07,  0.09,  0.06  up 0+07:31:44
12:59:37
90 processes:  1 running, 89 sleeping
CPU:  0.0% user,  0.0% nice,  0.0% system,  0.0% interrupt, 99.9% idle
Mem: 323M Active, 1656M Inact, 1001M Wired, 585M Buf, 21G Free
Swap: 24G Total, 24G Free

# bc
423959+82623+256273+5457329
6220184
6112118 - 6220184
-108066
___
freebsd-stable@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"