[[ + peter ]]

Folks, I spent quite a bit of time trying to figure out how to resolve maxusers scaling in a happy way for all.

I think I came up with a solution.

This solution should work for i386, and other 32 bit platforms, as well as scaling well for 64 bit (or higher) platforms that have virtually unlimited AND 64bit with limited kernel address space.

Here is how it works:

We calculate the maxusers value based on physical memory, and then clamp it down if physical memory exceeds kernel addressable memory.

The algorithm actually remains the same for all architectures, with the exception that machines with large kernel address space it is no longer clamped at 384.

I've attached a test program that lets you play with various values for VM_MIN_KERNEL_ADDRESS, VM_MAX_KERNEL_ADDRESS and physpages. (argv[1, 2, 3] respectively.)

Please give me your feedback.

Note a lot of the ugly types/casting is due to building this test program on i386 darwin. I will clean it up before committing.

I just want to make sure this the right direction.

I would like to commit this shortly.

-Alfred






On 10/25/12 6:50 AM, John Baldwin wrote:
On Thursday, October 25, 2012 4:05:51 am Konstantin Belousov wrote:
On Thu, Oct 25, 2012 at 01:46:21AM +0000, Alfred Perlstein wrote:
Author: alfred
Date: Thu Oct 25 01:46:20 2012
New Revision: 242029
URL: http://svn.freebsd.org/changeset/base/242029

Log:
   Allow autotune maxusers > 384 on 64 bit machines
A default install on large memory machines with multiple 10gigE interfaces
   were not being given enough mbufs to do full bandwidth TCP or NFS traffic.
To keep the value somewhat reasonable, we scale back the number of
   maxuers by 1/6 past the 384 point.  This gives us enough mbufs for most
   of our pretty basic 10gigE line-speed tests to complete.

Modified:
   head/sys/kern/subr_param.c

Modified: head/sys/kern/subr_param.c
==============================================================================
--- head/sys/kern/subr_param.c  Thu Oct 25 01:27:01 2012        (r242028)
+++ head/sys/kern/subr_param.c  Thu Oct 25 01:46:20 2012        (r242029)
@@ -278,8 +278,16 @@ init_param2(long physpages)
                maxusers = physpages / (2 * 1024 * 1024 / PAGE_SIZE);
                if (maxusers < 32)
                        maxusers = 32;
-               if (maxusers > 384)
-                       maxusers = 384;
+               /*
+                * Clips maxusers to 384 on machines with <= 4GB RAM or 32bit.
+                * Scales it down 6x for large memory machines.
+                */
+               if (maxusers > 384) {
+                       if (sizeof(void *) <= 4)
+                           maxusers = 384;
+                       else
+                           maxusers = 384 + ((maxusers - 384) / 6);
+               }
This is unbelievably weird way to express the '64bit'. The
#ifdef _LP64 is enough there instead of the runtime check.

Also, are you sure that all our 64bit arches do not have KVA limitations ?
There is an active thread on current@ with a different patch that uses a KVA
constant to derive 384 instead.  When we have updated tuning in the past
(e.g. adjusting the cap on maxvnodes), there was a bit of discussion rather
than a random drive by commit.  I think we should probably hold off on making
changes here and figure out what the right way to fix this is in the thread on
current@ instead.  Andre has already suggested divorcing mbuf tuning from
maxusers entirely for example.


#include <stdio.h>
#include <stdlib.h>
#include <sys/types.h>
#ifndef __FreeBSD__
#include <machine/types.h>
#endif

#define VM_MIN_KERNEL_ADDRESS atoull(argv[1])
#define VM_MAX_KERNEL_ADDRESS atoull(argv[2])
#define physpages atol(argv[3])
#define PAGE_SIZE (long long)4096
int maxusers;

unsigned long long
atoull(const char *str)
{
        unsigned long long ret;

        if (sscanf(str, "%llu", &ret) != 1) {
                fprintf(stderr, "sscanf: error\n");
                exit(1);
        }
        return ret;
}

int
main(int argc, char **argv) {
                u_int64_t maxkvapages;

                long maxusers_kva;
                long maxusers_phys;

                printf("min: %llu, max: %llu\n", VM_MIN_KERNEL_ADDRESS,
                        VM_MAX_KERNEL_ADDRESS);

                maxkvapages = (VM_MAX_KERNEL_ADDRESS - VM_MIN_KERNEL_ADDRESS)   
                 / (long long)PAGE_SIZE;

                printf("VM_ADDRESS_SPACE: %llu\n", (long long)((long 
long)VM_MAX_KERNEL_ADDRESS - (long long)VM_MIN_KERNEL_ADDRESS));
                printf("maxkvapages: %ld\n", (long)maxkvapages);
                maxusers_kva = maxkvapages * 3 / 4 / 512;
                printf("maxusers_kva: %ld\n", maxusers_kva);
                maxusers_phys = physpages / 512;
                printf("maxusers_phys: %ld\n", maxusers_phys);
                if (maxusers_phys > maxusers_kva) {
                        maxusers = maxusers_kva;
                } else {
                        maxusers = maxusers_phys;
                }
                printf("maxusers: %d\n", maxusers);
return 0;
}

_______________________________________________
svn-src-head@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"

Reply via email to