As requested a few times in private to make the results available, here
you go with what works for me. Hope this help some anyway.
Use what make sense to you based on your setup, hardware and traffic.
Final value in use after testing are now set as follow for me assuming a
good amount of memory to allow so many process to run. I use minimum
2GB, some have 4GB.
Recompile httpd with upper limits for process. I put 2048 to allow more
room in the future if needed, but I still want to be safe and limit the
process lower that that. If php is in use for example, static
compilation would improve, but I choose to keep the system as much as
possible as default for many reasons, including maintenance, support and
regular upgrades. Your choice may vary.
In fstab
========
A partition for the files used by the sites set with noatime set on it
to avoid the change in last access time for each files. Definitely
improve access time a lots under heavy load!
httpd logs could be on it's own partition as well, mounted softdep to
gain some efficiency in logs updates if very busy sites.
For httpd.conf
==============
Timeout 300
KeepAlive On
MaxKeepAliveRequests 100
KeepAliveTimeout 5
MinSpareServers 50
MaxSpareServers 100
StartServers 75
MaxClients 768
MaxRequestsPerChild 0
In sysctl.conf
==============
# Below are values added to improve performance of httpd after
# testing with http_load under parallel and rate setting.
kern.maxclusters=12000 # The maximum number of mbuf(9) clusters
# that may be allocated.
kern.maxfiles=4096 # The maximum number of open files that
# may be open in the system.
kern.maxproc=2048 # The maximum number of simultaneous
# processes the system will allow.
kern.seminfo.semmni=1024 # The maximum number of semaphore
# identifiers allowed.
kern.seminfo.semmns=4096 # The maximum number of semaphores
# allowed in the system.
kern.shminfo.shmall=16384 # The maximum amount of total shared
# memory allowed in the system (in
# pages).
kern.shminfo.shmmax=67108864 # The maximum shared memory segment size
# (in bytes).
kern.somaxconn=2048 # Upper bound on the number of half-open
# connections a process
# can allow to be associated with a
# socket, using listen(2).
net.inet.ip.maxqueue=1280 # Fragment flood protection. Sets the
# maximum number of
# unassembled IP fragments in the
# fragment queue.
net.inet.tcp.keepidle=30 # Time connection must be idle before
# keepalive sent.
net.inet.tcp.keepinittime=10 # Used by the syncache to timeout SYN
# request.
net.inet.tcp.keepintvl=30 # Interval between keepalive sent to
# remote machines.
net.inet.tcp.mssdflt=1452 # The maximum segment size that is used
# as default for non-local connections.
net.inet.tcp.recvspace=65535 # TCP receive buffer size.
net.inet.tcp.rstppslimit=400 # This variable specifies the maximum
# number of outgoing
# TCP RST packets per second. TCP RST
# packets exceeding
# this value are subject to rate
# limitation and will not go
# out from the node. A negative value
# disables rate limitation.
net.inet.tcp.sendspace=65535 # TCP Send buffer size.
net.inet.tcp.synbucketlimit=420 # The maximum number of entries allowed
# per hash bucket in
# the TCP SYN cache.
net.inet.tcp.syncachelimit=20510 # The maximum number of entries
# allowed in the TCP SYN
# cache.
