Re: gpart destroy, zpool destroy, zfs destroy under securelevel 3
On 26.05.2014 17:31, Vladimir Sharun wrote: > Hello FreeBSD community, > > Recently plays with securelevel and what I discover: no chance for > data to survive against remote root, except backups of course. Maybe > this log can be a proposal for raising securelevel further or include > securelevel support against the software which can deal with zfs and > GEOM labels ? Hi, if you have root privileges you can just write some random bytes in some places and this will be enough to break your system. So, restricting some gpart's or zpool's actions depending from securelevel looks like protection from kids. -- WBR, Andrey V. Elsukov ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re[2]: gpart destroy, zpool destroy, zfs destroy under securelevel 3
Hello, > if you have root privileges you can just write some random bytes in some > places and this will be enough to break your system. So, restricting > some gpart's or zpool's actions depending from securelevel looks like > protection from kids. Having root under securelevel 3 confirmed disallows you to: 1) Direct write to the block devices such as (a)da 2) Change rules and/or shutdown pf 3) Remove system flags such as schg, sunlnk I think your statement true in case of securelevel -1, we're talking about the highest one - 3, which shown in logs. ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: official pkg repo with WITHOUT_X11=true
On 29 May 2014, at 02:23, Bryan Drewery wrote: > As for skipping unneeded ports the best I can do is '-a' or "Build it all". > If a port is only needed for WITH_X11 then an IGNORE should be added to it > when WITHOUT_X11 is set to prevent wasting time on it. We can probably do a bit better by looking at the complete dependency graph and removing any ports that have unconditional dependencies on X. For a headless server, there's no reason to build any of the kde-* or gnome-* ports or, indeed, X itself. I suspect that we could easily trim 2/3 of the build time by omitting ports that have a GUI, GUI toolkits, and so on. Longer term, we may be able to share the build time a bit. Ports which don't have a WITHOUT_X11 flag and don't unconditionally depend on X11 can potentially be pre-seeded from the normal package build (if we can identify them). That only leaves the ports that actually have build-time conditional X support to build in the no-Xorg run. David ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: gpart destroy, zpool destroy, zfs destroy under securelevel 3
On 29.05.2014 12:56, Vladimir Sharun wrote: > Hello, > >> if you have root privileges you can just write some random bytes in some >> places and this will be enough to break your system. So, restricting >> some gpart's or zpool's actions depending from securelevel looks like >> protection from kids. > > Having root under securelevel 3 confirmed disallows you to: > 1) Direct write to the block devices such as (a)da > 2) Change rules and/or shutdown pf > 3) Remove system flags such as schg, sunlnk > > I think your statement true in case of securelevel -1, we're talking about > the highest one - 3, which shown in logs. Ok, you are right. But geom_dev restricts access only from user level applications. When GEOM object does access directly via GEOM methods this protection won't work. And it seems it isn't easy to fix, all classes should have own check. -- WBR, Andrey V. Elsukov signature.asc Description: OpenPGP digital signature
Memory blackhole in 11. Possibly libc.so.7?
uname: FreeBSD 11.0-CURRENT #0 r266393M: Sun May 18 13:04:00 2014 amd64 I'm also loading the Radeon_kms modules Upon system startup, memory profile is clean. I get locked memory (mem_wire) usage as: 9% before Radeon*.ko modules loaded 12% when slim is started (loads Radeon*.ko modules) 14% after I login through slim These are quite normal. However, after some usage (I don't know what exactly) mem_free drops significantly and causes display problems (Radeon has issues when memory falls very low, if I remember correctly). Upon observing low level of mem_free, I went to single user (shutdown now), also kldunload any modules not compiled into kernel (using custom built kernel). Final kldstat: Id Refs AddressSize Name 1 99 0x8020 941c18 kernel 21 0x80b42000 b0f0 linprocfs.ko 32 0x80b4e000 45ba0linux.ko 41 0x80b94000 2a38 linsysfs.ko 51 0x80b97000 243fc8 zfs.ko 61 0x80ddb000 2940 acl_nfs4.ko 72 0x80dde000 5d10 opensolaris.ko 81 0x80de4000 40a50snd_hda.ko 92 0x80e25000 756a0sound.ko 101 0x80e9b000 7550 umass.ko 115 0x80ea3000 3f7b0usb.ko 131 0x80f32000 19758ext2fs.ko 141 0x80f4c000 d050 ehci.ko 151 0x80f5a000 a3f0 ohci.ko 161 0x81011000 1adb ums.ko 171 0x81013000 951 pflog.ko 181 0x81014000 27641pf.ko 191 0x8103c000 108ffc radeonkms.ko 201 0x81145000 3fd39drm2.ko 212 0x81185000 ae68 agp.ko 224 0x8119 1776 iicbus.ko 231 0x81192000 d1a iic.ko 241 0x81193000 167d iicbb.ko 254 0x81195000 1bde firmware.ko 261 0x81197000 ac6 radeonkmsfw_RS780_pfp.ko 271 0x81198000 55c6 radeonkmsfw_RS780_me.ko 281 0x8119e000 dc6 radeonkmsfw_R600_rlc.ko 311 0x811d6000 9118 netgraph.ko 321 0x811e 160a ng_ether.ko But look at memory (70%), even though there are absolutely no processes running. SYSTEM MEMORY INFORMATION: mem_wire:2735349760 ( 2608MB) [ 70%] Wired: disabled for paging out mem_active: + 8638464 ( 8MB) [ 0%] Active: recently referenced mem_inactive:+137596928 (131MB) [ 3%] Inactive: recently not referenced mem_cache: + 34885632 ( 33MB) [ 0%] Cached: almost avail. for allocation mem_free:+970948608 (925MB) [ 24%] Free: fully available for allocation mem_gap_vm: + 413696 ( 0MB) [ 0%] Memory gap: UNKNOWN -- --- -- mem_all: = 3887833088 ( 3707MB) [100%] Total real memory managed mem_gap_sys: +123482112 (117MB)Memory gap: Kernel?! -- --- mem_phys:= 4011315200 ( 3825MB)Total real memory available mem_gap_hw: +283652096 (270MB)Memory gap: Segment Mappings?! -- --- mem_hw: = 4294967296 ( 4096MB)Total real memory installed SYSTEM MEMORY SUMMARY: mem_used:3151536128 ( 3005MB) [ 73%] Logically used memory mem_avail: + 1143431168 ( 1090MB) [ 26%] Logically available memory -- --- -- mem_total: = 4294967296 ( 4096MB) [100%] Logically total memory I don't know if the lsof dump in single user mode will be of any help, but it seems like lib/libc.so.7 has something to do with it: COMMANDPID USER FD TYPE DEVICE SIZE/OFF NODE NAME kernel 0 root cwd VDIR 123,1479344360 30 4 / kernel 0 root rtd VDIR 123,1479344360 30 4 / init 1 root cwd VDIR 123,1479344360 30 4 / init 1 root rtd VDIR 123,1479344360 30 4 / init 1 root txt VREG 123,1479344360 946272 9945 /sbin/init ng_queue 2186 root cwd VDIR 123,1479344360 30 4 / ng_queue 2186 root rtd VDIR 123,1479344360 30 4 / sh 10935 root cwd VDIR 114,1970143416148 13 /home/user sh 10935 root rtd VDIR 123,1479344360 30 4 / sh 10935 root txt VREG 123,1479344360 145616 9796 /bin/sh sh 10935 root txt VREG 123,1479344360 118360 9731 /libexec/ld-elf.so.1 sh 10935 root txt VREG 123,1479344360 163328 9700 /lib/libedit.so.7 sh 10935 root txt VREG 123,1479344360 326264 9666 /lib/libncurses.so.8 sh 10935 root txt VREG 123,14793443601597040 9648 /lib/libc.so.7 sh 10935 root0u VCHR0,4 0t4591427 4 /dev/console sh 10935 root1u VCHR0,4 0t4591427 4 /dev/conso
Re[2]: gpart destroy, zpool destroy, zfs destroy under securelevel 3
Hello, > Ok, you are right. But geom_dev restricts access only from user level > applications. When GEOM object does access directly via GEOM methods > this protection won't work. And it seems it isn't easy to fix, all > classes should have own check. Thank you for better clarification. This is the goal I mentioned in first email: GEOM & ZFS layers/subsystems are securelevel ignorant. ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: newcons and beeping X
Are there any problems with MFC? -- View this message in context: http://freebsd.1045724.n5.nabble.com/newcons-and-beeping-X-tp5906883p5916170.html Sent from the freebsd-current mailing list archive at Nabble.com. ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
[head tinderbox] failure on amd64/amd64
TB --- 2014-05-29 11:00:39 - tinderbox 2.22 running on freebsd-current.sentex.ca TB --- 2014-05-29 11:00:39 - FreeBSD freebsd-current.sentex.ca 9.2-STABLE FreeBSD 9.2-STABLE #0 r263721: Tue Mar 25 09:27:39 EDT 2014 d...@freebsd-current.sentex.ca:/usr/obj/usr/src/sys/GENERIC amd64 TB --- 2014-05-29 11:00:39 - starting HEAD tinderbox run for amd64/amd64 TB --- 2014-05-29 11:00:39 - cleaning the object tree TB --- 2014-05-29 11:04:27 - /usr/local/bin/svn stat --no-ignore /src TB --- 2014-05-29 11:04:30 - At svn revision 266832 TB --- 2014-05-29 11:04:31 - building world TB --- 2014-05-29 11:04:31 - CROSS_BUILD_TESTING=YES TB --- 2014-05-29 11:04:31 - MAKEOBJDIRPREFIX=/obj TB --- 2014-05-29 11:04:31 - MAKESYSPATH=/src/share/mk TB --- 2014-05-29 11:04:31 - PATH=/usr/bin:/usr/sbin:/bin:/sbin TB --- 2014-05-29 11:04:31 - SRCCONF=/dev/null TB --- 2014-05-29 11:04:31 - TARGET=amd64 TB --- 2014-05-29 11:04:31 - TARGET_ARCH=amd64 TB --- 2014-05-29 11:04:31 - TZ=UTC TB --- 2014-05-29 11:04:31 - __MAKE_CONF=/dev/null TB --- 2014-05-29 11:04:31 - cd /src TB --- 2014-05-29 11:04:31 - /usr/bin/make -B buildworld >>> Building an up-to-date bmake(1) >>> World build started on Thu May 29 11:04:38 UTC 2014 >>> Rebuilding the temporary build tree >>> stage 1.1: legacy release compatibility shims >>> stage 1.2: bootstrap tools >>> stage 2.1: cleaning up the object tree >>> stage 2.2: rebuilding the object tree >>> stage 2.3: build tools >>> stage 3: cross tools [...] c++ -O2 -pipe -I/src/lib/clang/libllvmcore/../../../contrib/llvm/include -I/src/lib/clang/libllvmcore/../../../contrib/llvm/tools/clang/include -I/src/lib/clang/libllvmcore/../../../contrib/llvm/lib/IR -I. -I/src/lib/clang/libllvmcore/../../../contrib/llvm/../../lib/clang/include -DLLVM_ON_UNIX -DLLVM_ON_FREEBSD -D__STDC_LIMIT_MACROS -D__STDC_CONSTANT_MACROS -fno-strict-aliasing -DLLVM_DEFAULT_TARGET_TRIPLE=\"x86_64-unknown-freebsd11.0\" -DLLVM_HOST_TRIPLE=\"x86_64-unknown-freebsd11.0\" -DDEFAULT_SYSROOT=\"/obj/amd64.amd64/src/tmp\" -I/obj/amd64.amd64/src/tmp/legacy/usr/include -fno-exceptions -fno-rtti -c /src/lib/clang/libllvmcore/../../../contrib/llvm/lib/IR/DebugLoc.cpp -o DebugLoc.o c++ -O2 -pipe -I/src/lib/clang/libllvmcore/../../../contrib/llvm/include -I/src/lib/clang/libllvmcore/../../../contrib/llvm/tools/clang/include -I/src/lib/clang/libllvmcore/../../../contrib/llvm/lib/IR -I. -I/src/lib/clang/libllvmcore/../../../contrib/llvm/../../lib/clang/include -DLLVM_ON_UNIX -DLLVM_ON_FREEBSD -D__STDC_LIMIT_MACROS -D__STDC_CONSTANT_MACROS -fno-strict-aliasing -DLLVM_DEFAULT_TARGET_TRIPLE=\"x86_64-unknown-freebsd11.0\" -DLLVM_HOST_TRIPLE=\"x86_64-unknown-freebsd11.0\" -DDEFAULT_SYSROOT=\"/obj/amd64.amd64/src/tmp\" -I/obj/amd64.amd64/src/tmp/legacy/usr/include -fno-exceptions -fno-rtti -c /src/lib/clang/libllvmcore/../../../contrib/llvm/lib/IR/Dominators.cpp -o Dominators.o c++ -O2 -pipe -I/src/lib/clang/libllvmcore/../../../contrib/llvm/include -I/src/lib/clang/libllvmcore/../../../contrib/llvm/tools/clang/include -I/src/lib/clang/libllvmcore/../../../contrib/llvm/lib/IR -I. -I/src/lib/clang/libllvmcore/../../../contrib/llvm/../../lib/clang/include -DLLVM_ON_UNIX -DLLVM_ON_FREEBSD -D__STDC_LIMIT_MACROS -D__STDC_CONSTANT_MACROS -fno-strict-aliasing -DLLVM_DEFAULT_TARGET_TRIPLE=\"x86_64-unknown-freebsd11.0\" -DLLVM_HOST_TRIPLE=\"x86_64-unknown-freebsd11.0\" -DDEFAULT_SYSROOT=\"/obj/amd64.amd64/src/tmp\" -I/obj/amd64.amd64/src/tmp/legacy/usr/include -fno-exceptions -fno-rtti -c /src/lib/clang/libllvmcore/../../../contrib/llvm/lib/IR/Function.cpp -o Function.o c++ -O2 -pipe -I/src/lib/clang/libllvmcore/../../../contrib/llvm/include -I/src/lib/clang/libllvmcore/../../../contrib/llvm/tools/clang/include -I/src/lib/clang/libllvmcore/../../../contrib/llvm/lib/IR -I. -I/src/lib/clang/libllvmcore/../../../contrib/llvm/../../lib/clang/include -DLLVM_ON_UNIX -DLLVM_ON_FREEBSD -D__STDC_LIMIT_MACROS -D__STDC_CONSTANT_MACROS -fno-strict-aliasing -DLLVM_DEFAULT_TARGET_TRIPLE=\"x86_64-unknown-freebsd11.0\" -DLLVM_HOST_TRIPLE=\"x86_64-unknown-freebsd11.0\" -DDEFAULT_SYSROOT=\"/obj/amd64.amd64/src/tmp\" -I/obj/amd64.amd64/src/tmp/legacy/usr/include -fno-exceptions -fno-rtti -c /src/lib/clang/libllvmcore/../../../contrib/llvm/lib/IR/GCOV.cpp -o GCOV.o c++ -O2 -pipe -I/src/lib/clang/libllvmcore/../../../contrib/llvm/include -I/src/lib/clang/libllvmcore/../../../contrib/llvm/tools/clang/include -I/src/lib/clang/libllvmcore/../../../contrib/llvm/lib/IR -I. -I/src/lib/clang/libllvmcore/../../../contrib/llvm/../../lib/clang/include -DLLVM_ON_UNIX -DLLVM_ON_FREEBSD -D__STDC_LIMIT_MACROS -D__STDC_CONSTANT_MACROS -fno-strict-aliasing -DLLVM_DEFAULT_TARGET_TRIPLE=\"x86_64-unknown-freebsd11.0\" -DLLVM_HOST_TRIPLE=\"x86_64-unknown-freebsd11.0\" -DDEFAULT_SYSROOT=\"/obj/amd64.amd64/src/tmp\" -I/obj/amd64.amd64/src/tmp/legacy/usr/include -fno-exceptions -fno-rtti -c /src/lib/clang/libllvmcore/../../../contrib/llvm/lib/IR/GVMat
Re: official pkg repo with WITHOUT_X11=true
> On May 29, 2014, at 4:19, David Chisnall wrote: > >> On 29 May 2014, at 02:23, Bryan Drewery wrote: >> >> As for skipping unneeded ports the best I can do is '-a' or "Build it all". >> If a port is only needed for WITH_X11 then an IGNORE should be added to it >> when WITHOUT_X11 is set to prevent wasting time on it. > > We can probably do a bit better by looking at the complete dependency graph > and removing any ports that have unconditional dependencies on X. For a > headless server, there's no reason to build any of the kde-* or gnome-* ports > or, indeed, X itself. I suspect that we could easily trim 2/3 of the build > time by omitting ports that have a GUI, GUI toolkits, and so on. Yeah. My point was more that poudriere can't do that now and I would rather not add all that special-case logic to it. Clever make.conf logic might be able to do it. > > Longer term, we may be able to share the build time a bit. Ports which don't > have a WITHOUT_X11 flag and don't unconditionally depend on X11 can > potentially be pre-seeded from the normal package build (if we can identify > them). That only leaves the ports that actually have build-time conditional > X support to build in the no-Xorg run. Yup! I have a patch for that in the works. > > David ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: RFT vidcontrol for vt(4)
On Mon, 12 May 2014 17:10:54 +0200 Claude Buisson wrote: > On 05/12/2014 16:14, Aleksandr Rybalko wrote: > > On Mon, 12 May 2014 15:35:48 +0200 > > Claude Buisson wrote: > > > >> On 03/11/2014 15:27, Aleksandr Rybalko wrote: > >>> Hello hackers! > >>> > >>> Here is link to the patch[1] for vidcontrol that makes it to know if it > >>> run w/ or w/o vt(4) and if vt(4) is present, then: > >>> 1. screen map feature disabled (vt(4) use Unicode, so screen map not > >>> needed). > >>> 2. enable to load fornt from /usr/share/vt/fonts/ dir. PLease put > >>> "gallant" font[2] to /usr/share/vt/fonts/. > >>> > >>> Looks like it works fine, but maybe I forgot something :) > >>> So please test it in your own environment. > >>> > >>> Big thanks to Ed for preparing that font file! > >>> > >>> 1. > >>> http://people.freebsd.org/~ray/newcons/vidcontrol_for_vt_2014-03-11.patch > >>> 2. http://people.freebsd.org/~emaste/newcons/gallant.fnt > >>> > >>> Thanks! > >>> > >>> WBW > >>> > >> > >> Hi, > >> > >> I applied this patch on a 10.0-STABLE r264390 with a Radeon Mobility X300 > >> (M22) > >> 5460, and DRM2. > >> > >> I could load an home made terminus 16x32 font (my eyes are too old to work > >> with > >> a 8x16 font on a 1920x1200 17" screen), by > >> > >> - looping on each ttyvN in a rc.local script > >> > >> or > >> > >> - adding allscreens_flags="-f terminus-u32" to rc.conf (for > >> /etc/rc.d/syscons > >> consumption) > > > > Cool! > > Are you like to share that font? > > > > Off course. > In fact I generated 12x24, 14x28 and 16x32 fonts from terminus-font-4.38 by > using the tools/tools/vt/fontcvt utility. > I will send you these fonts offlist (because the FreeBSD lists do not like > attachments). > > >> > >> I just discovered than scrolling back in console mode works ONLY on ttyv0. > > > > Huh, it's surprising me. Before your mail I think we have problem with > > scrollback on vty0, but not on others. :) > > Looks like I have to concentrate and fix them all at once. > > > > I confirm that scrollback works on ttyv0, but not on others ttyv when a font > is > loaded. > > >> > >> What I have to do to get scrolling on every ttyvN ? > >> > >> The only way to get the system working in normal VGA mode (640x480) (not > >> loading > >> the drm2 and radeon kms modules by loader.conf) is by configuring the BIOS > >> to > >> not do display expansion - which leads to the same ridiculously small > >> font.. > >> > >> And of course, kbdmux keeps being mandatory to be able to load a keymap. > > > > Yeah, I still remember. :) > > > > This could be noted in the newly born vt(4) man page .. > > >> > >> TIA > >> > >> Claude Buisson > >> > > > > Many thanks Claude! > > > > WBW > > > > CBu > Hello Claude! Looks like nobody care about this patch, only you did test for it :) So I commit it to HEAD, if you need it to be MFCed, I will do it a week later. Many thanks for your help!!! WBW -- Aleksandr Rybalko ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: Memory blackhole in 11. Possibly libc.so.7?
On Thursday, May 29, 2014 5:41:03 am Beeblebrox wrote: > uname: FreeBSD 11.0-CURRENT #0 r266393M: Sun May 18 13:04:00 2014 amd64 > I'm also loading the Radeon_kms modules > > Upon system startup, memory profile is clean. I get locked memory (mem_wire) > usage as: > 9%before Radeon*.ko modules loaded > 12% when slim is started (loads Radeon*.ko modules) > 14% after I login through slim > These are quite normal. However, after some usage (I don't know what > exactly) mem_free drops significantly and causes display problems (Radeon > has issues when memory falls very low, if I remember correctly). > > Upon observing low level of mem_free, I went to single user (shutdown now), > also kldunload any modules not compiled into kernel (using custom built > kernel). Final kldstat: > Id Refs AddressSize Name > 1 99 0x8020 941c18 kernel > 21 0x80b42000 b0f0 linprocfs.ko > 32 0x80b4e000 45ba0linux.ko > 41 0x80b94000 2a38 linsysfs.ko > 51 0x80b97000 243fc8 zfs.ko > 61 0x80ddb000 2940 acl_nfs4.ko > 72 0x80dde000 5d10 opensolaris.ko > 81 0x80de4000 40a50snd_hda.ko > 92 0x80e25000 756a0sound.ko > 101 0x80e9b000 7550 umass.ko > 115 0x80ea3000 3f7b0usb.ko > 131 0x80f32000 19758ext2fs.ko > 141 0x80f4c000 d050 ehci.ko > 151 0x80f5a000 a3f0 ohci.ko > 161 0x81011000 1adb ums.ko > 171 0x81013000 951 pflog.ko > 181 0x81014000 27641pf.ko > 191 0x8103c000 108ffc radeonkms.ko > 201 0x81145000 3fd39drm2.ko > 212 0x81185000 ae68 agp.ko > 224 0x8119 1776 iicbus.ko > 231 0x81192000 d1a iic.ko > 241 0x81193000 167d iicbb.ko > 254 0x81195000 1bde firmware.ko > 261 0x81197000 ac6 radeonkmsfw_RS780_pfp.ko > 271 0x81198000 55c6 radeonkmsfw_RS780_me.ko > 281 0x8119e000 dc6 radeonkmsfw_R600_rlc.ko > 311 0x811d6000 9118 netgraph.ko > 321 0x811e 160a ng_ether.ko > > But look at memory (70%), even though there are absolutely no processes > running. SYSTEM MEMORY INFORMATION: > mem_wire:2735349760 ( 2608MB) [ 70%] Wired: disabled for paging > out > mem_active: + 8638464 ( 8MB) [ 0%] Active: recently referenced > mem_inactive:+137596928 (131MB) [ 3%] Inactive: recently not > referenced > mem_cache: + 34885632 ( 33MB) [ 0%] Cached: almost avail. for > allocation > mem_free:+970948608 (925MB) [ 24%] Free: fully available for > allocation > mem_gap_vm: + 413696 ( 0MB) [ 0%] Memory gap: UNKNOWN > -- --- -- > mem_all: = 3887833088 ( 3707MB) [100%] Total real memory managed > mem_gap_sys: +123482112 (117MB)Memory gap: Kernel?! > -- --- > mem_phys:= 4011315200 ( 3825MB)Total real memory available > mem_gap_hw: +283652096 (270MB)Memory gap: Segment > Mappings?! > -- --- > mem_hw: = 4294967296 ( 4096MB)Total real memory installed > > SYSTEM MEMORY SUMMARY: > mem_used:3151536128 ( 3005MB) [ 73%] Logically used memory > mem_avail: + 1143431168 ( 1090MB) [ 26%] Logically available memory > -- --- -- > mem_total: = 4294967296 ( 4096MB) [100%] Logically total memory > > I don't know if the lsof dump in single user mode will be of any help, but > it seems like lib/libc.so.7 has something to do with it: Why do you think libc.so.7 has anything to do with this? wired memory is usually only allocated in the kernel, so if anything you are looking at some sort of memory leak in a driver (maybe the radeon driver?) -- John Baldwin ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: Memory blackhole in 11. Possibly libc.so.7?
On Thu, 29 May 2014 09:08:10 -0400 John Baldwin wrote: > On Thursday, May 29, 2014 5:41:03 am Beeblebrox wrote: > > uname: FreeBSD 11.0-CURRENT #0 r266393M: Sun May 18 13:04:00 2014 > > amd64 I'm also loading the Radeon_kms modules > > > > > > I don't know if the lsof dump in single user mode will be of any > > help, but it seems like lib/libc.so.7 has something to do with it: > > > Why do you think libc.so.7 has anything to do with this? wired > memory is usually only allocated in the kernel, so if anything you > are looking at some sort of memory leak in a driver (maybe the radeon > driver?) > > -- > John Baldwin Aren't ZFS buffers accounted as kernel wired memory? -- Alexander Kabaev signature.asc Description: PGP signature
Re: official pkg repo with WITHOUT_X11=true
On 2014-05-29 08:42, Bryan Drewery wrote: > >> On May 29, 2014, at 4:19, David Chisnall wrote: >> >>> On 29 May 2014, at 02:23, Bryan Drewery wrote: >>> >>> As for skipping unneeded ports the best I can do is '-a' or "Build it all". >>> If a port is only needed for WITH_X11 then an IGNORE should be added to it >>> when WITHOUT_X11 is set to prevent wasting time on it. >> >> We can probably do a bit better by looking at the complete dependency graph >> and removing any ports that have unconditional dependencies on X. For a >> headless server, there's no reason to build any of the kde-* or gnome-* >> ports or, indeed, X itself. I suspect that we could easily trim 2/3 of the >> build time by omitting ports that have a GUI, GUI toolkits, and so on. > > Yeah. My point was more that poudriere can't do that now and I would rather > not add all that special-case logic to it. Clever make.conf logic might be > able to do it. > >> >> Longer term, we may be able to share the build time a bit. Ports which >> don't have a WITHOUT_X11 flag and don't unconditionally depend on X11 can >> potentially be pre-seeded from the normal package build (if we can identify >> them). That only leaves the ports that actually have build-time conditional >> X support to build in the no-Xorg run. > > Yup! I have a patch for that in the works. > That would be a great improvement for the 'sets' feature in poudriere. Almost all of my different sets have some overlap. Although this would either require the 'queue' system you are working on, to say build sets X, Y, and Z, and if there is any overlap share it. Or, augment the set feature with an additional flag to specify a 'parent' set or something, -z myset -Z commonset to tell it an already built set where it can steal packages from. >> >> David > ___ > freebsd-current@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-current > To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org" > -- Allan Jude signature.asc Description: OpenPGP digital signature
Re: Memory blackhole in 11. Possibly libc.so.7?
On 2014-05-29 09:57, Alexander Kabaev wrote: > On Thu, 29 May 2014 09:08:10 -0400 > John Baldwin wrote: > >> On Thursday, May 29, 2014 5:41:03 am Beeblebrox wrote: >>> uname: FreeBSD 11.0-CURRENT #0 r266393M: Sun May 18 13:04:00 2014 >>> amd64 I'm also loading the Radeon_kms modules >>> > >>> >>> I don't know if the lsof dump in single user mode will be of any >>> help, but it seems like lib/libc.so.7 has something to do with it: >> >> >> Why do you think libc.so.7 has anything to do with this? wired >> memory is usually only allocated in the kernel, so if anything you >> are looking at some sort of memory leak in a driver (maybe the radeon >> driver?) >> >> -- >> John Baldwin > > Aren't ZFS buffers accounted as kernel wired memory? > > -- > Alexander Kabaev > Yes, the ZFS ARC cache is counted as wired memory. You might want to consider setting a lower vfs.zfs.arc_max in /boot/loader.conf The default is all memory less 1GB, or 95% of memory, whichever is more, and this is likely too high for your usage case. It is infact too high for every use case except a server that does nothing but act as a file server. -- Allan Jude signature.asc Description: OpenPGP digital signature
Re: Memory blackhole in 11. Possibly libc.so.7?
>> Why do you think libc.so.7 has anything to do with this? Well, because there are two instances of it running in the lsof dump, with several possible "child process" candidates. Why would they be hanging around when practically everything has been killed? Radeon*.ko modules are a very strong possibility, as I knew before I posted. Unfortunately I cannot kldunload those modules once laded because then tty's go dark and I must reboot. I send the Radeon developer some of my crash reports once and a while, so nothing to add there. Have not heard from him in a while, and I d't want to disturb him unnecessarily. And yes, as Alexander points out, zfs is also a possibility. - FreeBSD-11-current_amd64_root-on-zfs_RadeonKMS -- View this message in context: http://freebsd.1045724.n5.nabble.com/Memory-blackhole-in-11-Possibly-libc-so-7-tp5916161p5916224.html Sent from the freebsd-current mailing list archive at Nabble.com. ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: RFT vidcontrol for vt(4)
On 05/29/2014 15:17, Aleksandr Rybalko wrote:
On Mon, 12 May 2014 17:10:54 +0200
Claude Buisson wrote:
On 05/12/2014 16:14, Aleksandr Rybalko wrote:
On Mon, 12 May 2014 15:35:48 +0200
Claude Buisson wrote:
On 03/11/2014 15:27, Aleksandr Rybalko wrote:
Hello hackers!
Here is link to the patch[1] for vidcontrol that makes it to know if it
run w/ or w/o vt(4) and if vt(4) is present, then:
1. screen map feature disabled (vt(4) use Unicode, so screen map not
needed).
2. enable to load fornt from /usr/share/vt/fonts/ dir. PLease put
"gallant" font[2] to /usr/share/vt/fonts/.
Looks like it works fine, but maybe I forgot something :)
So please test it in your own environment.
Big thanks to Ed for preparing that font file!
1.
http://people.freebsd.org/~ray/newcons/vidcontrol_for_vt_2014-03-11.patch
2. http://people.freebsd.org/~emaste/newcons/gallant.fnt
Thanks!
WBW
Hi,
I applied this patch on a 10.0-STABLE r264390 with a Radeon Mobility X300 (M22)
5460, and DRM2.
I could load an home made terminus 16x32 font (my eyes are too old to work with
a 8x16 font on a 1920x1200 17" screen), by
- looping on each ttyvN in a rc.local script
or
- adding allscreens_flags="-f terminus-u32" to rc.conf (for /etc/rc.d/syscons
consumption)
Cool!
Are you like to share that font?
Off course.
In fact I generated 12x24, 14x28 and 16x32 fonts from terminus-font-4.38 by
using the tools/tools/vt/fontcvt utility.
I will send you these fonts offlist (because the FreeBSD lists do not like
attachments).
I just discovered than scrolling back in console mode works ONLY on ttyv0.
Huh, it's surprising me. Before your mail I think we have problem with
scrollback on vty0, but not on others. :)
Looks like I have to concentrate and fix them all at once.
I confirm that scrollback works on ttyv0, but not on others ttyv when a font is
loaded.
What I have to do to get scrolling on every ttyvN ?
The only way to get the system working in normal VGA mode (640x480) (not loading
the drm2 and radeon kms modules by loader.conf) is by configuring the BIOS to
not do display expansion - which leads to the same ridiculously small font..
And of course, kbdmux keeps being mandatory to be able to load a keymap.
Yeah, I still remember. :)
This could be noted in the newly born vt(4) man page ..
TIA
Claude Buisson
Many thanks Claude!
WBW
CBu
Hello Claude!
Looks like nobody care about this patch, only you did test for it :)
So I commit it to HEAD, if you need it to be MFCed, I will do it a week
later.
Many thanks for your help!!!
WBW
Hello !
I (sort of) solved the scroll problem by patching
sys/dev/vt/font/vt_font_default.c
to compile my 16x32 font into the kernel.
Clearly some more work is needed to make the modified vidcontrol a finished
utility and to have the proper rc.conf knob to load a custom font at boot.
And it must also be possible to generate a kernel with a custom font, without
having to patch the source.
I also found that
tools/tools/vt/mkkfont/mkkfont.c
does not put a comma after the final "}" of .vf_map, so one must apply the
attached patch.
Keep the work going !
CBu
--- tools/tools/vt/mkkfont/mkkfont.c.orig 2014-03-06 19:30:56.0
+0100
+++ tools/tools/vt/mkkfont/mkkfont.c2014-05-15 18:45:42.0 +0200
@@ -139,7 +139,7 @@
else
printf("\t\t\t\tNULL,\n");
}
- printf("\t\t\t\t }\n");
+ printf("\t\t\t\t },\n");
printf("\t.vf_map_count\t\t= { %u, %u, %u, %u },\n",
be32toh(fh->map_count[0]),
be32toh(fh->map_count[1]),
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: Memory blackhole in 11. Possibly libc.so.7?
On Thursday, May 29, 2014 9:57:22 am Alexander Kabaev wrote: > On Thu, 29 May 2014 09:08:10 -0400 > John Baldwin wrote: > > > On Thursday, May 29, 2014 5:41:03 am Beeblebrox wrote: > > > uname: FreeBSD 11.0-CURRENT #0 r266393M: Sun May 18 13:04:00 2014 > > > amd64 I'm also loading the Radeon_kms modules > > > > > > > > > > I don't know if the lsof dump in single user mode will be of any > > > help, but it seems like lib/libc.so.7 has something to do with it: > > > > > > Why do you think libc.so.7 has anything to do with this? wired > > memory is usually only allocated in the kernel, so if anything you > > are looking at some sort of memory leak in a driver (maybe the radeon > > driver?) > > > > -- > > John Baldwin > > Aren't ZFS buffers accounted as kernel wired memory? Ah, yes, using ZFS this might be "normal". -- John Baldwin ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: RFT vidcontrol for vt(4)
On Thu, May 29, 2014 at 7:56 AM, Claude Buisson wrote: > On 05/29/2014 15:17, Aleksandr Rybalko wrote: > >> On Mon, 12 May 2014 17:10:54 +0200 >> Claude Buisson wrote: >> >> On 05/12/2014 16:14, Aleksandr Rybalko wrote: >>> On Mon, 12 May 2014 15:35:48 +0200 Claude Buisson wrote: On 03/11/2014 15:27, Aleksandr Rybalko wrote: > >> Hello hackers! >> >> Here is link to the patch[1] for vidcontrol that makes it to know if >> it >> run w/ or w/o vt(4) and if vt(4) is present, then: >> 1. screen map feature disabled (vt(4) use Unicode, so screen map not >> needed). >> 2. enable to load fornt from /usr/share/vt/fonts/ dir. PLease put >> "gallant" font[2] to /usr/share/vt/fonts/. >> >> Looks like it works fine, but maybe I forgot something :) >> So please test it in your own environment. >> >> Big thanks to Ed for preparing that font file! >> >> 1. >> http://people.freebsd.org/~ray/newcons/vidcontrol_for_vt_ >> 2014-03-11.patch >> 2. http://people.freebsd.org/~emaste/newcons/gallant.fnt >> >> Thanks! >> >> WBW >> >> > Hi, > > I applied this patch on a 10.0-STABLE r264390 with a Radeon Mobility > X300 (M22) > 5460, and DRM2. > > I could load an home made terminus 16x32 font (my eyes are too old to > work with > a 8x16 font on a 1920x1200 17" screen), by > > - looping on each ttyvN in a rc.local script > > or > > - adding allscreens_flags="-f terminus-u32" to rc.conf (for > /etc/rc.d/syscons > consumption) > Cool! Are you like to share that font? >>> Off course. >>> In fact I generated 12x24, 14x28 and 16x32 fonts from terminus-font-4.38 >>> by >>> using the tools/tools/vt/fontcvt utility. >>> I will send you these fonts offlist (because the FreeBSD lists do not >>> like >>> attachments). >>> >>> > I just discovered than scrolling back in console mode works ONLY on > ttyv0. > Huh, it's surprising me. Before your mail I think we have problem with scrollback on vty0, but not on others. :) Looks like I have to concentrate and fix them all at once. >>> I confirm that scrollback works on ttyv0, but not on others ttyv when a >>> font is >>> loaded. >>> >>> > What I have to do to get scrolling on every ttyvN ? > > The only way to get the system working in normal VGA mode (640x480) > (not loading > the drm2 and radeon kms modules by loader.conf) is by configuring the > BIOS to > not do display expansion - which leads to the same ridiculously small > font.. > > And of course, kbdmux keeps being mandatory to be able to load a > keymap. > Yeah, I still remember. :) >>> This could be noted in the newly born vt(4) man page .. >>> >>> > TIA > > Claude Buisson > > Many thanks Claude! WBW >>> CBu >>> >>> >> Hello Claude! >> >> Looks like nobody care about this patch, only you did test for it :) >> So I commit it to HEAD, if you need it to be MFCed, I will do it a week >> later. >> >> Many thanks for your help!!! >> >> WBW >> >> > Hello ! > > I (sort of) solved the scroll problem by patching > sys/dev/vt/font/vt_font_default.c > to compile my 16x32 font into the kernel. > > Clearly some more work is needed to make the modified vidcontrol a finished > utility and to have the proper rc.conf knob to load a custom font at boot. > And it must also be possible to generate a kernel with a custom font, > without > having to patch the source. > > I also found that > tools/tools/vt/mkkfont/mkkfont.c > does not put a comma after the final "}" of .vf_map, so one must apply the > attached patch. > > Keep the work going ! > > CBu > Actually, I did test vidcontrol patches, but everything just seemed to work fine, so I failed to report anything. I guess reporting that nothing happened is a bit boring, but still important. -- R. Kevin Oberman, Network Engineer, Retired E-mail: rkober...@gmail.com ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: Memory blackhole in 11. Possibly libc.so.7?
On 2014-05-29 12:04, John Baldwin wrote: > On Thursday, May 29, 2014 9:57:22 am Alexander Kabaev wrote: >> On Thu, 29 May 2014 09:08:10 -0400 >> John Baldwin wrote: >> >>> On Thursday, May 29, 2014 5:41:03 am Beeblebrox wrote: uname: FreeBSD 11.0-CURRENT #0 r266393M: Sun May 18 13:04:00 2014 amd64 I'm also loading the Radeon_kms modules >> I don't know if the lsof dump in single user mode will be of any help, but it seems like lib/libc.so.7 has something to do with it: >>> >>> >>> Why do you think libc.so.7 has anything to do with this? wired >>> memory is usually only allocated in the kernel, so if anything you >>> are looking at some sort of memory leak in a driver (maybe the radeon >>> driver?) >>> >>> -- >>> John Baldwin >> >> Aren't ZFS buffers accounted as kernel wired memory? > > Ah, yes, using ZFS this might be "normal". > sysctl kstat.zfs.misc.arcstats.size will tell you how much memory the ARC is using, it will likely be 'most' of the wired memory. 'top' gives a decent display as well: Mem: 8896K Active, 205M Inact, 5764M Wired, 1926M Free ARC: 2939M Total, 39M MFU, 2182M MRU, 1936K Anon, 136M Header, 581M Other -- Allan Jude signature.asc Description: OpenPGP digital signature
system data diagram
Hey all, http://www.brendangregg.com/Perf/linuxperftools.png is a neat diagram. Who wants to make one for FreeBSD? It'll even make its way to our website & documentation :) -- Eitan Adler ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: Processor cores not properly detected/activated?
On 28 May 2014 10:58, John Baldwin wrote: > On Wednesday, May 28, 2014 1:51:28 pm Adrian Chadd wrote: >> On 28 May 2014 06:56, John Baldwin wrote: >> >> >> > Userland cpusets only default to 128 (CPU_MAXSIZE in ). >> > Changing MAXCPU to even 128 is unfortunately a potential KBI change since >> > it >> > changes the size of 'cpuset_t'. We can certainly bump these in HEAD for >> > 11, >> > but we might not be able to MFC them without introducing ABI breakage. >> > (The cpuset APIs do allow the size of cpuset_t to change as the size is >> > encoded in the API calls, so there is that, it's more that if some public >> > structure embeds a cpuset_t in the kernel that we would have problems. I >> > thought 'struct pcpu' did, but it does not.) >> > >> > Hmm, smp_rendezvous() accepts a cpuset_t as its first argument (and is a >> > public symbol used by kernel modules such as dtrace). 'struct rmlock' also >> > embeds a cpuset_t. So, I think we can't bump cpuset_t without breaking >> > the KBI. We can bump it in HEAD however. (Note, if re@ signed off, we >> > could >> > perhaps merge to 10, but we tend to be very hesitant about breaking the >> > KBI.) >> > One thing we could do safely is bump the userland cpuset size to 256 in 10. >> > It's really only MAXCPU that is problematic. >> > >> > In particular, I propose we bump the userland cpuset_t size to 256 now (and >> > go ahead and merge that to 10). In HEAD only we can bump MAXCPU for amd64 >> > to 256. >> >> Since 11 is going to be around for a few years, can we experiment >> bumping it up to something compute-cluster-computer-sized just to get >> it over with? Something stupid, like 4096 or something? > > It costs wired memory to increase it for the kernel. The userland set size > can be increased rather arbitrarily, so we don't need to make it but so large > as it is easy to bump later (even with a branch). Well, what about making the API/KBI use cpuset_t pointers for things rather than including it as a bitmask? Do you think there'd be a noticable performance overhead for the bits where it's indirecting through a pointer to get to the bitmask data? -a ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: Memory blackhole in 11. Possibly libc.so.7?
On Thursday, May 29, 2014 10:42:18 am Beeblebrox wrote: > >> Why do you think libc.so.7 has anything to do with this? > > Well, because there are two instances of it running in the lsof dump, with > several possible "child process" candidates. Why would they be hanging > around when practically everything has been killed? Everything that is dynamically pulls in libc.so.7. This includes /bin/sh, etc. However, those mappings are not _wired_ memory. Those would just be active or inactive pages. > Radeon*.ko modules are a very strong possibility, as I knew before I posted. > Unfortunately I cannot kldunload those modules once laded because then tty's > go dark and I must reboot. I send the Radeon developer some of my crash > reports once and a while, so nothing to add there. Have not heard from him > in a while, and I d't want to disturb him unnecessarily. > > And yes, as Alexander points out, zfs is also a possibility. ZFS is almost certainly the cause of the wired memory usage. -- John Baldwin ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: Processor cores not properly detected/activated?
On Thursday, May 29, 2014 12:53:54 pm Adrian Chadd wrote: > On 28 May 2014 10:58, John Baldwin wrote: > > On Wednesday, May 28, 2014 1:51:28 pm Adrian Chadd wrote: > >> On 28 May 2014 06:56, John Baldwin wrote: > >> > >> > >> > Userland cpusets only default to 128 (CPU_MAXSIZE in ). > >> > Changing MAXCPU to even 128 is unfortunately a potential KBI change > >> > since it > >> > changes the size of 'cpuset_t'. We can certainly bump these in HEAD for > >> > 11, > >> > but we might not be able to MFC them without introducing ABI breakage. > >> > (The cpuset APIs do allow the size of cpuset_t to change as the size is > >> > encoded in the API calls, so there is that, it's more that if some public > >> > structure embeds a cpuset_t in the kernel that we would have problems. I > >> > thought 'struct pcpu' did, but it does not.) > >> > > >> > Hmm, smp_rendezvous() accepts a cpuset_t as its first argument (and is a > >> > public symbol used by kernel modules such as dtrace). 'struct rmlock' > >> > also > >> > embeds a cpuset_t. So, I think we can't bump cpuset_t without breaking > >> > the KBI. We can bump it in HEAD however. (Note, if re@ signed off, we > >> > could > >> > perhaps merge to 10, but we tend to be very hesitant about breaking the > >> > KBI.) > >> > One thing we could do safely is bump the userland cpuset size to 256 in > >> > 10. > >> > It's really only MAXCPU that is problematic. > >> > > >> > In particular, I propose we bump the userland cpuset_t size to 256 now > >> > (and > >> > go ahead and merge that to 10). In HEAD only we can bump MAXCPU for > >> > amd64 > >> > to 256. > >> > >> Since 11 is going to be around for a few years, can we experiment > >> bumping it up to something compute-cluster-computer-sized just to get > >> it over with? Something stupid, like 4096 or something? > > > > It costs wired memory to increase it for the kernel. The userland set size > > can be increased rather arbitrarily, so we don't need to make it but so > > large > > as it is easy to bump later (even with a branch). > > Well, what about making the API/KBI use cpuset_t pointers for things > rather than including it as a bitmask? Do you think there'd be a > noticable performance overhead for the bits where it's indirecting > through a pointer to get to the bitmask data? The wired memory is not due to cpuset_t. The wired memory usage is due to things that do 'struct foo foo_bits[MAXCPU]'. The KBI issues I mentioned above are 'struct rmlock' (so now you want any rmlock users to malloc space, or you want rmlock_init() call malloc? (that seems like a bad idea)). The other one is smp_rendezvous. Plus, it's not just a pointer, you really need a (pointer, size_t) tuple similar to what cpuset_getaffinity(), etc. use. -- John Baldwin ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: Processor cores not properly detected/activated?
On 29 May 2014 10:18, John Baldwin wrote: >> > It costs wired memory to increase it for the kernel. The userland set size >> > can be increased rather arbitrarily, so we don't need to make it but so >> > large >> > as it is easy to bump later (even with a branch). >> >> Well, what about making the API/KBI use cpuset_t pointers for things >> rather than including it as a bitmask? Do you think there'd be a >> noticable performance overhead for the bits where it's indirecting >> through a pointer to get to the bitmask data? > > The wired memory is not due to cpuset_t. The wired memory usage is due to > things > that do 'struct foo foo_bits[MAXCPU]'. The KBI issues I mentioned above are > 'struct rmlock' (so now you want any rmlock users to malloc space, or you > want rmlock_init() call malloc? (that seems like a bad idea)). The other one > is smp_rendezvous. Plus, it's not just a pointer, you really need a (pointer, > size_t) tuple similar to what cpuset_getaffinity(), etc. use. Why would calling malloc be a problem? Except for the initial setup of things, anything dynamically allocating structs with embedded things like rmlocks are already dynamically allocating them via malloc or uma. There's a larger fundamental problem with malloc, fragmentation and getting the required larger allocations for things. But even a 4096 CPU box would require a 512 byte malloc. That shouldn't be that hard to do. It'd just be from some memory that isn't close to the rest of the lock state. -a ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: Processor cores not properly detected/activated?
On Thursday, May 29, 2014 2:24:45 pm Adrian Chadd wrote: > On 29 May 2014 10:18, John Baldwin wrote: > > >> > It costs wired memory to increase it for the kernel. The userland set > >> > size > >> > can be increased rather arbitrarily, so we don't need to make it but so > >> > large > >> > as it is easy to bump later (even with a branch). > >> > >> Well, what about making the API/KBI use cpuset_t pointers for things > >> rather than including it as a bitmask? Do you think there'd be a > >> noticable performance overhead for the bits where it's indirecting > >> through a pointer to get to the bitmask data? > > > > The wired memory is not due to cpuset_t. The wired memory usage is due to > > things > > that do 'struct foo foo_bits[MAXCPU]'. The KBI issues I mentioned above are > > 'struct rmlock' (so now you want any rmlock users to malloc space, or you > > want rmlock_init() call malloc? (that seems like a bad idea)). The other > > one > > is smp_rendezvous. Plus, it's not just a pointer, you really need a > > (pointer, > > size_t) tuple similar to what cpuset_getaffinity(), etc. use. > > Why would calling malloc be a problem? Except for the initial setup of > things, anything dynamically allocating structs with embedded things > like rmlocks are already dynamically allocating them via malloc or > uma. > > There's a larger fundamental problem with malloc, fragmentation and > getting the required larger allocations for things. But even a 4096 > CPU box would require a 512 byte malloc. That shouldn't be that hard > to do. It'd just be from some memory that isn't close to the rest of > the lock state. Other similar APIs like mtx_init() don't call malloc(), so it would be unusual behavior. However, we have several other problems before we can move beyond 256 anyway (like pf). -- John Baldwin ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: Processor cores not properly detected/activated?
On Thu, May 29, 2014 at 02:44:19PM -0400, John Baldwin wrote: > On Thursday, May 29, 2014 2:24:45 pm Adrian Chadd wrote: > > On 29 May 2014 10:18, John Baldwin wrote: > > > > >> > It costs wired memory to increase it for the kernel. The userland set > > >> > size > > >> > can be increased rather arbitrarily, so we don't need to make it but > > >> > so large > > >> > as it is easy to bump later (even with a branch). > > >> > > >> Well, what about making the API/KBI use cpuset_t pointers for things > > >> rather than including it as a bitmask? Do you think there'd be a > > >> noticable performance overhead for the bits where it's indirecting > > >> through a pointer to get to the bitmask data? > > > > > > The wired memory is not due to cpuset_t. The wired memory usage is due > > > to things > > > that do 'struct foo foo_bits[MAXCPU]'. The KBI issues I mentioned above > > > are > > > 'struct rmlock' (so now you want any rmlock users to malloc space, or you > > > want rmlock_init() call malloc? (that seems like a bad idea)). The > > > other one > > > is smp_rendezvous. Plus, it's not just a pointer, you really need a > > > (pointer, > > > size_t) tuple similar to what cpuset_getaffinity(), etc. use. > > > > Why would calling malloc be a problem? Except for the initial setup of > > things, anything dynamically allocating structs with embedded things > > like rmlocks are already dynamically allocating them via malloc or > > uma. > > > > There's a larger fundamental problem with malloc, fragmentation and > > getting the required larger allocations for things. But even a 4096 > > CPU box would require a 512 byte malloc. That shouldn't be that hard > > to do. It'd just be from some memory that isn't close to the rest of > > the lock state. > > Other similar APIs like mtx_init() don't call malloc(), so it would be > unusual behavior. However, we have several other problems before we can > move beyond 256 anyway (like pf). What is pf ? We definitely have a problem with the legacy APIC mode, we must start using x2APIC, I believe. pgpX1AfZL6bKi.pgp Description: PGP signature
cpuid_t typedef? (was Re: Processor cores not properly detected/activated?)
On 29 May 2014 11:44, John Baldwin wrote: > On Thursday, May 29, 2014 2:24:45 pm Adrian Chadd wrote: >> On 29 May 2014 10:18, John Baldwin wrote: >> >> >> > It costs wired memory to increase it for the kernel. The userland set >> >> > size >> >> > can be increased rather arbitrarily, so we don't need to make it but so >> >> > large >> >> > as it is easy to bump later (even with a branch). >> >> >> >> Well, what about making the API/KBI use cpuset_t pointers for things >> >> rather than including it as a bitmask? Do you think there'd be a >> >> noticable performance overhead for the bits where it's indirecting >> >> through a pointer to get to the bitmask data? >> > >> > The wired memory is not due to cpuset_t. The wired memory usage is due to >> > things >> > that do 'struct foo foo_bits[MAXCPU]'. The KBI issues I mentioned above >> > are >> > 'struct rmlock' (so now you want any rmlock users to malloc space, or you >> > want rmlock_init() call malloc? (that seems like a bad idea)). The other >> > one >> > is smp_rendezvous. Plus, it's not just a pointer, you really need a >> > (pointer, >> > size_t) tuple similar to what cpuset_getaffinity(), etc. use. >> >> Why would calling malloc be a problem? Except for the initial setup of >> things, anything dynamically allocating structs with embedded things >> like rmlocks are already dynamically allocating them via malloc or >> uma. >> >> There's a larger fundamental problem with malloc, fragmentation and >> getting the required larger allocations for things. But even a 4096 >> CPU box would require a 512 byte malloc. That shouldn't be that hard >> to do. It'd just be from some memory that isn't close to the rest of >> the lock state. > > Other similar APIs like mtx_init() don't call malloc(), so it would be > unusual behavior. However, we have several other problems before we can > move beyond 256 anyway (like pf). Maybe behaviour has to change over time. :( anyway. Besides all of this - I'm thinking of just introducing: typedef uint32_t cpuid_t; .. then once we've converted all the users, we can make NOCPU something other than 255 (which is the other limiting factor here..) Any objections? -a ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: RFT vidcontrol for vt(4)
On Thu, 29 May 2014 09:11:04 -0700 Kevin Oberman wrote: > On Thu, May 29, 2014 at 7:56 AM, Claude Buisson > wrote: > > > On 05/29/2014 15:17, Aleksandr Rybalko wrote: > > > >> On Mon, 12 May 2014 17:10:54 +0200 > >> Claude Buisson wrote: > >> > >> On 05/12/2014 16:14, Aleksandr Rybalko wrote: > >>> > On Mon, 12 May 2014 15:35:48 +0200 > Claude Buisson wrote: > > On 03/11/2014 15:27, Aleksandr Rybalko wrote: > > > >> Hello hackers! > >> > >> Here is link to the patch[1] for vidcontrol that makes it to > >> know if it > >> run w/ or w/o vt(4) and if vt(4) is present, then: > >> 1. screen map feature disabled (vt(4) use Unicode, so screen > >> map not needed). > >> 2. enable to load fornt from /usr/share/vt/fonts/ dir. PLease > >> put "gallant" font[2] to /usr/share/vt/fonts/. > >> > >> Looks like it works fine, but maybe I forgot something :) > >> So please test it in your own environment. > >> > >> Big thanks to Ed for preparing that font file! > >> > >> 1. > >> http://people.freebsd.org/~ray/newcons/vidcontrol_for_vt_ > >> 2014-03-11.patch > >> 2. http://people.freebsd.org/~emaste/newcons/gallant.fnt > >> > >> Thanks! > >> > >> WBW > >> > >> > > Hi, > > > > I applied this patch on a 10.0-STABLE r264390 with a Radeon > > Mobility X300 (M22) > > 5460, and DRM2. > > > > I could load an home made terminus 16x32 font (my eyes are too > > old to work with > > a 8x16 font on a 1920x1200 17" screen), by > > > > - looping on each ttyvN in a rc.local script > > > > or > > > > - adding allscreens_flags="-f terminus-u32" to rc.conf (for > > /etc/rc.d/syscons > > consumption) > > > > Cool! > Are you like to share that font? > > > >>> Off course. > >>> In fact I generated 12x24, 14x28 and 16x32 fonts from > >>> terminus-font-4.38 by > >>> using the tools/tools/vt/fontcvt utility. > >>> I will send you these fonts offlist (because the FreeBSD lists do > >>> not like > >>> attachments). > >>> > >>> > > I just discovered than scrolling back in console mode works > > ONLY on ttyv0. > > > > Huh, it's surprising me. Before your mail I think we have > problem with scrollback on vty0, but not on others. :) > Looks like I have to concentrate and fix them all at once. > > > >>> I confirm that scrollback works on ttyv0, but not on others ttyv > >>> when a font is > >>> loaded. > >>> > >>> > > What I have to do to get scrolling on every ttyvN ? > > > > The only way to get the system working in normal VGA mode > > (640x480) (not loading > > the drm2 and radeon kms modules by loader.conf) is by > > configuring the BIOS to > > not do display expansion - which leads to the same ridiculously > > small font.. > > > > And of course, kbdmux keeps being mandatory to be able to load a > > keymap. > > > > Yeah, I still remember. :) > > > >>> This could be noted in the newly born vt(4) man page .. > >>> > >>> > > TIA > > > > Claude Buisson > > > > > Many thanks Claude! > > WBW > > > >>> CBu > >>> > >>> > >> Hello Claude! > >> > >> Looks like nobody care about this patch, only you did test for > >> it :) So I commit it to HEAD, if you need it to be MFCed, I will > >> do it a week later. > >> > >> Many thanks for your help!!! > >> > >> WBW > >> > >> > > Hello ! > > > > I (sort of) solved the scroll problem by patching > > sys/dev/vt/font/vt_font_default.c > > to compile my 16x32 font into the kernel. > > > > Clearly some more work is needed to make the modified vidcontrol a > > finished utility and to have the proper rc.conf knob to load a > > custom font at boot. And it must also be possible to generate a > > kernel with a custom font, without > > having to patch the source. > > > > I also found that > > tools/tools/vt/mkkfont/mkkfont.c > > does not put a comma after the final "}" of .vf_map, so one must > > apply the attached patch. > > > > Keep the work going ! > > > > CBu > > > > Actually, I did test vidcontrol patches, but everything just seemed > to work fine, so I failed to report anything. I guess reporting that > nothing happened is a bit boring, but still important. > > -- > R. Kevin Oberman, Network Engineer, Retired > E-mail: rkober...@gmail.com Late is better than never :) Thanks Kevin! WBW -- Aleksandr Rybalko ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: cpuid_t typedef? (was Re: Processor cores not properly detected/activated?)
On Thursday, May 29, 2014 4:05:49 pm Adrian Chadd wrote: > On 29 May 2014 11:44, John Baldwin wrote: > > On Thursday, May 29, 2014 2:24:45 pm Adrian Chadd wrote: > >> On 29 May 2014 10:18, John Baldwin wrote: > >> > >> >> > It costs wired memory to increase it for the kernel. The userland > >> >> > set size > >> >> > can be increased rather arbitrarily, so we don't need to make it but > >> >> > so large > >> >> > as it is easy to bump later (even with a branch). > >> >> > >> >> Well, what about making the API/KBI use cpuset_t pointers for things > >> >> rather than including it as a bitmask? Do you think there'd be a > >> >> noticable performance overhead for the bits where it's indirecting > >> >> through a pointer to get to the bitmask data? > >> > > >> > The wired memory is not due to cpuset_t. The wired memory usage is due > >> > to things > >> > that do 'struct foo foo_bits[MAXCPU]'. The KBI issues I mentioned above > >> > are > >> > 'struct rmlock' (so now you want any rmlock users to malloc space, or you > >> > want rmlock_init() call malloc? (that seems like a bad idea)). The > >> > other one > >> > is smp_rendezvous. Plus, it's not just a pointer, you really need a > >> > (pointer, > >> > size_t) tuple similar to what cpuset_getaffinity(), etc. use. > >> > >> Why would calling malloc be a problem? Except for the initial setup of > >> things, anything dynamically allocating structs with embedded things > >> like rmlocks are already dynamically allocating them via malloc or > >> uma. > >> > >> There's a larger fundamental problem with malloc, fragmentation and > >> getting the required larger allocations for things. But even a 4096 > >> CPU box would require a 512 byte malloc. That shouldn't be that hard > >> to do. It'd just be from some memory that isn't close to the rest of > >> the lock state. > > > > Other similar APIs like mtx_init() don't call malloc(), so it would be > > unusual behavior. However, we have several other problems before we can > > move beyond 256 anyway (like pf). > > Maybe behaviour has to change over time. :( > > anyway. Besides all of this - I'm thinking of just introducing: > > typedef uint32_t cpuid_t; > > .. then once we've converted all the users, we can make NOCPU > something other than 255 (which is the other limiting factor here..) > > Any objections? This one is a bit harder as you'll have to do shims for kinfo_proc, but I think this is fine. You could also just use u_int, but a new foo_t isn't that bad I guess. -- John Baldwin ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: Processor cores not properly detected/activated?
On Thursday, May 29, 2014 3:27:57 pm Konstantin Belousov wrote: > On Thu, May 29, 2014 at 02:44:19PM -0400, John Baldwin wrote: > > On Thursday, May 29, 2014 2:24:45 pm Adrian Chadd wrote: > > > On 29 May 2014 10:18, John Baldwin wrote: > > > > > > >> > It costs wired memory to increase it for the kernel. The userland > > > >> > set size > > > >> > can be increased rather arbitrarily, so we don't need to make it but > > > >> > so large > > > >> > as it is easy to bump later (even with a branch). > > > >> > > > >> Well, what about making the API/KBI use cpuset_t pointers for things > > > >> rather than including it as a bitmask? Do you think there'd be a > > > >> noticable performance overhead for the bits where it's indirecting > > > >> through a pointer to get to the bitmask data? > > > > > > > > The wired memory is not due to cpuset_t. The wired memory usage is due > > > > to things > > > > that do 'struct foo foo_bits[MAXCPU]'. The KBI issues I mentioned > > > > above are > > > > 'struct rmlock' (so now you want any rmlock users to malloc space, or > > > > you > > > > want rmlock_init() call malloc? (that seems like a bad idea)). The > > > > other one > > > > is smp_rendezvous. Plus, it's not just a pointer, you really need a > > > > (pointer, > > > > size_t) tuple similar to what cpuset_getaffinity(), etc. use. > > > > > > Why would calling malloc be a problem? Except for the initial setup of > > > things, anything dynamically allocating structs with embedded things > > > like rmlocks are already dynamically allocating them via malloc or > > > uma. > > > > > > There's a larger fundamental problem with malloc, fragmentation and > > > getting the required larger allocations for things. But even a 4096 > > > CPU box would require a 512 byte malloc. That shouldn't be that hard > > > to do. It'd just be from some memory that isn't close to the rest of > > > the lock state. > > > > Other similar APIs like mtx_init() don't call malloc(), so it would be > > unusual behavior. However, we have several other problems before we can > > move beyond 256 anyway (like pf). > > What is pf ? The firewall, though it might be fixable without too much trouble: #define PFID_CPUBITS8 #define PFID_CPUSHIFT (sizeof(uint64_t) * NBBY - PFID_CPUBITS) #define PFID_CPUMASK((uint64_t)((1 << PFID_CPUBITS) - 1) << PFID_CPUSHIFT) #define PFID_MAXID (~PFID_CPUMASK) CTASSERT((1 << PFID_CPUBITS) >= MAXCPU); In theory we can just bump up PFID_CPUBITS to 32, though I'm not sure how many bits PFID_MAXID should have (e.g. does that cap your rules, or does that cap your state entries, etc.)? > We definitely have a problem with the legacy APIC mode, we must start > using x2APIC, I believe. Correct. Handling X2APIC entries isn't that hard, but right now the x86 MP code uses an array indexed by APIC ID to map to CPU IDs early on and we'll need to probably replace that with a flat array indexed by CPU ID and resort to linear searches to check for dupes, etc. The first thing we would need would be a machine that actually did X2APIC and created X2APIC MADT entries, etc. Shouldn't be too hard to simulate this in bhyve though. -- John Baldwin ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: cpuid_t typedef? (was Re: Processor cores not properly detected/activated?)
On 29 May 2014 13:18, John Baldwin wrote: >> anyway. Besides all of this - I'm thinking of just introducing: >> >> typedef uint32_t cpuid_t; >> >> .. then once we've converted all the users, we can make NOCPU >> something other than 255 (which is the other limiting factor here..) >> >> Any objections? > > This one is a bit harder as you'll have to do shims for kinfo_proc, but > I think this is fine. You could also just use u_int, but a new foo_t > isn't that bad I guess. I don't think I'd modify any userland-facing ABI/KBI's just yet. I'm just worried that 11.0-REL will come out before we have made a decent inroads into this and we _can't_ support > 254 CPUs. -a ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: cpuid_t typedef? (was Re: Processor cores not properly detected/activated?)
On Thursday, May 29, 2014 5:09:05 pm Adrian Chadd wrote: > On 29 May 2014 13:18, John Baldwin wrote: > > >> anyway. Besides all of this - I'm thinking of just introducing: > >> > >> typedef uint32_t cpuid_t; > >> > >> .. then once we've converted all the users, we can make NOCPU > >> something other than 255 (which is the other limiting factor here..) > >> > >> Any objections? > > > > This one is a bit harder as you'll have to do shims for kinfo_proc, but > > I think this is fine. You could also just use u_int, but a new foo_t > > isn't that bad I guess. > > I don't think I'd modify any userland-facing ABI/KBI's just yet. I'm > just worried that 11.0-REL will come out before we have made a decent > inroads into this and we _can't_ support > 254 CPUs. Eh, that's one of the biggies to do actually. Kind of pointless to update td_oncpu/lastcpu and not fix kinfo_proc at the same time. You'll just have to add new int fields and populate the old ones with sane values for CPUs < 255. -- John Baldwin ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: cpuid_t typedef? (was Re: Processor cores not properly detected/activated?)
On 29 May 2014 14:29, John Baldwin wrote: > On Thursday, May 29, 2014 5:09:05 pm Adrian Chadd wrote: >> On 29 May 2014 13:18, John Baldwin wrote: >> >> >> anyway. Besides all of this - I'm thinking of just introducing: >> >> >> >> typedef uint32_t cpuid_t; >> >> >> >> .. then once we've converted all the users, we can make NOCPU >> >> something other than 255 (which is the other limiting factor here..) >> >> >> >> Any objections? >> > >> > This one is a bit harder as you'll have to do shims for kinfo_proc, but >> > I think this is fine. You could also just use u_int, but a new foo_t >> > isn't that bad I guess. >> >> I don't think I'd modify any userland-facing ABI/KBI's just yet. I'm >> just worried that 11.0-REL will come out before we have made a decent >> inroads into this and we _can't_ support > 254 CPUs. > > Eh, that's one of the biggies to do actually. Kind of pointless to > update td_oncpu/lastcpu and not fix kinfo_proc at the same time. You'll > just have to add new int fields and populate the old ones with sane values > for CPUs < 255. Ugh. Ok. I was too deep in the trenches of device drivers and other ancillary things doing bad things to char/short with cpu ids when walking things. I totally missed kinfo_proc. I'll go think about it a bit more. -a ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: RES: KQueue vs Select (NetMap)
On 29.05.2014 06:57, Fred Pedrisa wrote: > Hello, > > There are 4 threads, and a total of 32 FDs. What do you think ? > > -Mensagem original- > De: owner-freebsd-curr...@freebsd.org > [mailto:owner-freebsd-curr...@freebsd.org] Em nome de Adrian Chadd > Enviada em: quinta-feira, 29 de maio de 2014 01:52 > Para: Fred Pedrisa > Cc: freebsd-current; Jan Bramkamp > Assunto: Re: KQueue vs Select (NetMap) > > If your netmap thread(s) just have one or two FDs in some low range (say, > under FD 8 or 10) - no. > > If you have a whole bunch of active FDs and your netmap threads get FDs that > are high - then yes. select() operates on a bitmap of FD numbers. So if your > netmap FD is like, FD 8 and it's the highest FD that you're interested in, > select() only has to scan up to that FD. So it scans up to 8 FDs. If you > have a very active program and it has thousands of FDs open, select() has to > check all the FDs in the bitmap to see if they're set before getting to your > netmap FD. If your threads use just a handful of small FDs than you shouldn't see any performance difference between select()/poll() and kqueue(). But kqueue() can block on multiple event types. This can simply your netmap threads main loop. It sometimes even enables you to get by with just one type of main loop instead of multiple different main loops for different interfaces e.g. one for timers, one for sockets and one for files. ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: official pkg repo with WITHOUT_X11=true
On Thu, May 29, 2014 at 4:20 PM, Allan Jude wrote: > On 2014-05-29 08:42, Bryan Drewery wrote: > > > >> On May 29, 2014, at 4:19, David Chisnall wrote: > >> > >>> On 29 May 2014, at 02:23, Bryan Drewery wrote: > >>> > >>> As for skipping unneeded ports the best I can do is '-a' or "Build it > all". > >>> If a port is only needed for WITH_X11 then an IGNORE should be added > to it > >>> when WITHOUT_X11 is set to prevent wasting time on it. > >> > >> We can probably do a bit better by looking at the complete dependency > graph and removing any ports that have unconditional dependencies on X. > For a headless server, there's no reason to build any of the kde-* or > gnome-* ports or, indeed, X itself. I suspect that we could easily trim > 2/3 of the build time by omitting ports that have a GUI, GUI toolkits, and > so on. > > > > Yeah. My point was more that poudriere can't do that now and I would > rather not add all that special-case logic to it. Clever make.conf logic > might be able to do it. > > > >> > >> Longer term, we may be able to share the build time a bit. Ports which > don't have a WITHOUT_X11 flag and don't unconditionally depend on X11 can > potentially be pre-seeded from the normal package build (if we can identify > them). That only leaves the ports that actually have build-time > conditional X support to build in the no-Xorg run. > > > > Yup! I have a patch for that in the works. > > > > That would be a great improvement for the 'sets' feature in poudriere. > Almost all of my different sets have some overlap. Although this would > either require the 'queue' system you are working on, to say build sets > X, Y, and Z, and if there is any overlap share it. Or, augment the set > feature with an additional flag to specify a 'parent' set or something, > -z myset -Z commonset to tell it an already built set where it can steal > packages from. > > >> > >> David > > ___ > > freebsd-current@freebsd.org mailing list > > http://lists.freebsd.org/mailman/listinfo/freebsd-current > > To unsubscribe, send any mail to " > freebsd-current-unsubscr...@freebsd.org" > > > > > -- > Allan Jude > > Having a "parent" set would be nice, yes. I maintain two repos for several FreeBSD-versions. Being able to pull some of the deps from packages instead of blindingly building would be nice. Best regards Andreas ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
RES: RES: KQueue vs Select (NetMap)
Ok. Thanks for the enlightenment :) -Mensagem original- De: owner-freebsd-curr...@freebsd.org [mailto:owner-freebsd-curr...@freebsd.org] Em nome de Jan Bramkamp Enviada em: quinta-feira, 29 de maio de 2014 18:55 Para: 'freebsd-current' Assunto: Re: RES: KQueue vs Select (NetMap) On 29.05.2014 06:57, Fred Pedrisa wrote: > Hello, > > There are 4 threads, and a total of 32 FDs. What do you think ? > > -Mensagem original- > De: owner-freebsd-curr...@freebsd.org > [mailto:owner-freebsd-curr...@freebsd.org] Em nome de Adrian Chadd > Enviada em: quinta-feira, 29 de maio de 2014 01:52 > Para: Fred Pedrisa > Cc: freebsd-current; Jan Bramkamp > Assunto: Re: KQueue vs Select (NetMap) > > If your netmap thread(s) just have one or two FDs in some low range > (say, under FD 8 or 10) - no. > > If you have a whole bunch of active FDs and your netmap threads get > FDs that are high - then yes. select() operates on a bitmap of FD > numbers. So if your netmap FD is like, FD 8 and it's the highest FD > that you're interested in, > select() only has to scan up to that FD. So it scans up to 8 FDs. If > you have a very active program and it has thousands of FDs open, > select() has to check all the FDs in the bitmap to see if they're set > before getting to your netmap FD. If your threads use just a handful of small FDs than you shouldn't see any performance difference between select()/poll() and kqueue(). But kqueue() can block on multiple event types. This can simply your netmap threads main loop. It sometimes even enables you to get by with just one type of main loop instead of multiple different main loops for different interfaces e.g. one for timers, one for sockets and one for files. ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org" ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: Memory blackhole in 11. Possibly libc.so.7?
I'm replying late, but last night I almost had a complete meltdown of the system. I did a partial "pkg upgrade" for the packages that managed to get built by poudriere, then all hell broke loose. Screen lock-ups, random reboots, and several hard reboots later I decided to do a fresh buildworld/buildkernel and we seem to be back at some level of normality. It makes no sense, I know; and I would not be able to tell you what it was that went wrong. What I can tell you is that * While compiling, code would compile for a while, then freeze, then continue where it left off. "# top -P" was showing that during these freeze-ups, of the 4 cores on my system, 3 were mostly idle (80% to 95%) while one core would be at or near 0% idle. Not always the same core, and when one core got freed-up, you could have another core drop to 0%-8% idle. I thought maybe radeon was using cpu instead of gpu but that was debunked when same behavior was observed while compiling my regular kernel without radeon / xorg loaded. * Initially a debug-kernel was built with below, but screen goes black when radeon / xorg is loaded. (include GENERIC \ ident KERNDEBUG \ nooptions INET6 \ options KDB \ options DDB \ options GDB \ options INVARIANTS \ options INVARIANT_SUPPORT \ options WITNESS \ options DEBUG_LOCKS \ options DEBUG_VFS_LOCKS \ options DIAGNOSTIC) * I have modified my zfs-related entries in loader.conf, and latest is: vfs.zfs.trim.enabled=1 #Enable_Trim vfs.zfs.prefetch_disable=0 #I have 4G of Ram #vfs.zfs.arc_min="512M" #Ram 4GB => 512, Ram 8GB => 1024 #vfs.zfs.arc_max="1536M" #TotalRam x 0.5 - 512 MB #vm.kmem_size="6G" #Ram x 1.5 #vfs.zfs.vdev_max_pending="1" Currently, kstat.zfs.misc.arcstats.size: 2012423600 and the number seems to hang around in that vicinity. Separate ZFS question then: I have 2 sata-III HDD's (64G SSD + 7200rpm spindle) and 3 zpools. System is my personal desktop running sql, http server etc but no commercial load. tank-b: root, usr, var on SSD tank-d: home, all data, sql-db, NFS exported PXE root tank-a: all code compile (poudriere, world), located nearest to the center of spindle HDD + has 3GB ZIL from SSD drive. Q1: Is it ok to assume that arcstat.size will not change much regardless of the number of zpools? Q2: I have 3GB free space on the SSD reserved for an L2ARC, but decided it was not necessary after reading that this would be mostly useful for a commercial web server for example. Was my assessment incorrect and will the system benefit from a 3GB (or larger?) L2ARC on SSD? If so, which pool (not sure tank-b makes sense since it's already fully on the same SSD): Regards. - FreeBSD-11-current_amd64_root-on-zfs_RadeonKMS -- View this message in context: http://freebsd.1045724.n5.nabble.com/Memory-blackhole-in-11-Possibly-libc-so-7-tp5916161p5916406.html Sent from the freebsd-current mailing list archive at Nabble.com. ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
