Unusual threading behavior on single processes
I have question about a performance quirk on OpenBSD, but I'm not really sure how to address it, or what the root cause even is; that being how multithreaded applications (libpthread?) behave (notably, games). I have tested many applications, the behavior is the same in all of them, but I'll talk about OpenMW (an open-source game engine for morrowind) since I have the most useful information about how this program is threaded. By default, OpenMW uses 4 threads (cited here: https://openmw.readthedocs.io/en/stable/reference/modding/settings/cells.html), one for main/generic processing, one for graphics, one for audio, and one for preloading terrain. You can see this if you look at the thread usage under top while running the game; however, this is exactly where my question comes into play. Instead of each thread processing the game independently with their own limits, each thread is "capped" to the total limit of one thread (I.E. instead of openmw's process using 100% of 4 threads, or 400% cpu in top, instead the process uses 25% across 4 threads, or 100% cpu in top). I tested this using GENERIC instead of GENERIC.MP as well, and get identical performance on the one thread; it's almost like pthreads is acting as a placeholder of sorts and not actually improving performance where it should. Is it a lock (spin is at 0)? A placeholder? A limitation of how Ryzen SMP is implemented? I'd be happy to do any additional testing, I have a fresh -current source tree ready dmesg OpenBSD 6.6-current (GENERIC.MP) #75: Tue Mar 24 12:56:37 MDT 2020 dera...@amd64.openbsd.org:/usr/src/sys/arch/amd64/compile/GENERIC.MP real mem = 16603250688 (15834MB) avail mem = 16087437312 (15342MB) mpath0 at root scsibus0 at mpath0: 256 targets mainbus0 at root bios0 at mainbus0: SMBIOS rev. 3.1 @ 0x986ec000 (62 entries) bios0: vendor LENOVO version "R0UET76W (1.56 )" date 11/05/2019 bios0: LENOVO 20KVCTO1WW acpi0 at bios0: ACPI 5.0 acpi0: sleep states S0 S3 S4 S5 acpi0: tables DSDT FACP SSDT SSDT CRAT CDIT UEFI MSDM BATB HPET APIC MCFG SBST WSMT IVRS FPDT SSDT SSDT SSDT UEFI SSDT acpi0: wakeup devices GPP0(S3) GPP1(S3) GPP2(S3) GPP3(S3) GPP4(S3) GPP5(S3) GPP6(S3) GP17(S3) XHC0(S3) XHC1(S3) GP18(S3) LID_(S3) SLPB(S3) acpitimer0 at acpi0: 3579545 Hz, 32 bits acpihpet0 at acpi0: 14318180 Hz acpimadt0 at acpi0 addr 0xfee0: PC-AT compat cpu0 at mainbus0: apid 0 (boot processor) cpu0: AMD Ryzen 5 2500U with Radeon Vega Mobile Gfx, 1996.61 MHz, 17-11-00 cpu0: FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,MMX,FXSR,SSE,SSE2,HTT,SSE3,PCLMUL,MWAIT,SSSE3,FMA3,CX16,SSE4.1,SSE4.2,MOVBE,POPCNT,AES,XSAVE,AVX,F16C,RDRAND,NXE,MMXX,FFXSR,PAGE1GB,RDTSCP,LONG,LAHF,CMPLEG,SVM,EAPICSP,AMCR8,ABM,SSE4A,MASSE,3DNOWP,OSVW,SKINIT,TCE,TOPEXT,CPCTR,DBKP,PCTRL3,MWAITX,ITSC,FSGSBASE,BMI1,AVX2,SMEP,BMI2,RDSEED,ADX,SMAP,CLFLUSHOPT,SHA,IBPB,XSAVEOPT,XSAVEC,XGETBV1,XSAVES cpu0: 64KB 64b/line 4-way I-cache, 32KB 64b/line 8-way D-cache, 512KB 64b/line 8-way L2 cache, 4MB 64b/line 16-way L3 cache cpu0: ITLB 64 4KB entries fully associative, 64 4MB entries fully associative cpu0: DTLB 64 4KB entries fully associative, 64 4MB entries fully associative cpu0: smt 0, core 0, package 0 mtrr: Pentium Pro MTRR support, 8 var ranges, 88 fixed ranges cpu0: apic clock running at 24MHz cpu0: mwait min=64, max=64, C-substates=1.1, IBE cpu1 at mainbus0: apid 1 (application processor) cpu1: AMD Ryzen 5 2500U with Radeon Vega Mobile Gfx, 1996.23 MHz, 17-11-00 cpu1: FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,MMX,FXSR,SSE,SSE2,HTT,SSE3,PCLMUL,MWAIT,SSSE3,FMA3,CX16,SSE4.1,SSE4.2,MOVBE,POPCNT,AES,XSAVE,AVX,F16C,RDRAND,NXE,MMXX,FFXSR,PAGE1GB,RDTSCP,LONG,LAHF,CMPLEG,SVM,EAPICSP,AMCR8,ABM,SSE4A,MASSE,3DNOWP,OSVW,SKINIT,TCE,TOPEXT,CPCTR,DBKP,PCTRL3,MWAITX,ITSC,FSGSBASE,BMI1,AVX2,SMEP,BMI2,RDSEED,ADX,SMAP,CLFLUSHOPT,SHA,IBPB,XSAVEOPT,XSAVEC,XGETBV1,XSAVES cpu1: 64KB 64b/line 4-way I-cache, 32KB 64b/line 8-way D-cache, 512KB 64b/line 8-way L2 cache, 4MB 64b/line 16-way L3 cache cpu1: ITLB 64 4KB entries fully associative, 64 4MB entries fully associative cpu1: DTLB 64 4KB entries fully associative, 64 4MB entries fully associative cpu1: smt 1, core 0, package 0 cpu2 at mainbus0: apid 2 (application processor) cpu2: AMD Ryzen 5 2500U with Radeon Vega Mobile Gfx, 1996.23 MHz, 17-11-00 cpu2: FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,MMX,FXSR,SSE,SSE2,HTT,SSE3,PCLMUL,MWAIT,SSSE3,FMA3,CX16,SSE4.1,SSE4.2,MOVBE,POPCNT,AES,XSAVE,AVX,F16C,RDRAND,NXE,MMXX,FFXSR,PAGE1GB,RDTSCP,LONG,LAHF,CMPLEG,SVM,EAPICSP,AMCR8,ABM,SSE4A,MASSE,3DNOWP,OSVW,SKINIT,TCE,TOPEXT,CPCTR,DBKP,PCTRL3,MWAITX,ITSC,FSGSBASE,BMI1,AVX2,SMEP,BMI2,RDSEED,ADX,SMAP,CLFLUSHOPT,SHA,IBPB,XSAVEOPT,XSAVEC,XGETBV1,XSAVES cpu2: 64KB 64b/line 4-way I-cache, 32KB 64b/line 8-way D-cache, 512KB 64b/line 8-way L2 cache, 4MB 64b/line 16-way L3 cache cpu2: ITLB 64 4KB entries fully associative, 64 4MB entries fully associative cpu2: DTLB 6
Re: Unusual threading behavior on single processes
Thank you, your information was very helpful. I compiled and ran malloc_duel and it's working as intended. I wasn't aware of the -H flag for top, and I can see programs are threading as you say, though the bottleneck to my poor performance is still a mystery. I took some screen captures so you can see what I'm seeing: Xonotic: https://0x0.st/iBCD.png OpenMW: https://0x0.st/iBC5.png Terraria: (fnaify if curious, thanks thfr :>) https://0x0.st/iMrr.png In the case of OpenMW, the bottleneck actually seems pretty obvious with what top -H reports. I don't really know what to say about the other examples. I would break out a profiling tool at this stage, but the results of testing with top -H have left me with no idea where the bottleneck is (except openmw where it might actually be CPU); digging through systat hasn't really given me any revelations either. :/ If anyone has a hunch where I should check, or if you need me to test a different software, I'd be more than happy to. Regards, Stefmorino On Sat, Mar 28, 2020, at 09:00:21AM +, Otto Moerbeek wrote: > On Fri, Mar 27, 2020 at 09:03:40PM +, Stefmorino wrote: >> I have question about a performance quirk on OpenBSD, but I'm not really sure >> how to address it, or what the root cause even is; that being how >> multithreaded >> applications (libpthread?) behave (notably, games). >> >> I have tested many applications, the behavior is the same in all of them, but >> I'll talk about OpenMW (an open-source game engine for morrowind) since I >> have >> the most useful information about how this program is threaded. By default, >> OpenMW uses 4 threads (cited here: >> https://openmw.readthedocs.io/en/stable/reference/modding/settings/cells.html), >> one for main/generic processing, one for graphics, one for audio, and one for >> preloading terrain. You can see this if you look at the thread usage under >> top >> while running the game; however, this is exactly where my question comes into >> play. Instead of each thread processing the game independently with their own >> limits, each thread is "capped" to the total limit of one thread (I.E. >> instead >> of openmw's process using 100% of 4 threads, or 400% cpu in top, instead the >> process uses 25% across 4 threads, or 100% cpu in top). I tested this using >> GENERIC instead of GENERIC.MP as well, and get identical performance on the >> one >> thread; it's almost like pthreads is acting as a placeholder of sorts and not >> actually improving performance where it should. >> >> Is it a lock (spin is at 0)? A placeholder? A limitation of how Ryzen SMP is >> implemented? > > Hard to tell, no idea what that game engine does. But this not a > general problem, e.g. the malloc_duel regress test > (/usr/src/regress/lib/libpthread/malloc_duel). I see > 100% as well > with other multi-threaded programs. > > 32013 otto 600 6020K 1552K onproc/3 - 1:07 228.81% > malloc_due > > Wild guess: it could be that you program actually does not do real > threading, but userland threading. Check with top -H if it really > creates threads. You should see multiple threads having the same PID. > or all thraeds are using a resource that cannot be shared. > > -Otto >> >> I'd be happy to do any additional testing, I have a fresh -current source >> tree >> ready >> >> dmesg >> OpenBSD 6.6-current (GENERIC.MP) #75: Tue Mar 24 12:56:37 MDT 2020 >> dera...@amd64.openbsd.org:/usr/src/sys/arch/amd64/compile/GENERIC.MP >> real mem = 16603250688 (15834MB) >> avail mem = 16087437312 (15342MB) >> mpath0 at root >> scsibus0 at mpath0: 256 targets >> mainbus0 at root >> bios0 at mainbus0: SMBIOS rev. 3.1 @ 0x986ec000 (62 entries) >> bios0: vendor LENOVO version "R0UET76W (1.56 )" date 11/05/2019 >> bios0: LENOVO 20KVCTO1WW >> acpi0 at bios0: ACPI 5.0 >> acpi0: sleep states S0 S3 S4 S5 >> acpi0: tables DSDT FACP SSDT SSDT CRAT CDIT UEFI MSDM BATB HPET APIC MCFG >> SBST WSMT IVRS FPDT SSDT SSDT SSDT UEFI SSDT >> acpi0: wakeup devices GPP0(S3) GPP1(S3) GPP2(S3) GPP3(S3) GPP4(S3) GPP5(S3) >> GPP6(S3) GP17(S3) XHC0(S3) XHC1(S3) GP18(S3) LID_(S3) SLPB(S3) >> acpitimer0 at acpi0: 3579545 Hz, 32 bits >> acpihpet0 at acpi0: 14318180 Hz >> acpimadt0 at acpi0 addr 0xfee0: PC-AT compat >> cpu0 at mainbus0: apid 0 (boot processor) >> cpu0: AMD Ryzen 5 2500U with Radeon Vega Mobile Gfx, 1996.61 MHz, 17-11-00 >> cpu0: >> FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUS
Thinkpad E585 usage report
For those unaware, the E585 is a 2017 AMD Ryzen Thinkpad with Vega mobile graphics (similar to the A485 but cheaper). Due to its newness, I wasn't even sure if it was going to boot when I ordered it, but for the most part, everything works decently well. There are a few major problems, but nothing that make the laptop by any means unusable. Important: *Wireless card is entirely non-functional, but is recognized by the kernel. dmesg snippet: `vendor "Atheros", unknown product 0x0042 (class network subclass miscellaneous, rev 0x31) at pci4 dev 0 function 0 not configured` *CPU does not seems to boost clock, and as such cannot get close to its true effective speed; at least if the value reported by sysctl hw.cpuspeed is to be trusted. *The typical AMDGPU problems (cursor buffer is messed up without SWcursor, random hangs/freezes, some recoverable, others not) (considerably less stable than other AMD hardware I've tried on the driver, seems to be more stable with aperture enabled) Less important: *X always graphically bugs out (and occasionally freezes) after suspend/hibernation and needs to be restarted (note switching to a tty, suspending, and switching back to X does not affect this behavior in any way) suspending and hibernating while in wscons works as intended *Mute and brightness controls do not work at all (mixerctl can be used for muting still) Notes: *apm(8) and apmd(8) both work, including suspend and hibernate, as well as all battery-related functions and setperf modes *Integrated webcam works with video(1) *Increasing and decreasing the audio with the function keys works despite mute not working *Ethernet works through re(4) *Anything not listed in this section presumably works; this is just here to remove confusion from the previous parts All in all I'm very impressed. dmesg: ``` OpenBSD 6.6-current (GENERIC.MP) #413: Tue Oct 29 20:11:52 MDT 2019 dera...@amd64.openbsd.org:/usr/src/sys/arch/amd64/compile/GENERIC.MP real mem = 16621588480 (15851MB) avail mem = 16105111552 (15359MB) mpath0 at root scsibus0 at mpath0: 256 targets mainbus0 at root bios0 at mainbus0: SMBIOS rev. 3.1 @ 0x986ea000 (62 entries) bios0: vendor LENOVO version "R0UET74W (1.54 )" date 07/23/2019 bios0: LENOVO 20KVCTO1WW acpi0 at bios0: ACPI 5.0 acpi0: sleep states S0 S3 S4 S5 acpi0: tables DSDT FACP SSDT SSDT CRAT CDIT UEFI MSDM BATB HPET APIC MCFG SBST WSMT IVRS FPDT SSDT SSDT SSDT UEFI SSDT acpi0: wakeup devices GPP0(S3) GPP1(S3) GPP2(S3) GPP3(S3) GPP4(S3) GPP5(S3) GPP6(S3) GP17(S3) XHC0(S3) XHC1(S3) GP18(S3) LID_(S3) SLPB(S3) acpitimer0 at acpi0: 3579545 Hz, 32 bits acpihpet0 at acpi0: 14318180 Hz acpimadt0 at acpi0 addr 0xfee0: PC-AT compat cpu0 at mainbus0: apid 0 (boot processor) cpu0: AMD Ryzen 5 2500U with Radeon Vega Mobile Gfx, 1996.54 MHz, 17-11-00 cpu0: FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,MMX,FXSR,SSE,SSE2,HTT,SSE3,PCLMUL,MWAIT,SSSE3,FMA3,CX16,SSE4.1,SSE4.2,MOVBE,POPCNT,AES,XSAVE,AVX,F16C,RDRAND,NXE,MMXX,FFXSR,PAGE1GB,RDTSCP,LONG,LAHF,CMPLEG,SVM,EAPICSP,AMCR8,ABM,SSE4A,MASSE,3DNOWP,OSVW,SKINIT,TCE,TOPEXT,CPCTR,DBKP,PCTRL3,MWAITX,ITSC,FSGSBASE,BMI1,AVX2,SMEP,BMI2,RDSEED,ADX,SMAP,CLFLUSHOPT,SHA,IBPB,XSAVEOPT,XSAVEC,XGETBV1,XSAVES cpu0: 64KB 64b/line 4-way I-cache, 32KB 64b/line 8-way D-cache, 512KB 64b/line 8-way L2 cache, 4MB 64b/line 16-way L3 cache cpu0: ITLB 64 4KB entries fully associative, 64 4MB entries fully associative cpu0: DTLB 64 4KB entries fully associative, 64 4MB entries fully associative cpu0: smt 0, core 0, package 0 mtrr: Pentium Pro MTRR support, 8 var ranges, 88 fixed ranges cpu0: apic clock running at 24MHz cpu0: mwait min=64, max=64, C-substates=1.1, IBE cpu1 at mainbus0: apid 1 (application processor) cpu1: AMD Ryzen 5 2500U with Radeon Vega Mobile Gfx, 1996.23 MHz, 17-11-00 cpu1: FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,MMX,FXSR,SSE,SSE2,HTT,SSE3,PCLMUL,MWAIT,SSSE3,FMA3,CX16,SSE4.1,SSE4.2,MOVBE,POPCNT,AES,XSAVE,AVX,F16C,RDRAND,NXE,MMXX,FFXSR,PAGE1GB,RDTSCP,LONG,LAHF,CMPLEG,SVM,EAPICSP,AMCR8,ABM,SSE4A,MASSE,3DNOWP,OSVW,SKINIT,TCE,TOPEXT,CPCTR,DBKP,PCTRL3,MWAITX,ITSC,FSGSBASE,BMI1,AVX2,SMEP,BMI2,RDSEED,ADX,SMAP,CLFLUSHOPT,SHA,IBPB,XSAVEOPT,XSAVEC,XGETBV1,XSAVES cpu1: 64KB 64b/line 4-way I-cache, 32KB 64b/line 8-way D-cache, 512KB 64b/line 8-way L2 cache, 4MB 64b/line 16-way L3 cache cpu1: ITLB 64 4KB entries fully associative, 64 4MB entries fully associative cpu1: DTLB 64 4KB entries fully associative, 64 4MB entries fully associative cpu1: smt 1, core 0, package 0 cpu2 at mainbus0: apid 2 (application processor) cpu2: AMD Ryzen 5 2500U with Radeon Vega Mobile Gfx, 1996.23 MHz, 17-11-00 cpu2: FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,MMX,FXSR,SSE,SSE2,HTT,SSE3,PCLMUL,MWAIT,SSSE3,FMA3,CX16,SSE4.1,SSE4.2,MOVBE,POPCNT,AES,XSAVE,AVX,F16C,RDRAND,NXE,MMXX,FFXSR,PAGE1GB,RDTSCP,LONG,LAHF,CMPLEG,SVM,EAPICSP,AMCR8,ABM,SSE4A,MASSE,3DNOWP,OSVW,SKINIT,T
For people having stability issues on amdgpu and/or gallium
On all my machines utilizing AMDGPU, I've had issues with stability, lots of hang ups and freezes, correlating to video memory in some way (my low memory vega iGPU hangs far quicker and in many more scenarios than my dGPU in my desktop). After my inexperienced (but extensive) testing, I've figured out this is likely a bug in mesa's gallium implementation rather than the kernel driver, and disabling opengl in the environment fixed my instabilities entirely. This can be done by disabling amdgpu at the kernel level and booting with VESAFB/EFIFB, or by disabling opengl in your environment. The former is simple enough (documented in config(8)), the latter I've found to work through setting the mesa variable LIBGL_ALWAYS_SOFTWARE=1 somewhere in your environment. I've also tried setting `Option "Accel" "off"` under the Device section in xorg.conf, but this results in a sigabort on my tested systems, or graphical errors in the case of modesetting. I have not yet tested the stability of vulkan, but I plan to. Hope this helps anyone having stability issues! :>
AMDGPU report on R9 Fury X
wscons was the proper resolution, no clipping on either side (1920x1080), and device was properly identified as Fiji. Inside of X, the cursor was incorrect unless SWcursor was set in xorg.conf (off by 1 error?); ShadowPrimary works properly. Suspend and hibernate were both broken, suspend would hang at a black screen upon hitting the power button, and hibernate would boot to a graphical artifact covering the whole screen and freeze (X was running). X was very unstable, resulting in freezes that required resets; though I could prevent damage to the filesystem by temporarily holding the power button for ~3 seconds and the system will properly shutdown (trying to enter wscons, kill cwm with C-M-S-q did not work, C-M-Del did not work). As such, these freezes don't seem to be kernel panics. Vulkan worked as intended, OpenGL mostly worked, though when testing SuperTuxKart graphical artifacts made the game unplayable. outside of SuperTuxKart, the cursor, and the freezes, nothing out of the ordinary was notable in Minetest, Terraria, OpenMW, mpv, ffmpeg/ffplay, Firefox, Chromium, xterm, and cwm. This was tested on amdgpu-firmware-20190312. OpenBSD 6.5-current (GENERIC.MP) #8: Thu Jul 4 20:14:31 EDT 2019 stan@pwuffy.OwOpenBSD.local:/usr/src/sys/arch/amd64/compile/GENERIC.MP real mem = 17099194368 (16307MB) avail mem = 16571428864 (15803MB) mpath0 at root scsibus0 at mpath0: 256 targets mainbus0 at root bios0 at mainbus0: SMBIOS rev. 3.0 @ 0x8f9a8000 (94 entries) bios0: vendor American Megatrends Inc. version "3802" date 03/15/2018 bios0: ASUSTeK COMPUTER INC. Z170-A acpi0 at bios0: ACPI 6.0 acpi0: sleep states S0 S3 S4 S5 acpi0: tables DSDT FACP APIC FPDT DBG2 MCFG SSDT FIDT SSDT SSDT HPET SSDT SSDT UEFI SSDT LPIT WSMT SSDT SSDT DBGP acpi0: wakeup devices PEGP(S4) PEG0(S4) PEGP(S4) PEG1(S4) PEGP(S4) PEG2(S4) SIO1(S3) PS2K(S4) PS2M(S4) UAR1(S4) PXSX(S4) RP09(S4) PXSX(S4) RP10(S4) PXSX(S4) RP11(S4) [...] acpitimer0 at acpi0: 3579545 Hz, 24 bits acpimadt0 at acpi0 addr 0xfee0: PC-AT compat cpu0 at mainbus0: apid 0 (boot processor) cpu0: Intel(R) Core(TM) i7-6700K CPU @ 4.00GHz, 4012.31 MHz, 06-5e-03 cpu0: FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,EST,TM2,SSSE3,SDBG,FMA3,CX16,xTPR,PDCM,PCID,SSE4.1,SSE4.2,x2APIC,MOVBE,POPCNT,DEADLINE,AES,XSAVE,AVX,F16C,RDRAND,NXE,PAGE1GB,RDTSCP,LONG,LAHF,ABM,3DNOWP,PERF,ITSC,FSGSBASE,TSC_ADJUST,SGX,BMI1,HLE,AVX2,SMEP,BMI2,ERMS,INVPCID,RTM,MPX,RDSEED,ADX,SMAP,CLFLUSHOPT,PT,MD_CLEAR,TSXFA,IBRS,IBPB,STIBP,L1DF,SSBD,SENSOR,ARAT,XSAVEOPT,XSAVEC,XGETBV1,XSAVES,MELTDOWN cpu0: 256KB 64b/line 8-way L2 cache cpu0: smt 0, core 0, package 0 mtrr: Pentium Pro MTRR support, 10 var ranges, 88 fixed ranges cpu0: apic clock running at 24MHz cpu0: mwait min=64, max=64, C-substates=0.2.1.2.4.1, IBE cpu1 at mainbus0: apid 2 (application processor) cpu1: Intel(R) Core(TM) i7-6700K CPU @ 4.00GHz, 4010.80 MHz, 06-5e-03 cpu1: FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,EST,TM2,SSSE3,SDBG,FMA3,CX16,xTPR,PDCM,PCID,SSE4.1,SSE4.2,x2APIC,MOVBE,POPCNT,DEADLINE,AES,XSAVE,AVX,F16C,RDRAND,NXE,PAGE1GB,RDTSCP,LONG,LAHF,ABM,3DNOWP,PERF,ITSC,FSGSBASE,TSC_ADJUST,SGX,BMI1,HLE,AVX2,SMEP,BMI2,ERMS,INVPCID,RTM,MPX,RDSEED,ADX,SMAP,CLFLUSHOPT,PT,MD_CLEAR,TSXFA,IBRS,IBPB,STIBP,L1DF,SSBD,SENSOR,ARAT,XSAVEOPT,XSAVEC,XGETBV1,XSAVES,MELTDOWN cpu1: 256KB 64b/line 8-way L2 cache cpu1: smt 0, core 1, package 0 cpu2 at mainbus0: apid 4 (application processor) cpu2: Intel(R) Core(TM) i7-6700K CPU @ 4.00GHz, 4010.80 MHz, 06-5e-03 cpu2: FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,EST,TM2,SSSE3,SDBG,FMA3,CX16,xTPR,PDCM,PCID,SSE4.1,SSE4.2,x2APIC,MOVBE,POPCNT,DEADLINE,AES,XSAVE,AVX,F16C,RDRAND,NXE,PAGE1GB,RDTSCP,LONG,LAHF,ABM,3DNOWP,PERF,ITSC,FSGSBASE,TSC_ADJUST,SGX,BMI1,HLE,AVX2,SMEP,BMI2,ERMS,INVPCID,RTM,MPX,RDSEED,ADX,SMAP,CLFLUSHOPT,PT,MD_CLEAR,TSXFA,IBRS,IBPB,STIBP,L1DF,SSBD,SENSOR,ARAT,XSAVEOPT,XSAVEC,XGETBV1,XSAVES,MELTDOWN cpu2: 256KB 64b/line 8-way L2 cache cpu2: smt 0, core 2, package 0 cpu3 at mainbus0: apid 6 (application processor) cpu3: Intel(R) Core(TM) i7-6700K CPU @ 4.00GHz, 4010.80 MHz, 06-5e-03 cpu3: FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,EST,TM2,SSSE3,SDBG,FMA3,CX16,xTPR,PDCM,PCID,SSE4.1,SSE4.2,x2APIC,MOVBE,POPCNT,DEADLINE,AES,XSAVE,AVX,F16C,RDRAND,NXE,PAGE1GB,RDTSCP,LONG,LAHF,ABM,3DNOWP,PERF,ITSC,FSGSBASE,TSC_ADJUST,SGX,BMI1,HLE,AVX2,SMEP,BMI2,ERMS,INVPCID,RTM,MPX,RDSEED,ADX,SMAP,CLFLUSHOPT,PT,MD_CLEAR,TSXFA,IBRS,IBPB,STIBP,L1DF,SSBD,SENSOR,ARAT,XSAVEOPT,XSAVEC,XGETBV1,XSAVES,MELTDOWN cpu3: 256KB 64b/line 8-way L2 cache cpu3: smt 0, core 3, package 0
