Re: PCIe hotplug
On Mon, Jul 23, 2012 at 12:45:14AM -0700, Julian Elischer wrote: > On 7/22/12 9:11 PM, Warner Losh wrote: > >On Jul 22, 2012, at 9:12 PM, Alexander Kabaev wrote: > > > >>On Sun, 22 Jul 2012 20:22:33 -0600 > >>Scott Long wrote: > >> > >>>On Jul 20, 2012, at 8:04 PM, Julian Elischer wrote: > >>> > Is anyone looking at PCIe hotplug support? > > I'm especially interested if anyone has a strategy for device > re-insertion and reassociating the reinserted device with its old > device_t so that it gets the same unit number.. (assumes access to > a serial number or similar) Even if it is put back into a different > slot. > > >>>Would the PCI system be responsible for figuring out this serial > >>>number? I don't think that it can, but it's a question to answer, I > >>>guess. If it can't then it's up to the driver to generate a unique > >>>cookie that would be stored by the PCI subsystem. This cookie would > >>>have to be based off of data that can be retrieved from the PCI > >>>config space and/or VPD space, since anything more would require > >>>resource allocation, which is only allowed in the DEV_ATTACH phase, > >>>and once you've hit that phase you've already pretty much sealed the > >>>deal on unit number assignment. > >>> > >>>So what would probably happen is that the PCI layer provides a ring > >>>buffer of cookie storage and a set of accessors for the drivers. The > >>>cookies would map to a key-value pair with the device unit name and > >>>number. During probe, a driver can look at PCI config space and > >>>generate a cookie. That cookie can then be communicated up to the > >>>PCI layer for storage. Maybe the driver calls a match routine that > >>>returns a unit number on match and a store on failure, then the > >>>driver calls a set_unit_number accessor. Only the driver that wins > >>>the bid would win the unit number reassignment or cookie storage. Or > >>>maybe the driver passes the cookie up as part of its return code, and > >>>the match and unit assignment happens automatically. Drivers that > >>>don't want to participate in this simply wouldn't, and everything > >>>would continue to operate the same way. The two sticky parts are > >>>rogue/buggy drivers that abuse the api and cause a flood of cookies > >>>to be generated, and questions on when a unit number is eligible for > >>>reuse. For the first one, a ring buffer of cookies would solve the > >>>immediate problem, but you might still have some risk of drivers > >>>selectively wrapping the buffer for whatever accidental or evil > >>>purpose. For the second problem, maybe a unit number stays > >>>persistent only if the PCIe hot remove mechanism requests it, and > >>>then only until the ring-buffer wraps. > >>> > >>>Scott > >>> > >>I do not think the whole problem as depicted by Julian is even worth > >>solving. Why keeping any data for the device that might _never_ come > >>back? What if the device hierarchy just starts from the PCI-e and > >>extends upwards and user still holds on to some vestiges of a previous > >>device chain (say, by keeping a character control device sharing the > >>same unit number open, common practice)? Reusing unit number is much > >>trickier then, and might not be even possible. So, before one jumps > >>into 'how', can we agree on 'why' first? When device goes away, it is > >>not just this device's device_t that is disappearing, it is a whole > >>tree rooted at that device. I see no point in trying to reconstruct > >>that. > >There's a reason that PC Card and CardBus never supported this at all. > >The assumption was that reconnecting devices is so cheap that it isn't > >worth the bother. This is true for all but some specialized devices > >today: network information is easy to reconstruct, storage drives are easy > >to reconfigure (since we already fail all in-flight transactions when the > >device goes away), etc. I can see some advantage to having storage cope, > >but there already geom classes that can help people code when drives can > >go away. > > > >>PCI-e hotplug proper is very much orthogonal to the question of unit > >>numbering and IS worth supporting. > >Yes. totally agreed. > > I'm not saying that it's vitally important but was wondering if people > had a strategy for it.. > i.e. is it a question worth worrying about? > > In a separate forum Warner and I (yeah I know I'm answering Warner, > but I'm addressing the others) discussed the feasibility of surviving > an "oops pulled the wrong card" event with regards to a particular > flash memory card. I was just carrying that forwards as a thought > experiment (There is actually a strategy that sounds feasible). > > The problem of getting a serial number out of the BAR space during > probe is also possibly solvable in our case but the question of how > long to remember a device is legitimate an My answer would be that > 1/ a particular driver would be able to specify whether it could > handle
[head tinderbox] failure on i386/pc98
TB --- 2012-07-24 18:30:00 - tinderbox 2.9 running on freebsd-current.sentex.ca TB --- 2012-07-24 18:30:00 - FreeBSD freebsd-current.sentex.ca 8.3-PRERELEASE FreeBSD 8.3-PRERELEASE #0: Mon Mar 26 13:54:12 EDT 2012 d...@freebsd-current.sentex.ca:/usr/obj/usr/src/sys/GENERIC amd64 TB --- 2012-07-24 18:30:00 - starting HEAD tinderbox run for i386/pc98 TB --- 2012-07-24 18:30:00 - cleaning the object tree TB --- 2012-07-24 18:30:00 - cvsupping the source tree TB --- 2012-07-24 18:30:00 - /usr/bin/csup -z -r 3 -g -L 1 -h cvsup.sentex.ca /tinderbox/HEAD/i386/pc98/supfile TB --- 2012-07-24 18:31:06 - building world TB --- 2012-07-24 18:31:06 - CROSS_BUILD_TESTING=YES TB --- 2012-07-24 18:31:06 - MAKEOBJDIRPREFIX=/obj TB --- 2012-07-24 18:31:06 - PATH=/usr/bin:/usr/sbin:/bin:/sbin TB --- 2012-07-24 18:31:06 - SRCCONF=/dev/null TB --- 2012-07-24 18:31:06 - TARGET=pc98 TB --- 2012-07-24 18:31:06 - TARGET_ARCH=i386 TB --- 2012-07-24 18:31:06 - TZ=UTC TB --- 2012-07-24 18:31:06 - __MAKE_CONF=/dev/null TB --- 2012-07-24 18:31:06 - cd /src TB --- 2012-07-24 18:31:06 - /usr/bin/make -B buildworld >>> World build started on Tue Jul 24 18:31:06 UTC 2012 >>> Rebuilding the temporary build tree >>> stage 1.1: legacy release compatibility shims >>> stage 1.2: bootstrap tools >>> stage 2.1: cleaning up the object tree >>> stage 2.2: rebuilding the object tree >>> stage 2.3: build tools >>> stage 3: cross tools >>> stage 4.1: building includes >>> stage 4.2: building libraries [...] c++ -O2 -pipe -I/src/lib/clang/libclangarcmigrate/../../../contrib/llvm/include -I/src/lib/clang/libclangarcmigrate/../../../contrib/llvm/tools/clang/include -I/src/lib/clang/libclangarcmigrate/../../../contrib/llvm/tools/clang/lib/ARCMigrate -I. -I/src/lib/clang/libclangarcmigrate/../../../contrib/llvm/../../lib/clang/include -DLLVM_ON_UNIX -DLLVM_ON_FREEBSD -D__STDC_LIMIT_MACROS -D__STDC_CONSTANT_MACROS -fno-strict-aliasing -DLLVM_DEFAULT_TARGET_TRIPLE=\"i386-unknown-freebsd10.0\" -DDEFAULT_SYSROOT=\"\" -fstack-protector -fno-exceptions -fno-rtti -c /src/lib/clang/libclangarcmigrate/../../../contrib/llvm/tools/clang/lib/ARCMigrate/TransGCAttrs.cpp -o TransGCAttrs.o c++ -O2 -pipe -I/src/lib/clang/libclangarcmigrate/../../../contrib/llvm/include -I/src/lib/clang/libclangarcmigrate/../../../contrib/llvm/tools/clang/include -I/src/lib/clang/libclangarcmigrate/../../../contrib/llvm/tools/clang/lib/ARCMigrate -I. -I/src/lib/clang/libclangarcmigrate/../../../contrib/llvm/../../lib/clang/include -DLLVM_ON_UNIX -DLLVM_ON_FREEBSD -D__STDC_LIMIT_MACROS -D__STDC_CONSTANT_MACROS -fno-strict-aliasing -DLLVM_DEFAULT_TARGET_TRIPLE=\"i386-unknown-freebsd10.0\" -DDEFAULT_SYSROOT=\"\" -fstack-protector -fno-exceptions -fno-rtti -c /src/lib/clang/libclangarcmigrate/../../../contrib/llvm/tools/clang/lib/ARCMigrate/TransGCCalls.cpp -o TransGCCalls.o c++ -O2 -pipe -I/src/lib/clang/libclangarcmigrate/../../../contrib/llvm/include -I/src/lib/clang/libclangarcmigrate/../../../contrib/llvm/tools/clang/include -I/src/lib/clang/libclangarcmigrate/../../../contrib/llvm/tools/clang/lib/ARCMigrate -I. -I/src/lib/clang/libclangarcmigrate/../../../contrib/llvm/../../lib/clang/include -DLLVM_ON_UNIX -DLLVM_ON_FREEBSD -D__STDC_LIMIT_MACROS -D__STDC_CONSTANT_MACROS -fno-strict-aliasing -DLLVM_DEFAULT_TARGET_TRIPLE=\"i386-unknown-freebsd10.0\" -DDEFAULT_SYSROOT=\"\" -fstack-protector -fno-exceptions -fno-rtti -c /src/lib/clang/libclangarcmigrate/../../../contrib/llvm/tools/clang/lib/ARCMigrate/TransProperties.cpp -o TransProperties.o /src/lib/clang/libclangarcmigrate/../../../contrib/llvm/tools/clang/include/clang/AST/RecursiveASTVisitor.h: In member function 'bool clang::RecursiveASTVisitor::TraverseDeclRefExpr(clang::DeclRefExpr*) [with Derived = ::PropertiesRewriter::PlusOneAssign]': /src/lib/clang/libclangarcmigrate/../../../contrib/llvm/tools/clang/include/clang/AST/RecursiveASTVisitor.h:1901: internal compiler error: in var_ann, at tree-flow-inline.h:128 Please submit a full bug report, with preprocessed source if appropriate. See http://gcc.gnu.org/bugs.html> for instructions. *** Error code 1 Stop in /src/lib/clang/libclangarcmigrate. *** Error code 1 Stop in /src/lib/clang. *** Error code 1 Stop in /src/lib. *** Error code 1 Stop in /src. *** Error code 1 Stop in /src. *** Error code 1 Stop in /src. *** Error code 1 Stop in /src. TB --- 2012-07-24 19:19:37 - WARNING: /usr/bin/make returned exit code 1 TB --- 2012-07-24 19:19:37 - ERROR: failed to build world TB --- 2012-07-24 19:19:37 - 1996.82 user 374.08 system 2977.05 real http://tinderbox.freebsd.org/tinderbox-head-HEAD-i386-pc98.full ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
RFC: use EM_LEGACY_IRQ in if_lem.c ?
if_lem.c ("lem", one of the e1000 drivers) has 2 possible interrupt modes:
EM_LEGACY_IRQ uses the standard dispatch mechanism, whereas
FAST_INTR has a custom handler that signals a taskqueue to do the job.
I have no idea which actual hardware uses it (all of my Intel 1G
cards use either "em" or "igb"), but "lem" is the driver used in
qemu, and there the EM_LEGACY_IRQ gives approx 10% higher packet
rates than the other.
Any objections if i change the default to EM_LEGACY_IRQ ?
cheers
luigi
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: -current build failure
On 23 Jul 2012, at 20:53, David Chisnall wrote: > On 23 Jul 2012, at 20:18, Konstantin Belousov wrote: > >> Longer description is that pc_curthread is offset 0 if %gs-based. >> The dereferenced pointer point to the struct thread, which contains >> td_proc pointer at offset 8. Instead, clang seems to dereference >> td_proc from offset 8 based on %gs, or something similar. > > This appears to be a bug in the LLVM X86 back end. It is performing an > invalid fold of the two loads. I have filed this bug: > > http://llvm.org/bugs/show_bug.cgi?id=13438 And fixed it in LLVM r160687. Since it's a single-line change, we can probably pull it into our version. dim: http://llvm.org/viewvc/llvm-project?view=rev&revision=160687 David ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: RFC: use EM_LEGACY_IRQ in if_lem.c ?
Interesting, lem is all the non-pcie hardware, and if you see better
performance
out of the LEGACY path then I'm OK with changing the default.
Jack
On Tue, Jul 24, 2012 at 1:20 PM, Luigi Rizzo wrote:
> if_lem.c ("lem", one of the e1000 drivers) has 2 possible interrupt modes:
> EM_LEGACY_IRQ uses the standard dispatch mechanism, whereas
> FAST_INTR has a custom handler that signals a taskqueue to do the job.
>
> I have no idea which actual hardware uses it (all of my Intel 1G
> cards use either "em" or "igb"), but "lem" is the driver used in
> qemu, and there the EM_LEGACY_IRQ gives approx 10% higher packet
> rates than the other.
>
> Any objections if i change the default to EM_LEGACY_IRQ ?
>
> cheers
> luigi
> ___
> freebsd-current@freebsd.org mailing list
> http://lists.freebsd.org/mailman/listinfo/freebsd-current
> To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
>
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: -current build failure
On Tue, Jul 24, 2012 at 10:08:13PM +0100, David Chisnall wrote: > On 23 Jul 2012, at 20:53, David Chisnall wrote: > > > On 23 Jul 2012, at 20:18, Konstantin Belousov wrote: > > > >> Longer description is that pc_curthread is offset 0 if %gs-based. > >> The dereferenced pointer point to the struct thread, which contains > >> td_proc pointer at offset 8. Instead, clang seems to dereference > >> td_proc from offset 8 based on %gs, or something similar. > > > > This appears to be a bug in the LLVM X86 back end. It is performing an > > invalid fold of the two loads. I have filed this bug: > > > > http://llvm.org/bugs/show_bug.cgi?id=13438 > > And fixed it in LLVM r160687. Since it's a single-line change, we can > probably pull it into our version. > > dim: http://llvm.org/viewvc/llvm-project?view=rev&revision=160687 As kan rightfully notes, the assumption that &%fs:0 == *%fs:0 holds for userspace on amd64, and the same is true for %gs userspace on i386. The change you committed to clang/llvm/whatever it called just breaks useful optimization for FreeBSD. Sigh. pgpDN70yT4jG3.pgp Description: PGP signature
