Hi, all.
Resurrecting this thread from six months ago, I apologize for not having more
time to dig into it, but in light of recent findings, I see numerous other
drivers and other parts of the code that have comments to the effect that
"FreeBSD doesn't support interrupts" and they effectively #ifdef out the
attempt.
Could this be as simple as needing to do the same in vmxnet3? Empirically,
ignoring the error from rte_intr_enable() allows the driver to work normally,
for what that's worth.
Thanks,
lew
----- On Dec 6, 2021, at 6:08 AM, Konstantin Ananyev
konstantin.anan...@intel.com wrote:
>> -----Original Message-----
>> From: Richardson, Bruce <bruce.richard...@intel.com>
>> Sent: Monday, December 6, 2021 9:17 AM
>> To: Lewis Donzis <l...@perftech.com>
>> Cc: dev <dev@dpdk.org>; Wang, Yong <yongw...@vmware.com>; Ananyev, Konstantin
>> <konstantin.anan...@intel.com>
>> Subject: Re: vmxnet3 no longer functional on DPDK 21.11
>>
>> On Sun, Dec 05, 2021 at 07:52:33PM -0600, Lewis Donzis wrote:
>> >
>> >
>> > ----- On Nov 30, 2021, at 7:42 AM, Bruce Richardson
>> > bruce.richard...@intel.com
>> > wrote:
>> >
>> > > On Mon, Nov 29, 2021 at 02:45:15PM -0600, Lewis Donzis wrote:
>> > >> Hello.
>> > >> We just upgraded from 21.08 to 21.11 and it's rather astounding the
>> > >> number of incompatible changes in three months. Not a big deal, just
>> > >> kind of a surprise, that's all.
>> > >> Anyway, the problem is that the vmxnet3 driver is no longer
>> > >> functional
>> > >> on FreeBSD.
>> > >> In drivers/net/vmxnet3/vmxnet3_ethdev.c, vmxnet3_dev_start() gets an
>> > >> error calling rte_intr_enable(). So it logs "interrupt enable
>> > >> failed"
>> > >> and returns an error.
>> > >> In lib/eal/freebsd/eal_interrupts.c, rte_intr_enable() is returning
>> > >> an
>> > >> error because rte_intr_dev_fd_get(intr_handle) is returning -1.
>> > >> I don't see how that could ever return anything other than -1 since
>> > >> it
>> > >> appears that there is no code that ever calls rte_intr_dev_fd_set()
>> > >> with a value other than -1 on FreeBSD. Also weird to me is that even
>> > >> if it didn't get an error, the switch statement that follows looks
>> > >> like
>> > >> it will return an error in every case.
>> > >> Nonetheless, it worked in 21.08, and I can't quite see why the
>> > >> difference, so I must be missing something.
>> > >> For the moment, I just commented the "return -EIO" in
>> > >> vmxnet3_ethdev.c,
>> > >> and it's now working again, but that's obviously not the correct
>> > >> solution.
>> > >> Can someone who's knowledgable about this mechanism perhaps explain a
>> > >> little bit about what's going on? I'll be happy to help
>> > >> troubleshoot.
>> > >> It seems like it must be something simple, but I just don't see it
>> > >> yet.
>> > >
>> > > Hi
>> > >
>> > > if you have the chance, it would be useful if you could use "git bisect"
>> > > to
>> > > identify the commit in 21.11 that broke this driver. Looking through the
>> > > logs for 21.11 I can't identify any particular likely-looking commit, so
>> > > bisect is likely a good way to start looking into this.
>> > >
>> > > Regards,
>> > > /Bruce
>> >
>> > Hi, Bruce. git bisect is very time-consuming and very cool!
>> >
>> > I went back to 21.08, about 1100 commits, and worked through the process,
>> > but
>> > then I realized that I had forgotten to run ninja on one of
>> the steps, so I did it again.
>> >
>> > I also re-checked it after the bisect, just to make sure that
>> > c87d435a4d79739c0cec2ed280b94b41cb908af7 is good, and
>> 7a0935239b9eb817c65c03554a9954ddb8ea5044 is bad.
>> >
>> > Thanks,
>> > lew
>> >
>>
>> Many thanks for taking the time to do this. Adding Konstantin to thread as
>> author of the commit you identified. Konstantin, any thoughts on this
>> issue?
>
> Hmm, that's looks really strange to me.
> So to clarify, it fails at:
> static int
> vmxnet3_dev_start(struct rte_eth_dev *dev)
> {
> ...
> line 1695:
> if (rte_intr_enable(dev->intr_handle) < 0) {
> PMD_INIT_LOG(ERR, "interrupt enable failed");
> return -EIO;
> }
>
> Right?
>
> The strange thing here is that 7a0935239b9e
> doesn't change dev_start or rte_intr code in any way.
> All it does - change rte_eth_rx_burst/rte_eth_tx_burst and other fast-path
> functions.
> Anyway, if git blames that commit, let's try to figure out what is going on.
> Unfortunately, I don't have freebsd with vmxnet3, so will need to rely on your
> help here.
> As the first thing can you try to run testpmd build with last good commit
> (c87d435a4d79)
> and then testpmd build with bad commit applied and collect for both cases:
> - contents of 'struct rte_eth_dev' and ' rte_eth_dev->intr_handle' for
> your vmxnet3 port
> - debug log output (--log-level=eal,debug --log-level=pmd,debug)
>
> Konstantin
