hi,
I have tested below dpdk version, after dpdk-17.05 version, ixgbe nic would send fail. the logs is same as below. dpdk-17.02 ---- ok dpdk-17.02.1 ---- ok dpdk-17.05 ---- nok dpdk-17.05.1 ---- nok dpdk-17.08 ---- nok PMD: ixgbe_xmit_pkts(): port_id=0 queue_id=0 tx_tail=63 nb_tx=11 PMD: ixgbe_xmit_cleanup(): TX descriptor 95 is not done(port=0 queue=0) PMD: ixgbe_xmit_pkts(): port_id=0 queue_id=0 pktlen=205 tx_first=63 tx_last=63 PMD: ixgbe_xmit_pkts(): Not enough free TX descriptors nb_used= 1 nb_free= 0 (port=0 queue=0) PMD: ixgbe_xmit_cleanup(): TX descriptor 95 is not done(port=0 queue=0) tx burst failed, ret 0 , n 16, Success PMD: ixgbe_xmit_cleanup(): TX descriptor 95 is not done(port=0 queue=0) PMD: ixgbe_xmit_pkts(): port_id=0 queue_id=0 pktlen=205 tx_first=63 tx_last=63 PMD: ixgbe_xmit_pkts(): Not enough free TX descriptors nb_used= 1 nb_free= 0 (port=0 queue=0) PMD: ixgbe_xmit_cleanup(): TX descriptor 95 is not done(port=0 queue=0) tx burst failed, ret 0 , n 16, Success PMD: ixgbe_xmit_cleanup(): TX descriptor 95 is not done(port=0 queue=0) PMD: ixgbe_xmit_pkts(): port_id=0 queue_id=0 pktlen=54 tx_first=63 tx_last=63 PMD: ixgbe_xmit_pkts(): Not enough free TX descriptors nb_used= 1 nb_free= 0 (port=0 queue=0) PMD: ixgbe_xmit_cleanup(): TX descriptor 95 is not done(port=0 queue=0) tx burst failed, ret 0 , n 5, Success PMD: ixgbe_xmit_cleanup(): TX descriptor 95 is not done(port=0 queue=0) -- Best Regards, zimeiw At 2017-08-23 17:04:44, "Ferruh Yigit" <ferruh.yi...@intel.com> wrote: >On 8/23/2017 6:23 AM, zimeiw wrote: >> hi, >> >> Any comments? >> >> is it possible NIC haredware issue? or NIC firmware version isn't match >> dpdk driver? > >AFAIK for there isn't this kind of dependency for ixgbe. ixgbe driver >maintainer cc'ed for more information. > >> >> -- >> Best Regards, >> zimeiw >> >> At 2017-08-23 08:52:38, "zimeiw" <zim...@163.com> wrote: >> >> hi, >> >> "disable tso" logs were printed by my app. >> >> dpdk-17.05.1 is download from dpdk website, it is stable release. >> >> >> -- >> Best Regards, >> zimeiw >> >> >> At 2017-08-22 22:23:54, "Ferruh Yigit" <ferruh.yi...@intel.com >> <mailto:ferruh.yi...@intel.com>> wrote: >> >On 8/22/2017 2:51 PM, zimeiw wrote: >> >> hi, >> >> ===== >> >> check the dpdk default config, CONFIG_RTE_ETHDEV_TX_PREPARE_NOOP >> isn't >> >> enable. >> >> # >> >> # Turn off Tx preparation stage >> >> # >> >> # Warning: rte_ethdev_tx_prepare() can be safely disabled only if >> using a >> >> # driver which do not implement any Tx preparation. >> >> # >> >> CONFIG_RTE_ETHDEV_TX_PREPARE_NOOP=n >> >> >> >> ==== >> >> I enable IXGBE debug for dpdk-17.05.1 and dpdk-17.02. the logs are in >> >> the attachment. >> >> app with dpdk-17.02 works ok. >> >> app with dpdk-17.05.1 has below issue. it seems TX descriptor 95 >> can't >> >> be handled by NIC? >> > >> >This is interesting, no idea what is going on, trying dpdk-17.05 can be >> >an option, to be sure if this issue is in the original release or in the >> >stable tree? >> > >> >btw, in 17.05.01, there are "disable tso" logs, do you know where they >> >are coming from? >> > >> >> >> >> >> >> PMD: ixgbe_xmit_pkts(): port_id=0 queue_id=0 tx_tail=63 nb_tx=11 >> >> PMD: ixgbe_xmit_cleanup(): TX descriptor 95 is not done(port=0 >> queue=0) >> > >> >Descriptor Done bit is not set, HW says descriptor is still in use, so >> >cleaning descriptor fails. >> > >> >> PMD: ixgbe_xmit_pkts(): port_id=0 queue_id=0 pktlen=205 tx_first=63 >> >> tx_last=63 >> >> PMD: ixgbe_xmit_pkts(): Not enough free TX descriptors nb_used= 1 >> >> nb_free= 0 (port=0 queue=0) >> > >> >And after a while there is no free descriptor left and Tx fails. >> > >> >> PMD: ixgbe_xmit_cleanup(): TX descriptor 95 is not done(port=0 >> queue=0) >> >> tx burst failed, ret 0 , n 16, nb_prep 0 , Success >> >> PMD: ixgbe_xmit_cleanup(): TX descriptor 95 is not done(port=0 >> queue=0) >> >> PMD: ixgbe_xmit_pkts(): port_id=0 queue_id=0 pktlen=205 tx_first=63 >> >> tx_last=63 >> >> PMD: ixgbe_xmit_pkts(): Not enough free TX descriptors nb_used= 1 >> >> nb_free= 0 (port=0 queue=0) >> >> PMD: ixgbe_xmit_cleanup(): TX descriptor 95 is not done(port=0 >> queue=0) >> >> tx burst failed, ret 0 , n 16, nb_prep 0 , Success >> >> PMD: ixgbe_xmit_cleanup(): TX descriptor 95 is not done(port=0 >> queue=0) >> >> PMD: ixgbe_xmit_pkts(): port_id=0 queue_id=0 pktlen=54 tx_first=63 >> >> tx_last=63 >> >> PMD: ixgbe_xmit_pkts(): Not enough free TX descriptors nb_used= 1 >> >> nb_free= 0 (port=0 queue=0) >> >> PMD: ixgbe_xmit_cleanup(): TX descriptor 95 is not done(port=0 >> queue=0) >> >> tx burst failed, ret 0 , n 5, nb_prep 0 , Success >> >> >> >> >> >> >> >> -- >> >> Best Regards, >> >> zimeiw >> >> >> >> >> >> At 2017-08-22 20:13:57, "Ferruh Yigit" <ferruh.yi...@intel.com >> <mailto:ferruh.yi...@intel.com>> wrote: >> >>>On 8/22/2017 12:00 PM, zimeiw wrote: >> >>>> hi, >> >>>> >> >>>> i set .txq_flags = 0. >> >>>> and change the "send burst" code as below. >> >>>> and printf some logs. >> >>>> according to the logs, rte_eth_tx_burst() return 0, no error happen >> in >> >>>> rte_eth_tx_burst(). >> >>>> >> >>>> ==================== >> >>>> struct rte_mbuf **m_table; >> >>>> int ret; >> >>>> uint16_t queueid; >> >>>> uint16_t nb_prep; >> >>>> >> >>>> queueid = qconf->tx_queue_id[port]; >> >>>> m_table = (struct rte_mbuf **)qconf->tx_mbufs[port].m_table; >> >>>> >> >>>> nb_prep = rte_eth_tx_prepare(port, queueid, m_table, n); >> >>>> if (nb_prep != n) >> >>>> printf("Preparing packet burst %d to transmit failed: >> %s\n", >> >>>> nb_prep, rte_strerror(rte_errno)); >> >>> >> >>>m->ol_flags should request offloading (PKT_TX_...), otherwise prepare >> >>>does not do the work. >> >>> >> >>>but m->ol_flags is not changed, if this is working with 17.02, issue >> can >> >>>be something else. >> >>> >> >>>You may try debugging ixgbe PMD to figure out at which point it gives >> an >> >>>error. >> >>> >> >>>> >> >>>> ret = rte_eth_tx_burst(port, queueid, m_table, nb_prep); >> >>>> if (unlikely(ret < n)) >> >>>> { >> >>>> printf("tx burst failed, ret %d , n %d, nb_prep %d , %s \n", >> >>>> ret, n, nb_prep, rte_strerror(rte_errno) ); >> >>>> do >> >>>> { >> >>>> rte_pktmbuf_free(m_table[ret]); >> >>>> } while (++ret < n); >> >>>> } >> >>>> ======================= >> >>>> >> >>>> =====log====== >> >>>> Checking link status done >> >>>> Port 0 Link Up - speed 10000 Mbps - full-duplex >> >>>> USER8: main loop on lcore 1 >> >>>> USER8: -- lcoreid=1 portid=0 rxqueueid=0 >> >>>> nb ports 1 hz: 1698027100 >> >>>> tx burst failed, ret 2 , n 16, nb_prep 16 , Success >> >>>> tx burst failed, ret 0 , n 16, nb_prep 16 , Success >> >>>> tx burst failed, ret 0 , n 16, nb_prep 16 , Success >> >>>> tx burst failed, ret 0 , n 16, nb_prep 16 , Success >> >>>> tx burst failed, ret 0 , n 16, nb_prep 16 , Success >> >>>> tx burst failed, ret 0 , n 16, nb_prep 16 , Success >> >>>> tx burst failed, ret 0 , n 16, nb_prep 16 , Success >> >>>> tx burst failed, ret 0 , n 16, nb_prep 16 , Success >> >>>> tx burst failed, ret 0 , n 16, nb_prep 16 , Success >> >>>> tx burst failed, ret 0 , n 16, nb_prep 16 , Success >> >>>> >> >>>> ==========NIC firware version ======= >> >>>> # ethtool -i enp2s0f1 >> >>>> driver: ixgbe >> >>>> version: 4.2.1-k >> >>>> firmware-version: 0x61bd0001 >> >>>> expansion-rom-version: >> >>>> bus-info: 0000:02:00.1 >> >>>> supports-statistics: yes >> >>>> supports-test: yes >> >>>> supports-eeprom-access: yes >> >>>> supports-register-dump: yes >> >>>> supports-priv-flags: no >> >>>> >> >>>> >> >>>> -- >> >>>> Best Regards, >> >>>> zimeiw >> >>>> >> >>>> >> >>>> At 2017-08-22 17:31:56, "Ferruh Yigit" <ferruh.yi...@intel.com >> <mailto:ferruh.yi...@intel.com>> wrote: >> >>>>>On 8/22/2017 9:30 AM, zimeiw wrote: >> >>>>>> hi, >> >>>>>> >> >>>>>> no use rte_eth_tx_prepare()? >> >>>>>> >> >>>>>> I set txq_flags value as below, may it cause this issue? >> >>>>>> >> >>>>>> .txq_flags = ~ETH_TXQ_FLAGS_NOXSUMS >> >>>>> >> >>>>>So you are enabling all csum offloading, for Tx path this requires >> >>>>>packets to be prepared before sending to the NIC. >> >>>>> >> >>>>>Can you please test with rte_eth_tx_prepare()? testpmd has the >> sample usage. >> >>>>> >> >>>>>Thanks, >> >>>>>ferruh >> >>>>> >> >>>>>> >> >>>>>> -- >> >>>>>> Best Regards, >> >>>>>> zimeiw >> >>>>>> >> >>>>>> >> >>>>>> At 2017-08-22 15:49:56, "Ferruh Yigit" <ferruh.yi...@intel.com >> <mailto:ferruh.yi...@intel.com>> wrote: >> >>>>>>>On 8/22/2017 5:45 AM, zimeiw wrote: >> >>>>>>>> hi, >> >>>>>>>> My test env is dell T430 server, NIC is intel 82599ES. >> >>>>>>>> I integrate my app with dpdk-17.05.1 version. after do some tcp >> traffic testing, app can receive packets from the 82599 NIC, but 82599 NIC >> send packets failed. >> >>>>>>>> But my app with dpdk-17.02 works well. >> >>>>>>>> >> >>>>>>>> >> >>>>>>>> If any change with ixgbe driver? or need some additional >> configuration for the NIC? >> >>>>>>> >> >>>>>>>tx_pkt_prepare added but it has been added on 17.02 which seems >> working >> >>>>>>>for your case. >> >>>>>>> >> >>>>>>>Just to double check, are you using rte_eth_tx_prepare()? >> >>>>>>> >> >>>>>>>> >> >>>>>>>> >> >>>>>>>> >> >>>>>>>> Thanks. >> >>>>>>>> >> >>>>>>>> >> >>>>>>>> -- >> >>>>>>>> >> >>>>>>>> Best Regards, >> >>>>>>>> zimeiw >> >>>>>>>> >> >>>>>>> >> >>>>>> >> >>>>>> >> >>>>>> >> >>>>>> >> >>>>>> >> >>>>> >> >>>> >> >>>> >> >>>> >> >>>> >> >>>> >> >>> >> >> >> >> >> >> >> >> >> >> >> > >> >> >> >> >> >> >> >> >> >
