[dpdk-dev] bypass event for reboot
Hi Konstantin, could you please comment on this. thanks, Sharath On Fri, Apr 4, 2014 at 6:12 PM, Sharath wrote: > Hi! > > I have been able to use the below API to set the bypass event on power off > and power on > > * int rte_eth_dev_bypass_event_store (uint8_t port, uint32_t event, > uint32_t state) > > > > is there a similar way to set the bypass state on "reboot" event ? > > can anyone please let me know about this. > > thanks in advance > Sharath. >
[dpdk-dev] [PATCH v3 1/2] ixgbe: release software locked semaphores on initialization
It may happen that DPDK application gets killed while having
acquired locks on the ethernet hardware, causing these locks to
be never released. On next restart of the application, DPDK
skip those ports because it can not acquire the lock,
this may cause some ports (or even complete board if SMBI is locked)
to be inaccessible from DPDK application until reboot of the
hardware.
This patch release locks that are supposed to be locked due to
an improper exit of the application.
Signed-off-by: Didier Pallard
---
v3: Move code outside from base driver
lib/librte_pmd_ixgbe/ixgbe_ethdev.c | 41 ---
1 file changed, 38 insertions(+), 3 deletions(-)
diff --git a/lib/librte_pmd_ixgbe/ixgbe_ethdev.c
b/lib/librte_pmd_ixgbe/ixgbe_ethdev.c
index fdf6c1d..637fa88 100644
--- a/lib/librte_pmd_ixgbe/ixgbe_ethdev.c
+++ b/lib/librte_pmd_ixgbe/ixgbe_ethdev.c
@@ -587,6 +587,38 @@ ixgbe_dcb_init(struct ixgbe_hw *hw,struct ixgbe_dcb_config
*dcb_config)
}
/*
+ * Ensure that all locks are released before first NVM or PHY access
+ */
+static void
+ixgbe_swfw_lock_reset(struct ixgbe_hw *hw) {
+ uint16_t mask;
+
+ /*
+ * Phy lock should not fail in this early stage. If this is the case,
+ * it is due to an improper exit of the application.
+ * So force the release of the faulty lock. Release of common lock
+ * is done automatically by swfw_sync function.
+ */
+ mask = IXGBE_GSSR_PHY0_SM << hw->bus.func;
+ if (ixgbe_acquire_swfw_semaphore(hw, mask) < 0) {
+ DEBUGOUT1("SWFW phy%d lock released", hw->bus.func);
+ }
+ ixgbe_release_swfw_semaphore(hw, mask);
+
+ /*
+ * Those one are more tricky since they are common to all ports; but
+ * swfw_sync retries last long enough (1s) to be almost sure that if
+ * lock can not be taken it is due to an improper lock of the
+ * semaphore.
+ */
+ mask = IXGBE_GSSR_EEP_SM | IXGBE_GSSR_MAC_CSR_SM | IXGBE_GSSR_SW_MNG_SM;
+ if (ixgbe_acquire_swfw_semaphore(hw, mask) < 0) {
+ DEBUGOUT("SWFW common locks released");
+ }
+ ixgbe_release_swfw_semaphore(hw, mask);
+}
+
+/*
* This function is based on code in ixgbe_attach() in ixgbe/ixgbe.c.
* It returns 0 on success.
*/
@@ -643,6 +675,12 @@ eth_ixgbe_dev_init(__attribute__((unused)) struct
eth_driver *eth_drv,
return -EIO;
}
+ /* pick up the PCI bus settings for reporting later */
+ ixgbe_get_bus_info(hw);
+
+ /* Unlock any pending hardware semaphore */
+ ixgbe_swfw_lock_reset(hw);
+
/* Initialize DCB configuration*/
memset(dcb_config, 0, sizeof(struct ixgbe_dcb_config));
ixgbe_dcb_init(hw,dcb_config);
@@ -700,9 +738,6 @@ eth_ixgbe_dev_init(__attribute__((unused)) struct
eth_driver *eth_drv,
/* disable interrupt */
ixgbe_disable_intr(hw);
- /* pick up the PCI bus settings for reporting later */
- ixgbe_get_bus_info(hw);
-
/* reset mappings for queue statistics hw counters*/
ixgbe_reset_qstat_mappings(hw);
--
1.7.10.4
[dpdk-dev] [PATCH v3 2/2] igb: release software locked semaphores on initialization
It may happen that DPDK application gets killed while having
acquired locks on the ethernet hardware, causing these locks to
be never released. On next restart of the application, DPDK
skip those ports because it can not acquire the lock,
this may cause some ports (or even complete board if SMBI is locked)
to be inaccessible from DPDK application until reboot of the
hardware.
This patch release locks that are supposed to be locked due to
an improper exit of the application.
Signed-off-by: Didier Pallard
---
v3: Move code outside from base driver
lib/librte_pmd_e1000/igb_ethdev.c | 69 -
1 file changed, 68 insertions(+), 1 deletion(-)
diff --git a/lib/librte_pmd_e1000/igb_ethdev.c
b/lib/librte_pmd_e1000/igb_ethdev.c
index 184e7d6..673b4de 100644
--- a/lib/librte_pmd_e1000/igb_ethdev.c
+++ b/lib/librte_pmd_e1000/igb_ethdev.c
@@ -320,6 +320,61 @@ igb_identify_hardware(struct rte_eth_dev *dev)
}
static int
+igb_reset_swfw_lock(struct e1000_hw *hw)
+{
+ int ret_val;
+
+ /*
+* Do mac ops initialization manually here, since we will need
+* some function pointers set by this call.
+*/
+ ret_val = e1000_init_mac_params(hw);
+ if (ret_val)
+ return ret_val;
+
+ /*
+* SMBI lock should not fail in this early stage. If this is the case,
+* it is due to an improper exit of the application.
+* So force the release of the faulty lock.
+*/
+ if (e1000_get_hw_semaphore_generic(hw) < 0) {
+ DEBUGOUT("SMBI lock released");
+ }
+ e1000_put_hw_semaphore_generic(hw);
+
+ if (hw->mac.ops.acquire_swfw_sync != NULL) {
+ uint16_t mask;
+
+ /*
+* Phy lock should not fail in this early stage. If this is the
case,
+* it is due to an improper exit of the application.
+* So force the release of the faulty lock.
+*/
+ mask = E1000_SWFW_PHY0_SM << hw->bus.func;
+ if (hw->bus.func > E1000_FUNC_1)
+ mask <<= 2;
+ if (hw->mac.ops.acquire_swfw_sync(hw, mask) < 0) {
+ DEBUGOUT1("SWFW phy%d lock released", hw->bus.func);
+ }
+ hw->mac.ops.release_swfw_sync(hw, mask);
+
+ /*
+* This one is more tricky since it is common to all ports; but
+* swfw_sync retries last long enough (1s) to be almost sure
that if
+* lock can not be taken it is due to an improper lock of the
+* semaphore.
+*/
+ mask = E1000_SWFW_EEP_SM;
+ if (hw->mac.ops.acquire_swfw_sync(hw, mask) < 0) {
+ DEBUGOUT("SWFW common locks released");
+ }
+ hw->mac.ops.release_swfw_sync(hw, mask);
+ }
+
+ return E1000_SUCCESS;
+}
+
+static int
eth_igb_dev_init(__attribute__((unused)) struct eth_driver *eth_drv,
struct rte_eth_dev *eth_dev)
{
@@ -348,13 +403,25 @@ eth_igb_dev_init(__attribute__((unused)) struct
eth_driver *eth_drv,
hw->hw_addr= (void *)pci_dev->mem_resource[0].addr;
igb_identify_hardware(eth_dev);
- if (e1000_setup_init_funcs(hw, TRUE) != E1000_SUCCESS) {
+ if (e1000_setup_init_funcs(hw, FALSE) != E1000_SUCCESS) {
error = -EIO;
goto err_late;
}
e1000_get_bus_info(hw);
+ /* Reset any pending lock */
+ if (igb_reset_swfw_lock(hw) != E1000_SUCCESS) {
+ error = -EIO;
+ goto err_late;
+ }
+
+ /* Finish initialization */
+ if (e1000_setup_init_funcs(hw, TRUE) != E1000_SUCCESS) {
+ error = -EIO;
+ goto err_late;
+ }
+
hw->mac.autoneg = 1;
hw->phy.autoneg_wait_to_complete = 0;
hw->phy.autoneg_advertised = E1000_ALL_SPEED_DUPLEX;
--
1.7.10.4
[dpdk-dev] Using DPDK in a multiprocess environment
Hi, I'd like to split DPDK application functionality into a setup (primary) process and business logic (secondary) processes. The secondary processes access the hardware queues directly (exclusive queue per process) and not through software rings. I'm running into an initialization problem: - The primary starts and sets up memory and ports and then goes to sleep waiting for termination signal - Secondary processes fail when probing the PCI bus for devices (required, otherwise I get 0 ports visible in the secondary) The error is directly related to the secondary failing to get the *same* virtual address for mmap'ing the UIO device fd's. The reason is that the secondary processes has considerably more shared objects loaded and some of these are loaded and mapped into addresses which the primary used to map UIO fd's. The pci_map_resource() (linuxapp/eal_pci.c) code explicitly requires that the secondary processes get the same mmap'ed address as given to the primary. 1) Is this behavior (same mmap address) required? 2) If so, is there a workaround to cause PCI areas of UIO devices to be mapped to the same location in arbitrary processes? The samples work just fine since all primary and secondary processes have similar set and load order for .so's Using v1.6 on Ubuntu 12.04 64b, ixgbe devices, 1GB hugepages, ASLR disabled. Thanks, Etai
[dpdk-dev] Using DPDK in a multiprocess environment
Jeff, Thanks for the quick reply. I'll see if calling eal_init earlier resolves the problem I'm seeing. I'm not sure this will resolve the issue if shared objects are loaded before main() starts... I understand the rationale for having the same mbuf addresses across processes. And indeed they're mapped just fine (--virt-addr also gives some control over the mapping?). I was wondering if the same logic applies to the mapping of device PCI addresses. Are they shared or passed around between processes in the same way? Thanks again for the quick response, Etai ?? 8 2014 18:54, "Shaw, Jeffrey B" ???: > Have you tried calling "rte_eal_init()" closer to the beginning of the > program in your secondary process (i.e. the first thing in main())? > > The same mmap address is required. The reason is simple, if process A > thinks the virtual address of an mbuf is 123, and process B thinks the > virtual address of the same mbuf is 456, either process may segmentation > fault, accessing mbuf memory that is not actually mapped into the processes > address space. > > Jeff > > -Original Message- > From: dev [mailto:dev-bounces at dpdk.org] On Behalf Of Etai Lev Ran > Sent: Tuesday, April 08, 2014 8:13 AM > To: dev at dpdk.org > Subject: [dpdk-dev] Using DPDK in a multiprocess environment > > Hi, > > > > I'd like to split DPDK application functionality into a setup (primary) > process and business logic (secondary) processes. > > The secondary processes access the hardware queues directly (exclusive > queue per process) and not through software rings. > > > > I'm running into an initialization problem: > > - The primary starts and sets up memory and ports and then goes to > sleep waiting for termination signal > > - Secondary processes fail when probing the PCI bus for devices > (required, otherwise I get 0 ports visible in the secondary) > > > > The error is directly related to the secondary failing to get the *same* > virtual address for mmap'ing the UIO device fd's. > > The reason is that the secondary processes has considerably more shared > objects loaded and some of these are > > loaded and mapped into addresses which the primary used to map UIO fd's. > > The pci_map_resource() (linuxapp/eal_pci.c) code explicitly requires that > the secondary processes get the same mmap'ed > > address as given to the primary. > > > > 1) Is this behavior (same mmap address) required? > > 2) If so, is there a workaround to cause PCI areas of UIO devices to > be > mapped to the same location in arbitrary processes? > > > > The samples work just fine since all primary and secondary processes have > similar set and load order for .so's > > > > Using v1.6 on Ubuntu 12.04 64b, ixgbe devices, 1GB hugepages, ASLR > disabled. > > > > Thanks, > > Etai > > > >
[dpdk-dev] Using DPDK in a multiprocess environment
Have you tried calling "rte_eal_init()" closer to the beginning of the program in your secondary process (i.e. the first thing in main())? The same mmap address is required. The reason is simple, if process A thinks the virtual address of an mbuf is 123, and process B thinks the virtual address of the same mbuf is 456, either process may segmentation fault, accessing mbuf memory that is not actually mapped into the processes address space. Jeff -Original Message- From: dev [mailto:dev-boun...@dpdk.org] On Behalf Of Etai Lev Ran Sent: Tuesday, April 08, 2014 8:13 AM To: dev at dpdk.org Subject: [dpdk-dev] Using DPDK in a multiprocess environment Hi, I'd like to split DPDK application functionality into a setup (primary) process and business logic (secondary) processes. The secondary processes access the hardware queues directly (exclusive queue per process) and not through software rings. I'm running into an initialization problem: - The primary starts and sets up memory and ports and then goes to sleep waiting for termination signal - Secondary processes fail when probing the PCI bus for devices (required, otherwise I get 0 ports visible in the secondary) The error is directly related to the secondary failing to get the *same* virtual address for mmap'ing the UIO device fd's. The reason is that the secondary processes has considerably more shared objects loaded and some of these are loaded and mapped into addresses which the primary used to map UIO fd's. The pci_map_resource() (linuxapp/eal_pci.c) code explicitly requires that the secondary processes get the same mmap'ed address as given to the primary. 1) Is this behavior (same mmap address) required? 2) If so, is there a workaround to cause PCI areas of UIO devices to be mapped to the same location in arbitrary processes? The samples work just fine since all primary and secondary processes have similar set and load order for .so's Using v1.6 on Ubuntu 12.04 64b, ixgbe devices, 1GB hugepages, ASLR disabled. Thanks, Etai
[dpdk-dev] Using DPDK in a multiprocess environment
Etai, If this doesn?t work, then you will need to change the virtual address range that is used by DPDK. By default this is set dynamically, however; with DPDK 1.6you can change it to any region in the virtual address space you want. The problem you have is what you stated, the secondary process is built with more shared libraries, which load upon application start, and are occupying the region that DPDK allocates in the primary for shared regions. In DPDK version 1.6 there is an option to change the base address. It is --base-virtaddr With this option you can set the base address for where the huge pages are mapped into the process virtual address space. This is all implemented within $DPDK_DIR/lib/librte_eal/linuxapp/eal/eal_memory.c Gerald On 4/8/14, 9:07 AM, "elevran" wrote: >Jeff, > >Thanks for the quick reply. > >I'll see if calling eal_init earlier resolves the problem I'm seeing. I'm >not sure this will resolve the issue if shared objects are loaded before >main() starts... > >I understand the rationale for having the same mbuf addresses across >processes. And indeed they're mapped just fine (--virt-addr also gives >some >control over the mapping?). >I was wondering if the same logic applies to the mapping of device PCI >addresses. Are they shared or passed around between processes in the same >way? > >Thanks again for the quick response, >Etai >?? 8 2014 18:54, "Shaw, Jeffrey B" >???: > >> Have you tried calling "rte_eal_init()" closer to the beginning of the >> program in your secondary process (i.e. the first thing in main())? >> >> The same mmap address is required. The reason is simple, if process A >> thinks the virtual address of an mbuf is 123, and process B thinks the >> virtual address of the same mbuf is 456, either process may segmentation >> fault, accessing mbuf memory that is not actually mapped into the >>processes >> address space. >> >> Jeff >> >> -Original Message- >> From: dev [mailto:dev-bounces at dpdk.org] On Behalf Of Etai Lev Ran >> Sent: Tuesday, April 08, 2014 8:13 AM >> To: dev at dpdk.org >> Subject: [dpdk-dev] Using DPDK in a multiprocess environment >> >> Hi, >> >> >> >> I'd like to split DPDK application functionality into a setup (primary) >> process and business logic (secondary) processes. >> >> The secondary processes access the hardware queues directly (exclusive >> queue per process) and not through software rings. >> >> >> >> I'm running into an initialization problem: >> >> - The primary starts and sets up memory and ports and then >>goes to >> sleep waiting for termination signal >> >> - Secondary processes fail when probing the PCI bus for devices >> (required, otherwise I get 0 ports visible in the secondary) >> >> >> >> The error is directly related to the secondary failing to get the *same* >> virtual address for mmap'ing the UIO device fd's. >> >> The reason is that the secondary processes has considerably more shared >> objects loaded and some of these are >> >> loaded and mapped into addresses which the primary used to map UIO fd's. >> >> The pci_map_resource() (linuxapp/eal_pci.c) code explicitly requires >>that >> the secondary processes get the same mmap'ed >> >> address as given to the primary. >> >> >> >> 1) Is this behavior (same mmap address) required? >> >> 2) If so, is there a workaround to cause PCI areas of UIO devices >>to >> be >> mapped to the same location in arbitrary processes? >> >> >> >> The samples work just fine since all primary and secondary processes >>have >> similar set and load order for .so's >> >> >> >> Using v1.6 on Ubuntu 12.04 64b, ixgbe devices, 1GB hugepages, ASLR >> disabled. >> >> >> >> Thanks, >> >> Etai >> >> >> >>
