[dpdk-dev] Interrupt is catched in kernel, but not handled
Hello, This is not a specific dpdk question, but more related to using of uio_pci_generic. Unfortunately, I don't find any examples of using the generic driver from userspace, so this is my last hope to reach a suggestion. I installed centos8 , and I use uio_pci_generic, in order to communicate with FPGA (Xilinx) through PCIe in userspace. The cpu is ATOM cpu. I first install the driver: echo "10ee 0007" > /sys/bus/pci/drivers/uio_pci_generic/new_id Reading from BAR works perfectly but there is an issue with interrupts. I use userspace application which wait for interrupt, exactly as described in code example here: https://www.kernel.org/doc/html/v4.1...uio-howto.html I than trigger an interrupt from FPGA, but no print from the userspace application is given and there is an exception: irq 23: nobody cared (try booting with the "irqpoll" option) [ 91.030760] CPU: 2 PID: 0 Comm: swapper/2 Not tainted 4.18.16 #6 [ 91.037037] Hardware name: /conga-MA5, BIOS MA50R000 10/30/2019 [ 91.043302] Call Trace: [ 91.045881] [ 91.048002] dump_stack+0x5c/0x80 [ 91.051464] __report_bad_irq+0x35/0xaf [ 91.055465] note_interrupt.cold.9+0xa/0x63 [ 91.059823] handle_irq_event_percpu+0x68/0x70 [ 91.064470] handle_irq_event+0x37/0x57 [ 91.068481] handle_fasteoi_irq+0x97/0x150 [ 91.072758] handle_irq+0x1a/0x30 [ 91.076230] do_IRQ+0x44/0xd0 [ 91.079333] common_interrupt+0xf/0xf [ 91.083154] [ 91.085360] RIP: 0010:cpuidle_enter_state+0x7d/0x220 [ 91.090563] Code: e8 18 1a 45 00 41 89 c4 e8 d0 50 b1 ff 65 8b 3d d9 db e5 5f e8 44 4f b1 ff 31 ff 48 89 c3 e8 ea 61 b1 ff fb 66 0f 1f 44 00 00 <48> b8 ff ff ff ff f3 01 00 00 4c 29 eb ba ff ff ff 7f 48 89 d9 48 [ 91.110283] RSP: 0018:b20e806b7ea8 EFLAGS: 0286 ORIG_RAX: ffda [ 91.118203] RAX: 90133fd214c0 RBX: 0015316faa09 RCX: 001f [ 91.125662] RDX: 0015316faa09 RSI: RDI: [ 91.133133] RBP: 90133fd2b300 R08: 011741eb842c R09: 0006 [ 91.140605] R10: R11: 90133fd205a8 R12: 0001 [ 91.148068] R13: 0015316f98b9 R14: 0001 R15: [ 91.155523] ? cpuidle_enter_state+0x76/0x220 [ 91.160088] do_idle+0x221/0x260 [ 91.163470] cpu_startup_entry+0x6a/0x70 [ 91.167588] start_secondary+0x1a4/0x1f0 [ 91.171676] secondary_startup_64+0xb7/0xc0 [ 91.176043] handlers: [ 91.178419] [] uio_interrupt [ 91.183054] Disabling IRQ #23 So, I started debugging this issue in kernel uio_pci_generic.c code, and according to the prints below, it seems that irq is catched but not delivered to userspace: static irqreturn_t irqhandler(int irq, struct uio_info *info) { struct uio_pci_generic_dev *gdev = to_uio_pci_generic_dev(info); printk("i'm here 1\n"); <<--- this is printed if (!pci_check_and_mask_intx(gdev->pdev)) return IRQ_NONE; printk("i'm here 2\n"); <<--- but this is NOT printed /* UIO core will signal the user process. */ return IRQ_HANDLED; } Reading documentation, I see that pci_check_and_mask_intx is actually checking if interrupt bit is set in configuration space. Since it seems to return 0 ,it should mean that it finds that this bit is not enabled. But how can it be that irq is triggered and status bit is not enabled in configuration space ? The device appear as following with lspci -vv: 02:00.0 RAM memory: Xilinx Corporation Default PCIe endpoint ID Subsystem: Xilinx Corporation Default PCIe endpoint ID Control: I/O- Mem+ BusMaster- SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx- Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- SERR-
[dpdk-dev] [PATCH 0/9] updates for hns3 PMD driver
From: "Wei Hu (Xavier)" This series are updates for hns3 ethernet PMD driver. Hao Chen (4): net/hns3: support Rx interrupt net/hns3: optimize RSS's default algorithm net/hns3: remove the redundant function call net/hns3: remove the unused macros Hongbo Zheng (1): net/hns3: get link state change through mailbox Huisong Li (1): net/hns3: modify custom macro Wei Hu (Xavier) (3): net/hns3: modify the return value of enable msix net/hns3: remove the redundant variable initialization net/hns3: remove the unnecessary assignment doc/guides/nics/features/hns3.ini| 1 + doc/guides/nics/features/hns3_vf.ini | 1 + drivers/net/hns3/hns3_cmd.h | 28 + drivers/net/hns3/hns3_ethdev.c | 176 --- drivers/net/hns3/hns3_ethdev.h | 2 +- drivers/net/hns3/hns3_ethdev_vf.c| 167 ++--- drivers/net/hns3/hns3_mbx.c | 37 ++ drivers/net/hns3/hns3_mbx.h | 21 drivers/net/hns3/hns3_regs.h | 3 + drivers/net/hns3/hns3_rss.c | 14 ++- drivers/net/hns3/hns3_rss.h | 10 -- drivers/net/hns3/hns3_rxtx.c | 58 - drivers/net/hns3/hns3_rxtx.h | 4 + 13 files changed, 473 insertions(+), 49 deletions(-) -- 2.23.0
[dpdk-dev] [PATCH 0/9] updates for hns3 PMD driver
From: "Wei Hu (Xavier)" This series are updates for hns3 ethernet PMD driver. Hao Chen (4): net/hns3: support Rx interrupt net/hns3: optimize RSS's default algorithm net/hns3: remove the redundant function call net/hns3: remove the unused macros Hongbo Zheng (1): net/hns3: get link state change through mailbox Huisong Li (1): net/hns3: modify custom macro Wei Hu (Xavier) (3): net/hns3: modify the return value of enable msix net/hns3: remove the redundant variable initialization net/hns3: remove the unnecessary assignment doc/guides/nics/features/hns3.ini| 1 + doc/guides/nics/features/hns3_vf.ini | 1 + drivers/net/hns3/hns3_cmd.h | 28 + drivers/net/hns3/hns3_ethdev.c | 176 --- drivers/net/hns3/hns3_ethdev.h | 2 +- drivers/net/hns3/hns3_ethdev_vf.c| 167 ++--- drivers/net/hns3/hns3_mbx.c | 37 ++ drivers/net/hns3/hns3_mbx.h | 21 drivers/net/hns3/hns3_regs.h | 3 + drivers/net/hns3/hns3_rss.c | 14 ++- drivers/net/hns3/hns3_rss.h | 10 -- drivers/net/hns3/hns3_rxtx.c | 58 - drivers/net/hns3/hns3_rxtx.h | 4 + 13 files changed, 473 insertions(+), 49 deletions(-) -- 2.23.0
[dpdk-dev] [PATCH 1/9] net/hns3: support Rx interrupt
From: Hao Chen
This patch supports of receive packets through interrupt mode for hns3
PF/VF driver. The following ops functions should be implemented defined
in struct eth_dev_ops:
rx_queue_intr_enable
rx_queue_intr_disable
rx_queue_count
Signed-off-by: Hao Chen
Signed-off-by: Wei Hu (Xavier)
---
doc/guides/nics/features/hns3.ini| 1 +
doc/guides/nics/features/hns3_vf.ini | 1 +
drivers/net/hns3/hns3_cmd.h | 28 +
drivers/net/hns3/hns3_ethdev.c | 160 --
drivers/net/hns3/hns3_ethdev_vf.c| 166 ---
drivers/net/hns3/hns3_mbx.h | 13 +++
drivers/net/hns3/hns3_regs.h | 3 +
drivers/net/hns3/hns3_rxtx.c | 51
drivers/net/hns3/hns3_rxtx.h | 4 +
9 files changed, 405 insertions(+), 22 deletions(-)
diff --git a/doc/guides/nics/features/hns3.ini
b/doc/guides/nics/features/hns3.ini
index 6df789ed1..cd5c08a9d 100644
--- a/doc/guides/nics/features/hns3.ini
+++ b/doc/guides/nics/features/hns3.ini
@@ -5,6 +5,7 @@
;
[Features]
Link status = Y
+Rx interrupt = Y
MTU update = Y
Jumbo frame = Y
Promiscuous mode = Y
diff --git a/doc/guides/nics/features/hns3_vf.ini
b/doc/guides/nics/features/hns3_vf.ini
index 41497c4c2..fd00ac3e2 100644
--- a/doc/guides/nics/features/hns3_vf.ini
+++ b/doc/guides/nics/features/hns3_vf.ini
@@ -5,6 +5,7 @@
;
[Features]
Link status = Y
+Rx interrupt = Y
MTU update = Y
Jumbo frame = Y
Unicast MAC filter = Y
diff --git a/drivers/net/hns3/hns3_cmd.h b/drivers/net/hns3/hns3_cmd.h
index be0ecbe86..897dc1420 100644
--- a/drivers/net/hns3/hns3_cmd.h
+++ b/drivers/net/hns3/hns3_cmd.h
@@ -209,6 +209,10 @@ enum hns3_opcode_type {
/* SFP command */
HNS3_OPC_SFP_GET_SPEED = 0x7104,
+ /* Interrupts commands */
+ HNS3_OPC_ADD_RING_TO_VECTOR = 0x1503,
+ HNS3_OPC_DEL_RING_TO_VECTOR = 0x1504,
+
/* Error INT commands */
HNS3_QUERY_MSIX_INT_STS_BD_NUM = 0x1513,
HNS3_QUERY_CLEAR_ALL_MPF_MSIX_INT = 0x1514,
@@ -673,6 +677,30 @@ struct hns3_tqp_map_cmd {
uint8_t rsv[18];
};
+#define HNS3_RING_TYPE_B 0
+#define HNS3_RING_TYPE_TX 0
+#define HNS3_RING_TYPE_RX 1
+#define HNS3_RING_GL_IDX_S 0
+#define HNS3_RING_GL_IDX_M GENMASK(1, 0)
+#define HNS3_RING_GL_RX0
+#define HNS3_RING_GL_TX1
+
+#define HNS3_VECTOR_ELEMENTS_PER_CMD 10
+
+#define HNS3_INT_TYPE_S0
+#define HNS3_INT_TYPE_MGENMASK(1, 0)
+#define HNS3_TQP_ID_S 2
+#define HNS3_TQP_ID_M GENMASK(12, 2)
+#define HNS3_INT_GL_IDX_S 13
+#define HNS3_INT_GL_IDX_M GENMASK(14, 13)
+struct hns3_ctrl_vector_chain_cmd {
+ uint8_t int_vector_id;
+ uint8_t int_cause_num;
+ uint16_t tqp_type_and_id[HNS3_VECTOR_ELEMENTS_PER_CMD];
+ uint8_t vfid;
+ uint8_t rsv;
+};
+
struct hns3_config_max_frm_size_cmd {
uint16_t max_frm_size;
uint8_t min_frm_size;
diff --git a/drivers/net/hns3/hns3_ethdev.c b/drivers/net/hns3/hns3_ethdev.c
index 72315718a..bf0ab458f 100644
--- a/drivers/net/hns3/hns3_ethdev.c
+++ b/drivers/net/hns3/hns3_ethdev.c
@@ -2021,6 +2021,40 @@ hns3_check_dcb_cfg(struct rte_eth_dev *dev)
return hns3_check_mq_mode(dev);
}
+static int
+hns3_bind_ring_with_vector(struct rte_eth_dev *dev, uint8_t vector_id,
+ bool mmap, uint16_t queue_id)
+{
+ struct hns3_hw *hw = HNS3_DEV_PRIVATE_TO_HW(dev->data->dev_private);
+ struct hns3_cmd_desc desc;
+ struct hns3_ctrl_vector_chain_cmd *req =
+ (struct hns3_ctrl_vector_chain_cmd *)desc.data;
+ enum hns3_cmd_status status;
+ enum hns3_opcode_type op;
+ uint16_t tqp_type_and_id = 0;
+
+ op = mmap ? HNS3_OPC_ADD_RING_TO_VECTOR : HNS3_OPC_DEL_RING_TO_VECTOR;
+ hns3_cmd_setup_basic_desc(&desc, op, false);
+ req->int_vector_id = vector_id;
+
+ hns3_set_field(tqp_type_and_id, HNS3_INT_TYPE_M, HNS3_INT_TYPE_S,
+ HNS3_RING_TYPE_RX);
+ hns3_set_field(tqp_type_and_id, HNS3_TQP_ID_M, HNS3_TQP_ID_S, queue_id);
+ hns3_set_field(tqp_type_and_id, HNS3_INT_GL_IDX_M, HNS3_INT_GL_IDX_S,
+ HNS3_RING_GL_RX);
+ req->tqp_type_and_id[0] = rte_cpu_to_le_16(tqp_type_and_id);
+
+ req->int_cause_num = 1;
+ status = hns3_cmd_send(hw, &desc, 1);
+ if (status) {
+ hns3_err(hw, "Map TQP %d fail, vector_id is %d, status is %d.",
+queue_id, vector_id, status);
+ return -EIO;
+ }
+
+ return 0;
+}
+
static int
hns3_dev_configure(struct rte_eth_dev *dev)
{
@@ -4020,15 +4054,83 @@ hns3_do_start(struct hns3_adapter *hns, bool
reset_queue)
}
static int
-hns3_dev_start(struct rte_eth_dev *eth_dev)
+hns3_map_rx_interrupt(struct rte_eth_dev *dev)
[dpdk-dev] [PATCH 2/9] net/hns3: get link state change through mailbox
From: Hongbo Zheng
When link down occurs, firmware adds the function of sending message
to PF driver through mailbox, hns3 PMD driver can recognize link state
change faster through the message.
Signed-off-by: Hongbo Zheng
Signed-off-by: Wei Hu (Xavier)
---
drivers/net/hns3/hns3_ethdev.c | 8 ++--
drivers/net/hns3/hns3_ethdev.h | 1 +
drivers/net/hns3/hns3_mbx.c| 37 ++
drivers/net/hns3/hns3_mbx.h| 8
4 files changed, 52 insertions(+), 2 deletions(-)
diff --git a/drivers/net/hns3/hns3_ethdev.c b/drivers/net/hns3/hns3_ethdev.c
index bf0ab458f..3c591be51 100644
--- a/drivers/net/hns3/hns3_ethdev.c
+++ b/drivers/net/hns3/hns3_ethdev.c
@@ -218,6 +218,8 @@ hns3_interrupt_handler(void *param)
hns3_schedule_reset(hns);
} else if (event_cause == HNS3_VECTOR0_EVENT_RST)
hns3_schedule_reset(hns);
+ else if (event_cause == HNS3_VECTOR0_EVENT_MBX)
+ hns3_dev_handle_mbx_msg(hw);
else
hns3_err(hw, "Received unknown event");
@@ -3806,14 +3808,16 @@ hns3_get_mac_link_status(struct hns3_hw *hw)
return !!link_status;
}
-static void
+void
hns3_update_link_status(struct hns3_hw *hw)
{
int state;
state = hns3_get_mac_link_status(hw);
- if (state != hw->mac.link_status)
+ if (state != hw->mac.link_status) {
hw->mac.link_status = state;
+ hns3_warn(hw, "Link status change to %s!", state ? "up" :
"down");
+ }
}
static void
diff --git a/drivers/net/hns3/hns3_ethdev.h b/drivers/net/hns3/hns3_ethdev.h
index e9a3fe410..004cd75a9 100644
--- a/drivers/net/hns3/hns3_ethdev.h
+++ b/drivers/net/hns3/hns3_ethdev.h
@@ -631,6 +631,7 @@ int hns3_dev_filter_ctrl(struct rte_eth_dev *dev,
enum rte_filter_op filter_op, void *arg);
bool hns3_is_reset_pending(struct hns3_adapter *hns);
bool hns3vf_is_reset_pending(struct hns3_adapter *hns);
+void hns3_update_link_status(struct hns3_hw *hw);
static inline bool
is_reset_pending(struct hns3_adapter *hns)
diff --git a/drivers/net/hns3/hns3_mbx.c b/drivers/net/hns3/hns3_mbx.c
index c1647af4b..26807bc4b 100644
--- a/drivers/net/hns3/hns3_mbx.c
+++ b/drivers/net/hns3/hns3_mbx.c
@@ -282,6 +282,40 @@ hns3_update_resp_position(struct hns3_hw *hw, uint32_t
resp_msg)
resp->tail = tail;
}
+static void
+hns3_link_fail_parse(struct hns3_hw *hw, uint8_t link_fail_code)
+{
+ switch (link_fail_code) {
+ case HNS3_MBX_LF_NORMAL:
+ break;
+ case HNS3_MBX_LF_REF_CLOCK_LOST:
+ hns3_warn(hw, "Reference clock lost!");
+ break;
+ case HNS3_MBX_LF_XSFP_TX_DISABLE:
+ hns3_warn(hw, "SFP tx is disabled!");
+ break;
+ case HNS3_MBX_LF_XSFP_ABSENT:
+ hns3_warn(hw, "SFP is absent!");
+ break;
+ default:
+ hns3_warn(hw, "Unknown fail code:%u!", link_fail_code);
+ break;
+ }
+}
+
+static void
+hns3_handle_link_change_event(struct hns3_hw *hw,
+ struct hns3_mbx_pf_to_vf_cmd *req)
+{
+#define LINK_STATUS_OFFSET 1
+#define LINK_FAIL_CODE_OFFSET 2
+
+ if (!req->msg[LINK_STATUS_OFFSET])
+ hns3_link_fail_parse(hw, req->msg[LINK_FAIL_CODE_OFFSET]);
+
+ hns3_update_link_status(hw);
+}
+
void
hns3_dev_handle_mbx_msg(struct hns3_hw *hw)
{
@@ -335,6 +369,9 @@ hns3_dev_handle_mbx_msg(struct hns3_hw *hw)
hns3_mbx_handler(hw);
break;
+ case HNS3_MBX_PUSH_LINK_STATUS:
+ hns3_handle_link_change_event(hw, req);
+ break;
default:
hns3_err(hw,
"VF received unsupported(%d) mbx msg from PF",
diff --git a/drivers/net/hns3/hns3_mbx.h b/drivers/net/hns3/hns3_mbx.h
index d1a6bfead..3722c8760 100644
--- a/drivers/net/hns3/hns3_mbx.h
+++ b/drivers/net/hns3/hns3_mbx.h
@@ -41,6 +41,7 @@ enum HNS3_MBX_OPCODE {
HNS3_MBX_GET_QID_IN_PF, /* (VF -> PF) get queue id in pf */
HNS3_MBX_HANDLE_VF_TBL = 38,/* (VF -> PF) store/clear hw cfg tbl */
+ HNS3_MBX_PUSH_LINK_STATUS = 201, /* (IMP -> PF) get port link status */
};
/* below are per-VF mac-vlan subcodes */
@@ -64,6 +65,13 @@ enum hns3_mbx_tbl_cfg_subcode {
HNS3_MBX_VPORT_LIST_CLEAR = 0,
};
+enum hns3_mbx_link_fail_subcode {
+ HNS3_MBX_LF_NORMAL = 0,
+ HNS3_MBX_LF_REF_CLOCK_LOST,
+ HNS3_MBX_LF_XSFP_TX_DISABLE,
+ HNS3_MBX_LF_XSFP_ABSENT,
+};
+
#define HNS3_MBX_MAX_MSG_SIZE 16
#define HNS3_MBX_MAX_RESP_DATA_SIZE8
#define HNS3_MBX_RING_MAP_BASIC_MSG_NUM3
--
2.23.0
[dpdk-dev] Cannot use MLX5 groups : cannot create jump action.: Operation not supported
Hi all, I'm struggling to use the group feature with MLX5. I'm using 19.08.2 for now (while https://bugs.dpdk.org/show_bug.cgi?id=371 gets fixed). I use the latest OFED (no inbox driver this time). I have dv_flow_en=1, but I still get "cannot create jump action.: Operation not supported". when I want to use the jump action or "cannot create table: Cannot allocate memory" when trying to add a rule to table 1, all of this with testpmd but my own app does not work better. sudo ~/dpdk-skylake/x86_64-native-linuxapp-gcc/app/testpmd –l 0-3 -w :17:00.0,dv_flow_en=1 –n 6 -- -i EAL: Detected 36 lcore(s) EAL: Detected 2 NUMA nodes EAL: Multi-process socket /var/run/dpdk/rte/mp_socket EAL: Selected IOVA mode 'VA' EAL: Probing VFIO support... EAL: PCI device :17:00.0 on NUMA socket 0 EAL: probe driver: 15b3:1017 net_mlx5 net_mlx5: can not get IB device "mlx5_0" ports number Interactive-mode selected testpmd: create a new mbuf pool : n=283456, size=2176, socket=1 testpmd: preferred mempool ops selected: ring_mp_mc testpmd: create a new mbuf pool : n=283456, size=2176, socket=0 testpmd: preferred mempool ops selected: ring_mp_mc Warning! port-topology=paired and odd forward ports number, the last port will pair with itself. Configuring Port 0 (socket 0) Port 0: 50:6B:4B:43:8A:DA Checking link statuses... Done testpmd> flow create 0 group 1 ingress pattern eth / end actions queue index 0 / end Caught error type 1 (cause unspecified): cannot create table: Cannot allocate memory testpmd> flow validate 0 group 1 ingress pattern eth / end actions queue index 0 / end Flow rule validated testpmd> flow create 0 group 0 ingress pattern eth / end actions jump group 1 / end Caught error type 16 (specific action): cannot create jump action.: Operation not supported testpmd> Thanks, Tom
Re: [dpdk-dev] Cannot use MLX5 groups : cannot create jump action.: Operation not supported
Solved. The error was due to that NIC not having UCTX_EN enabled in the firmware. Maybe a less cryptic message/warning would be helpful though? DV was enabled so I did not thought the problem could be below. A warning when the user explicitly asks for dv_flow_en=1 but UCTX_EN is not enabled maybe? Thanks, Tom Le 30/11/2019 à 12:19, Tom Barbette a écrit : Hi all, I'm struggling to use the group feature with MLX5. I'm using 19.08.2 for now (while https://bugs.dpdk.org/show_bug.cgi?id=371 gets fixed). I use the latest OFED (no inbox driver this time). I have dv_flow_en=1, but I still get "cannot create jump action.: Operation not supported". when I want to use the jump action or "cannot create table: Cannot allocate memory" when trying to add a rule to table 1, all of this with testpmd but my own app does not work better. sudo ~/dpdk-skylake/x86_64-native-linuxapp-gcc/app/testpmd –l 0-3 -w :17:00.0,dv_flow_en=1 –n 6 -- -i EAL: Detected 36 lcore(s) EAL: Detected 2 NUMA nodes EAL: Multi-process socket /var/run/dpdk/rte/mp_socket EAL: Selected IOVA mode 'VA' EAL: Probing VFIO support... EAL: PCI device :17:00.0 on NUMA socket 0 EAL: probe driver: 15b3:1017 net_mlx5 net_mlx5: can not get IB device "mlx5_0" ports number Interactive-mode selected testpmd: create a new mbuf pool : n=283456, size=2176, socket=1 testpmd: preferred mempool ops selected: ring_mp_mc testpmd: create a new mbuf pool : n=283456, size=2176, socket=0 testpmd: preferred mempool ops selected: ring_mp_mc Warning! port-topology=paired and odd forward ports number, the last port will pair with itself. Configuring Port 0 (socket 0) Port 0: 50:6B:4B:43:8A:DA Checking link statuses... Done testpmd> flow create 0 group 1 ingress pattern eth / end actions queue index 0 / end Caught error type 1 (cause unspecified): cannot create table: Cannot allocate memory testpmd> flow validate 0 group 1 ingress pattern eth / end actions queue index 0 / end Flow rule validated testpmd> flow create 0 group 0 ingress pattern eth / end actions jump group 1 / end Caught error type 16 (specific action): cannot create jump action.: Operation not supported testpmd> Thanks, Tom
Re: [dpdk-dev] Interrupt is catched in kernel, but not handled
On Sat, 30 Nov 2019 10:40:38 +0200 Ranran wrote: > Hello, > > This is not a specific dpdk question, but more related to using of > uio_pci_generic. > Unfortunately, I don't find any examples of using the generic driver > from userspace, so this is my last hope to reach a suggestion. > > I installed centos8 , and I use uio_pci_generic, in order to > communicate with FPGA (Xilinx) through PCIe in userspace. > > The cpu is ATOM cpu. > > I first install the driver: > echo "10ee 0007" > /sys/bus/pci/drivers/uio_pci_generic/new_id > > Reading from BAR works perfectly but there is an issue with interrupts. > > I use userspace application which wait for interrupt, exactly as > described in code example here: > https://www.kernel.org/doc/html/v4.1...uio-howto.html > > > I than trigger an interrupt from FPGA, but no print from the userspace > application is given and there is an exception: > > irq 23: nobody cared (try booting with the "irqpoll" option) > [ 91.030760] CPU: 2 PID: 0 Comm: swapper/2 Not tainted 4.18.16 #6 > [ 91.037037] Hardware name: /conga-MA5, BIOS MA50R000 10/30/2019 > [ 91.043302] Call Trace: > [ 91.045881] > [ 91.048002] dump_stack+0x5c/0x80 > [ 91.051464] __report_bad_irq+0x35/0xaf > [ 91.055465] note_interrupt.cold.9+0xa/0x63 > [ 91.059823] handle_irq_event_percpu+0x68/0x70 > [ 91.064470] handle_irq_event+0x37/0x57 > [ 91.068481] handle_fasteoi_irq+0x97/0x150 > [ 91.072758] handle_irq+0x1a/0x30 > [ 91.076230] do_IRQ+0x44/0xd0 > [ 91.079333] common_interrupt+0xf/0xf > [ 91.083154] > [ 91.085360] RIP: 0010:cpuidle_enter_state+0x7d/0x220 > [ 91.090563] Code: e8 18 1a 45 00 41 89 c4 e8 d0 50 b1 ff 65 8b 3d d9 > db e5 5f e8 44 4f b1 ff 31 ff 48 89 c3 e8 ea 61 b1 ff fb 66 0f 1f 44 > 00 00 <48> b8 ff ff ff ff f3 01 00 00 4c 29 eb ba ff ff ff 7f 48 89 d9 > 48 > [ 91.110283] RSP: 0018:b20e806b7ea8 EFLAGS: 0286 ORIG_RAX: > ffda > [ 91.118203] RAX: 90133fd214c0 RBX: 0015316faa09 RCX: 001f > [ 91.125662] RDX: 0015316faa09 RSI: RDI: > [ 91.133133] RBP: 90133fd2b300 R08: 011741eb842c R09: 0006 > [ 91.140605] R10: R11: 90133fd205a8 R12: 0001 > [ 91.148068] R13: 0015316f98b9 R14: 0001 R15: > [ 91.155523] ? cpuidle_enter_state+0x76/0x220 > [ 91.160088] do_idle+0x221/0x260 > [ 91.163470] cpu_startup_entry+0x6a/0x70 > [ 91.167588] start_secondary+0x1a4/0x1f0 > [ 91.171676] secondary_startup_64+0xb7/0xc0 > [ 91.176043] handlers: > [ 91.178419] [] uio_interrupt > [ 91.183054] Disabling IRQ #23 > > > So, I started debugging this issue in kernel uio_pci_generic.c code, > and according to the prints below, it seems that irq is catched but > not delivered to userspace: > > > > static irqreturn_t irqhandler(int irq, struct uio_info *info) > { > struct uio_pci_generic_dev *gdev = to_uio_pci_generic_dev(info); > printk("i'm here 1\n"); <<--- this is printed > if (!pci_check_and_mask_intx(gdev->pdev)) > return IRQ_NONE; > printk("i'm here 2\n"); <<--- but this is NOT printed > /* UIO core will signal the user process. */ > return IRQ_HANDLED; > } > Reading documentation, I see that pci_check_and_mask_intx is actually > checking if interrupt bit is set in configuration space. Since it > seems to return 0 ,it should mean that it finds that this bit is not > enabled. > But how can it be that irq is triggered and status bit is not enabled > in configuration space ? > > The device appear as following with lspci -vv: > > > 02:00.0 RAM memory: Xilinx Corporation Default PCIe endpoint ID > Subsystem: Xilinx Corporation Default PCIe endpoint ID > Control: I/O- Mem+ BusMaster- SpecCycle- MemWINV- VGASnoop- ParErr- > Stepping- SERR- FastB2B- DisINTx- > Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- > SERR- Interrupt: pin A routed to IRQ 23 > Region 0: Memory at 9120 (32-bit, non-prefetchable) [size=1M] > Capabilities: [40] Power Management version 3 > Flags: PMEClk- DSI+ D1- D2- AuxCurrent=0mA PME(D0-,D1-,D2-,D3hot-,D3cold-) > Status: D0 NoSoftRst+ PME-Enable- DSel=0 DScale=0 PME- > Capabilities: [48] MSI: Enable- Count=1/1 Maskable- 64bit+ > Address: Data: > Capabilities: [58] Express (v1) Endpoint, MSI 00 > DevCap: MaxPayload 256 bytes, PhantFunc 1, Latency L0s <64ns, L1 <1us > ExtTag+ AttnBtn- AttnInd- PwrInd- RBE+ FLReset- SlotPowerLimit 10.000W > DevCtl: Report errors: Correctable- Non-Fatal- Fatal- Unsupported- > RlxdOrd+ ExtTag+ PhantFunc- AuxPwr- NoSnoop+ > MaxPayload 256 bytes, MaxReadReq 512 bytes > DevSta: CorrErr- UncorrErr- FatalErr- UnsuppReq- AuxPwr- TransPend- > LnkCap: Port #0, Speed 2.5GT/s, Width x1, ASPM L0s, Exit Latency L0s unlimited > ClockPM- Surprise- LLActRep- BwNot- ASPMOptComp- > LnkCtl: ASPM Disabled; RCB 64 bytes Disabled- CommClk+ > ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt- > LnkSta: Speed 2.5GT/s, Width x1, TrErr- Train- SlotClk+ DLActive- > BWMgmt- ABWMg
Re: [dpdk-dev] Interrupt is catched in kernel, but not handled
בתאריך שבת, 30 בנוב׳ 2019, 18:51, מאת Stephen Hemminger < step...@networkplumber.org>: > On Sat, 30 Nov 2019 10:40:38 +0200 > Ranran wrote: > > > Hello, > > > > This is not a specific dpdk question, but more related to using of > > uio_pci_generic. > > Unfortunately, I don't find any examples of using the generic driver > > from userspace, so this is my last hope to reach a suggestion. > > > > I installed centos8 , and I use uio_pci_generic, in order to > > communicate with FPGA (Xilinx) through PCIe in userspace. > > > > The cpu is ATOM cpu. > > > > I first install the driver: > > echo "10ee 0007" > /sys/bus/pci/drivers/uio_pci_generic/new_id > > > > Reading from BAR works perfectly but there is an issue with interrupts. > > > > I use userspace application which wait for interrupt, exactly as > > described in code example here: > > https://www.kernel.org/doc/html/v4.1...uio-howto.html > > > > > > I than trigger an interrupt from FPGA, but no print from the userspace > > application is given and there is an exception: > > > > irq 23: nobody cared (try booting with the "irqpoll" option) > > [ 91.030760] CPU: 2 PID: 0 Comm: swapper/2 Not tainted 4.18.16 #6 > > [ 91.037037] Hardware name: /conga-MA5, BIOS MA50R000 10/30/2019 > > [ 91.043302] Call Trace: > > [ 91.045881] > > [ 91.048002] dump_stack+0x5c/0x80 > > [ 91.051464] __report_bad_irq+0x35/0xaf > > [ 91.055465] note_interrupt.cold.9+0xa/0x63 > > [ 91.059823] handle_irq_event_percpu+0x68/0x70 > > [ 91.064470] handle_irq_event+0x37/0x57 > > [ 91.068481] handle_fasteoi_irq+0x97/0x150 > > [ 91.072758] handle_irq+0x1a/0x30 > > [ 91.076230] do_IRQ+0x44/0xd0 > > [ 91.079333] common_interrupt+0xf/0xf > > [ 91.083154] > > [ 91.085360] RIP: 0010:cpuidle_enter_state+0x7d/0x220 > > [ 91.090563] Code: e8 18 1a 45 00 41 89 c4 e8 d0 50 b1 ff 65 8b 3d d9 > > db e5 5f e8 44 4f b1 ff 31 ff 48 89 c3 e8 ea 61 b1 ff fb 66 0f 1f 44 > > 00 00 <48> b8 ff ff ff ff f3 01 00 00 4c 29 eb ba ff ff ff 7f 48 89 d9 > > 48 > > [ 91.110283] RSP: 0018:b20e806b7ea8 EFLAGS: 0286 ORIG_RAX: > > ffda > > [ 91.118203] RAX: 90133fd214c0 RBX: 0015316faa09 RCX: > 001f > > [ 91.125662] RDX: 0015316faa09 RSI: RDI: > > > [ 91.133133] RBP: 90133fd2b300 R08: 011741eb842c R09: > 0006 > > [ 91.140605] R10: R11: 90133fd205a8 R12: > 0001 > > [ 91.148068] R13: 0015316f98b9 R14: 0001 R15: > > > [ 91.155523] ? cpuidle_enter_state+0x76/0x220 > > [ 91.160088] do_idle+0x221/0x260 > > [ 91.163470] cpu_startup_entry+0x6a/0x70 > > [ 91.167588] start_secondary+0x1a4/0x1f0 > > [ 91.171676] secondary_startup_64+0xb7/0xc0 > > [ 91.176043] handlers: > > [ 91.178419] [] uio_interrupt > > [ 91.183054] Disabling IRQ #23 > > > > > > So, I started debugging this issue in kernel uio_pci_generic.c code, > > and according to the prints below, it seems that irq is catched but > > not delivered to userspace: > > > > > > > > static irqreturn_t irqhandler(int irq, struct uio_info *info) > > { > > struct uio_pci_generic_dev *gdev = to_uio_pci_generic_dev(info); > > printk("i'm here 1\n"); <<--- this is printed > > if (!pci_check_and_mask_intx(gdev->pdev)) > > return IRQ_NONE; > > printk("i'm here 2\n"); <<--- but this is NOT printed > > /* UIO core will signal the user process. */ > > return IRQ_HANDLED; > > } > > Reading documentation, I see that pci_check_and_mask_intx is actually > > checking if interrupt bit is set in configuration space. Since it > > seems to return 0 ,it should mean that it finds that this bit is not > > enabled. > > But how can it be that irq is triggered and status bit is not enabled > > in configuration space ? > > > > The device appear as following with lspci -vv: > > > > > > 02:00.0 RAM memory: Xilinx Corporation Default PCIe endpoint ID > > Subsystem: Xilinx Corporation Default PCIe endpoint ID > > Control: I/O- Mem+ BusMaster- SpecCycle- MemWINV- VGASnoop- ParErr- > > Stepping- SERR- FastB2B- DisINTx- > > Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- > > SERR- > Interrupt: pin A routed to IRQ 23 > > Region 0: Memory at 9120 (32-bit, non-prefetchable) [size=1M] > > Capabilities: [40] Power Management version 3 > > Flags: PMEClk- DSI+ D1- D2- AuxCurrent=0mA > PME(D0-,D1-,D2-,D3hot-,D3cold-) > > Status: D0 NoSoftRst+ PME-Enable- DSel=0 DScale=0 PME- > > Capabilities: [48] MSI: Enable- Count=1/1 Maskable- 64bit+ > > Address: Data: > > Capabilities: [58] Express (v1) Endpoint, MSI 00 > > DevCap: MaxPayload 256 bytes, PhantFunc 1, Latency L0s <64ns, L1 <1us > > ExtTag+ AttnBtn- AttnInd- PwrInd- RBE+ FLReset- SlotPowerLimit 10.000W > > DevCtl: Report errors: Correctable- Non-Fatal- Fatal- Unsupported- > > RlxdOrd+ ExtTag+ PhantFunc- AuxPwr- NoSnoop+ > > MaxPayload 256 bytes, MaxReadReq 512 bytes > > DevSta: CorrErr- UncorrErr- FatalErr- UnsuppReq- AuxPwr- TransPend- > > LnkCap: Port
