Re: [PATCH v3 net-next 07/21] nvme-tcp: Add DDP data-path

2021-02-03 Thread Sagi Grimberg
Given how much ddp code there is can you split it into a separate file? mmm, do we need to check the preferences or get to a consensus among the maintainers for that one? Not sure if moving it would be better here. Given that the ddp code is working directly on nvme-tcp structs we'll need a

Re: [PATCH v3 net-next 06/21] nvme-tcp: Add DDP offload control path

2021-02-03 Thread Sagi Grimberg
On 2/1/21 2:04 AM, Boris Pismenny wrote: This commit introduces direct data placement offload to NVME TCP. There is a context per queue, which is established after the handshake using the tcp_ddp_sk_add/del NDOs. Additionally, a resynchronization routine is used to assist hardware recovery fr

Re: [PATCH v3 net-next 09/21] nvme-tcp: Deal with netdevice DOWN events

2021-02-03 Thread Sagi Grimberg
@@ -2930,6 +2931,27 @@ static struct nvme_ctrl *nvme_tcp_create_ctrl(struct device *dev, return ERR_PTR(ret); } +static int nvme_tcp_netdev_event(struct notifier_block *this, +unsigned long event, void *ptr) +{ + struct net_device *ndev = net

Re: [PATCH v3 net-next 08/21] nvme-tcp : Recalculate crc in the end of the capsule

2021-02-03 Thread Sagi Grimberg
On 2/1/21 2:04 AM, Boris Pismenny wrote: From: Ben Ben-ishay crc offload of the nvme capsule. Check if all the skb bits are on, and if not recalculate the crc in SW and check it. This patch reworks the receive-side crc calculation to always run at the end, so as to keep a single flow for bo

Re: [PATCH v3 net-next 07/21] nvme-tcp: Add DDP data-path

2021-02-03 Thread Sagi Grimberg
Given how much ddp code there is can you split it into a separate file? mmm, do we need to check the preferences or get to a consensus among the maintainers for that one? Not sure if moving it would be better here. Given that the ddp code is working directly on nvme-tcp structs we'll need a

Re: [PATCH v3 net-next 07/21] nvme-tcp: Add DDP data-path

2021-02-03 Thread Sagi Grimberg
On 2/1/21 2:04 AM, Boris Pismenny wrote: Introduce the NVMe-TCP DDP data-path offload. Using this interface, the NIC hardware will scatter TCP payload directly to the BIO pages according to the command_id in the PDU. To maintain the correctness of the network stack, the driver is expected to c

Re: [PATCH v1 net-next 00/15] nvme-tcp receive offloads

2021-01-14 Thread Sagi Grimberg
Hey Boris, sorry for some delays on my end... I saw some long discussions on this set with David, what is the status here? The main purpose of this series is to address these. I'll take some more look into the patches, but if you addressed the feedback from the last iteration I don't expe

Re: [PATCH v1 net-next 00/15] nvme-tcp receive offloads

2021-01-13 Thread Sagi Grimberg
Hey Boris, sorry for some delays on my end... I saw some long discussions on this set with David, what is the status here? I'll take some more look into the patches, but if you addressed the feedback from the last iteration I don't expect major issues with this patch set (at least from nvme-tcp

Re: [PATCH net-next RFC v1 05/10] nvme-tcp: Add DDP offload control path

2020-11-09 Thread Sagi Grimberg
diff --git a/drivers/nvme/host/tcp.c b/drivers/nvme/host/tcp.c index 8f4f29f18b8c..06711ac095f2 100644 --- a/drivers/nvme/host/tcp.c +++ b/drivers/nvme/host/tcp.c @@ -62,6 +62,7 @@ enum nvme_tcp_queue_flags { NVME_TCP_Q_ALLOCATED= 0, NVME_TCP_Q_LIVE = 1, NVME

Re: [PATCH net-next RFC v1 06/10] nvme-tcp: Add DDP data-path

2020-11-09 Thread Sagi Grimberg
static int nvme_tcp_recv_data(struct nvme_tcp_queue *queue, struct sk_buff *skb, @@ -1115,6 +1222,7 @@ static int nvme_tcp_try_send_cmd_pdu(struct nvme_tcp_request *req) bool inline_data = nvme_tcp_has_inline_data(req); u8 hdgst = nvme_tcp_hdgst_len(queue); int len

Re: [PATCH rdma v2] RDMA: Add rdma_connect_locked()

2020-10-27 Thread Sagi Grimberg
Reviewed-by: Sagi Grimberg

Re: [PATCH net-next RFC v1 00/10] nvme-tcp receive offloads

2020-10-08 Thread Sagi Grimberg
On 9/30/20 9:20 AM, Boris Pismenny wrote: This series adds support for nvme-tcp receive offloads which do not mandate the offload of the network stack to the device. Instead, these work together with TCP to offload: 1. copy from SKB to the block layer buffers 2. CRC verification for received P

Re: [PATCH net-next RFC v1 01/10] iov_iter: Skip copy in memcpy_to_page if src==dst

2020-10-08 Thread Sagi Grimberg
You probably want Al to have a look at this..

Re: [PATCH net-next RFC v1 06/10] nvme-tcp: Add DDP data-path

2020-10-08 Thread Sagi Grimberg
  static   int nvme_tcp_offload_socket(struct nvme_tcp_queue *queue,   struct nvme_tcp_config *config) @@ -630,6 +720,7 @@ static void nvme_tcp_error_recovery(struct nvme_ctrl *ctrl)   static int nvme_tcp_process_nvme_cqe(struct nvme_tcp_queue *queue,   struct nvme_com

Re: [PATCH net-next RFC v1 08/10] nvme-tcp: Deal with netdevice DOWN events

2020-10-08 Thread Sagi Grimberg
On 9/30/20 9:20 AM, Boris Pismenny wrote: From: Or Gerlitz For ddp setup/teardown and resync, the offloading logic uses HW resources at the NIC driver such as SQ and CQ. These resources are destroyed when the netdevice does down and hence we must stop using them before the NIC driver destro

Re: [PATCH net-next RFC v1 07/10] nvme-tcp : Recalculate crc in the end of the capsule

2020-10-08 Thread Sagi Grimberg
crc offload of the nvme capsule. Check if all the skb bits are on, and if not recalculate the crc in SW and check it. Can you clarify in the patch description that this is only for pdu data digest and not header digest? This patch reworks the receive-side crc calculation to always run at t

Re: [PATCH net-next RFC v1 06/10] nvme-tcp: Add DDP data-path

2020-10-08 Thread Sagi Grimberg
bool nvme_tcp_resync_request(struct sock *sk, u32 seq, u32 flags); +void nvme_tcp_ddp_teardown_done(void *ddp_ctx); const struct tcp_ddp_ulp_ops nvme_tcp_ddp_ulp_ops __read_mostly = { + .resync_request = nvme_tcp_resync_request, + .ddp_teardown_done = nvme_tcp_ddp

Re: [PATCH net-next RFC v1 05/10] nvme-tcp: Add DDP offload control path

2020-10-08 Thread Sagi Grimberg
On 9/30/20 9:20 AM, Boris Pismenny wrote: This commit introduces direct data placement offload to NVME TCP. There is a context per queue, which is established after the handshake using the tcp_ddp_sk_add/del NDOs. Additionally, a resynchronization routine is used to assist hardware recovery f

Re: [PATCH net-next RFC v1 04/10] net/tls: expose get_netdev_for_sock

2020-10-08 Thread Sagi Grimberg
Reviewed-by: Sagi Grimberg

Re: [PATCH net-next RFC v1 03/10] net: Introduce crc offload for tcp ddp ulp

2020-10-08 Thread Sagi Grimberg
offload engine does crc32c, do you see this extended to other crc types in the future? Other than this the patch looks good to me, Reviewed-by: Sagi Grimberg

Re: [PATCH net-next RFC v1 02/10] net: Introduce direct data placement tcp offload

2020-10-08 Thread Sagi Grimberg
+ * tcp_ddp.h + * Author: Boris Pismenny + * Copyright (C) 2020 Mellanox Technologies. + */ +#ifndef _TCP_DDP_H +#define _TCP_DDP_H + +#include Why is blkdev.h needed? +#include +#include +#include + +/* limits returned by the offload driver, zero means don't care */ +struct tc

Re: remove kernel_setsockopt and kernel_getsockopt

2020-05-13 Thread Sagi Grimberg
looks quite promising: 42 files changed, 721 insertions(+), 799 deletions(-) For the nvme-tcp bits, Acked-by: Sagi Grimberg

Re: [for-next V2 10/10] RDMA/core: Provide RDMA DIM support for ULPs

2019-07-03 Thread Sagi Grimberg
Hi Sagi, I'm not sharing your worries about bad out-of-the-box experience for a number of reasons. First of all, this code is part of upstream kernel and will take time till users actually start to use it as is and not as part of some distro backports or MOFED packages. True, but I am still

Re: [for-next V2 10/10] RDMA/core: Provide RDMA DIM support for ULPs

2019-07-01 Thread Sagi Grimberg
Hey Idan, " Please don't. This is a bad choice to opt it in by default." I disagree here. I'd prefer Linux to have good out of the box experience (e.g. reach 100G in 4K NVMeOF on Intel servers) with the default parameters. Especially since Yamin have shown it is beneficial / not hurting in te

Re: [for-next V2 08/10] linux/dim: Implement rdma_dim

2019-06-25 Thread Sagi Grimberg
+void rdma_dim(struct dim *dim, u64 completions) +{ + struct dim_sample *curr_sample = &dim->measuring_sample; + struct dim_stats curr_stats; + u32 nevents; + + dim_update_sample_with_comps(curr_sample->event_ctr + 1, +curr_sample->pkt

Re: [for-next V2 05/10] linux/dim: Rename externally used net_dim members

2019-06-25 Thread Sagi Grimberg
Question, do any other nics use or plan to use this? Reviewed-by: Sagi Grimberg

Re: [for-next V2 04/10] linux/dim: Rename net_dim_sample() to net_dim_update_sample()

2019-06-25 Thread Sagi Grimberg
Reviewed-by: Sagi Grimberg

Re: [for-next V2 03/10] linux/dim: Rename externally exposed macros

2019-06-25 Thread Sagi Grimberg
This can be squashed to the prev one, otherwise Reviewed-by: Sagi Grimberg

Re: [for-next V2 02/10] linux/dim: Remove "net" prefix from internal DIM members

2019-06-25 Thread Sagi Grimberg
Reviewed-by: Sagi Grimberg

Re: [for-next V2 01/10] linux/dim: Move logic to dim.h

2019-06-25 Thread Sagi Grimberg
Reviewed-by: Sagi Grimberg

Re: [for-next V2 09/10] RDMA/nldev: Added configuration of RDMA dynamic interrupt moderation to netlink

2019-06-25 Thread Sagi Grimberg
On 6/25/19 1:57 PM, Saeed Mahameed wrote: From: Yamin Friedman Added parameter in ib_device for enabling dynamic interrupt moderation so that it can be configured in userspace using rdma tool. In order to set dim for an ib device the command is: rdma dev set [DEV] dim [on|off] Please set on

Re: [for-next V2 10/10] RDMA/core: Provide RDMA DIM support for ULPs

2019-06-25 Thread Sagi Grimberg
+static int ib_poll_dim_handler(struct irq_poll *iop, int budget) +{ + struct ib_cq *cq = container_of(iop, struct ib_cq, iop); + struct dim *dim = cq->dim; + int completed; + + completed = __ib_process_cq(cq, budget, cq->wc, IB_POLL_BATCH); + if (completed < budg

Re: [PATCH 6/7] net/core: Declare a local function 'static'

2019-03-21 Thread Sagi Grimberg
This can be dropped. Dave merged a fix for it already On 3/21/19 3:19 PM, Bart Van Assche wrote: This patch avoids that the compiler complains about a missing prototype when building with W=1. Cc: Sagi Grimberg Cc: Christoph Hellwig Signed-off-by: Bart Van Assche --- net/core/datagram.c

Re: [PATCH 7/7] net/utils: Use strlcpy() instead of open-coding it

2019-03-21 Thread Sagi Grimberg
Reviewed-by: Sagi Grimberg

[PATCH v5 02/13] datagram: open-code copy_page_to_iter

2018-12-03 Thread Sagi Grimberg
From: Sagi Grimberg This will be useful to consolidate skb_copy_and_hash_datagram_iter and skb_copy_and_csum_datagram to a single code path. Acked-by: David S. Miller Signed-off-by: Sagi Grimberg --- net/core/datagram.c | 9 ++--- 1 file changed, 6 insertions(+), 3 deletions(-) diff

[PATCH v5 03/13] iov_iter: pass void csum pointer to csum_and_copy_to_iter

2018-12-03 Thread Sagi Grimberg
From: Sagi Grimberg The single caller to csum_and_copy_to_iter is skb_copy_and_csum_datagram and we are trying to unite its logic with skb_copy_datagram_iter by passing a callback to the copy function that we want to apply. Thus, we need to make the checksum pointer private to the function

[PATCH v5 01/13] ath6kl: add ath6kl_ prefix to crypto_type

2018-12-03 Thread Sagi Grimberg
From: Sagi Grimberg Prevent a namespace conflict as in following patches as skbuff.h will include the crypto API. Acked-by: David S. Miller Cc: Kalle Valo Signed-off-by: Sagi Grimberg --- drivers/net/wireless/ath/ath6kl/cfg80211.c | 2 +- drivers/net/wireless/ath/ath6kl/common.h | 2

[PATCH v5 00/13] TCP transport binding for NVMe over Fabrics

2018-12-03 Thread Sagi Grimberg
le patches Sagi Grimberg (13): ath6kl: add ath6kl_ prefix to crypto_type datagram: open-code copy_page_to_iter iov_iter: pass void csum pointer to csum_and_copy_to_iter datagram: consolidate datagram copy to iter helpers iov_iter: introduce hash_and_copy_to_iter helper datagram:

[PATCH v5 04/13] datagram: consolidate datagram copy to iter helpers

2018-12-03 Thread Sagi Grimberg
From: Sagi Grimberg skb_copy_datagram_iter and skb_copy_and_csum_datagram are essentialy the same but with a couple of differences: The first is the copy operation used which either a simple copy or a csum_and_copy, and the second are the behavior on the "short copy" path where simply

[PATCH v5 05/13] iov_iter: introduce hash_and_copy_to_iter helper

2018-12-03 Thread Sagi Grimberg
From: Sagi Grimberg Allow consumers that want to use iov iterator helpers and also update a predefined hash calculation online when copying data. This is useful when copying incoming network buffers to a local iterator and calculate a digest on the incoming stream. nvme-tcp host driver that will

[PATCH v5 06/13] datagram: introduce skb_copy_and_hash_datagram_iter helper

2018-12-03 Thread Sagi Grimberg
From: Sagi Grimberg Introduce a helper to copy datagram into an iovec iterator but also update a predefined hash. This is useful for consumers of skb_copy_datagram_iter to also support inflight data digest without having to finish to copy and only then traverse the iovec and calculate the digest

[PATCH v5 07/13] nvmet: Add install_queue callout

2018-12-03 Thread Sagi Grimberg
From: Sagi Grimberg nvmet-tcp will implement it to allocate queue commands which are only known at nvmf connect time (sq size). Reviewed-by: Christoph Hellwig Signed-off-by: Sagi Grimberg --- drivers/nvme/target/fabrics-cmd.c | 10 ++ drivers/nvme/target/nvmet.h | 1 + 2 files

[PATCH v5 10/13] nvme-tcp: Add protocol header

2018-12-03 Thread Sagi Grimberg
From: Sagi Grimberg Signed-off-by: Sagi Grimberg --- include/linux/nvme-tcp.h | 189 +++ include/linux/nvme.h | 1 + 2 files changed, 190 insertions(+) create mode 100644 include/linux/nvme-tcp.h diff --git a/include/linux/nvme-tcp.h b/include/linux

[PATCH v5 08/13] nvme-fabrics: allow user passing header digest

2018-12-03 Thread Sagi Grimberg
From: Sagi Grimberg Header digest is a nvme-tcp specific feature, but nothing prevents other transports reusing the concept so do not associate with tcp transport solely. Reviewed-by: Christoph Hellwig Signed-off-by: Sagi Grimberg --- drivers/nvme/host/fabrics.c | 5 + drivers/nvme/host

[PATCH v5 13/13] nvme-tcp: add NVMe over TCP host driver

2018-12-03 Thread Sagi Grimberg
From: Sagi Grimberg This patch implements the NVMe over TCP host driver. It can be used to connect to remote NVMe over Fabrics subsystems over good old TCP/IP. The driver implements the TP 8000 of how nvme over fabrics capsules and data are encapsulated in nvme-tcp pdus and exchaged on top of a

[PATCH v5 11/13] nvmet-tcp: add NVMe over TCP target driver

2018-12-03 Thread Sagi Grimberg
From: Sagi Grimberg This patch implements the TCP transport driver for the NVMe over Fabrics target stack. This allows exporting NVMe over Fabrics functionality over good old TCP/IP. The driver implements the TP 8000 of how nvme over fabrics capsules and data are encapsulated in nvme-tcp pdus

[PATCH v5 09/13] nvme-fabrics: allow user passing data digest

2018-12-03 Thread Sagi Grimberg
From: Sagi Grimberg Data digest is a nvme-tcp specific feature, but nothing prevents other transports reusing the concept so do not associate with tcp transport solely. Reviewed-by: Christoph Hellwig Signed-off-by: Sagi Grimberg --- drivers/nvme/host/fabrics.c | 5 + drivers/nvme/host

[PATCH v5 12/13] nvmet: allow configfs tcp trtype configuration

2018-12-03 Thread Sagi Grimberg
From: Sagi Grimberg Reviewed-by: Max Gurtovoy Reviewed-by: Christoph Hellwig Signed-off-by: Sagi Grimberg --- drivers/nvme/target/configfs.c | 1 + 1 file changed, 1 insertion(+) diff --git a/drivers/nvme/target/configfs.c b/drivers/nvme/target/configfs.c index db2cb64be7ba..618bbd006544

Re: [PATCH v4 00/13] TCP transport binding for NVMe over Fabrics

2018-11-30 Thread Sagi Grimberg
What is the plan ahead here? I think the nvme code looks pretty reasonable now (I'll do another pass at nitpicking), but we need the networking stuff sorted out with at least ACKs, or a merge through the networking tree and then a shared branch we can pull in. I would think that having Dave

Re: [PATCH v4 00/13] TCP transport binding for NVMe over Fabrics

2018-11-29 Thread Sagi Grimberg
What is the plan ahead here? I think the nvme code looks pretty reasonable now (I'll do another pass at nitpicking), but we need the networking stuff sorted out with at least ACKs, or a merge through the networking tree and then a shared branch we can pull in. I would think that having Dave

Re: [PATCH v4 11/13] nvmet-tcp: add NVMe over TCP target driver

2018-11-29 Thread Sagi Grimberg
+static inline void nvmet_tcp_put_cmd(struct nvmet_tcp_cmd *cmd) +{ +    if (unlikely(cmd == &cmd->queue->connect)) +    return; if you don't return connect cmd to the list please don't add it to it in the first place (during alloc_cmd). and if you use it once, we might think of a clean

[PATCH v4 03/13] iov_iter: pass void csum pointer to csum_and_copy_to_iter

2018-11-27 Thread Sagi Grimberg
From: Sagi Grimberg The single caller to csum_and_copy_to_iter is skb_copy_and_csum_datagram and we are trying to unite its logic with skb_copy_datagram_iter by passing a callback to the copy function that we want to apply. Thus, we need to make the checksum pointer private to the function

[PATCH v4 10/13] nvme-tcp: Add protocol header

2018-11-27 Thread Sagi Grimberg
From: Sagi Grimberg Signed-off-by: Sagi Grimberg --- include/linux/nvme-tcp.h | 189 +++ include/linux/nvme.h | 1 + 2 files changed, 190 insertions(+) create mode 100644 include/linux/nvme-tcp.h diff --git a/include/linux/nvme-tcp.h b/include/linux

[PATCH v4 00/13] TCP transport binding for NVMe over Fabrics

2018-11-27 Thread Sagi Grimberg
nks to the members of the Fabrics Linux Driver team that helped development, testing and benchmarking this work. Gitweb code is available at: git://git.infradead.org/nvme.git nvme-tcp Sagi Grimberg (13): ath6kl: add ath6kl_ prefix to crypto_type datagram: open-code copy_page_to_iter iov_

[PATCH v4 11/13] nvmet-tcp: add NVMe over TCP target driver

2018-11-27 Thread Sagi Grimberg
From: Sagi Grimberg This patch implements the TCP transport driver for the NVMe over Fabrics target stack. This allows exporting NVMe over Fabrics functionality over good old TCP/IP. The driver implements the TP 8000 of how nvme over fabrics capsules and data are encapsulated in nvme-tcp pdus

[PATCH v4 12/13] nvmet: allow configfs tcp trtype configuration

2018-11-27 Thread Sagi Grimberg
From: Sagi Grimberg Reviewed-by: Max Gurtovoy Reviewed-by: Christoph Hellwig Signed-off-by: Sagi Grimberg --- drivers/nvme/target/configfs.c | 1 + 1 file changed, 1 insertion(+) diff --git a/drivers/nvme/target/configfs.c b/drivers/nvme/target/configfs.c index db2cb64be7ba..618bbd006544

[PATCH v4 04/13] datagram: consolidate datagram copy to iter helpers

2018-11-27 Thread Sagi Grimberg
From: Sagi Grimberg skb_copy_datagram_iter and skb_copy_and_csum_datagram are essentialy the same but with a couple of differences: The first is the copy operation used which either a simple copy or a csum_and_copy, and the second are the behavior on the "short copy" path where simply

[PATCH v4 05/13] iov_iter: introduce hash_and_copy_to_iter helper

2018-11-27 Thread Sagi Grimberg
From: Sagi Grimberg Allow consumers that want to use iov iterator helpers and also update a predefined hash calculation online when copying data. This is useful when copying incoming network buffers to a local iterator and calculate a digest on the incoming stream. nvme-tcp host driver that will

[PATCH v4 07/13] nvmet: Add install_queue callout

2018-11-27 Thread Sagi Grimberg
From: Sagi Grimberg nvmet-tcp will implement it to allocate queue commands which are only known at nvmf connect time (sq size). Reviewed-by: Christoph Hellwig Signed-off-by: Sagi Grimberg --- drivers/nvme/target/fabrics-cmd.c | 10 ++ drivers/nvme/target/nvmet.h | 1 + 2 files

[PATCH v4 09/13] nvme-fabrics: allow user passing data digest

2018-11-27 Thread Sagi Grimberg
From: Sagi Grimberg Data digest is a nvme-tcp specific feature, but nothing prevents other transports reusing the concept so do not associate with tcp transport solely. Reviewed-by: Christoph Hellwig Signed-off-by: Sagi Grimberg --- drivers/nvme/host/fabrics.c | 5 + drivers/nvme/host

[PATCH v4 02/13] datagram: open-code copy_page_to_iter

2018-11-27 Thread Sagi Grimberg
From: Sagi Grimberg This will be useful to consolidate skb_copy_and_hash_datagram_iter and skb_copy_and_csum_datagram to a single code path. Signed-off-by: Sagi Grimberg --- net/core/datagram.c | 9 ++--- 1 file changed, 6 insertions(+), 3 deletions(-) diff --git a/net/core/datagram.c b

[PATCH v4 13/13] nvme-tcp: add NVMe over TCP host driver

2018-11-27 Thread Sagi Grimberg
From: Sagi Grimberg This patch implements the NVMe over TCP host driver. It can be used to connect to remote NVMe over Fabrics subsystems over good old TCP/IP. The driver implements the TP 8000 of how nvme over fabrics capsules and data are encapsulated in nvme-tcp pdus and exchaged on top of a

[PATCH v4 06/13] datagram: introduce skb_copy_and_hash_datagram_iter helper

2018-11-27 Thread Sagi Grimberg
From: Sagi Grimberg Introduce a helper to copy datagram into an iovec iterator but also update a predefined hash. This is useful for consumers of skb_copy_datagram_iter to also support inflight data digest without having to finish to copy and only then traverse the iovec and calculate the digest

[PATCH v4 08/13] nvme-fabrics: allow user passing header digest

2018-11-27 Thread Sagi Grimberg
From: Sagi Grimberg Header digest is a nvme-tcp specific feature, but nothing prevents other transports reusing the concept so do not associate with tcp transport solely. Reviewed-by: Christoph Hellwig Signed-off-by: Sagi Grimberg --- drivers/nvme/host/fabrics.c | 5 + drivers/nvme/host

[PATCH v4 01/13] ath6kl: add ath6kl_ prefix to crypto_type

2018-11-27 Thread Sagi Grimberg
From: Sagi Grimberg Prevent a namespace conflict as in following patches as skbuff.h will include the crypto API. Cc: Kalle Valo Signed-off-by: Sagi Grimberg --- drivers/net/wireless/ath/ath6kl/cfg80211.c | 2 +- drivers/net/wireless/ath/ath6kl/common.h | 2 +- drivers/net/wireless/ath

Re: [PATCH v3 13/13] nvme-tcp: add NVMe over TCP host driver

2018-11-26 Thread Sagi Grimberg
This looks odd.  It's not really the timeout handlers job to call nvme_end_request here. Well.. if we are not yet LIVE, we will not trigger error recovery, which means nothing will complete this command so something needs to do it... I think that we need it for rdma too.. yes we do. and w

Re: [PATCH nvme-cli v3 15/13] nvme: Add TCP transport

2018-11-26 Thread Sagi Grimberg
The cli parts good to me. Is there any forseeable issue if I apply these ahead of the kernel integration? I don't see any, but just want to confirm since its all in one series. I don't see an issue, note that its on top of the sqflow patch [1] that I've been meaning to ping you on. [PATCH nvme-

Re: [PATCH v3 11/13] nvmet-tcp: add NVMe over TCP target driver

2018-11-25 Thread Sagi Grimberg
+ if (!cmd || queue->state == NVMET_TCP_Q_DISCONNECTING) { + cmd = nvmet_tcp_fetch_send_command(queue); + if (unlikely(!cmd)) + return 0; + } + + if (cmd->state == NVMET_TCP_SEND_DATA_PDU) { + ret = nvmet_try_send

Re: [PATCH v3 13/13] nvme-tcp: add NVMe over TCP host driver

2018-11-25 Thread Sagi Grimberg
+static enum nvme_tcp_recv_state nvme_tcp_recv_state(struct nvme_tcp_queue *queue) +{ + return (queue->pdu_remaining) ? NVME_TCP_RECV_PDU : + (queue->ddgst_remaining) ? NVME_TCP_RECV_DDGST : + NVME_TCP_RECV_DATA; +} This just seems to be used in a single sw

Re: [PATCH mlx5-next] RDMA/mlx5: Don't use cached IRQ affinity mask

2018-08-17 Thread Sagi Grimberg
Hi Jason, The new patchworks doesn't grab patches inlined in messages, so you will need to resend it. Yes, just wanted to to add Steve's tested by as its going to lists that did not follow this thread. Also, can someone remind me what the outcome is here? Does it supersede Leon's patch: htt

Re: [PATCH mlx5-next] RDMA/mlx5: Don't use cached IRQ affinity mask

2018-08-17 Thread Sagi Grimberg
Hey Sagi, The patch works allowing connections for the various affinity mappings below: One comp_vector per core across all cores, starting with numa-local cores: Thanks Steve, is this your "Tested by:" tag?

Re: [PATCH mlx5-next] RDMA/mlx5: Don't use cached IRQ affinity mask

2018-08-16 Thread Sagi Grimberg
arranged such that a vector is not assigned to any distinct cpu, we map it to a cpu that is on the same node. If numa affinity can not be sufficed, we map it to any unmapped cpu we can find. Then, map the remaining cpus in the possible cpumap naively. Signed-off-by: Sagi Grimberg --- Steve, can you

Re: [PATCH mlx5-next] RDMA/mlx5: Don't use cached IRQ affinity mask

2018-07-31 Thread Sagi Grimberg
Hi Max, Yes, since nvmf is the only user of this function. Still waiting for comments on the suggested patch :) Sorry for the late response (but I'm on vacation so I have an excuse ;)) I'm thinking that we should avoid trying to find an assignment when stuff like irqbalance daemon is running

Re: [PATCH mlx5-next] RDMA/mlx5: Don't use cached IRQ affinity mask

2018-07-18 Thread Sagi Grimberg
IMO we must fulfil the user wish to connect to N queues and not reduce it because of affinity overlaps. So in order to push Leon's patch we must also fix the blk_mq_rdma_map_queues to do a best effort mapping according the affinity and map the rest in naive way (in that way we will *always* map

Re: [PATCH mlx5-next] RDMA/mlx5: Don't use cached IRQ affinity mask

2018-07-16 Thread Sagi Grimberg
Hi, I've tested this patch and seems problematic at this moment. Problematic how? what are you seeing? maybe this is because of the bug that Steve mentioned in the NVMe mailing list. Sagi mentioned that we should fix it in the NVMe/RDMA initiator and I'll run his suggestion as well. Is y

Re: [PATCH mlx5-next] RDMA/mlx5: Don't use cached IRQ affinity mask

2018-07-16 Thread Sagi Grimberg
Leon, I'd like to see a tested-by tag for this (at least until I get some time to test it). The patch itself looks fine to me.

Re: [net 02/11] net/mlx5: Fix get vector affinity helper function

2018-01-14 Thread Sagi Grimberg
IRQ vector affinity using legacy IRQ API, following smp_affinity read procfs implementation. Fixes: 231243c82793 ("Revert mlx5: move affinity hints assignments to generic code") Fixes: a435393acafb ("mlx5: move affinity hints assignments to generic code") Cc: Sagi Grimberg Cc

Re: [net 02/14] Revert "mlx5: move affinity hints assignments to generic code"

2017-12-26 Thread Sagi Grimberg
Are you sure it won't get populated at all ? even if you manually set IRQ affinity via sysfs ? Yes, the msi_desc affinity is not initialized without the affinity descriptor passed (which is what PCI_IRQ_AFFINITY is for). Anyway we can implement this driver helper function to return the IRQ

Re: [net 02/14] Revert "mlx5: move affinity hints assignments to generic code"

2017-12-25 Thread Sagi Grimberg
Before the offending commit, mlx5 core did the IRQ affinity itself, and it seems that the new generic code have some drawbacks and one of them is the lack for user ability to modify irq affinity after the initial affinity values got assigned. The issue is still being discussed and a solution in

Re: [RFD] Managed interrupt affinities [ Was: mlx5 broken affinity ]

2017-11-13 Thread Sagi Grimberg
Can you explain what do you mean by "subsystem"? I thought that the subsystem would be the irq subsystem (which means you are the one to provide the needed input :) ) and the driver would pass in something like msi_irq_ops to pci_alloc_irq_vectors() if it supports the driver requirements that yo

Re: [RFD] Managed interrupt affinities [ Was: mlx5 broken affinity ]

2017-11-13 Thread Sagi Grimberg
Do you know if any exist? Would it make sense to have a survey to understand if anyone relies on it? From what I've seen so far, drivers that were converted simply worked with the non-managed facility and didn't have any special code for it. Perhaps Christoph can comment as he convert most of

Re: [RFD] Managed interrupt affinities [ Was: mlx5 broken affinity ]

2017-11-13 Thread Sagi Grimberg
Hi Thomas, What can be done with some time to work on? The managed mechanism consists of 3 pieces: 1) Vector spreading 2) Managed vector allocation, which becomes a guaranteed reservation in 4.15 due of the big rework of the vector management code. Non managed interrupts get a

Re: mlx5 broken affinity

2017-11-09 Thread Sagi Grimberg
The early discussion of the managed facility came to the conclusion that it will manage this stuff completely to allow fixed association of 'queue / interrupt / corresponding memory' to a single CPU or a set of CPUs. That removes a lot of 'affinity' handling magic from the driver and utilizes th

Re: mlx5 broken affinity

2017-11-09 Thread Sagi Grimberg
Again, I think Jes or others can provide more information. Sagi, I believe Jes is not trying to argue about what initial affinity values you give to the driver, We have a very critical regression that is afflicting Live systems today and common tools that already exists in various distros, suc

Re: mlx5 broken affinity

2017-11-09 Thread Sagi Grimberg
Thomas, Because the user sometimes knows better based on statically assigned loads, or the user wants consistency across kernels. It's great that the system is better at allocating this now, but we also need to allow for a user to change it. Like anything on Linux, a user wanting to blow off his

Re: mlx5 broken affinity

2017-11-07 Thread Sagi Grimberg
Depending on the machine and the number of queues this might even result in completely losing the ability to suspend/hibernate because the number of available vectors on CPU0 is not sufficient to accomodate all queue interrupts. Would it be possible to keep the managed facility until a user ov

Re: mlx5 broken affinity

2017-11-05 Thread Sagi Grimberg
This wasn't to start a debate about which allocation method is the perfect solution. I am perfectly happy with the new default, the part that is broken is to take away the user's option to reassign the affinity. That is a bug and it needs to be fixed! Well, I would really want to wait for Tho

Re: mlx5 broken affinity

2017-11-02 Thread Sagi Grimberg
This wasn't to start a debate about which allocation method is the perfect solution. I am perfectly happy with the new default, the part that is broken is to take away the user's option to reassign the affinity. That is a bug and it needs to be fixed! Well, I would really want to wait for Tho

Re: mlx5 broken affinity

2017-11-02 Thread Sagi Grimberg
I vaguely remember Nacking Sagi's patch as we knew it would break mlx5e netdev affinity assumptions. I remember that argument. Still the series found its way in. Of course it maid its way in, it was acked by three different maintainers, and I addressed all of Saeed's comments. That series m

Re: mlx5 broken affinity

2017-11-02 Thread Sagi Grimberg
Jes, I am all in favor of making the automatic setup better, but assuming an automatic setup is always right seems problematic. There could be workloads where you may want to assign affinity explicitly. Adding Thomas to the thread. My understanding that the thought is to prevent user-space fr

Re: mlx5 broken affinity

2017-11-01 Thread Sagi Grimberg
Hi, Hi Jes, The below patch seems to have broken PCI IRQ affinity assignments for mlx5. I wouldn't call it breaking IRQ affinity assignments. It just makes them automatic. Prior to this patch I could echo a value to /proc/irq//smp_affinity and it would get assigned. With this patch applied

Re: [PATCH v2 08/15] nvmet: make config_item_type const

2017-10-17 Thread Sagi Grimberg
Acked-by: Sagi Grimberg

Re: [PATCH] net/smc: mark as BROKEN due to remote memory exposure

2017-05-14 Thread Sagi Grimberg
Hi Dave, this patch has not been superceeded by anything, can you explain why it has been marked as such in patchworks? I think you're being overbearing by requiring this to be marked BROKEN and I would like you to explore other ways with the authors to fix whatever perceived problems you thin

Re: net/smc and the RDMA core

2017-05-04 Thread Sagi Grimberg
if you can point out specific issues, we will be happy to work with you to get them addressed! Hello Ursula, My list of issues that I would like to see addressed can be found below. Doug, Christoph and others may have additional inputs. The issues that have not yet been mentioned in other e-m

Re: [PATCH block-tree] net: off by one in inet6_pton()

2017-04-20 Thread Sagi Grimberg
Thanks Dan, Reviewed-by: Sagi Grimberg

Re: [PATCH rfc 5/6] block: Add rdma affinity based queue mapping helper

2017-04-06 Thread Sagi Grimberg
shouldn't you include and like in commit 8ec2ef2b66ea2f that fixes blk-mq-pci.c ? Not really. We can lose these from blk-mq-pci.c as well. +#include +#include +#include +#include +#include "blk-mq.h" Is this include needed ? You're right, I can just keep: +#include +#include +#in

Re: [PATCH rfc 0/6] Automatic affinity settings for nvme over rdma

2017-04-06 Thread Sagi Grimberg
Hi Sagi, Hey Max, the patchset looks good and of course we can add support for more drivers in the future. have you run some performance testing with the nvmf initiator ? I'm limited by the target machine in terms of IOPs, but the host shows ~10% cpu usage decrease, and latency improves slig

Re: [PATCH rfc 6/6] nvme-rdma: use intelligent affinity based queue mappings

2017-04-06 Thread Sagi Grimberg
Use the geneic block layer affinity mapping helper. Also, generic nr_io_queues = min(opts->nr_io_queues, num_online_cpus()); + nr_io_queues = min_t(unsigned int, nr_io_queues, + ibdev->num_comp_vectors); + Add a comment here? Will do

Re: [PATCH rfc 2/6] mlx5: move affinity hints assignments to generic code

2017-04-06 Thread Sagi Grimberg
static int mlx5e_get_cpu(struct mlx5e_priv *priv, int ix) { - return cpumask_first(priv->mdev->priv.irq_info[ix].mask); + return cpumask_first(pci_irq_get_affinity(priv->mdev->pdev, + MLX5_EQ_VEC_COMP_BASE + ix)); This looks ok for now, but if we look at the c

[PATCH rfc 3/6] RDMA/core: expose affinity mappings per completion vector

2017-04-02 Thread Sagi Grimberg
This will allow ULPs to intelligently locate threads based on completion vector cpu affinity mappings. In case the driver does not expose a get_vector_affinity callout, return NULL so the caller can maintain a fallback logic. Signed-off-by: Sagi Grimberg --- include/rdma/ib_verbs.h | 24

  1   2   >