Re: [PATCH RFC] target/user: Add double ring buffers support.

2017-02-14 Thread Xiubo Li
The struct tcmu_cmd_entry {} size is fixed 44 bytes without iovec[], and the size of struct iovec[N] is about 16 bytes * N. The cmd entry size will be [44B, N *16 + 44B], and the data size will be [0, N * 4096], so the ratio of sizeof(cmd entry): sizeof(entry datas) == (N * 16 + 44)Bytes : (N * 4

Re: [PATCH RFC] target/user: Add double ring buffers support.

2017-02-14 Thread Andy Grover
On 02/13/2017 09:50 PM, Xiubo Li wrote: > The struct tcmu_cmd_entry {} size is fixed 44 bytes without iovec[], and > the size of struct iovec[N] is about 16 bytes * N. > > The cmd entry size will be [44B, N *16 + 44B], and the data size will be > [0, N * 4096], so the ratio of sizeof(cmd entry): s

Re: [PATCH RFC] target/user: Add double ring buffers support.

2017-02-13 Thread Xiubo Li
Yes I think it's now clear we need more buffer space to avoid bottlenecks for high iops. The initial design kept it simple with the 1MB vmalloc'd space but anticipated greater would be needed. It should not be necessary to change userspace or the TCMU ABI to handle growing the buffer for fast de

Re: [PATCH RFC] target/user: Add double ring buffers support.

2017-02-13 Thread Xiubo Li
On 2017年02月14日 01:42, Andy Grover wrote: On 02/12/2017 05:38 PM, Xiubo Li wrote: On 2017年02月11日 02:02, Andy Grover wrote: 1. increase the region mmap()ed by userspace, TCMU_RING_SIZE, from 1MB to 1GB or larger 2. Don't vmalloc() the whole thing, instead vmalloc for the cmd ring portion, and d

Re: [PATCH RFC] target/user: Add double ring buffers support.

2017-02-13 Thread Andy Grover
On 02/13/2017 01:50 AM, Xiubo Li wrote: There is one new scheme in my mind: Yes I think it's now clear we need more buffer space to avoid bottlenecks for high iops. The initial design kept it simple with the 1MB vmalloc'd space but anticipated greater would be needed. It should not be necessar

Re: [PATCH RFC] target/user: Add double ring buffers support.

2017-02-13 Thread Andy Grover
On 02/12/2017 05:38 PM, Xiubo Li wrote: On 2017年02月11日 02:02, Andy Grover wrote: 1. increase the region mmap()ed by userspace, TCMU_RING_SIZE, from 1MB to 1GB or larger 2. Don't vmalloc() the whole thing, instead vmalloc for the cmd ring portion, and dynamically alloc pages for the data area as

Re: [PATCH RFC] target/user: Add double ring buffers support.

2017-02-13 Thread Xiubo Li
Hi Andy, There is one new scheme in my mind: Yes I think it's now clear we need more buffer space to avoid bottlenecks for high iops. The initial design kept it simple with the 1MB vmalloc'd space but anticipated greater would be needed. It should not be necessary to change userspace or the

Re: [PATCH RFC] target/user: Add double ring buffers support.

2017-02-10 Thread Andy Grover
On 02/09/2017 10:48 PM, lixi...@cmss.chinamobile.com wrote: For now the tcmu is based on UIO framework and only using the map0 with fixed ring buffer size. This will work fine mostly, but when we are using the 10GBASE NIC, the ring buffer is too small to deal with the incoming iscsi cmds. We can