Hi Mina,
I recommend you cc linux-mm and Matthew Wilcox on these two patches also.
David
Linus Torvalds wrote:
> From a quick look, what you were doing was checking that the patterns
> you set up in user space came through ok. Dammit, what's wrong with
> just using read()/write() on a pipe, or splice, or whatever. It will
> test exactly the same iov_iter thing.
I was trying to make
Add kunit tests to benchmark 256MiB copies to a UBUF iterator and an IOVEC
iterator. This attaches a userspace VM with a mapped file in it
temporarily to the test thread.
Signed-off-by: David Howells
cc: Andrew Morton
cc: Christoph Hellwig
cc: Christian Brauner
cc: Jens Axboe
cc: Al Viro
Add a kunit test to benchmark an xarray containing 256MiB of data getting
decanted into 256-page BVEC iterators that get copied from - modelling
bufferage being drawn from the pagecache and batched up for I/O.
Signed-off-by: David Howells
cc: Christoph Hellwig
cc: Christian Brauner
cc: Jens
with mmap
- basically it has to emulated part of execve(). Doing so requires access
to additional core symbols: mm_alloc(), vm_area_alloc(), insert_vm_struct()
and arch_pick_mmap_layout(). See the iov_kunit_create_user_buf() function
added in the patch.
Signed-off-by: David Howells
cc: Andrew
Add kunit tests to benchmark 256MiB copies to a KVEC iterator, a BVEC
iterator, an XARRAY iterator and to a loop that allocates 256-page BVECs
and fills them in (similar to a maximal bio struct being set up).
Signed-off-by: David Howells
cc: Christoph Hellwig
cc: Christian Brauner
cc: Jens
Add copy kunit tests for ITER_UBUF- and ITER_IOVEC-type iterators. This
attaches a userspace VM with a mapped file in it temporarily to the test
thread.
Signed-off-by: David Howells
cc: Andrew Morton
cc: Christoph Hellwig
cc: Christian Brauner
cc: Jens Axboe
cc: Al Viro
cc: Matthew Wilcox
and not in a module?
Signed-off-by: David Howells
cc: Andrew Morton
cc: Christoph Hellwig
cc: Christian Brauner
cc: Jens Axboe
cc: Al Viro
cc: Matthew Wilcox
cc: David Hildenbrand
cc: John Hubbard
cc: Brendan Higgins
cc: David Gow
cc: Huacai Chen
cc: WANG Xuerui
cc: Heiko Carstens
cc: Vasi
Make the BVEC-testing functions use the consolidated pattern checking
functions to reduce the amount of duplicated code.
Signed-off-by: David Howells
cc: Christoph Hellwig
cc: Christian Brauner
cc: Jens Axboe
cc: Al Viro
cc: David Hildenbrand
cc: John Hubbard
cc: Brendan Higgins
cc: David
Consolidate the test vector struct in the kunit tests so that the bvec
pattern check helpers can share with the kvec check helpers.
Signed-off-by: David Howells
cc: Christoph Hellwig
cc: Christian Brauner
cc: Jens Axboe
cc: Al Viro
cc: David Hildenbrand
cc: John Hubbard
cc: Brendan Higgins
Consolidate some of the repeated code snippets into helper functions to
reduce the line count.
Signed-off-by: David Howells
cc: Christoph Hellwig
cc: Christian Brauner
cc: Jens Axboe
cc: Al Viro
cc: David Hildenbrand
cc: John Hubbard
cc: Brendan Higgins
cc: David Gow
cc: linux-kselftest
shirn
Signed-off-by: David Howells
cc: Christoph Hellwig
cc: Christian Brauner
cc: Jens Axboe
cc: Al Viro
cc: David Hildenbrand
cc: John Hubbard
cc: Brendan Higgins
cc: David Gow
cc: linux-kselftest@vger.kernel.org
cc: kunit-...@googlegroups.com
cc: linux...@kvack.org
cc: linux
30400.203330-1-dhowe...@redhat.com/ #
v2
Link: https://lore.kernel.org/r/20230922113038.1135236-1-dhowe...@redhat.com/ #
v3
David Howells (10):
iov_iter: Fix some checkpatch complaints in kunit tests
iov_iter: Consolidate some of the repeated code into helpers
iov_iter: Consolidate the test vector
Add kunit tests to benchmark 256MiB copies to a KVEC iterator, a BVEC
iterator, an XARRAY iterator and to a loop that allocates 256-page BVECs
and fills them in (similar to a maximal bio struct being set up).
Signed-off-by: David Howells
cc: Christoph Hellwig
cc: Christian Brauner
cc: Jens
Add a kunit test to benchmark an xarray containing 256MiB of data getting
decanted into 256-page BVEC iterators that get copied from - modelling
bufferage being drawn from the pagecache and batched up for I/O.
Signed-off-by: David Howells
cc: Christoph Hellwig
cc: Christian Brauner
cc: Jens
Add kunit tests to benchmark 256MiB copies to a UBUF iterator and an IOVEC
iterator. This attaches a userspace VM with a mapped file in it
temporarily to the test thread.
Signed-off-by: David Howells
cc: Andrew Morton
cc: Christoph Hellwig
cc: Christian Brauner
cc: Jens Axboe
cc: Al Viro
Consolidate some of the repeated code snippets into helper functions to
reduce the line count.
Signed-off-by: David Howells
cc: Christoph Hellwig
cc: Christian Brauner
cc: Jens Axboe
cc: Al Viro
cc: David Hildenbrand
cc: John Hubbard
cc: Brendan Higgins
cc: David Gow
cc: linux-kselftest
with mmap
- basically it has to emulated part of execve(). Doing so requires access
to additional core symbols: mm_alloc(), vm_area_alloc(), insert_vm_struct()
and arch_pick_mmap_layout(). See the iov_kunit_create_user_buf() function
added in the patch.
Signed-off-by: David Howells
cc: Andrew
Consolidate the test vector struct in the kunit tests so that the bvec
pattern check helpers can share with the kvec check helpers.
Signed-off-by: David Howells
cc: Christoph Hellwig
cc: Christian Brauner
cc: Jens Axboe
cc: Al Viro
cc: David Hildenbrand
cc: John Hubbard
cc: Brendan Higgins
Add copy kunit tests for ITER_UBUF- and ITER_IOVEC-type iterators. This
attaches a userspace VM with a mapped file in it temporarily to the test
thread.
Signed-off-by: David Howells
cc: Andrew Morton
cc: Christoph Hellwig
cc: Christian Brauner
cc: Jens Axboe
cc: Al Viro
cc: Matthew Wilcox
and not in a module?
Signed-off-by: David Howells
cc: Andrew Morton
cc: Christoph Hellwig
cc: Christian Brauner
cc: Jens Axboe
cc: Al Viro
cc: Matthew Wilcox
cc: David Hildenbrand
cc: John Hubbard
cc: Brendan Higgins
cc: David Gow
cc: Huacai Chen
cc: WANG Xuerui
cc: Heiko Carstens
cc: Vasi
shirn
Signed-off-by: David Howells
cc: Christoph Hellwig
cc: Christian Brauner
cc: Jens Axboe
cc: Al Viro
cc: David Hildenbrand
cc: John Hubbard
cc: Brendan Higgins
cc: David Gow
cc: linux-kselftest@vger.kernel.org
cc: kunit-...@googlegroups.com
cc: linux...@kvack.org
cc: linux
Make the BVEC-testing functions use the consolidated pattern checking
functions to reduce the amount of duplicated code.
Signed-off-by: David Howells
cc: Christoph Hellwig
cc: Christian Brauner
cc: Jens Axboe
cc: Al Viro
cc: David Hildenbrand
cc: John Hubbard
cc: Brendan Higgins
cc: David
30400.203330-1-dhowe...@redhat.com/ #
v2
David Howells (10):
iov_iter: Fix some checkpatch complaints in kunit tests
iov_iter: Consolidate some of the repeated code into helpers
iov_iter: Consolidate the test vector struct in the kunit tests
iov_iter: Consolidate bvec pattern checking
iov_iter:
David Howells wrote:
> Hi Jens,
>
> Can you consider taking this through the block tree?
Sorry, I forgot to remove the 'RFC' tag in the subject.
David
David Howells wrote:
> diff --git a/arch/loongarch/include/asm/page.h
> b/arch/loongarch/include/asm/page.h
> index 63f137ce82a4..c7c5f5b4c0d3 100644
> --- a/arch/loongarch/include/asm/page.h
> +++ b/arch/loongarch/include/asm/page.h
> @@ -32,6 +32,7 @@
>
> #include
Consolidate the test vector struct in the kunit tests so that the bvec
pattern check helpers can share with the kvec check helpers.
Signed-off-by: David Howells
cc: Christoph Hellwig
cc: Christian Brauner
cc: Jens Axboe
cc: Al Viro
cc: David Hildenbrand
cc: John Hubbard
cc: Brendan Higgins
with mmap
- basically it has to emulated part of execve(). Doing so requires access
to additional core symbols: mm_alloc(), vm_area_alloc(), insert_vm_struct()
and arch_pick_mmap_layout(). See the iov_kunit_create_user_buf() function
added in the patch.
Signed-off-by: David Howells
cc: Andrew
Add kunit tests to benchmark 256MiB copies to a UBUF iterator and an IOVEC
iterator. This attaches a userspace VM with a mapped file in it
temporarily to the test thread.
Signed-off-by: David Howells
cc: Andrew Morton
cc: Christoph Hellwig
cc: Christian Brauner
cc: Jens Axboe
cc: Al Viro
and not in a module?
Signed-off-by: David Howells
cc: Andrew Morton
cc: Christoph Hellwig
cc: Christian Brauner
cc: Jens Axboe
cc: Al Viro
cc: Matthew Wilcox
cc: David Hildenbrand
cc: John Hubbard
cc: Brendan Higgins
cc: David Gow
cc: Huacai Chen
cc: WANG Xuerui
cc: Heiko Carstens
cc: Vasi
Consolidate some of the repeated code snippets into helper functions to
reduce the line count.
Signed-off-by: David Howells
cc: Christoph Hellwig
cc: Christian Brauner
cc: Jens Axboe
cc: Al Viro
cc: David Hildenbrand
cc: John Hubbard
cc: Brendan Higgins
cc: David Gow
cc: linux-kselftest
Add copy kunit tests for ITER_UBUF- and ITER_IOVEC-type iterators. This
attaches a userspace VM with a mapped file in it temporarily to the test
thread.
Signed-off-by: David Howells
cc: Andrew Morton
cc: Christoph Hellwig
cc: Christian Brauner
cc: Jens Axboe
cc: Al Viro
cc: Matthew Wilcox
Make the BVEC-testing functions use the consolidated pattern checking
functions to reduce the amount of duplicated code.
Signed-off-by: David Howells
cc: Christoph Hellwig
cc: Christian Brauner
cc: Jens Axboe
cc: Al Viro
cc: David Hildenbrand
cc: John Hubbard
cc: Brendan Higgins
cc: David
Add kunit tests to benchmark 256MiB copies to a KVEC iterator, a BVEC
iterator, an XARRAY iterator and to a loop that allocates 256-page BVECs
and fills them in (similar to a maximal bio struct being set up).
Signed-off-by: David Howells
cc: Christoph Hellwig
cc: Christian Brauner
cc: Jens
he iteration is
not not negligible.
- Switch the benchmarking to use copy_from_iter() so that only a single
page is needed in the userspace buffer (as it can be shared R/O), not
256MiB's worth.
Link: https://lore.kernel.org/r/20230914221526.3153402-1-dhowe...@redhat.com/ #
v1
David How
shirn
Signed-off-by: David Howells
cc: Christoph Hellwig
cc: Christian Brauner
cc: Jens Axboe
cc: Al Viro
cc: David Hildenbrand
cc: John Hubbard
cc: Brendan Higgins
cc: David Gow
cc: linux-kselftest@vger.kernel.org
cc: kunit-...@googlegroups.com
cc: linux...@kvack.org
cc: linux
David Laight wrote:
> You could also just not do the copy!
> Although you need (say) asm volatile("\n",:::"memory") to
> stop it all being completely optimised away.
> That might show up a difference in the 'out_of_line' test
> where 15% on top on the data copies is massive - it may be
> that the
David Laight wrote:
> I was thinking of import_iovec() - or whatever its current
> name is.
That doesn't actually access the buffer described by the iovec[].
> That really needs a single structure that contains the iov_iter
> and the cache[] (which the caller pretty much always allocates
> in t
David Laight wrote:
> Isn't that going to be completely dominated by the cache fills
> from memory?
>
> I'd have thought you'd need to use something with a lot of
> small fragments so that the iteration code dominates the copy.
Okay, if I switch it to using MAP_ANON for the big 256MiB buffer, s
David Laight wrote:
> > > Some measurements can be made using readv() and writev()
> > > on /dev/zero and /dev/null.
> >
> > Forget /dev/null; that doesn't actually engage any iteration code. The same
> > for writing to /dev/zero. Reading from /dev/zero does its own iteration
> > thing
> > ra
David Laight wrote:
> > Add kunit tests to benchmark 256MiB copies to a UBUF iterator and an IOVEC
> > iterator. This attaches a userspace VM with a mapped file in it
> > temporarily to the test thread.
>
> Isn't that going to be completely dominated by the cache fills
> from memory?
Yes... b
Add kunit tests to benchmark 256MiB copies to a UBUF iterator and an IOVEC
iterator. This attaches a userspace VM with a mapped file in it
temporarily to the test thread.
Signed-off-by: David Howells
cc: Andrew Morton
cc: Christoph Hellwig
cc: Christian Brauner
cc: Jens Axboe
cc: Al Viro
with mmap
- basically it has to emulated part of execve(). Doing so requires access
to additional core symbols: mm_alloc(), vm_area_alloc(), insert_vm_struct()
and arch_pick_mmap_layout(). See the iov_kunit_create_user_buf() function
added in the patch.
Signed-off-by: David Howells
cc: Andrew
and not in a module?
Signed-off-by: David Howells
cc: Andrew Morton
cc: Christoph Hellwig
cc: Christian Brauner
cc: Jens Axboe
cc: Al Viro
cc: Matthew Wilcox
cc: David Hildenbrand
cc: John Hubbard
cc: Brendan Higgins
cc: David Gow
cc: linux...@kvack.org
cc: linux-fsde...@vger.kernel.org
cc
Make the BVEC-testing functions use the consolidated pattern checking
functions to reduce the amount of duplicated code.
Signed-off-by: David Howells
cc: Christoph Hellwig
cc: Christian Brauner
cc: Jens Axboe
cc: Al Viro
cc: David Hildenbrand
cc: John Hubbard
cc: Brendan Higgins
cc: David
Add copy kunit tests for ITER_UBUF- and ITER_IOVEC-type iterators. This
attaches a userspace VM with a mapped file in it temporarily to the test
thread.
Signed-off-by: David Howells
cc: Andrew Morton
cc: Christoph Hellwig
cc: Christian Brauner
cc: Jens Axboe
cc: Al Viro
cc: Matthew Wilcox
Add kunit tests to benchmark 256MiB copies to a KVEC iterator, a BVEC
iterator, an XARRAY iterator and to a loop that allocates 256-page BVECs
and fills them in (similar to a maximal bio struct being set up).
Signed-off-by: David Howells
cc: Christoph Hellwig
cc: Christian Brauner
cc: Jens
Consolidate some of the repeated code snippets into helper functions to
reduce the line count.
Signed-off-by: David Howells
cc: Christoph Hellwig
cc: Christian Brauner
cc: Jens Axboe
cc: Al Viro
cc: David Hildenbrand
cc: John Hubbard
cc: Brendan Higgins
cc: David Gow
cc: linux-kselftest
Consolidate the test vector struct in the kunit tests so that the bvec
pattern check helpers can share with the kvec check helpers.
Signed-off-by: David Howells
cc: Christoph Hellwig
cc: Christian Brauner
cc: Jens Axboe
cc: Al Viro
cc: David Hildenbrand
cc: John Hubbard
cc: Brendan Higgins
shirn
Signed-off-by: David Howells
cc: Christoph Hellwig
cc: Christian Brauner
cc: Jens Axboe
cc: Al Viro
cc: David Hildenbrand
cc: John Hubbard
cc: Brendan Higgins
cc: David Gow
cc: linux-kselftest@vger.kernel.org
cc: kunit-...@googlegroups.com
cc: linux...@kvack.org
cc: linux
ddev 15 uS
iov_kunit_benchmark_xarray: avg 3582 uS, stddev 13 uS
I've pushed the patches here also:
https://git.kernel.org/pub/scm/linux/kernel/git/dhowells/linux-fs.git/log/?h=iov-kunit
David
David Howells (9):
iov_iter: Fix some checkpatch complaints in kunit tests
iov_iter: Consolid
51 matches
Mail list logo
