On Thu, Jul 30, 2015 at 09:27:44AM -0400, Jeff Cody wrote:
> On Thu, Jul 30, 2015 at 09:41:08AM +0300, Vasiliy Tolstov wrote:
> > 2015-07-29 12:31 GMT+03:00 Liu Yuan <namei.u...@gmail.com>:
> > > Technically, it won't affect the performance because index updates are
> > > not range
> > > but concrete in terms of underlying 4M block size. Only 2 or 3 indexes in
> > > a
> > > range will be updated and 90+% updates will be only 1. So if 2 updates
> > > stride a
> > > large range, it will actually worse the performance of sheepdog because
> > > many
> > > additional unref of object will be executed by sheep internally.
> > >
> > > It is not a performance problem but more the right fix. Even with your
> > > patch,
> > > updates of inode can overlap. You just don't allow overlapped requests go
> > > to
> > > sheepdog, which is a overkill approach. IMHO, we should only adjust to
> > > avoid
> > > the overlapped inode updates, which can be done easily and incrementally
> > > on top
> > > of old code, rather than take on a complete new untested overkill
> > > mechanism. So
> > > what we get from your patch? Covering the problem and lock every requests?
> > >
> > > Your patch actually fix nothing but just cover the problem by slowing
> > > down the
> > > request and even with your patch, the problem still exists because inode
> > > updates
> > > can overlap. Your commit log doesn't explain what is the real problem and
> > > why
> > > your fix works. This is not your toy project that can commit whatever you
> > > want.
> > >
> > >> BTW, sheepdog project was already forked, why don't you fork the block
> > >> driver, too?
> > >
> > > What makes you think you own the block driver?
> > >
> > > We forked the sheepdog project because it is low quality of code partly
> > > and mostly
> > > some company tries to make it a private project. It is not as open source
> > > friendly
> > > as before and that is the main reason Kazutaka and I chose to fork the
> > > sheepdog
> > > project. But this doesn't mean we need to fork the QEMU project, it is an
> > > open source project and not your home toy.
> > >
> > > Kazutaka and I are the biggest contributers of both sheepdog and QEMU
> > > sheepdog
> > > block driver for years, so I think I am eligible to review the patch and
> > > responsible to suggest the right fix. If you are pissed off when someone
> > > else
> > > have other opinions, you can just fork the code and play with it at home
> > > or you
> > > follow the rule of open source project.
> >
> >
> > Jeff Cody, please be the judge, patch from Hitoshi solved my problem
> > that i emailed in sheepdog list (i have test environment with 8 hosts
> > on each 6 SSD disks and infiniband interconnect between hosts) before
> > Hitoshi patch, massive writing to sheepdog storage breaks file system
> > and corrupt it.
> > After the patch i don't see issues.
> >
>
> I'd rather see some sort consensus amongst Liu, Hitoshi, yourself, or
> others more intimately familiar with sheepdog.
>
> Right now, we have Hitoshi's patch in the main git repo, slated for
> 2.4 release (which is Monday). It sounds, from Liu's email, as this
> may not fix the root cause.
>
> Vasiliy said he would test Liu's patch; if he can confirm this new
> patch fix, then I would be inclined to use Liu's patch, based on the
> detailed analysis of the issue in the commit message.
>
This is my performance comparison on top of latest QEMU with my latop with SSD.
sheepdog cluster run with 3 nodes with '-n' to get best volume performance.
QEMU command:
qemu-system-x86_64 -m 1024 --enable-kvm \
-drive
file=debian_squeeze_amd64_standard.qcow2,cache=writeback,if=virtio \
-drive file=sheepdog:test,if=virtio
sheepdog:test is created as 'dog vdi create test 80G'
I test both time for mkfs and iops for fio write.
fio.conf:
[global]
ioengine=libaio
direct=1
thread=1
norandommap=1
runtime=60
size=300M
directory=/mnt
[write4k-rand]
stonewall
group_reporting
bs=4k
rw=randwrite
numjobs=8
iodepth=32
Resualt:
================================================
sheep formated with -c 2:1 (erasure coding)
mkfs fio
Yuan 0.069 4578
Hitosh 0.071 3722
sheep formarted with -c 2 (replication)
mkfs fio
Yuan 0.074 6873
Hitosh 0.081 6174
================================================
Thanks,
Yuan
