Justin, NVMe drives have their own IO queueing mechanism and there is a huge performance difference vs the linux queue. Next to properly configured file system and scheduler try setting "scsi_mod.use_blk_mq=1" in grub cmdline. If you are looking for a BFQ scheduler, its probably a module so you will need to load it.
Best, Matija On Tue, Jan 9, 2018 at 1:17 AM, Nate McCall <n...@thelastpickle.com> wrote: > >> >> >> In regards to setting read ahead, how is this set for nvme drives? Also, >> below is our compression settings for the table… It’s the same as our tests >> that we are doing against SAS SSDs so I don’t think the compression >> settings would be the issue… >> >> >> > > Check blockdev --report between the old and the new servers to see if > there is a difference. Are there other deltas in the disk layouts between > the old and new servers (ie. LVM, mdadm, etc.)? > > You can control read ahead via 'blockdev --setra' or via poking the > kernel: /sys/block/[YOUR DRIVE]/queue/read_ahead_kb > > In both cases, changes are instantaneous so you can do it on a canary and > monitor for effect. > > Also, i'd be curious to know (since you have this benchmark setup) if you > got the degradation you are currently seeing if you set concurrent_reads > and concurrent_writes back to their defaults. > > > -- > ----------------- > Nate McCall > Wellington, NZ > @zznate > > CTO > Apache Cassandra Consulting > http://www.thelastpickle.com >
