Re: [ceph-users] Reaching aio-max-nr on Ubuntu 16.04 with Luminous

2017-08-30 Thread Thomas Bennett
Hi Dan, Great! Thanks for the feedback. Much appreciated. For completion here is my ansible role: - name: Increase aio-max-nr for bluestore sysctl: name: fs.aio-max-nr value: 1048576 sysctl_file: /etc/sysctl.d/ceph-tuning.conf sysctl_set: yes Cheers, Tom On Wed, Aug 30, 2017

Re: [ceph-users] Reaching aio-max-nr on Ubuntu 16.04 with Luminous

2017-08-30 Thread Dan van der Ster
Hi Thomas, Yes we set it to a million. >From our puppet manifest: # need to increase aio-max-nr to allow many bluestore devs sysctl { 'fs.aio-max-nr': val => '1048576' } Cheers, Dan On Aug 30, 2017 9:53 AM, "Thomas Bennett" wrote: > > Hi, > > I've been testing out Lum

[ceph-users] Reaching aio-max-nr on Ubuntu 16.04 with Luminous

2017-08-30 Thread Thomas Bennett
Hi, I've been testing out Luminous and I've noticed that at some point the number of osds per nodes was limited by aio-max-nr. By default its set to 65536 in Ubuntu 16.04 Has anyone else experienced this issue? fs.aio-nr currently sitting at 196608 with 48 osds. I have 48 osd's per node so I've