Re: [ceph-users] Note about rbd_aio_write usage

2017-07-07 Thread Jason Dillaman
On Fri, Jul 7, 2017 at 2:48 AM, Piotr Dałek wrote: > Is this: > https://github.com/yuyuyu101/ceph/commit/794b49b5b860c538a349bdadb16bb6ae97ad9c20#commitcomment-15707924 > the issue you mention? Because at this point I'm considering switching to > C++ API and passing static bufferptr buried in my b

Re: [ceph-users] Note about rbd_aio_write usage

2017-07-06 Thread Piotr Dałek
On 17-07-06 09:39 PM, Jason Dillaman wrote: On Thu, Jul 6, 2017 at 3:25 PM, Piotr Dałek wrote: Is that deep copy an equivalent of what Jewel librbd did at unspecified point of time, or extra one? It's equivalent / replacement -- not an additional copy. This was changed to support scatter/gath

Re: [ceph-users] Note about rbd_aio_write usage

2017-07-06 Thread Jason Dillaman
On Thu, Jul 6, 2017 at 3:25 PM, Piotr Dałek wrote: > Is that deep copy an equivalent of what > Jewel librbd did at unspecified point of time, or extra one? It's equivalent / replacement -- not an additional copy. This was changed to support scatter/gather IO API methods which the latest version o

Re: [ceph-users] Note about rbd_aio_write usage

2017-07-06 Thread Jason Dillaman
On Thu, Jul 6, 2017 at 11:46 AM, Piotr Dałek wrote: > How about a hybrid solution? Keep the old rbd_aio_write contract (don't copy > the buffer with the assumption that it won't change) and instead of > constructing bufferlist containing bufferptr to copied data, construct a > bufferlist containin

Re: [ceph-users] Note about rbd_aio_write usage

2017-07-06 Thread Piotr Dałek
On 17-07-06 04:40 PM, Jason Dillaman wrote: On Thu, Jul 6, 2017 at 10:22 AM, Piotr Dałek wrote: So I really see two problems here: lack of API docs and backwards-incompatible change in API behavior. Docs are always in need of update, so any pull requests would be greatly appreciated. However

Re: [ceph-users] Note about rbd_aio_write usage

2017-07-06 Thread Jason Dillaman
On Thu, Jul 6, 2017 at 10:22 AM, Piotr Dałek wrote: > So I really see two problems here: lack of API docs and > backwards-incompatible change in API behavior. Docs are always in need of update, so any pull requests would be greatly appreciated. However, I disagree that the behavior has substanti

Re: [ceph-users] Note about rbd_aio_write usage

2017-07-06 Thread Piotr Dałek
On 17-07-06 03:43 PM, Jason Dillaman wrote: I've learned the hard way that pre-luminous, even if it copies the buffer, it does so too late. In my specific case, my FUSE module does enter the write call and issues rbd_aio_write there, then exits the write - expecting the buffer provided by FUSE to

Re: [ceph-users] Note about rbd_aio_write usage

2017-07-06 Thread Jason Dillaman
The correct (POSIX-style) program behavior should treat the buffer as immutable until the IO operation completes. It is never safe to assume the buffer can be re-used while the IO is in-flight. You should not add any logic to assume the buffer is safely copied prior to the completion of the IO. On

Re: [ceph-users] Note about rbd_aio_write usage

2017-07-06 Thread Piotr Dałek
On 17-07-06 03:03 PM, Jason Dillaman wrote: On Thu, Jul 6, 2017 at 8:26 AM, Piotr Dałek wrote: Hi, If you're using "rbd_aio_write()" in your code, be aware of the fact that before Luminous release, this function expects buffer to remain unchanged until write op ends, and on Luminous and later

Re: [ceph-users] Note about rbd_aio_write usage

2017-07-06 Thread Jason Dillaman
Pre-Luminous also copies the provided buffer when using the C API -- it just copies it at a later point and not immediately. The eventual goal is to eliminate the copy completely, but that requires some additional plumbing work deep down within the librados messenger layer. On Thu, Jul 6, 2017 at