On Wed, Jul 31, 2019 at 10:31 PM nokia ceph <nokiacephus...@gmail.com> wrote: > > Thank you Greg, > > Another question , we need to give new destination object , so that we can > read them separately in parallel with src object . This function resides in > objector.h , seems to be like internal and can it be used in interface level > and can we use this in our client ? Currently we use librados.h in our client > to communicate with ceph cluster.
copy_from is an ObjectOperations and exposed via the librados C++ api like all the others. It may not be in the simple <function_name>(<object>, <data>, ....) interfaces. It may also not be in the C API? > Also any equivalent librados api for the command rados -p poolname <src > object> <dst object> It's using the copy_from command we're discussing here. You can look at the source as an example: https://github.com/ceph/ceph/blob/master/src/tools/rados/rados.cc#L497 -Greg > > Thanks, > Muthu > > On Wed, Jul 31, 2019 at 11:13 PM Gregory Farnum <gfar...@redhat.com> wrote: >> >> >> >> On Wed, Jul 31, 2019 at 1:32 AM nokia ceph <nokiacephus...@gmail.com> wrote: >>> >>> Hi Greg, >>> >>> We were trying to implement this however having issues in assigning the >>> destination object name with this api. >>> There is a rados command "rados -p <poolname> cp <src obj> <dst obj>" , is >>> there any librados api equivalent to this ? >> >> >> The copyfrom operation, like all other ops, is directed to a specific >> object. The object you run it on is the destination; it copies the specified >> “src” object into itself. >> -Greg >> >>> >>> Thanks, >>> Muthu >>> >>> On Fri, Jul 5, 2019 at 4:00 PM nokia ceph <nokiacephus...@gmail.com> wrote: >>>> >>>> Thank you Greg, we will try this out . >>>> >>>> Thanks, >>>> Muthu >>>> >>>> On Wed, Jul 3, 2019 at 11:12 PM Gregory Farnum <gfar...@redhat.com> wrote: >>>>> >>>>> Well, the RADOS interface doesn't have a great deal of documentation >>>>> so I don't know if I can point you at much. >>>>> >>>>> But if you look at Objecter.h, you see that the ObjectOperation has >>>>> this function: >>>>> void copy_from(object_t src, snapid_t snapid, object_locator_t >>>>> src_oloc, version_t src_version, unsigned flags, unsigned >>>>> src_fadvise_flags) >>>>> >>>>> src: the object to copy from >>>>> snapid: if you want to copy a specific snap instead of HEAD >>>>> src_oloc: the object locator for the object >>>>> src_version: the version of the object to copy from (helps identify if >>>>> it was updated in the meantime) >>>>> flags: probably don't want to set these, but see >>>>> PrimaryLogPG::_copy_some for the choices >>>>> src_fadvise_flags: these are the fadvise flags we have in various >>>>> places that let you specify things like not to cache the data. >>>>> Probably leave them unset. >>>>> >>>>> -Greg >>>>> >>>>> >>>>> >>>>> On Wed, Jul 3, 2019 at 2:47 AM nokia ceph <nokiacephus...@gmail.com> >>>>> wrote: >>>>> > >>>>> > Hi Greg, >>>>> > >>>>> > Can you please share the api details for COPY_FROM or any reference >>>>> > document? >>>>> > >>>>> > Thanks , >>>>> > Muthu >>>>> > >>>>> > On Wed, Jul 3, 2019 at 4:12 AM Brad Hubbard <bhubb...@redhat.com> wrote: >>>>> >> >>>>> >> On Wed, Jul 3, 2019 at 4:25 AM Gregory Farnum <gfar...@redhat.com> >>>>> >> wrote: >>>>> >> > >>>>> >> > I'm not sure how or why you'd get an object class involved in doing >>>>> >> > this in the normal course of affairs. >>>>> >> > >>>>> >> > There's a copy_from op that a client can send and which copies an >>>>> >> > object from another OSD into the target object. That's probably the >>>>> >> > primitive you want to build on. Note that the OSD doesn't do much >>>>> >> >>>>> >> Argh! yes, good idea. We really should document that! >>>>> >> >>>>> >> > consistency checking (it validates that the object version matches an >>>>> >> > input, but if they don't it just returns an error) so the client >>>>> >> > application is responsible for any locking needed. >>>>> >> > -Greg >>>>> >> > >>>>> >> > On Tue, Jul 2, 2019 at 3:49 AM Brad Hubbard <bhubb...@redhat.com> >>>>> >> > wrote: >>>>> >> > > >>>>> >> > > Yes, this should be possible using an object class which is also a >>>>> >> > > RADOS client (via the RADOS API). You'll still have some client >>>>> >> > > traffic as the machine running the object class will still need to >>>>> >> > > connect to the relevant primary osd and send the write (presumably >>>>> >> > > in >>>>> >> > > some situations though this will be the same machine). >>>>> >> > > >>>>> >> > > On Tue, Jul 2, 2019 at 4:08 PM nokia ceph >>>>> >> > > <nokiacephus...@gmail.com> wrote: >>>>> >> > > > >>>>> >> > > > Hi Brett, >>>>> >> > > > >>>>> >> > > > I think I was wrong here in the requirement description. It is >>>>> >> > > > not about data replication , we need same content stored in >>>>> >> > > > different object/name. >>>>> >> > > > We store video contents inside the ceph cluster. And our new >>>>> >> > > > requirement is we need to store same content for different users >>>>> >> > > > , hence need same content in different object name . if client >>>>> >> > > > sends write request for object x and sets number of copies as >>>>> >> > > > 100, then cluster has to clone 100 copies of object x and store >>>>> >> > > > it as object x1, objectx2,etc. Currently this is done in the >>>>> >> > > > client side where objectx1, object x2...objectx100 are cloned >>>>> >> > > > inside the client and write request sent for all 100 objects >>>>> >> > > > which we want to avoid to reduce network consumption. >>>>> >> > > > >>>>> >> > > > Similar usecases are rbd snapshot , radosgw copy . >>>>> >> > > > >>>>> >> > > > Is this possible in object class ? >>>>> >> > > > >>>>> >> > > > thanks, >>>>> >> > > > Muthu >>>>> >> > > > >>>>> >> > > > >>>>> >> > > > On Mon, Jul 1, 2019 at 7:58 PM Brett Chancellor >>>>> >> > > > <bchancel...@salesforce.com> wrote: >>>>> >> > > >> >>>>> >> > > >> Ceph already does this by default. For each replicated pool, >>>>> >> > > >> you can set the 'size' which is the number of copies you want >>>>> >> > > >> Ceph to maintain. The accepted norm for replicas is 3, but you >>>>> >> > > >> can set it higher if you want to incur the performance penalty. >>>>> >> > > >> >>>>> >> > > >> On Mon, Jul 1, 2019, 6:01 AM nokia ceph >>>>> >> > > >> <nokiacephus...@gmail.com> wrote: >>>>> >> > > >>> >>>>> >> > > >>> Hi Brad, >>>>> >> > > >>> >>>>> >> > > >>> Thank you for your response , and we will check this video as >>>>> >> > > >>> well. >>>>> >> > > >>> Our requirement is while writing an object into the cluster , >>>>> >> > > >>> if we can provide number of copies to be made , the network >>>>> >> > > >>> consumption between client and cluster will be only for one >>>>> >> > > >>> object write. However , the cluster will clone/copy multiple >>>>> >> > > >>> objects and stores inside the cluster. >>>>> >> > > >>> >>>>> >> > > >>> Thanks, >>>>> >> > > >>> Muthu >>>>> >> > > >>> >>>>> >> > > >>> On Fri, Jun 28, 2019 at 9:23 AM Brad Hubbard >>>>> >> > > >>> <bhubb...@redhat.com> wrote: >>>>> >> > > >>>> >>>>> >> > > >>>> On Thu, Jun 27, 2019 at 8:58 PM nokia ceph >>>>> >> > > >>>> <nokiacephus...@gmail.com> wrote: >>>>> >> > > >>>> > >>>>> >> > > >>>> > Hi Team, >>>>> >> > > >>>> > >>>>> >> > > >>>> > We have a requirement to create multiple copies of an >>>>> >> > > >>>> > object and currently we are handling it in client side to >>>>> >> > > >>>> > write as separate objects and this causes huge network >>>>> >> > > >>>> > traffic between client and cluster. >>>>> >> > > >>>> > Is there possibility of cloning an object to multiple >>>>> >> > > >>>> > copies using librados api? >>>>> >> > > >>>> > Please share the document details if it is feasible. >>>>> >> > > >>>> >>>>> >> > > >>>> It may be possible to use an object class to accomplish what >>>>> >> > > >>>> you want >>>>> >> > > >>>> to achieve but the more we understand what you are trying to >>>>> >> > > >>>> do, the >>>>> >> > > >>>> better the advice we can offer (at the moment your >>>>> >> > > >>>> description sounds >>>>> >> > > >>>> like replication which is already part of RADOS as you know). >>>>> >> > > >>>> >>>>> >> > > >>>> More on object classes from Cephalocon Barcelona in May this >>>>> >> > > >>>> year: >>>>> >> > > >>>> https://www.youtube.com/watch?v=EVrP9MXiiuU >>>>> >> > > >>>> >>>>> >> > > >>>> > >>>>> >> > > >>>> > Thanks, >>>>> >> > > >>>> > Muthu >>>>> >> > > >>>> > _______________________________________________ >>>>> >> > > >>>> > ceph-users mailing list >>>>> >> > > >>>> > ceph-users@lists.ceph.com >>>>> >> > > >>>> > http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com >>>>> >> > > >>>> >>>>> >> > > >>>> >>>>> >> > > >>>> >>>>> >> > > >>>> -- >>>>> >> > > >>>> Cheers, >>>>> >> > > >>>> Brad >>>>> >> > > >>> >>>>> >> > > >>> _______________________________________________ >>>>> >> > > >>> ceph-users mailing list >>>>> >> > > >>> ceph-users@lists.ceph.com >>>>> >> > > >>> http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com >>>>> >> > > >>>>> >> > > >>>>> >> > > >>>>> >> > > -- >>>>> >> > > Cheers, >>>>> >> > > Brad >>>>> >> > > _______________________________________________ >>>>> >> > > ceph-users mailing list >>>>> >> > > ceph-users@lists.ceph.com >>>>> >> > > http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com >>>>> >> >>>>> >> >>>>> >> >>>>> >> -- >>>>> >> Cheers, >>>>> >> Brad _______________________________________________ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
