Well, just from a quick skim, libradosstriper.h has a function
rados_striper_set_object_layout_object_size(rados_striper_t striper,
unsigned int object_size)
and libradosstriper.hpp has one in RadosStriper
set_object_layout_object_size(unsigned int object_size);
So I imagine you specify it with those the same way you've set the stripe
unit and counts.
On Sat, Oct 7, 2017 at 12:38 PM Alexander Kushnirenko <kushnire...@gmail.com>
wrote:
> Hi, Gregory!
>
> It turns out that this error is internal CEPH feature. I wrote standalone
> program to create 132M object in striper mode. It works only for 4M
> stripe. If you set stripe_unit = 2M it still creates 4M stripe_unit.
> Anything bigger than 4M causes crash here
> <https://github.com/ceph/ceph/blob/master/src/osdc/Striper.cc#L64%5C>:
>
>
> __u32 object_size = layout->object_size;
> __u32 su = layout->stripe_unit;
> __u32 stripe_count = layout->stripe_count;
> assert(object_size >= su); <------------
>
> I'm curious where it gets layout->object_size for object that is just been
> created.
>
> As I understod striper mode was created by CERN guys. In there document
> <https://indico.cern.ch/event/542464/contributions/2202295/attachments/1289543/1919853/HEPUsageOfCeph.pdf>
> they recommend 8M stripe_unit. But it does not work in luminous.
>
> Created I/O context.
> Connected to pool backup with rados_striper_create
> Stripe unit OK 8388608
> Stripe count OK 1
> /build/ceph-12.2.0/src/osdc/Striper.cc: In function 'static void
> Striper::file_to_extents(CephContext*, const char*, const file_layout_t*,
> uint64_t, uint64_t, uint64_t, std::map<object_t, std::vector<ObjectExtent>
> >&, uint64_t)' thread 7f13bd5c1e00 time 2017-10-07 21:44:58.654778
> /build/ceph-12.2.0/src/osdc/Striper.cc: 64: FAILED assert(object_size >=
> su)
> ceph version 12.2.0 (32ce2a3ae5239ee33d6150705cdb24d43bab910c) luminous
> (rc)
> 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char
> const*)+0x102) [0x7f13b3f3b332]
> 2: (Striper::file_to_extents(CephContext*, char const*, file_layout_t
> const*, unsigned long, unsigned long, unsigned long, std::map<object_t,
> std::vector<ObjectExtent, std::allocator<ObjectExtent> >,
> std::less<object_t>, std::allocator<std::pair<object_t const,
> std::vector<ObjectExtent, std::allocator<ObjectExtent> > > > >&, unsigned
> long)+0x1e1e) [0x7f13bce235ee]
> 3: (Striper::file_to_extents(CephContext*, char const*, file_layout_t
> const*, unsigned long, unsigned long, unsigned long,
> std::vector<ObjectExtent, std::allocator<ObjectExtent> >&, unsigned
> long)+0x51) [0x7f13bce23691]
> 4:
> (libradosstriper::RadosStriperImpl::internal_aio_write(std::__cxx11::basic_string<char,
> std::char_traits<char>, std::allocator<char> > const&,
> boost::intrusive_ptr<libradosstriper::MultiAioCompletionImpl>,
> ceph::buffer::list const&, unsigned long, unsigned long, ceph_file_layout
> const&)+0x224) [0x7f13bcda4184]
> 5:
> (libradosstriper::RadosStriperImpl::write_in_open_object(std::__cxx11::basic_string<char,
> std::char_traits<char>, std::allocator<char> > const&, ceph_file_layout
> const&, std::__cxx11::basic_string<char, std::char_traits<char>,
> std::allocator<char> > const&, ceph::buffer::list const&, unsigned long,
> unsigned long)+0x13c) [0x7f13bcda476c]
> 6:
> (libradosstriper::RadosStriperImpl::write(std::__cxx11::basic_string<char,
> std::char_traits<char>, std::allocator<char> > const&, ceph::buffer::list
> const&, unsigned long, unsigned long)+0xd5) [0x7f13bcda4bd5]
> 7: (rados_striper_write()+0xdb) [0x7f13bcd9ba0b]
> 8: (()+0x10fb) [0x55dd87b410fb]
> 9: (__libc_start_main()+0xf1) [0x7f13bc9d72b1]
> 10: (()+0xbca) [0x55dd87b40bca]
>
>
> On Fri, Sep 29, 2017 at 11:46 PM, Gregory Farnum <gfar...@redhat.com>
> wrote:
>
>> I haven't used the striper, but it appears to make you specify sizes,
>> stripe units, and stripe counts. I would expect you need to make sure that
>> the size is an integer multiple of the stripe unit. And it probably
>> defaults to a 4MB object if you don't specify one?
>>
>> On Fri, Sep 29, 2017 at 2:09 AM Alexander Kushnirenko <
>> kushnire...@gmail.com> wrote:
>>
>>> Hi,
>>>
>>> I'm trying to use CEPH-12.2.0 as storage for with Bareos-16.2.4 backup
>>> with libradosstriper1 support.
>>>
>>> Libradosstriber was suggested on this list to solve the problem, that
>>> current CEPH-12 discourages users from using object with very big size
>>> (>128MB). Bareos treat Rados Object as Volume and in CEPH-10 it created
>>> objects with very big size (10G and more). CEPH-10 allowed such behaviour,
>>> put recovery indeed take very long time. So stripping objects seems to be
>>> the right thing to do.
>>>
>>> Bareos supports libradosstriper and the code seems to work. But for some
>>> reason it run only with stripe_unit=4194304, which seems to be typical
>>> value for RadosGW for example. I tried several other values for
>>> stripe_unit, but the code exit with error.
>>>
>>> Is there a particular reason why only 4M size works? Can one use some
>>> CLI to test different stripe sizes?
>>>
>>> Basic flow of creating object in Bareos is the following:
>>> rados_ioctx_create(m_cluster, m_rados_poolname, &m_ctx);
>>> rados_striper_create(m_ctx, &m_striper);
>>> rados_striper_set_object_layout_stripe_unit(m_striper, m_stripe_unit);
>>> rados_striper_set_object_layout_stripe_count(m_striper, m_stripe_count);
>>> .....
>>> status = rados_striper_write(m_striper, m_virtual_filename, buffer,
>>> count, offset);
>>>
>>> Alexander
>>>
>>> _______________________________________________
>>> ceph-users mailing list
>>> ceph-users@lists.ceph.com
>>> http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
>>>
>>
>
_______________________________________________
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
