Hi
I had faced same issue with exactly same case. Just before our release :)
LMDB returns misaligned values and sometime Lists do not work properly.
because I used own Value class which creates a reader I implemented
workaround for the issue.
struct Value :public lmdb::value
{
private:
mutable void* aligned;
__declspec(align(sizeof(void*))) mutable byte buffer[4096];
void* getPointer() const
{
if ((uintptr_t)mv_data % sizeof(void*))
{
if (mv_size <= sizeof(buffer))
{
memcpy(buffer, mv_data, mv_size);
return buffer;
}
aligned = _aligned_malloc(mv_size, sizeof(void*));
memcpy(aligned, mv_data, mv_size);
return aligned;
}
return mv_data;
}
public:
auto createReader() const
{
return capnp::FlatArrayMessageReader(kj::ArrayPtr<const
capnp::word>((capnp::word*)getPointer(), mv_size / sizeof(capnp::word)));
}
Value():aligned(nullptr)
{}
~Value() {
if (aligned)
_aligned_free(aligned);
}
};
среда, 10 августа 2016 г., 2:39:00 UTC+3 пользователь Kenton Varda написал:
>
> If misalignment is rare, then it seems like doing a copy in those cases
> wouldn't be that bad? Or are the messages very large?
>
> It's surprising that a database designed to be memory-mapped would store
> data misaligned; that seems to defeat the purpose. :/
>
> It's hard for me to say whether performance is the only problem that would
> ever happen. For example, x86 has been adding advanced instructions like
> SSE that do have alignment requirements. What if we eventually decide we
> want to use these to optimize Cap'n Proto in some way? Loosening the
> alignment requirement seems risky. :/
>
> -Kenton
>
> On Tue, Aug 9, 2016 at 2:38 PM, <[email protected] <javascript:>> wrote:
>
>> We are reading from Lmdb database. In some (rare) cases we get misaligned
>> buffers. I would rather take the performance hit (in these rare cases). I
>> guess an alternative for me would be to realign the data (copy), but I
>> would be happier to know that nothing bad would happen inside the capnproto
>> except for the performance hit.
>>
>> I am developing on x64 architecture and there is no requirement to be
>> compatible to any other architecture. We usually don't run with asserts
>> enabled, so no issues were observed even with misaligned buffers. However,
>> I feel nervous since the API contract is violated.
>>
>> Thanks,
>>
>> - Yevgeni
>>
>> On Tuesday, August 9, 2016 at 2:19:43 PM UTC-7, Kenton Varda wrote:
>>>
>>> Hi Yevgeni,
>>>
>>> Even on architectures that support misaligned access, such access will
>>> be slower and non-atomic.
>>>
>>> Usually it is possible to change the code which initially receives the
>>> message in such a way that it will end up always aligned. When possible,
>>> this is usually a much better solution than taking a portability and
>>> performance hit.
>>>
>>> How are you reading the messages in your case such that they don't end
>>> up aligned?
>>>
>>> -Kenton
>>>
>>> On Thu, Jul 28, 2016 at 12:52 PM, <[email protected]> wrote:
>>>
>>>> When creating a FlatArrayMessageReader from a buffer which is not
>>>> aligned, I am getting an assertion violation: "Pointer section of struct
>>>> list element not aligned."
>>>>
>>>> I was reading this discussion on the matter:
>>>> https://groups.google.com/forum/#!msg/capnproto/RPqMlhkSdeo/z2O6JgztBAAJ
>>>>
>>>> I failed to understand however, whether anything will break within
>>>> capnproto library if non-aligned buffers are used or is this just a
>>>> protection for some architectures against SIGBUS violations?
>>>> I would really like to avoid memory copies just for the purpose of
>>>> alignment especially that my CPU architecture supports misaligned access.
>>>>
>>>> Do you think the assertion
>>>> at external/capnproto/c++/src/capnp/layout.c++:2732 should be changed?
>>>>
>>>> Thanks,
>>>>
>>>> - Yevgeni
>>>>
>>>> --
>>>> You received this message because you are subscribed to the Google
>>>> Groups "Cap'n Proto" group.
>>>> To unsubscribe from this group and stop receiving emails from it, send
>>>> an email to [email protected].
>>>> Visit this group at https://groups.google.com/group/capnproto.
>>>>
>>>
>>> --
>> You received this message because you are subscribed to the Google Groups
>> "Cap'n Proto" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to [email protected] <javascript:>.
>> Visit this group at https://groups.google.com/group/capnproto.
>>
>
>
--
You received this message because you are subscribed to the Google Groups
"Cap'n Proto" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
Visit this group at https://groups.google.com/group/capnproto.
