> On May 2, 2015, at 2:48 PM, Parth Oberoi <htrap...@gmail.com> wrote: > > Hello Everyone, > > My proposal[0] for adding Riak as a destination to Syslog-ng was accepted for > GSoC2015, so before my coding period begins i.e. 25th May, I wanted to do > some ground work and take some feedback from the potential users. > The proposal[0] contains all the initial implementation details which might > require improvement . > Therefore if some of you could take a look and give me some feedback on it, > would help me greatly. > Below are a few links to important snippets of the proposal. > > > [0] - > https://github.com/balabit/syslog-ng/wiki/GSoC2015-Proposal:-Riak-destination-(htrap) > > Goals for the project :- > https://github.com/balabit/syslog-ng/wiki/GSoC2015-Proposal:-Riak-destination-(htrap)#goals-for-the-project > > Development Tools :- > https://github.com/balabit/syslog-ng/wiki/GSoC2015-Proposal:-Riak-destination-(htrap)#development-tools > > Deliverables :- > https://github.com/balabit/syslog-ng/wiki/GSoC2015-Proposal:-Riak-destination-(htrap)#deliverables > > My scheduled Plan:- > https://github.com/balabit/syslog-ng/wiki/GSoC2015-Proposal:-Riak-destination-(htrap)#scheduled-plan---
Hi Parth, I’ve only had time to briefly read the overview and the goals. That said, this sounds great! I’m happy to assist in any way possible. I’m a bit concerned with your use of the set embedded in the map. Large objects have traditionally been a big problem in Riak due to the use of distributed Erlang and head of line blocking. I’m curious if you could elaborate on what type of data you will be storing in the set: how big you expect each item to be, how big you expect the map to be, and the overall layout of data inside of the data structure. I’ve provided some references on why these things might be problematic below. Thanks, - Chris [1] http://stackoverflow.com/questions/24389183/riak-returns-an-error-reading-large-object-of-size-when-a-mapreduce [2] http://stackoverflow.com/questions/10491156/is-riak-recommended-for-storing-and-streaming-large-binary-files [3] http://basho.com/riak-development-anti-patterns/ [4] http://dl.acm.org/citation.cfm?id=2596635 Christopher Meiklejohn Senior Software Engineer Basho Technologies, Inc. cmeiklej...@basho.com _______________________________________________ riak-users mailing list riak-users@lists.basho.com http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
