Re: Python client performance issue
Hi Mike,
perhaps you can try to upgrade the protocol buffers library to at least
version 2.3.0. This is from the changelog for that version:
Python
* 10-25 times faster than 2.2.0, still pure-Python.
Cheers,
Nico
Am Montag, den 14.02.2011, 19:35 -0500 schrieb Mike Stoddart:
> Will do when I get time. Would the REST API be any faster?
>
> Thanks
> Mike
>
> On Mon, Feb 14, 2011 at 7:01 PM, Thomas Burdick
> wrote:
> > I would highly recommend looking in to the cProfile and pstat module and
> > profile the code that is going slow. If your using the protocol buffer
> > client it could possibly be related to the fact that python protocol buffers
> > is extraordinarily slow and is well known to be slow. Profile until proven
> > guilty though.
> > Tom Burdick
> >
> > On Mon, Feb 14, 2011 at 7:09 AM, Mike Stoddart wrote:
> >>
> >> I added some code to my system to test writing data into Riak. I'm
> >> using the Python client library with protocol buffers. I'm writing a
> >> snapshot of my current data, which is one json object containing on
> >> average 60 individual json sub-objects. Each sub object contains about
> >> 22 values.
> >>
> >># Archived entry. ts is a formatted timestamp.
> >>entry = self._bucket.new(ts, data=data)
> >>entry.store()
> >>
> >># Now write the current entry.
> >>entry = self._bucket.new("current", data=data)
> >>entry.store()
> >>
> >> I'm writing the same data twice; the archived copy and the current
> >> copy, which I can easily retrieve later. Performance is lower than
> >> expected; top is showing a constant cpu usage of 10-12%.
> >>
> >> I haven't decided to use Riak; this is to help me decide. But for now
> >> are there any optimisations I can do here? A similiar test with
> >> MongoDB shows a steady cpu usage of 1%. The cpu usages are for my
> >> client, not Riak's own processes. The only differences in my test app
> >> is the code that writes the data to the database. Otherwise all other
> >> code is 100% the same between these two test apps.
> >>
> >> Any suggestions appreciated.
> >> Thanks
> >> Mike
> >>
> >> ___
> >> riak-users mailing list
> >> riak-users@lists.basho.com
> >> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
> >
> >
>
> ___
> riak-users mailing list
> riak-users@lists.basho.com
> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
___
riak-users mailing list
riak-users@lists.basho.com
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Re: Python client performance issue
python-riak-client already uses version 2.3.0. Adventurous types might want to
check out https://github.com/Greplin/fast-python-pb, which wraps the C/C++
protocol buffers library.
--
Andy Gross
Principal Architect
Basho Technologies, Inc.
On Tuesday, February 15, 2011 at 1:46 AM, Nico Meyer wrote:
> Hi Mike,
>
> perhaps you can try to upgrade the protocol buffers library to at least
> version 2.3.0. This is from the changelog for that version:
>
> Python
> * 10-25 times faster than 2.2.0, still pure-Python.
>
>
> Cheers,
> Nico
>
> Am Montag, den 14.02.2011, 19:35 -0500 schrieb Mike Stoddart:
> > Will do when I get time. Would the REST API be any faster?
> >
> > Thanks
> > Mike
> >
> > On Mon, Feb 14, 2011 at 7:01 PM, Thomas Burdick
> > wrote:
> > > I would highly recommend looking in to the cProfile and pstat module and
> > > profile the code that is going slow. If your using the protocol buffer
> > > client it could possibly be related to the fact that python protocol
> > > buffers
> > > is extraordinarily slow and is well known to be slow. Profile until proven
> > > guilty though.
> > > Tom Burdick
> > >
> > > On Mon, Feb 14, 2011 at 7:09 AM, Mike Stoddart wrote:
> > > >
> > > > I added some code to my system to test writing data into Riak. I'm
> > > > using the Python client library with protocol buffers. I'm writing a
> > > > snapshot of my current data, which is one json object containing on
> > > > average 60 individual json sub-objects. Each sub object contains about
> > > > 22 values.
> > > >
> > > > # Archived entry. ts is a formatted timestamp.
> > > > entry = self._bucket.new(ts, data=data)
> > > > entry.store()
> > > >
> > > > # Now write the current entry.
> > > > entry = self._bucket.new("current", data=data)
> > > > entry.store()
> > > >
> > > > I'm writing the same data twice; the archived copy and the current
> > > > copy, which I can easily retrieve later. Performance is lower than
> > > > expected; top is showing a constant cpu usage of 10-12%.
> > > >
> > > > I haven't decided to use Riak; this is to help me decide. But for now
> > > > are there any optimisations I can do here? A similiar test with
> > > > MongoDB shows a steady cpu usage of 1%. The cpu usages are for my
> > > > client, not Riak's own processes. The only differences in my test app
> > > > is the code that writes the data to the database. Otherwise all other
> > > > code is 100% the same between these two test apps.
> > > >
> > > > Any suggestions appreciated.
> > > > Thanks
> > > > Mike
> > > >
> > > > ___
> > > > riak-users mailing list
> > > > riak-users@lists.basho.com
> > > > http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
> >
> > ___
> > riak-users mailing list
> > riak-users@lists.basho.com
> > http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
>
>
>
> ___
> riak-users mailing list
> riak-users@lists.basho.com
> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
>
___
riak-users mailing list
riak-users@lists.basho.com
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Re: Python client performance issue
Hi Andy.
I am not quite sure what you mean, is the protobuf library included with
riak-python-client? Or are you talking about the version of the protobuf
compiler that was used to create riakclient_pb2.py from
riakclient.proto?
Cheers,
Nico
Am Dienstag, den 15.02.2011, 02:23 -0800 schrieb Andy Gross:
> python-riak-client already uses version 2.3.0. Adventurous types
> might want to check out https://github.com/Greplin/fast-python-pb,
> which wraps the C/C++ protocol buffers library.
>
> --
> Andy Gross
> Principal Architect
> Basho Technologies, Inc.
>
>
> On Tuesday, February 15, 2011 at 1:46 AM, Nico Meyer wrote:
>
> > Hi Mike,
> >
> > perhaps you can try to upgrade the protocol buffers library to at
> > least
> > version 2.3.0. This is from the changelog for that version:
> >
> > Python
> > * 10-25 times faster than 2.2.0, still pure-Python.
> >
> >
> > Cheers,
> > Nico
> >
> > Am Montag, den 14.02.2011, 19:35 -0500 schrieb Mike Stoddart:
> > > Will do when I get time. Would the REST API be any faster?
> > >
> > > Thanks
> > > Mike
> > >
> > > On Mon, Feb 14, 2011 at 7:01 PM, Thomas Burdick
> > > wrote:
> > > > I would highly recommend looking in to the cProfile and pstat
> > > > module and
> > > > profile the code that is going slow. If your using the protocol
> > > > buffer
> > > > client it could possibly be related to the fact that python
> > > > protocol buffers
> > > > is extraordinarily slow and is well known to be slow. Profile
> > > > until proven
> > > > guilty though.
> > > > Tom Burdick
> > > >
> > > > On Mon, Feb 14, 2011 at 7:09 AM, Mike Stoddart
> > > > wrote:
> > > > >
> > > > > I added some code to my system to test writing data into Riak.
> > > > > I'm
> > > > > using the Python client library with protocol buffers. I'm
> > > > > writing a
> > > > > snapshot of my current data, which is one json object
> > > > > containing on
> > > > > average 60 individual json sub-objects. Each sub object
> > > > > contains about
> > > > > 22 values.
> > > > >
> > > > > # Archived entry. ts is a formatted timestamp.
> > > > > entry = self._bucket.new(ts, data=data)
> > > > > entry.store()
> > > > >
> > > > > # Now write the current entry.
> > > > > entry = self._bucket.new("current", data=data)
> > > > > entry.store()
> > > > >
> > > > > I'm writing the same data twice; the archived copy and the
> > > > > current
> > > > > copy, which I can easily retrieve later. Performance is lower
> > > > > than
> > > > > expected; top is showing a constant cpu usage of 10-12%.
> > > > >
> > > > > I haven't decided to use Riak; this is to help me decide. But
> > > > > for now
> > > > > are there any optimisations I can do here? A similiar test
> > > > > with
> > > > > MongoDB shows a steady cpu usage of 1%. The cpu usages are for
> > > > > my
> > > > > client, not Riak's own processes. The only differences in my
> > > > > test app
> > > > > is the code that writes the data to the database. Otherwise
> > > > > all other
> > > > > code is 100% the same between these two test apps.
> > > > >
> > > > > Any suggestions appreciated.
> > > > > Thanks
> > > > > Mike
> > > > >
> > > > > ___
> > > > > riak-users mailing list
> > > > > riak-users@lists.basho.com
> > > > > http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
> > > > >
> > >
> > > ___
> > > riak-users mailing list
> > > riak-users@lists.basho.com
> > > http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
> > >
> >
> >
> >
> > ___
> > riak-users mailing list
> > riak-users@lists.basho.com
> > http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
> >
>
>
___
riak-users mailing list
riak-users@lists.basho.com
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Re: Python client performance issue
Sorry, I should have been more clear. The Python client depends on
"protobuf>=2.3.0" in setup.py, so people are already most likely using
protobufs-2.3.0.
- Andy
On Tuesday, February 15, 2011 at 3:09 AM, Nico Meyer wrote:
> Hi Andy.
>
> I am not quite sure what you mean, is the protobuf library included with
> riak-python-client? Or are you talking about the version of the protobuf
> compiler that was used to create riakclient_pb2.py from
> riakclient.proto?
>
> Cheers,
> Nico
>
> Am Dienstag, den 15.02.2011, 02:23 -0800 schrieb Andy Gross:
> > python-riak-client already uses version 2.3.0. Adventurous types
> > might want to check out https://github.com/Greplin/fast-python-pb,
> > which wraps the C/C++ protocol buffers library.
> >
> > --
> > Andy Gross
> > Principal Architect
> > Basho Technologies, Inc.
> >
> >
> > On Tuesday, February 15, 2011 at 1:46 AM, Nico Meyer wrote:
> >
> > > Hi Mike,
> > >
> > > perhaps you can try to upgrade the protocol buffers library to at
> > > least
> > > version 2.3.0. This is from the changelog for that version:
> > >
> > > Python
> > > * 10-25 times faster than 2.2.0, still pure-Python.
> > >
> > >
> > > Cheers,
> > > Nico
> > >
> > > Am Montag, den 14.02.2011, 19:35 -0500 schrieb Mike Stoddart:
> > > > Will do when I get time. Would the REST API be any faster?
> > > >
> > > > Thanks
> > > > Mike
> > > >
> > > > On Mon, Feb 14, 2011 at 7:01 PM, Thomas Burdick
> > > > wrote:
> > > > > I would highly recommend looking in to the cProfile and pstat
> > > > > module and
> > > > > profile the code that is going slow. If your using the protocol
> > > > > buffer
> > > > > client it could possibly be related to the fact that python
> > > > > protocol buffers
> > > > > is extraordinarily slow and is well known to be slow. Profile
> > > > > until proven
> > > > > guilty though.
> > > > > Tom Burdick
> > > > >
> > > > > On Mon, Feb 14, 2011 at 7:09 AM, Mike Stoddart
> > > > > wrote:
> > > > > >
> > > > > > I added some code to my system to test writing data into Riak.
> > > > > > I'm
> > > > > > using the Python client library with protocol buffers. I'm
> > > > > > writing a
> > > > > > snapshot of my current data, which is one json object
> > > > > > containing on
> > > > > > average 60 individual json sub-objects. Each sub object
> > > > > > contains about
> > > > > > 22 values.
> > > > > >
> > > > > > # Archived entry. ts is a formatted timestamp.
> > > > > > entry = self._bucket.new(ts, data=data)
> > > > > > entry.store()
> > > > > >
> > > > > > # Now write the current entry.
> > > > > > entry = self._bucket.new("current", data=data)
> > > > > > entry.store()
> > > > > >
> > > > > > I'm writing the same data twice; the archived copy and the
> > > > > > current
> > > > > > copy, which I can easily retrieve later. Performance is lower
> > > > > > than
> > > > > > expected; top is showing a constant cpu usage of 10-12%.
> > > > > >
> > > > > > I haven't decided to use Riak; this is to help me decide. But
> > > > > > for now
> > > > > > are there any optimisations I can do here? A similiar test
> > > > > > with
> > > > > > MongoDB shows a steady cpu usage of 1%. The cpu usages are for
> > > > > > my
> > > > > > client, not Riak's own processes. The only differences in my
> > > > > > test app
> > > > > > is the code that writes the data to the database. Otherwise
> > > > > > all other
> > > > > > code is 100% the same between these two test apps.
> > > > > >
> > > > > > Any suggestions appreciated.
> > > > > > Thanks
> > > > > > Mike
> > > > > >
> > > > > > ___
> > > > > > riak-users mailing list
> > > > > > riak-users@lists.basho.com
> > > > > > http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
> > > >
> > > > ___
> > > > riak-users mailing list
> > > > riak-users@lists.basho.com
> > > > http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
> > >
> > >
> > >
> > > ___
> > > riak-users mailing list
> > > riak-users@lists.basho.com
> > > http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
>
>
>
> ___
> riak-users mailing list
> riak-users@lists.basho.com
> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
>
___
riak-users mailing list
riak-users@lists.basho.com
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
redundant writers
Is riak suitable as a very reliable store where you have multiple feeds of streaming data that are at least theoretically identical? That is, can you count on writing multiple copies with the same keys at the same time to do something reasonable regardless of cluster partitioning? And is this a common usage scenario? -- Les Mikesell lesmikes...@gmail.com ___ riak-users mailing list riak-users@lists.basho.com http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Re: redundant writers
Riak maintains a single value per key and provides mechanisms (vector clocks) to detect/resolve conflicting values. In the proposed use case the multiple copies would overwrite each other and Riak, by default, would return a single value for a requested key. Behind the scenes Riak determines the appropriate value per key using vector clocks. More information about vector clocks is available here: http://blog.basho.com/2010/01/29/why-vector-clocks-are-easy/ http://blog.basho.com/2010/04/05/why-vector-clocks-are-hard/ Thanks, Dan Daniel Reverri Developer Advocate Basho Technologies, Inc. d...@basho.com On Tue, Feb 15, 2011 at 10:11 AM, Les Mikesell wrote: > Is riak suitable as a very reliable store where you have multiple feeds of > streaming data that are at least theoretically identical? That is, can you > count on writing multiple copies with the same keys at the same time to do > something reasonable regardless of cluster partitioning? And is this a > common usage scenario? > > -- > Les Mikesell > lesmikes...@gmail.com > > ___ > riak-users mailing list > riak-users@lists.basho.com > http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com > ___ riak-users mailing list riak-users@lists.basho.com http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Re: redundant writers
What about siblings? http://wiki.basho.com/REST-API.html (seach for sibling) On Tue, Feb 15, 2011 at 14:57, Dan Reverri wrote: > Riak maintains a single value per key and provides mechanisms (vector > clocks) to detect/resolve conflicting values. In the proposed use case the > multiple copies would overwrite each other and Riak, by default, would > return a single value for a requested key. > Behind the scenes Riak determines the appropriate value per key using vector > clocks. More information about vector clocks is available here: > http://blog.basho.com/2010/01/29/why-vector-clocks-are-easy/ > http://blog.basho.com/2010/04/05/why-vector-clocks-are-hard/ > Thanks, > Dan > Daniel Reverri > Developer Advocate > Basho Technologies, Inc. > d...@basho.com > > > On Tue, Feb 15, 2011 at 10:11 AM, Les Mikesell > wrote: >> >> Is riak suitable as a very reliable store where you have multiple feeds of >> streaming data that are at least theoretically identical? That is, can you >> count on writing multiple copies with the same keys at the same time to do >> something reasonable regardless of cluster partitioning? And is this a >> common usage scenario? >> >> -- >> Les Mikesell >> lesmikes...@gmail.com >> >> ___ >> riak-users mailing list >> riak-users@lists.basho.com >> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com > > > ___ > riak-users mailing list > riak-users@lists.basho.com > http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com > > ___ riak-users mailing list riak-users@lists.basho.com http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Re: redundant writers
In this scenario, the behavior I would want would be to only see one copy for each unique key, but to always have one even if one of the feeds or writers fails or the cluster is partitioned, then re-joined. This sounds good in theory, but what about the details? Is there likely to be a big performance hit from the normally-colliding writes? Will it take twice the disk space, the eventually clean up the duplicates? Would it be reasonable to do this to riak-search with something like news stories? On 2/15/2011 2:02 PM, Alexander Sicular wrote: What about siblings? http://wiki.basho.com/REST-API.html (seach for sibling) On Tue, Feb 15, 2011 at 14:57, Dan Reverri wrote: Riak maintains a single value per key and provides mechanisms (vector clocks) to detect/resolve conflicting values. In the proposed use case the multiple copies would overwrite each other and Riak, by default, would return a single value for a requested key. Behind the scenes Riak determines the appropriate value per key using vector clocks. More information about vector clocks is available here: http://blog.basho.com/2010/01/29/why-vector-clocks-are-easy/ http://blog.basho.com/2010/04/05/why-vector-clocks-are-hard/ Thanks, Dan Daniel Reverri Developer Advocate Basho Technologies, Inc. d...@basho.com On Tue, Feb 15, 2011 at 10:11 AM, Les Mikesell wrote: Is riak suitable as a very reliable store where you have multiple feeds of streaming data that are at least theoretically identical? That is, can you count on writing multiple copies with the same keys at the same time to do something reasonable regardless of cluster partitioning? And is this a common usage scenario? -- Les Mikesell lesmikes...@gmail.com ___ riak-users mailing list riak-users@lists.basho.com http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com ___ riak-users mailing list riak-users@lists.basho.com http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com ___ riak-users mailing list riak-users@lists.basho.com http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Re: redundant writers
Les, This is pretty much Dynamo 101 territory at this point and one of the tradeoffs with a distributed model. If you aren't familiar with Dynamo, Andy Gross (from Basho) gives an AWESOME walkthrough in an episode of TheNoSQLTapes: http://nosqltapes.com/video/understanding-dynamo-with-andy-gross Concurrent writes really do behave the same way they would during a network partition event. You can just stick your riak cluster behind a load balancer and go to town. On Tue, Feb 15, 2011 at 3:38 PM, Les Mikesell wrote: > In this scenario, the behavior I would want would be to only see one copy > for each unique key, but to always have one even if one of the feeds or > writers fails or the cluster is partitioned, then re-joined. This sounds > good in theory, but what about the details? Is there likely to be a big > performance hit from the normally-colliding writes? Will it take twice the > disk space, the eventually clean up the duplicates? Would it be reasonable > to do this to riak-search with something like news stories? > > > On 2/15/2011 2:02 PM, Alexander Sicular wrote: >> >> What about siblings? http://wiki.basho.com/REST-API.html (seach for >> sibling) >> >> On Tue, Feb 15, 2011 at 14:57, Dan Reverri wrote: >>> >>> Riak maintains a single value per key and provides mechanisms (vector >>> clocks) to detect/resolve conflicting values. In the proposed use case >>> the >>> multiple copies would overwrite each other and Riak, by default, would >>> return a single value for a requested key. >>> Behind the scenes Riak determines the appropriate value per key using >>> vector >>> clocks. More information about vector clocks is available here: >>> http://blog.basho.com/2010/01/29/why-vector-clocks-are-easy/ >>> http://blog.basho.com/2010/04/05/why-vector-clocks-are-hard/ >>> Thanks, >>> Dan >>> Daniel Reverri >>> Developer Advocate >>> Basho Technologies, Inc. >>> d...@basho.com >>> >>> >>> On Tue, Feb 15, 2011 at 10:11 AM, Les Mikesell >>> wrote: Is riak suitable as a very reliable store where you have multiple feeds of streaming data that are at least theoretically identical? That is, can you count on writing multiple copies with the same keys at the same time to do something reasonable regardless of cluster partitioning? And is this a common usage scenario? -- Les Mikesell lesmikes...@gmail.com ___ riak-users mailing list riak-users@lists.basho.com http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com >>> >>> >>> ___ >>> riak-users mailing list >>> riak-users@lists.basho.com >>> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com >>> >>> > > > ___ > riak-users mailing list > riak-users@lists.basho.com > http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com > ___ riak-users mailing list riak-users@lists.basho.com http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Memory usage
Hi, does somebody have some figures on Riak memory usage? I consider using it in a highly memory constrained environment (< 32 MiB), but I'm not sure whether Riak can handle low workloads under such tight constrains (it consumed > 20 MiB idle). Performance is not relevant to me, I just need partition tolerance, availability and persistence. Regards, Matthias-Christian ___ riak-users mailing list riak-users@lists.basho.com http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Riak Java Client
Gday Been messing around on some personal projects which use riak with Java and was interested to know if their was a roadmap or task list of the java client. Just wanted to know if there was any plans to build a new version in the future as I would be interested in contributing. I am currently working on a Log4J appender which writes directly to riak (see http://github.com/wolfeidau/riak-log4j-appender ). Currently using Netty network library to communicate with riak over HTTP as I don't need a complete riak client, and wanted to keep my dependencies as lean as possible. -- Regards, Mark Wolfe -- I am not young enough to know everything. -- ___ riak-users mailing list riak-users@lists.basho.com http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Re: Memory usage
You'll be constrained in the number of keys that you can store (using the bitcask backend). There's a good spreadsheet for capacity planning: https://spreadsheets.google.com/ccc?key=0Ak4OBkABJPsxdEowYXc2akxnYU9xNkJmbmZscnhaTFE&hl=en&authkey=CMHw8tYO I don't know how well Riak would behave otherwise in an environment that tight. I'd think you'd be limited to single key lookups, but I have been wrong before. -- Jeremiah Peschka Microsoft SQL Server MVP MCITP: Database Developer, DBA On Tuesday, February 15, 2011 at 4:34 PM, Matthias-Christian Ott wrote: > Hi, > > does somebody have some figures on Riak memory usage? I consider using > it in a highly memory constrained environment (< 32 MiB), but I'm not > sure whether Riak can handle low workloads under such tight constrains > (it consumed > 20 MiB idle). Performance is not relevant to me, I just > need partition tolerance, availability and persistence. > > Regards, > Matthias-Christian > > ___ > riak-users mailing list > riak-users@lists.basho.com > http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com > ___ riak-users mailing list riak-users@lists.basho.com http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Re: Memory usage
On Tue, Feb 15, 2011 at 04:44:58PM -0800, Jeremiah Peschka wrote: > You'll be constrained in the number of keys that you can store (using the > bitcask backend). There's a good spreadsheet for capacity planning: > https://spreadsheets.google.com/ccc?key=0Ak4OBkABJPsxdEowYXc2akxnYU9xNkJmbmZscnhaTFE&hl=en&authkey=CMHw8tYO The storage seems sufficient for me. > I don't know how well Riak would behave otherwise in an environment that > tight. I'd think you'd be limited to single key lookups, but I have been > wrong before. So you think it might not be possible to run map-reduce functions? Regards, Matthias-Christian ___ riak-users mailing list riak-users@lists.basho.com http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Re: Memory usage
I'm not sure. I think it would depend on the number of JavaScript MR engines you have running, the operations you're performing, and the amount of data that is being read into memory. -- Jeremiah Peschka Microsoft SQL Server MVP MCITP: Database Developer, DBA On Tuesday, February 15, 2011 at 5:10 PM, Matthias-Christian Ott wrote: > On Tue, Feb 15, 2011 at 04:44:58PM -0800, Jeremiah Peschka wrote: > > You'll be constrained in the number of keys that you can store (using the > > bitcask backend). There's a good spreadsheet for capacity planning: > > https://spreadsheets.google.com/ccc?key=0Ak4OBkABJPsxdEowYXc2akxnYU9xNkJmbmZscnhaTFE&hl=en&authkey=CMHw8tYO > > The storage seems sufficient for me. > > > I don't know how well Riak would behave otherwise in an environment that > > tight. I'd think you'd be limited to single key lookups, but I have been > > wrong before. > > So you think it might not be possible to run map-reduce functions? > > Regards, > Matthias-Christian > ___ riak-users mailing list riak-users@lists.basho.com http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Enhanced Querying for Ripple
Lately I've noticed the increased demand for a higher-level, abstract querying API in the Ripple document-mapper library -- something analogous to ActiveRecord + arel. At the current time, I'm preparing a strawman proposal for the feature that we can debate. I'd like to discuss this feature in-depth on Friday with interested members of the community and then set about implementing it. If you would like to participate in the conversation, please enter your email in the form I've linked to below. HOWEVER, I only ask that if you want to be involved in the conversation, **you should be willing to contribute some time to making this API happen** preferably by implementing and testing needed features. As much as I like being a code-hero, this is too big to do by myself. I will email the strawman proposal to participants by Thursday so you have time to review it. (We will likely meet on GoToMeeting, which requires Windows or Mac, sorry Linux users.) https://spreadsheets.google.com/a/basho.com/viewform?formkey=dDFKU0lNMkY0aVZyMFZ1Y244UXA1RHc6MQ Cheers! Sean Cribbs Developer Advocate Basho Technologies, Inc. http://basho.com/ ___ riak-users mailing list riak-users@lists.basho.com http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
