Re: Riak instead of memcached?

2011-02-16 Thread Jason J. W. Williams
My benchmarks with Riak against Redis showed Riak is fast, but it's never going to be as fast for caching as an in-memory only store that doesn't have to do coordination. That said if your data set is larger than RAM, Riak will likely be faster as it will scale to handle that better. -J On Wed,

Re: Key Expiry in Riak

2011-03-02 Thread Jason J. W. Williams
Are there any plans to make expiry setable via an HTTP header? -J On Wed, Mar 2, 2011 at 9:57 AM, Sean Cribbs wrote: > Innostore does not have an expiration option.  In general, you'll want to > use bitcask unless your operational requirements specifically call for > innostore. > Sean Cribbs >

Streaming map/reduce results

2011-03-02 Thread Jason J. W. Williams
Hello, When running map/reduce in streaming mode, since the results are from the last phase (reduce) what is the significance of the phase key? (e.g. {"phase":1,"data":[3]} ) -J ___ riak-users mailing list riak-users@lists.basho.com http://lists.basho.

Streaming map/reduce error doesn't use MIME type boundary

2011-03-02 Thread Jason J. W. Williams
Hello, I've been playing with implementing streaming support in the txRiak library. However, I've noticed that when a map reduce error occurs during a streaming map/reduce, Riak doesn't wrap the error in the boundary specified in the Content-Type header: https://gist.github.com/852270 That makes

Re: Streaming map/reduce results

2011-03-02 Thread Jason J. W. Williams
11 at 8:25 PM, Sean Cribbs wrote: > If your query has "keep":true on multiple phases, that phase key lets you > disambiguate results. > > Sean Cribbs > > On Mar 2, 2011, at 9:28 PM, "Jason J. W. Williams" > wrote: > >> Hello, >> >> Wh

Deciphering Key Filter Errors

2011-03-02 Thread Jason J. W. Williams
Hi, I'm experimenting using key filters to implement indexes. My approach is for each data key in bucket A, to create a new empty key in a dedicated index bucket where the original key name and value of the indexed field is encoded in the key name for a new index key. Data key looks like this: B

Re: Streaming map/reduce results

2011-03-03 Thread Jason J. W. Williams
passed in? Thought playing with streaming would >> help that, but looks like not. Thank you in advance. >> >> -J >> >> On Wed, Mar 2, 2011 at 8:25 PM, Sean Cribbs wrote: >>> If your query has "keep":true on multiple phases, that phase key lets yo

Re: Streaming map/reduce results

2011-03-03 Thread Jason J. W. Williams
't intuitive and are looking into > solutions. > > Sean Cribbs > Developer Advocate > Basho Technologies, Inc. > http://basho.com/ > > On Mar 3, 2011, at 9:44 AM, Jason J. W. Williams wrote: > >> Hi Sean, >> >> That's weird, because I'm se

Re: Deciphering Key Filter Errors

2011-03-03 Thread Jason J. W. Williams
If someone could help me understand just this error, that would help a lot: https://gist.github.com/852450 Thank you in advance. -J On Wed, Mar 2, 2011 at 11:55 PM, Jason J. W. Williams wrote: > Hi, > > I'm experimenting using key filters to implement indexes. My approach >

Log Tokenize Output from Key Filters

2011-03-03 Thread Jason J. W. Williams
Through trial and error I think I've found the cause of my error...the tokenize function. Is it possible to log the output of tokenize in a job to compare what I think it should be outputting to what it actually is? -J ___ riak-users mailing list riak-u

Re: Deciphering Key Filter Errors

2011-03-03 Thread Jason J. W. Williams
nth(2, ["helloworld"]). > ** exception error: no function clause matching lists:nth(1,[]) > It seems Riak should catch errors during the filtering process and discard > keys that cause errors. I will file a bug. > Thanks, > Dan > Daniel Reverri > Developer Advo

Re: Deciphering Key Filter Errors

2011-03-03 Thread Jason J. W. Williams
g the filtering process and discard >> keys that cause errors. I will file a bug. >> Thanks, >> Dan >> Daniel Reverri >> Developer Advocate >> Basho Technologies, Inc. >> d...@basho.com >> >> >> On Thu, Mar 3, 2011 at 10:28 AM, Jason J. W. W

Re: Deciphering Key Filter Errors

2011-03-03 Thread Jason J. W. Williams
o Technologies, Inc. > d...@basho.com > > > On Thu, Mar 3, 2011 at 12:32 PM, Jason J. W. Williams > wrote: >> >> ...btw I've found with the key filter, I have to supply a JavaScript >> map function if I want to use a JS reduce function. If I use just a JS >>

txRiakIdx: Twisted Riak library implementing transparent indexing

2011-03-08 Thread Jason J. W. Williams
Hi Guys, I've put together a library for my company's needs to implement secondary indexes in Riak transparently. It's built on top of Ray Cote's great txRIak library, and should drop in transparently anywhere txRiak is used. We've open sourced it in case it's useful for anyone else: https://gith

Key filters + link walking

2011-03-09 Thread Jason J. W. Williams
Is it possible to add a link phase when using key filters? Currently, just using key filters combined with a reduce phase (built-in reduce identity). Would like to add a link phase to pull all the linked keys and pass them to the identity reduce as well. Thank you in advance. -J _

Re: Key filters + link walking

2011-03-14 Thread Jason J. W. Williams
is greatly appreciated. -J On Mon, Mar 14, 2011 at 12:34 PM, Dan Reverri wrote: > Hi Jason, > > Yes, a link phase can be used after a key filter. > > Thanks, > Dan > > Daniel Reverri > Developer Advocate > Basho Technologies, Inc. > d...@basho.com > &g

Re: Key filters + link walking

2011-03-14 Thread Jason J. W. Williams
; Developer Advocate > Basho Technologies, Inc. > d...@basho.com > > > On Mon, Mar 14, 2011 at 11:54 AM, Jason J. W. Williams < > jasonjwwilli...@gmail.com> wrote: > >> The reason I ask, is when I add a link walk phase (with bucket set to the >> data bucket), my

Re: Key filters + link walking

2011-03-14 Thread Jason J. W. Williams
On Mon, Mar 14, 2011 at 2:01 PM, Dan Reverri wrote: > Hi Jason, > > The reduce_identity function expects a two element tuple or a two element > list; you can see this by looking at the pattern matching in the function: > https://github.com/basho/riak_kv/blob/master/src/riak_kv_mapreduce.erl#L141

Re: Riak Development Environments

2011-03-29 Thread Jason J. W. Williams
> > Anyone else care to share their setup? Or have any questions about > what your dev setup should look like? > We use 4GB RAM, 1x 2.0GHz core Cloudlayer instances at Softlayer for testing. Usually spin up 4-5 nodes. -J ___ riak-users mailing list riak

Re: EC2 and RIAK

2011-04-01 Thread Jason J. W. Williams
Hi David, If you want both good performance and reliability you might look at somebody like SoftLayer who will let you mix and match cloud and dedicated servers to meet your needs. Both they and Voxel have cloud servers that beat the pants off EC2. -J Sent via iPhone Is your e-mail Premiere

Re: EC2 and RIAK

2011-04-01 Thread Jason J. W. Williams
On Fri, Apr 1, 2011 at 11:00 AM, Mark Steele wrote: > I've done some rather disappointing tests with Riak using Rackspace cloud > servers. Much better off on dedicated hardware if you can find it. I haven't found anything except small web sites that RS Cloud has enough performance for. -J _

Erlang R14B02

2011-04-04 Thread Jason J. W. Williams
Any word on R14B02 support in Riak? Anxious to get it running again in Homebrew. -J ___ riak-users mailing list riak-users@lists.basho.com http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com

Re: Erlang R14B02

2011-04-04 Thread Jason J. W. Williams
Looks like it hasn't made it into Homebrew yet then: https://gist.github.com/902267 -J On Mon, Apr 4, 2011 at 1:36 PM, Andrew Thompson wrote: > On Mon, Apr 04, 2011 at 12:54:05PM -0600, Jason J. W. Williams wrote: >> Any word on R14B02 support in Riak? Anxious to get it ru

Re: Erlang R14B02

2011-04-04 Thread Jason J. W. Williams
ine. > > Sean Cribbs > Developer Advocate > Basho Technologies, Inc. > http://basho.com/ > > On Apr 4, 2011, at 3:36 PM, Andrew Thompson wrote: > >> On Mon, Apr 04, 2011 at 12:54:05PM -0600, Jason J. W. Williams wrote: >>> Any word on R14B02 support in Riak? A

Re: Secondary indexing

2011-04-22 Thread Jason J. W. Williams
We implemented a scheme for maintaining secondary indexes using MapReduce and KeyFilters. It's written in Python but there's a description of how the design works and a test script for validating other implementations: https://github.com/williamsjj/txriakidx Nor sure if that will help...it's reas

Re: Links vs Key Filters for Performance

2011-05-05 Thread Jason J. W. Williams
On Thu, May 5, 2011 at 4:49 PM, Andrew Berman wrote: > Ah, that makes sense.  So is it the case that using the link implementation > will always be faster?  Or are there cases where it makes more sense to use > a key filter? There's a practical limit to how many links you can walk before performa

Re: Problems starting riak

2011-06-04 Thread Jason J. W. Williams
Have you tried with Erlang R14B02? Sent via iPhone Is your email Premiere? On Jun 4, 2011, at 5:25, Alvaro Videla wrote: > Hi, > > I'm trying to build riak using the latest Erlang release built with these > options: > > ./configure --enable-smp-support --enable-darwin-64bit --enable-kernel-

Re: Problems starting riak

2011-06-05 Thread Jason J. W. Williams
e only compiles with latest Erlang. > > On Jun 4, 2011, at 10:48 PM, Jason J. W. Williams wrote: > >> Have you tried with Erlang R14B02? >> >> Sent via iPhone >> >> Is your email Premiere? >> >> On Jun 4, 2011, at 5:25, Alvaro Videla wrote: >

Re: SSDs instead of SAS.

2012-01-09 Thread Jason J. W. Williams
> > However, make sure you do the reading on the SSDs you're going to purchase > because not all SSDs are created equal. I had a client buy some smaller > OCZ-Vertex 3s recently which have a wear leveling issue (the 120GB drives > have fewer chips than the 240GB drives) that causes performance to f

Re: Riak Control installed when upgrading to 1.0.2?

2012-01-10 Thread Jason J. W. Williams
How does Riak Control differ from the management tools included in the enterprise version? -J On Tue, Jan 10, 2012 at 12:56 PM, Andrew Thompson wrote: > Riak control will be part of the 1.1 series, it is not present in 1.0.2 > or any of the 1.0 releases. > > Andrew > > __

Re: Riak Control installed when upgrading to 1.0.2?

2012-01-10 Thread Jason J. W. Williams
That's terrific. Thank you. -J On Tue, Jan 10, 2012 at 1:08 PM, David Smith wrote: > On Tue, Jan 10, 2012 at 12:58 PM, Jason J. W. Williams > wrote: > > How does Riak Control differ from the management tools included in the > > enterprise version? > > Riak C

Re: Riak Adoption - What can we do better?

2012-04-19 Thread Jason J. W. Williams
On Thu, Apr 19, 2012 at 11:37 AM, Zabrane Mickael wrote: > +1 for a solid book on advanced Riak usage. Better marketing of Mathias' excellent Riak Handbook would definitely help...would be great if it showed up on Amazon at least if not bn.com. Someday replication making it into the community ve

Replication in Riak open source

2010-05-26 Thread Jason J. W. Williams
Hello, Given replication is a built-in feature only in the EnterpriseDS product, how are folks using only the open source version of Riak handling replication? Writing to two clusters simultaneously or something else perhaps? Thank you in advance. -J

Memory Utilization

2010-05-26 Thread Jason J. W. Williams
Hi, We have a couple of projects we want to start small and to this point we've been considering MongoDB or Cassandra. Mongo's main drawback for us is it's extensive use of mmap, which can make it a bad neighbor vis-a-vis RAM usage if it has to co-exist with other parts of our stack. Cassandra's h

Re: Memory Utilization

2010-05-26 Thread Jason J. W. Williams
are some flags to the Erlang VM that can be tweaked. > > Sean Cribbs > Developer Advocate > Basho Technologies, Inc. > http://basho.com/ > > On May 26, 2010, at 6:23 PM, Jason J. W. Williams wrote: > >> Hi, >> >> We have a couple of projects we want

Re: Replication in Riak open source

2010-05-26 Thread Jason J. W. Williams
t; http://basho.com/ > > On May 26, 2010, at 6:17 PM, Jason J. W. Williams wrote: > >> Hello, >> >> Given replication is a built-in feature only in the EnterpriseDS >> product, how are folks using only the open source version of Riak >> handling replication?

Re: Memory Utilization

2010-05-27 Thread Jason J. W. Williams
s wrote: >> >> Yes, you can use low-level commands to recover lost data from the binlogs. >>  It's embedded InnoDB, not MySQL's driver, so not everything is exactly the >> same, but the technique is similar. >> >> Sean Cribbs >> Developer Advocate

Re: Replication in Riak open source

2010-05-27 Thread Jason J. W. Williams
Interesting. Makes sense. -J On Thu, May 27, 2010 at 5:43 AM, Sean Cribbs wrote: > Correct, the vector clocks would not necessarily be the same. > > Sean Cribbs > Developer Advocate > Basho Technologies, Inc. > http://basho.com/ > > On May 26, 2010, at 11:47 PM, Ja

Replica Distribution

2010-06-03 Thread Jason J. W. Williams
Hello, I was reading the "So what does N=3 really mean?" section of https://wiki.basho.com/display/RIAK/Replication#Replication-ReadRepair ... The part where it states that 3 replicas will not necessarily go to 3 physically separate nodes very much surprised me. Is there any guarantee at least tw

Re: [ANN] Riak Release 0.11.0

2010-06-09 Thread Jason J. W. Williams
Does the bitcask back end handle crash recovery as well as the InnoStore backend? -J On Wed, Jun 9, 2010 at 1:55 PM, Jon Meredith wrote: > Hello, Riak users.  We are excited to announce the release of Riak version > 0.11. > > Pre-built installations and source tarballs are available at: > http:/

Re: [ANN] Riak Release 0.11.0

2010-06-09 Thread Jason J. W. Williams
r than innostore, simply because we don't have enough data to > support it...yet. :) > I will say, however, that the append-only nature of bitcask minimizes the > opportunity to lose and/or corrupt data, not to mention obviating the need > for log files ala InnoDB. > D. > >

Re: [ANN] Riak Release 0.11.0

2010-06-11 Thread Jason J. W. Williams
Is it smart enough to coordinate with the other partitions to ensure not more than 25% (just a plug number) of the partitions are compacting at the same time? It would seem to me there's the possibility for a performance drop if you had the perfect storm of too many shards compacting at the same ti

Re: [ANN] Riak Release 0.11.0

2010-06-11 Thread Jason J. W. Williams
d. Also, a single node only compacts a single > partition at a time. > FWIW, in my own testing, with a 50/50 read/write mix, compaction (based on > fragmentation %) typically doesn't happen that often, particularly when you > have a cluster of machines. > Hope that helps. > D

Re: [ANN] Riak Release 0.11.0

2010-06-14 Thread Jason J. W. Williams
cally possible refers to having at least N physical nodes > available. Meaning you can't place 3 replicas on distinct nodes if you only > have 2 nodes. > Thanks, > Dan > Daniel Reverri > Developer Advocate > Basho Technologies, Inc. > d...@basho.com > > > On Wed, J

Re: Best way to back-up riak

2010-07-12 Thread Jason J. W. Williams
Hey Justin, Since Riak is lockless, what is the best approach to pulling a distributed FS snapshot of the bitcask files across nodes? I assume if they're not close to each other, you'll have an issue if you have to restore a cluster. -J On Sun, Jul 11, 2010 at 8:14 PM, Justin Sheehy wrote: > H

Re: Expected vs Actual Bucket Behavior

2010-07-21 Thread Jason J. W. Williams
Hmm...just created a new account to track this in the Basho bugzilla...seems not to recognize new accounts... -J On Wed, Jul 21, 2010 at 7:31 AM, Justin Sheehy wrote: > I think that we are all (myself included) getting two different issues > a bit mixed up in this discussion: > > 1: storing an i

Re: Best way to back-up riak

2010-07-27 Thread Jason J. W. Williams
Honestly the holding back of replication from Riak OS is the only real downside to the product. Reminds me of the old adage there's those that will never pay for OS software...and those that can't afford not to (because of things like Sox). -J Sent via iPhone Is your e-mail Premiere? On Jul

Re: riak deployment

2010-08-13 Thread Jason J. W. Williams
Joyent makes a great service, but they are WAY overly priced. I'd recommend a 512 SliceHost/Linode. Or a 1GB @ Voxel/SoftLayer. -J On Thu, Aug 12, 2010 at 10:40 AM, Alexander Sicular wrote: > - You can not tell riak where to place buckets. > - You could set the N val on a bucket to one, and you

Max Latency Advisable Between Nodes

2010-08-31 Thread Jason J. W. Williams
Hello, Has anyone done any testing on the maximum latency that can exist between Riak nodes before you start seeing issues? In the same vein how are folks using the open source product approaching replication between DCs? -J ___ riak-users mailing lis

Re: Pervasive replication

2010-10-05 Thread Jason J. W. Williams
Maybe the community can come up with a solution for those of us that can't afford or yet justify EnterpriseDS. -J Sent via iPhone Is your e-mail Premiere? On Oct 5, 2010, at 9:40, Dean Cookson wrote: > Andrew, > The best way to do what you describe is to have a cluster in each datacenter >

Re: Pervasive replication

2010-10-05 Thread Jason J. W. Williams
e to afford EnterpriseDS. Or is it prohibitively > expensive? > > I have no idea. But afaics that's Basho's revenue model. > > Francisco > > > 2010/10/5 Jason J. W. Williams : >> Maybe the community can come up with a solution for those of us that can&