Querying Multiple Indices

Are there any plans for allowing Riak to query multiple indices in one operation? If there are, how will these queries work? Kind regards Runar ___ riak-users mailing list riak-users@lists.basho.com http://lists.basho.com/mailman/listinfo/riak-users_lis

Should Riak have used dedicated nodes for secondary indices?

Siddharth Anand, says that secondary indices (for a key-value store) best is placed on a separate node, avoiding the need to look up 1 / N nodes during a query: "Systems that shard data based on a primary key will do well when routed by that key. When routed by a secondary key, the system will nee

Re: Should Riak have used dedicated nodes for secondary indices?

Good news! Riak doesn't use sharding. Data locality is critical in a distributed system. When you create an index, your structure looks something like: indexed_value:record_id Reading from an index requires locating indexed_value, finding all matching values, and then retrieving all matching rec

Re: Querying Multiple Indices

I have no idea if there are plans for this. Writing these queries yourself should be pretty trivial. Just spin up n threads, query n indexes, and then use some join algorithm [1] to merge the result sets together to produce the key list that you need to perform your lookups. [1]: https://en.wikip

Re: Cant load data after enabling indexing on bucket with precommit hook

I figured this out. So I'm using the https://github.com/basho/riak-php-client PHP Riak Client . I was using the setProperties method to set the commit hooks for the bucket to enable search. However, I was defining the bucket properties like so: '$prop_arr = array('precommit' => array('mod' => '

unsubscribe

From: jeremiah.pesc...@gmail.com Date: Wed, 25 Jan 2012 06:48:45 -0800 Subject: Re: Should Riak have used dedicated nodes for secondary indices? To: runar.jord...@gmail.com CC: riak-users@lists.basho.com Good news! Riak doesn't use sharding. Data locality is critical in a distributed system. W

Re: Querying Multiple Indices

We have always intended to have the ability to query multiple indexes at once, but the feature is not pinned on the roadmap yet. I'm sure there will be an announcement preceding its release. On Wed, Jan 25, 2012 at 9:50 AM, Jeremiah Peschka < jeremiah.pesc...@gmail.com> wrote: > I have no idea if

Re: Cant load data after enabling indexing on bucket with precommit hook

Also note that on 1.0 and later you can set the "search" bucket property to true and that will install the hook for you. On Wed, Jan 25, 2012 at 9:51 AM, charlesugwuh wrote: > I figured this out. > > So I'm using the https://github.com/basho/riak-php-client PHP Riak > Client . > I was using the

Re: unsubscribe

I think this is a reasonable question to ask, re the 1/N question during query time. This is currently an implementation detail of secondary indices in Riak. The document vs. term-based index partitioning seems to be a common debate. It almost feels like the emacs/vim flamewar of the distributed

secondary index read with limit?

I'm wondering if there's any way to achieve a limited secondary index read in Riak, i.e., up to N records in some key range. With that primitive, it seems like it would be possible for the user to implement things like pagination, incremental joins over multiple indices, etc. (For efficiency I gu

Re: secondary index read with limit?

Afaik, range queries the likes of which you suggest are not possible in riak because riak is an inherently unordered system. That being said, it is a major cause of concern for those looking at riak as a potential solution. One ultimately needs to implement a secondary ordered index elsewhere to

Re: secondary index read with limit?

On Jan 25, 2012, at 1:20 PM, Alexander Sicular wrote: > Afaik, range queries the likes of which you suggest are not possible in riak > because riak is an inherently unordered system. That being said, it is a > major cause of concern for those looking at riak as a potential solution. One > ulti

v1.0.3 search merge index preventing partition handoff?

Hello all, We are hitting an issue with a riak 1.0.3 cluster when adding new nodes to the ring. Specifically the handoff appears stuck and isn't making any progress. I have read a number of the threads on here and realize handoff will take a while, and have also tried attaching to the console an

Re: v1.0.3 search merge index preventing partition handoff?

Hi Ryan, >From the looks of the crash log, it seems that one of your merge index files >may be corrupt (did you run out of disk space, or crash a node?) At any rate, what seems to be happening is that the search vnode is in the middle of a handoff (presumably to the new machine), and while it i

Re: Querying Multiple Indices

It would be more efficient in disk reads if the keys output by one index search could somehow constrain the second one, and so on. On Wed, Jan 25, 2012 at 9:50 AM, Jeremiah Peschka wrote: I have no idea if there

Re: Querying Multiple Indices

See also: http://features.basho.com/entries/20516853-bitmap-index-queries On Wed, Jan 25, 2012 at 5:05 PM, Carl wrote: > It would be more efficient in disk reads if the keys output by one index > search could somehow constrain the second one, and so on. > > > On Wed, Jan 25, 2012 at 9:50 AM, Je

Re: riak sizing considerations

Sebastian, With all requests, Riak will attempt to read or write to all replicas regardless of the specified r or w value. The r and w values affect how many reads from or writes to partitions must be completed before the operation is considered successful. As a result, the get (read) and put

Re: riak sizing considerations

On Tuesday, January 24, 2012 at 5:21 AM, Sebastian Gerlach wrote: > Some other points in my considerations are the time required for a > reorganization after a new node are added to the cluster or a node has > been replaced. > > Sebastian, Sorry I missed this part of your question. The time

Re: how does vnode_vclock work?

Adam, There's a basic explanation in our wiki: http://wiki.basho.com/Vector-Clocks.html#VNode-Vector-Clocks. Requests to a vnode are processed in serial, but for a given bucket-key pair there's no guarantee that the same vnode will coordinate separate requests. Thus, two writes to the same b

Re: Fastest way to count number of items in Secondary Index query

Jeremy, Requests to 2i's HTTP interface will return keys rather the objects themselves. You may find better performance counting the returned keys rather than employing a M/R job. -- Ian Plosker Developer Advocate Basho Technologies On Monday, January 23, 2012 at 2:27 PM, Jeremy Raymond w

RE: Fastest way to count number of items in Secondary Index query

I’m using the Erlang PB interface in my application. The Erlang client gets the keys via this Input/Query in riakc_pb_socket:get_index/4 at https://github.com/basho/riak-erlang-client/blob/master/src/riakc_pb_socket.erl#L677 Input = {index, Bucket, Index, Key}, IdentityQuery = [{redu

Riak Recap for January 20 - 25

Morning, Afternoon, Evening To All - Coming to you live from ScaleConf in the stunningly-beautiful Cape Town, here's a short Recap from the last few days. Enjoy. Mark Community Manager Basho Technologies wiki.basho.com twitter.com/pharkmillups --- Riak Recap for