Re: mapreduce timeout error
thanks Russell .
On Wed, Sep 21, 2011 at 12:22 PM, Russell Brown wrote:
> Hi Sohil,
>
> On 21 Sep 2011, at 07:07, raghwani sohil wrote:
>
> Hi ,
>
> I am using *mapreduce *technique* *to access all keys between perticular
> period .
>
> This is my *keys.json* file .
>
> {"inputs":{"bucket":"test",
>"key_filters":[["matches" ,"2011_09_11"]]},
> "query":[{"map":{"language":"javascript",
> "source":"function(v) {
>return [v.key] ;
> }",
> "keep":true} } ]
> }
>
> I am invoking mapreduce call using curl
>
> curl -X POST -H "content-type: application/json"
> http://127.0.0.1:8098/mapred --data @keys.json
>
> but it gives me timeout error :- *{"error":"timeout"}*
>
> so Is there any way to resolve this problem ??
>
>
> Map reduce over an entire bucket is not recommended and can be slow. To
> lessen the likelihood of a timeout error you can add a 'timeout' parameter
> to the map reduce JSON.
>
> {"inputs":{"bucket":"test", "key_filters":[["matches" ,"2011_09_11"]]},
> "query":[{"map":{"language":"javascript", "source":"function(v) { return
> [v.key] ;}", "keep":true} } ], "timeout" : 6}
>
> You can read more about it on the wiki
> http://wiki.basho.com/MapReduce.html#MapReduce-via-the-HTTP-API
>
> However, it might be a better strategy to write to a new bucket per time
> period, or take advantage of riak search, or secondary indexes and use a
> range query.
>
> Cheers
>
> Russell
>
>
> thanks ,
> Sohil Raghwani .
>
> ___
> riak-users mailing list
> riak-users@lists.basho.com
> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
>
>
>
___
riak-users mailing list
riak-users@lists.basho.com
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Re: Search precommit hook bug?
On Tue, Sep 20, 2011 at 12:41 PM, Elias Levy
wrote:
> Now from what I've been able to find,{error,emfile} usually means you are
> out of file descriptors. Yes?
Yes. If you are running a default configuration, however, you will
need considerably more than 4096 file handles. You can tighten this
down by setting:
{max_open_files, 10}
in the eleveldb section of app.config. That will yield 10 open file
handles _per_ vnode -- on a out of the box setup that's 64 * 10 file
handles necessary.
D.
--
Dave Smith
Director, Engineering
Basho Technologies, Inc.
diz...@basho.com
___
riak-users mailing list
riak-users@lists.basho.com
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Riak 1.0.0 RC1 now available
All, We are pleased to announce the first release candidate for Riak 1.0.0 is now available. The packages are available on our downloads page: http://downloads.basho.com/riak/riak-1.0.0rc1/ As a release candidate, we consider this to be a functionally complete representation of Riak 1.0.0. From now until the 1.0.0 release, only critical bug fixes will be merged into the repository. We would like to thank everyone who took the time to download, install, and run the pre-releases. The Riak community has always been one of the great strengths of Riak, and this release period has been no different with feedback and bug reports we've been given. Thanks, The Basho team ___ riak-users mailing list riak-users@lists.basho.com http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Re: Search precommit hook bug?
On Wed, Sep 21, 2011 at 6:32 AM, David Smith wrote:
> On Tue, Sep 20, 2011 at 12:41 PM, Elias Levy
> wrote:
>
> > Now from what I've been able to find,{error,emfile} usually means you are
> > out of file descriptors. Yes?
>
> Yes. If you are running a default configuration, however, you will
> need considerably more than 4096 file handles.
So it seems. Tried again yesterday setting the limit to 10K. It almost
finish loading my 6 million objects. So even 10K was not sufficient. At
the moment the riak process in one of the nodes shows 8,814 open fds while
idle.
You can tighten this
> down by setting:
>
> {max_open_files, 10}
>
> in the eleveldb section of app.config. That will yield 10 open file
> handles _per_ vnode -- on a out of the box setup that's 64 * 10 file
> handles necessary.
>
Thanks. It makes sense now. I suppose all this will be documented once 1.0
is released. You may want to change the default value in the eleveldb
backend. Probably not a good idea to have a default that can result on the
system starving for file handles.
Any idea what kind of performance hit one takes by lowering the number of
open file handles?
We are using a ring size of 128. In a worse case scenario all those 128
vnodes would exist on a single node, so we need 128*max_open_files fd limit
per node.
What about replicas? Do they have their own vnode? Or does a ring with a
size of 128 can only have at most 128 vnodes regardless of replicas or
fallbacks?
Elias Levy
___
riak-users mailing list
riak-users@lists.basho.com
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Re: Riak Search Precommit Hook in 1.0
A few questions 1) Is this fix in the RC1 build that just came out today? 2) How exactly do search-cmd install, the "search" bucket property and the precommit hook fit together now? I'm a bit unclear on what I actually need to do to enable search on a bucket. Thanks. -- Greg Clipboard On Tuesday, September 20, 2011 at 3:42 PM, Greg Pascale wrote: > Man, it took a whole day after I complained about this to get it fixed? You > guys are such slackers! :) > > Thanks a bunch! > > -- > Greg > Clipboard > > On Tuesday, September 20, 2011 at 1:47 PM, Andrew Thompson wrote: > > > On Tue, Sep 20, 2011 at 10:53:28AM -0700, Greg Pascale wrote: > > > Hi Jon, > > > > > > Thanks for getting back to me. > > > > > > I'm doing full-text indexing, but I don't need to keep the text around > > > once my object has been indexed, so I wrote a second precommit hook to > > > remove the text from the object. This reduces the size of my objects > > > drastically (since text tends to be the largest part) and has given us > > > substantial perf gains in our tests. > > > > I just pushed a fix for this. It should be present in the next 1.0 > > prerelease. > > > > https://github.com/basho/riak_search/pull/87 > > > > Andrew > > > > ___ > > riak-users mailing list > > riak-users@lists.basho.com (mailto:riak-users@lists.basho.com) > > http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com > ___ riak-users mailing list riak-users@lists.basho.com http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Re: Riak Search Precommit Hook in 1.0
Hi Greg, The "install" command has been removed from "search-cmd". The new strategy is as follows: * The "search" bucket property must be "true" in order for the search precommit to be executed * If the "precommit" property does NOT include the search precommit hook it will be inserted at the beginning of the precommit list * If the "precommit" property does include the search precommit hook it will be left in it's defined location Thanks, Dan Daniel Reverri Developer Advocate Basho Technologies, Inc. d...@basho.com On Wed, Sep 21, 2011 at 11:09 AM, Greg Pascale wrote: > A few questions > > 1) Is this fix in the RC1 build that just came out today? > 2) How exactly do search-cmd install, the "search" bucket property and the > precommit hook fit together now? I'm a bit unclear on what I actually need > to do to enable search on a bucket. > > Thanks. > > -- > Greg > Clipboard > > On Tuesday, September 20, 2011 at 3:42 PM, Greg Pascale wrote: > > Man, it took a whole day after I complained about this to get it fixed? > You guys are such slackers! :) > > Thanks a bunch! > > -- > Greg > Clipboard > > On Tuesday, September 20, 2011 at 1:47 PM, Andrew Thompson wrote: > > On Tue, Sep 20, 2011 at 10:53:28AM -0700, Greg Pascale wrote: > > Hi Jon, > > Thanks for getting back to me. > > I'm doing full-text indexing, but I don't need to keep the text around once > my object has been indexed, so I wrote a second precommit hook to remove the > text from the object. This reduces the size of my objects drastically (since > text tends to be the largest part) and has given us substantial perf gains > in our tests. > > > I just pushed a fix for this. It should be present in the next 1.0 > prerelease. > > https://github.com/basho/riak_search/pull/87 > > Andrew > > ___ > riak-users mailing list > riak-users@lists.basho.com > http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com > > > > > ___ > riak-users mailing list > riak-users@lists.basho.com > http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com > > ___ riak-users mailing list riak-users@lists.basho.com http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Re: Riak Search Precommit Hook in 1.0
On 21 Sep 2011, at 19:09, Greg Pascale wrote:
> A few questions
>
> 1) Is this fix in the RC1 build that just came out today?
Yes it is.
> 2) How exactly do search-cmd install, the "search" bucket property and the
> precommit hook fit together now? I'm a bit unclear on what I actually need to
> do to enable search on a bucket.
To enable search on a bucket do any of the things you mentioned (add the hook,
set {'search': true}, run search-cmd install my_bucket.)
search-cmd install sets the 'search' bucket property to true. The presence of a
true search bucket property cause the search pre-commit hook to be added to the
bucket's properties. Likewise, if the pre-commit hook is present but the search
property is not, the search property is set to true (for example after an
upgrade.)
Have a look at the code in riak_search_kv_hook.erl[1] specifically the fixup/2
fun[2].
Cheers
Russell
[1]
https://github.com/basho/riak_search/blob/1.0.0rc1/src/riak_search_kv_hook.erl
[2]
https://github.com/basho/riak_search/blob/1.0.0rc1/src/riak_search_kv_hook.erl
>
> Thanks.
>
> --
> Greg
> Clipboard
>
> On Tuesday, September 20, 2011 at 3:42 PM, Greg Pascale wrote:
>
>> Man, it took a whole day after I complained about this to get it fixed? You
>> guys are such slackers! :)
>>
>> Thanks a bunch!
>>
>> --
>> Greg
>> Clipboard
>>
>> On Tuesday, September 20, 2011 at 1:47 PM, Andrew Thompson wrote:
>>
>>> On Tue, Sep 20, 2011 at 10:53:28AM -0700, Greg Pascale wrote:
Hi Jon,
Thanks for getting back to me.
I'm doing full-text indexing, but I don't need to keep the text around
once my object has been indexed, so I wrote a second precommit hook to
remove the text from the object. This reduces the size of my objects
drastically (since text tends to be the largest part) and has given us
substantial perf gains in our tests.
>>>
>>> I just pushed a fix for this. It should be present in the next 1.0
>>> prerelease.
>>>
>>> https://github.com/basho/riak_search/pull/87
>>>
>>> Andrew
>>>
>>> ___
>>> riak-users mailing list
>>> riak-users@lists.basho.com
>>> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
>>
>
> ___
> riak-users mailing list
> riak-users@lists.basho.com
> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
___
riak-users mailing list
riak-users@lists.basho.com
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Re: Riak Search Precommit Hook in 1.0
What Dan said.
He was quicker and more succinct.
On 21 Sep 2011, at 19:53, Russell Brown wrote:
>
> On 21 Sep 2011, at 19:09, Greg Pascale wrote:
>
>> A few questions
>>
>> 1) Is this fix in the RC1 build that just came out today?
>
> Yes it is.
>
>> 2) How exactly do search-cmd install, the "search" bucket property and the
>> precommit hook fit together now? I'm a bit unclear on what I actually need
>> to do to enable search on a bucket.
>
> To enable search on a bucket do any of the things you mentioned (add the
> hook, set {'search': true}, run search-cmd install my_bucket.)
>
> search-cmd install sets the 'search' bucket property to true. The presence of
> a true search bucket property cause the search pre-commit hook to be added to
> the bucket's properties. Likewise, if the pre-commit hook is present but the
> search property is not, the search property is set to true (for example after
> an upgrade.)
>
> Have a look at the code in riak_search_kv_hook.erl[1] specifically the
> fixup/2 fun[2].
>
> Cheers
>
> Russell
>
> [1]
> https://github.com/basho/riak_search/blob/1.0.0rc1/src/riak_search_kv_hook.erl
> [2]
> https://github.com/basho/riak_search/blob/1.0.0rc1/src/riak_search_kv_hook.erl
>
>>
>> Thanks.
>>
>> --
>> Greg
>> Clipboard
>>
>> On Tuesday, September 20, 2011 at 3:42 PM, Greg Pascale wrote:
>>
>>> Man, it took a whole day after I complained about this to get it fixed? You
>>> guys are such slackers! :)
>>>
>>> Thanks a bunch!
>>>
>>> --
>>> Greg
>>> Clipboard
>>>
>>> On Tuesday, September 20, 2011 at 1:47 PM, Andrew Thompson wrote:
>>>
On Tue, Sep 20, 2011 at 10:53:28AM -0700, Greg Pascale wrote:
> Hi Jon,
>
> Thanks for getting back to me.
>
> I'm doing full-text indexing, but I don't need to keep the text around
> once my object has been indexed, so I wrote a second precommit hook to
> remove the text from the object. This reduces the size of my objects
> drastically (since text tends to be the largest part) and has given us
> substantial perf gains in our tests.
I just pushed a fix for this. It should be present in the next 1.0
prerelease.
https://github.com/basho/riak_search/pull/87
Andrew
___
riak-users mailing list
riak-users@lists.basho.com
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
>>>
>>
>> ___
>> riak-users mailing list
>> riak-users@lists.basho.com
>> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
>
> ___
> riak-users mailing list
> riak-users@lists.basho.com
> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
___
riak-users mailing list
riak-users@lists.basho.com
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Querying Secondary Indexes
So far, the documentation on querying secondary indexes is using the HTTP APIs (see http://blog.basho.com/2011/09/14/Secondary-Indexes-in-Riak/). The riakclient.proto doesn't show anything that's overwhelmingly obvious. Do secondary indexes currently require either using the HTTP API or writing an MR job? --- Jeremiah Peschka - Founder, Brent Ozar PLF, LLC Microsoft SQL Server MVP ___ riak-users mailing list riak-users@lists.basho.com http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Re: Riak Search Precommit Hook in 1.0
On Wed, Sep 21, 2011 at 11:47 AM, Dan Reverri wrote: > Hi Greg, > > The "install" command has been removed from "search-cmd". The new strategy > is as follows: > * The "search" bucket property must be "true" in order for the search > precommit to be executed > * If the "precommit" property does NOT include the search precommit hook it > will be inserted at the beginning of the precommit list > Correction: the search precommit is added to the end by default. > * If the "precommit" property does include the search precommit hook it > will be left in it's defined location > > Thanks, > Dan > > Daniel Reverri > Developer Advocate > Basho Technologies, Inc. > d...@basho.com > > > > On Wed, Sep 21, 2011 at 11:09 AM, Greg Pascale wrote: > >> A few questions >> >> 1) Is this fix in the RC1 build that just came out today? >> 2) How exactly do search-cmd install, the "search" bucket property and the >> precommit hook fit together now? I'm a bit unclear on what I actually need >> to do to enable search on a bucket. >> >> Thanks. >> >> -- >> Greg >> Clipboard >> >> On Tuesday, September 20, 2011 at 3:42 PM, Greg Pascale wrote: >> >> Man, it took a whole day after I complained about this to get it fixed? >> You guys are such slackers! :) >> >> Thanks a bunch! >> >> -- >> Greg >> Clipboard >> >> On Tuesday, September 20, 2011 at 1:47 PM, Andrew Thompson wrote: >> >> On Tue, Sep 20, 2011 at 10:53:28AM -0700, Greg Pascale wrote: >> >> Hi Jon, >> >> Thanks for getting back to me. >> >> I'm doing full-text indexing, but I don't need to keep the text around >> once my object has been indexed, so I wrote a second precommit hook to >> remove the text from the object. This reduces the size of my objects >> drastically (since text tends to be the largest part) and has given us >> substantial perf gains in our tests. >> >> >> I just pushed a fix for this. It should be present in the next 1.0 >> prerelease. >> >> https://github.com/basho/riak_search/pull/87 >> >> Andrew >> >> ___ >> riak-users mailing list >> riak-users@lists.basho.com >> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com >> >> >> >> >> ___ >> riak-users mailing list >> riak-users@lists.basho.com >> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com >> >> > ___ riak-users mailing list riak-users@lists.basho.com http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Re: Querying Secondary Indexes
On 21 Sep 2011, at 19:57, Jeremiah Peschka wrote: > So far, the documentation on querying secondary indexes is using the HTTP > APIs (see http://blog.basho.com/2011/09/14/Secondary-Indexes-in-Riak/). The > riakclient.proto doesn't show anything that's overwhelmingly obvious. > > Do secondary indexes currently require either using the HTTP API or writing > an MR job? Yes. > --- > Jeremiah Peschka - Founder, Brent Ozar PLF, LLC > Microsoft SQL Server MVP > > > ___ > riak-users mailing list > riak-users@lists.basho.com > http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com ___ riak-users mailing list riak-users@lists.basho.com http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Re: Querying Secondary Indexes
Have you implemented any wrappers to make querying easier in the Java client or are you leaving that up to the users to find? --- Jeremiah Peschka - Founder, Brent Ozar PLF, LLC Microsoft SQL Server MVP On Sep 21, 2011, at 12:03 PM, Russell Brown wrote: > > On 21 Sep 2011, at 19:57, Jeremiah Peschka wrote: > >> So far, the documentation on querying secondary indexes is using the HTTP >> APIs (see http://blog.basho.com/2011/09/14/Secondary-Indexes-in-Riak/). The >> riakclient.proto doesn't show anything that's overwhelmingly obvious. >> >> Do secondary indexes currently require either using the HTTP API or writing >> an MR job? > > Yes. > >> --- >> Jeremiah Peschka - Founder, Brent Ozar PLF, LLC >> Microsoft SQL Server MVP >> >> >> ___ >> riak-users mailing list >> riak-users@lists.basho.com >> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com > ___ riak-users mailing list riak-users@lists.basho.com http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Re: Querying Secondary Indexes
On 21 Sep 2011, at 20:13, Jeremiah Peschka wrote: > Have you implemented any wrappers to make querying easier in the Java client > or are you leaving that up to the users to find? Yes, wrappers to make it easy. So in the java client I have added a fetchIndex method to the Bucket class. You specify the index name, the value (or start, end of a range) and then (behind the curtain) I either call the HTTP API for the HTTP transport, or generate a M/R query for the PB transport. The M/R query has a reduce phase that makes sure only the keys are returned to bring it inline with the HTTP API's return value. I also expose a simple builder for creating M/R jobs that have an Index Query as an input. So, as much as possible, I have made it so the user just calls fetchIndex and the client takes care of the 'how'. This test[1] illustrates it. Cheers Russell [1] https://github.com/basho/riak-java-client/blob/cd67a3ffc1e2cede0a8957a015a697eac2c2b421/src/test/java/com/basho/riak/client/itest/ITestBucket.java#L298 > --- > Jeremiah Peschka - Founder, Brent Ozar PLF, LLC > Microsoft SQL Server MVP > > On Sep 21, 2011, at 12:03 PM, Russell Brown wrote: > >> >> On 21 Sep 2011, at 19:57, Jeremiah Peschka wrote: >> >>> So far, the documentation on querying secondary indexes is using the HTTP >>> APIs (see http://blog.basho.com/2011/09/14/Secondary-Indexes-in-Riak/). The >>> riakclient.proto doesn't show anything that's overwhelmingly obvious. >>> >>> Do secondary indexes currently require either using the HTTP API or writing >>> an MR job? >> >> Yes. >> >>> --- >>> Jeremiah Peschka - Founder, Brent Ozar PLF, LLC >>> Microsoft SQL Server MVP >>> >>> >>> ___ >>> riak-users mailing list >>> riak-users@lists.basho.com >>> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com >> > > > ___ > riak-users mailing list > riak-users@lists.basho.com > http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com ___ riak-users mailing list riak-users@lists.basho.com http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Re: Querying Secondary Indexes
Fantastic. Good to know that my local spike makes sense to someone other than me. --- Jeremiah Peschka - Founder, Brent Ozar PLF, LLC Microsoft SQL Server MVP On Sep 21, 2011, at 12:21 PM, Russell Brown wrote: > > On 21 Sep 2011, at 20:13, Jeremiah Peschka wrote: > >> Have you implemented any wrappers to make querying easier in the Java client >> or are you leaving that up to the users to find? > > Yes, wrappers to make it easy. > > So in the java client I have added a fetchIndex method to the Bucket class. > You specify the index name, the value (or start, end of a range) and then > (behind the curtain) I either call the HTTP API for the HTTP transport, or > generate a M/R query for the PB transport. The M/R query has a reduce phase > that makes sure only the keys are returned to bring it inline with the HTTP > API's return value. > > I also expose a simple builder for creating M/R jobs that have an Index Query > as an input. > > So, as much as possible, I have made it so the user just calls fetchIndex and > the client takes care of the 'how'. > > This test[1] illustrates it. > > Cheers > > Russell > > [1] > https://github.com/basho/riak-java-client/blob/cd67a3ffc1e2cede0a8957a015a697eac2c2b421/src/test/java/com/basho/riak/client/itest/ITestBucket.java#L298 > >> --- >> Jeremiah Peschka - Founder, Brent Ozar PLF, LLC >> Microsoft SQL Server MVP >> >> On Sep 21, 2011, at 12:03 PM, Russell Brown wrote: >> >>> >>> On 21 Sep 2011, at 19:57, Jeremiah Peschka wrote: >>> So far, the documentation on querying secondary indexes is using the HTTP APIs (see http://blog.basho.com/2011/09/14/Secondary-Indexes-in-Riak/). The riakclient.proto doesn't show anything that's overwhelmingly obvious. Do secondary indexes currently require either using the HTTP API or writing an MR job? >>> >>> Yes. >>> --- Jeremiah Peschka - Founder, Brent Ozar PLF, LLC Microsoft SQL Server MVP ___ riak-users mailing list riak-users@lists.basho.com http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com >>> >> >> >> ___ >> riak-users mailing list >> riak-users@lists.basho.com >> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com > ___ riak-users mailing list riak-users@lists.basho.com http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Innostore 1.0.3 release to support Riak 1.0.0
To coincide with the Riak 1.0.0 release candidate, we have also released Innostore 1.0.3 binaries that work with the backend changes made in Riak 1.0.0. For this release, there are tar files containing the binaries for each of Riak's supported platforms.The tar files can be unpacked into the /usr/lib/riak directory as they were in previous releases. http://downloads.basho.com/innostore/innostore-1.0.3/ Thanks, Jared ___ riak-users mailing list riak-users@lists.basho.com http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Re: Riak Search Precommit Hook in 1.0
One more correction, the "install" command is not being removed. The "install" command simply sets the "search" property to true. Thanks, Dan Daniel Reverri Developer Advocate Basho Technologies, Inc. d...@basho.com On Wed, Sep 21, 2011 at 12:01 PM, Dan Reverri wrote: > On Wed, Sep 21, 2011 at 11:47 AM, Dan Reverri wrote: > >> Hi Greg, >> >> The "install" command has been removed from "search-cmd". The new strategy >> is as follows: >> * The "search" bucket property must be "true" in order for the search >> precommit to be executed >> * If the "precommit" property does NOT include the search precommit hook >> it will be inserted at the beginning of the precommit list >> > > Correction: the search precommit is added to the end by default. > > >> * If the "precommit" property does include the search precommit hook it >> will be left in it's defined location >> >> Thanks, >> Dan >> >> Daniel Reverri >> Developer Advocate >> Basho Technologies, Inc. >> d...@basho.com >> >> >> >> On Wed, Sep 21, 2011 at 11:09 AM, Greg Pascale wrote: >> >>> A few questions >>> >>> 1) Is this fix in the RC1 build that just came out today? >>> 2) How exactly do search-cmd install, the "search" bucket property and >>> the precommit hook fit together now? I'm a bit unclear on what I actually >>> need to do to enable search on a bucket. >>> >>> Thanks. >>> >>> -- >>> Greg >>> Clipboard >>> >>> On Tuesday, September 20, 2011 at 3:42 PM, Greg Pascale wrote: >>> >>> Man, it took a whole day after I complained about this to get it fixed? >>> You guys are such slackers! :) >>> >>> Thanks a bunch! >>> >>> -- >>> Greg >>> Clipboard >>> >>> On Tuesday, September 20, 2011 at 1:47 PM, Andrew Thompson wrote: >>> >>> On Tue, Sep 20, 2011 at 10:53:28AM -0700, Greg Pascale wrote: >>> >>> Hi Jon, >>> >>> Thanks for getting back to me. >>> >>> I'm doing full-text indexing, but I don't need to keep the text around >>> once my object has been indexed, so I wrote a second precommit hook to >>> remove the text from the object. This reduces the size of my objects >>> drastically (since text tends to be the largest part) and has given us >>> substantial perf gains in our tests. >>> >>> >>> I just pushed a fix for this. It should be present in the next 1.0 >>> prerelease. >>> >>> https://github.com/basho/riak_search/pull/87 >>> >>> Andrew >>> >>> ___ >>> riak-users mailing list >>> riak-users@lists.basho.com >>> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com >>> >>> >>> >>> >>> ___ >>> riak-users mailing list >>> riak-users@lists.basho.com >>> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com >>> >>> >> > ___ riak-users mailing list riak-users@lists.basho.com http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Re: Riak Search Precommit Hook in 1.0
Got it. Sounds like what I want to do is set search to true and also explicitly add the riak_search_kv_hook at the end of the precommit list. Thanks for all the help. -- Greg Clipboard On Wednesday, September 21, 2011 at 2:30 PM, Dan Reverri wrote: > One more correction, the "install" command is not being removed. The > "install" command simply sets the "search" property to true. > > Thanks, > Dan > > Daniel Reverri > Developer Advocate > Basho Technologies, Inc. > d...@basho.com (mailto:d...@basho.com) > > > On Wed, Sep 21, 2011 at 12:01 PM, Dan Reverri (mailto:d...@basho.com)> wrote: > > On Wed, Sep 21, 2011 at 11:47 AM, Dan Reverri > (mailto:d...@basho.com)> wrote: > > > Hi Greg, > > > > > > The "install" command has been removed from "search-cmd". The new > > > strategy is as follows: > > > * The "search" bucket property must be "true" in order for the search > > > precommit to be executed > > > * If the "precommit" property does NOT include the search precommit hook > > > it will be inserted at the beginning of the precommit list > > > > > > > > > Correction: the search precommit is added to the end by default. > > > > > * If the "precommit" property does include the search precommit hook it > > > will be left in it's defined location > > > > > > Thanks, > > > Dan > > > > > > Daniel Reverri > > > Developer Advocate > > > Basho Technologies, Inc. > > > d...@basho.com (mailto:d...@basho.com) > > > > > > > > > > > > On Wed, Sep 21, 2011 at 11:09 AM, Greg Pascale > > (mailto:g...@clipboard.com)> wrote: > > > > A few questions > > > > > > > > 1) Is this fix in the RC1 build that just came out today? > > > > 2) How exactly do search-cmd install, the "search" bucket property and > > > > the precommit hook fit together now? I'm a bit unclear on what I > > > > actually need to do to enable search on a bucket. > > > > > > > > Thanks. > > > > > > > > -- > > > > Greg > > > > Clipboard > > > > > > > > On Tuesday, September 20, 2011 at 3:42 PM, Greg Pascale wrote: > > > > > > > > > Man, it took a whole day after I complained about this to get it > > > > > fixed? You guys are such slackers! :) > > > > > > > > > > Thanks a bunch! > > > > > > > > > > -- > > > > > Greg > > > > > Clipboard > > > > > > > > > > On Tuesday, September 20, 2011 at 1:47 PM, Andrew Thompson wrote: > > > > > > > > > > > On Tue, Sep 20, 2011 at 10:53:28AM -0700, Greg Pascale wrote: > > > > > > > Hi Jon, > > > > > > > > > > > > > > Thanks for getting back to me. > > > > > > > > > > > > > > I'm doing full-text indexing, but I don't need to keep the text > > > > > > > around once my object has been indexed, so I wrote a second > > > > > > > precommit hook to remove the text from the object. This reduces > > > > > > > the size of my objects drastically (since text tends to be the > > > > > > > largest part) and has given us substantial perf gains in our > > > > > > > tests. > > > > > > > > > > > > I just pushed a fix for this. It should be present in the next 1.0 > > > > > > prerelease. > > > > > > > > > > > > https://github.com/basho/riak_search/pull/87 > > > > > > > > > > > > Andrew > > > > > > > > > > > > ___ > > > > > > riak-users mailing list > > > > > > riak-users@lists.basho.com (mailto:riak-users@lists.basho.com) > > > > > > http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com > > > > > > > > > > > > > > > > > ___ > > > > riak-users mailing list > > > > riak-users@lists.basho.com (mailto:riak-users@lists.basho.com) > > > > http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com > > > > > > > > > > ___ riak-users mailing list riak-users@lists.basho.com http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
