Re: REST query returns JSON properties as strings?

2012-07-14 Thread Sean Cribbs 
Hi (what's your name?),

What a search query returns to you is the "index document", or the result
of transforming the input before adding it to the search indexes; this also
means that deeply nested fields will be flattened. If you want the
original, you can use MapReduce to retrieve the documents resulting from a
search query.

On Fri, Jul 13, 2012 at 8:54 PM, 7zark7 <7za...@gmail.com> wrote:

> I would like to store JSON documents in a Riak 1.1.x bucket (using
> bitcask), and would like to search for them via REST using JSON property
> values.
>
> E.g. JSON docs with typed properties like this:
>
> {
>"type": "whatever",
>"owner": 123456,
>"text": "Blah blah blah",
>"enabled": true
> }
>
> However, the query response (/solr/mybucket/select?q=enabled:true&wt=json)
> returns each doc's properties as JSON strings, and not the original JSON
> type:
>
> {"responseHeader":{
> "status":0,
> "QTime":1,
> "params":{
> "q":"enabled:true",
> "q.op":"or",
> "filter":"",
> "wt":"json"
> }
> },"response":{
> "numFound":1,
> "start":0,
> "maxScore":"0.353553",
> "docs":[
> {
> "id":"Obdoy3XbJe3kdh1T2bfxJgISCCL",
> "index":"rest",
> *"fields":{*
> *"enabled": "true",*
> *"owner": "123456",*
> *"text": "Blah blah blah",*
> *"type": "whatever"*
> *},*
> "props":{}
> }
> ]
> }}
>
> This makes the results unusable for me, and I'm concerned I'd have to fall
> back to N calls back to retrieve each original doc, which is not feasible.
>
>
> I'm used to CouchDB+Lucene which behaves "correctly" to me.
>
> Is there some way to do what I'd like?  E.g. another query method or
> option?
>
> Thanks
>
>
>
> ___
> riak-users mailing list
> riak-users@lists.basho.com
> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
>
>


-- 
Sean Cribbs 
Software Engineer
Basho Technologies, Inc.
http://basho.com/
___
riak-users mailing list
riak-users@lists.basho.com
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com

Re: REST query returns JSON properties as strings?

2012-07-14 Thread Andrew Zeneski 
Sean,

One thing I have been meaning to ask is if we can change the character used 
when flattening the fields. I'd like to use a character besides _ so that I can 
identify the parent from a field that has _ in its name. I would probably 
change it to '.' personally.

Is this configurable?

Andrew

On Jul 14, 2012, at 11:31 AM, Sean Cribbs wrote:

> Hi (what's your name?),
> 
> What a search query returns to you is the "index document", or the result of 
> transforming the input before adding it to the search indexes; this also 
> means that deeply nested fields will be flattened. If you want the original, 
> you can use MapReduce to retrieve the documents resulting from a search query.
> 
> On Fri, Jul 13, 2012 at 8:54 PM, 7zark7 <7za...@gmail.com> wrote:
> I would like to store JSON documents in a Riak 1.1.x bucket (using bitcask), 
> and would like to search for them via REST using JSON property values.
> 
> E.g. JSON docs with typed properties like this:
> 
> {
>"type": "whatever",
>"owner": 123456,
>"text": "Blah blah blah",
>"enabled": true
> }
> 
> However, the query response (/solr/mybucket/select?q=enabled:true&wt=json) 
> returns each doc's properties as JSON strings, and not the original JSON type:
> 
> {"responseHeader":{
> "status":0,
> "QTime":1,
> "params":{
> "q":"enabled:true",
> "q.op":"or",
> "filter":"",
> "wt":"json"
> }
> },"response":{
> "numFound":1,
> "start":0,
> "maxScore":"0.353553",
> "docs":[
> {
> "id":"Obdoy3XbJe3kdh1T2bfxJgISCCL",
> "index":"rest",
> "fields":{
>   "enabled": "true",
>   "owner": "123456",
>   "text": "Blah blah blah",
> "type": "whatever"
> },
> "props":{}
> }
> ]
> }}
> 
> This makes the results unusable for me, and I'm concerned I'd have to fall 
> back to N calls back to retrieve each original doc, which is not feasible.
> 
> 
> I'm used to CouchDB+Lucene which behaves "correctly" to me.
> 
> Is there some way to do what I'd like?  E.g. another query method or option?
> 
> Thanks
> 
> 
> 
> ___
> riak-users mailing list
> riak-users@lists.basho.com
> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
> 
> 
> 
> 
> -- 
> Sean Cribbs 
> Software Engineer
> Basho Technologies, Inc.
> http://basho.com/
> 
> ___
> riak-users mailing list
> riak-users@lists.basho.com
> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com

___
riak-users mailing list
riak-users@lists.basho.com
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com

Re: REST query returns JSON properties as strings?

2012-07-14 Thread Sean Cribbs 
That is not currently configurable, no. However, you can implement your own
extractor to do so. I think the original reasoning for '_' over '.' was
that Lucene query syntax allows '.' to specify the index name. Having
multiple '.' in a field name might break the query parser.

On Sat, Jul 14, 2012 at 11:35 AM, Andrew Zeneski
wrote:

> Sean,
>
> One thing I have been meaning to ask is if we can change the character
> used when flattening the fields. I'd like to use a character besides _ so
> that I can identify the parent from a field that has _ in its name. I would
> probably change it to '.' personally.
>
> Is this configurable?
>
> Andrew
>
> On Jul 14, 2012, at 11:31 AM, Sean Cribbs wrote:
>
> Hi (what's your name?),
>
> What a search query returns to you is the "index document", or the result
> of transforming the input before adding it to the search indexes; this also
> means that deeply nested fields will be flattened. If you want the
> original, you can use MapReduce to retrieve the documents resulting from a
> search query.
>
> On Fri, Jul 13, 2012 at 8:54 PM, 7zark7 <7za...@gmail.com> wrote:
>
>> I would like to store JSON documents in a Riak 1.1.x bucket (using
>> bitcask), and would like to search for them via REST using JSON property
>> values.
>>
>> E.g. JSON docs with typed properties like this:
>>
>> {
>>"type": "whatever",
>>"owner": 123456,
>>"text": "Blah blah blah",
>>"enabled": true
>> }
>>
>> However, the query response
>> (/solr/mybucket/select?q=enabled:true&wt=json) returns each doc's
>> properties as JSON strings, and not the original JSON type:
>>
>> {"responseHeader":{
>> "status":0,
>> "QTime":1,
>> "params":{
>> "q":"enabled:true",
>> "q.op":"or",
>> "filter":"",
>> "wt":"json"
>> }
>> },"response":{
>> "numFound":1,
>> "start":0,
>> "maxScore":"0.353553",
>> "docs":[
>> {
>> "id":"Obdoy3XbJe3kdh1T2bfxJgISCCL",
>> "index":"rest",
>> *"fields":{*
>> *"enabled": "true",*
>> *"owner": "123456",*
>> *"text": "Blah blah blah",*
>> *"type": "whatever"*
>> *},*
>> "props":{}
>> }
>> ]
>> }}
>>
>> This makes the results unusable for me, and I'm concerned I'd have to
>> fall back to N calls back to retrieve each original doc, which is not
>> feasible.
>>
>>
>> I'm used to CouchDB+Lucene which behaves "correctly" to me.
>>
>> Is there some way to do what I'd like?  E.g. another query method or
>> option?
>>
>> Thanks
>>
>>
>>
>> ___
>> riak-users mailing list
>> riak-users@lists.basho.com
>> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
>>
>>
>
>
> --
> Sean Cribbs 
> Software Engineer
> Basho Technologies, Inc.
> http://basho.com/
>
> ___
> riak-users mailing list
> riak-users@lists.basho.com
> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
>
>
>


-- 
Sean Cribbs 
Software Engineer
Basho Technologies, Inc.
http://basho.com/
___
riak-users mailing list
riak-users@lists.basho.com
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com

SV: riak doesn't start on new Macbook Pro Retina

2012-07-14 Thread Erik Søe Sørensen 
If I remember correctly, this may be a symptom that the pipe file on /tmp which 
Riak wants to use does already exists but is owned by another user. This 
happens if Riak has previously been run by a different user.
If it's not that, I suggest you provide the logs from the failed "riak start" 
attempt for further diagnosis...

- Reply message -
Fra: "Steve Hull" 
Dato: lør., jul. 14, 2012 03:10
Emne: riak doesn't start on new Macbook Pro Retina
Til: "Alvaro Videla" 
Cc: "riak-users@lists.basho.com" 

Hi Alvaro!

Thanks for your quick response.

Well after trying `riak start`, I do notice some processes hanging around:


 0 [07-13 18:08] (master)~/Dev/prodar
! ps aux | grep riak
shull  19538   0.0  0.0  2434892512 s006  R+6:08PM   0:00.00 
grep --color riak

 0 [07-13 18:08] (master)~/Dev/prodar
! riak-1.1.4/bin/riak start
Riak failed to start within 15 seconds,
see the output of 'riak console' for more information.
If you want to wait longer, set the environment variable
WAIT_FOR_ERLANG to the number of seconds to wait.

 0 [07-13 18:08] (master)~/Dev/prodar
! ps aux | grep riak
shull  19585   0.5  0.2  2564380  36508 s008  Ss+   6:08PM   0:02.45 
/Users/shull/Dev/prodar/riak-1.1.4/erts-5.8.5/bin/beam.smp -K true -A 64 -W w 
-- -root /Users/shull/Dev/prodar/riak-1.1.4 -progname riak -- -home 
/Users/shull -- -boot /Users/shull/Dev/prodar/riak-1.1.4/releases/1.1.4/riak 
-embedded -config /Users/shull/Dev/prodar/riak-1.1.4/etc/app.config -pa 
./lib/basho-patches -name riak@127.0.0.1 -setcookie riak 
-- console
shull  19829   0.0  0.0  2434892572 s006  R+6:09PM   0:00.00 
grep --color riak
shull  19584   0.0  0.0  2434816248   ??  S 6:08PM   0:00.00 
/Users/shull/Dev/prodar/riak-1.1.4/erts-5.8.5/bin/run_erl -daemon 
/tmp//Users/shull/Dev/prodar/riak-1.1.4/ /Users/shull/Dev/prodar/riak-1.1.4/log 
exec /Users/shull/Dev/prodar/riak-1.1.4/bin/riak console
shull  19569   0.0  0.0  2434840244   ??  S 6:08PM   0:00.00 
/Users/shull/Dev/prodar/riak-1.1.4/erts-5.8.5/bin/epmd -daemon

If I kill these processes, `riak console` will start ok, but riak start never 
does.  :(

-S

On Fri, Jul 13, 2012 at 6:04 PM, Alvaro Videla 
mailto:videlalv...@gmail.com>> wrote:
You seem to be running Riak already or something in the same port.

See the "duplicate name" part of the error.

Regards,

Alvaro

Sent from my iPhone

On Jul 14, 2012, at 2:53 AM, Steve Hull 
mailto:st...@taskrabbit.com>> wrote:

Hi everyone!

I recently attended a users' group / conference at the Basho offices in SF, led 
by Casey.

Everything was working fine on my 2011 MBP, but when I got this new Retina MBP, 
I found that riak would not start properly.

! riak-1.1.4/bin/riak start
Riak failed to start within 15 seconds,
see the output of 'riak console' for more information.
If you want to wait longer, set the environment variable
WAIT_FOR_ERLANG to the number of seconds to wait.

! riak-1.1.4/bin/riak console
Exec: /Users/shull/Dev/prodar/riak-1.1.4/erts-5.8.5/bin/erlexec -boot 
/Users/shull/Dev/prodar/riak-1.1.4/releases/1.1.4/riak -embedded 
-config /Users/shull/Dev/prodar/riak-1.1.4/etc/app.config -pa 
./lib/basho-patches -args_file 
/Users/shull/Dev/prodar/riak-1.1.4/etc/vm.args -- console
Root: /Users/shull/Dev/prodar/riak-1.1.4
{error_logger,{{2012,7,13},{14,8,56}},"Protocol: ~p: register error: 
~p~n",["inet_tcp",{{badmatch,{error,duplicate_name}},[{inet_tcp_dist,listen,1},{net_kernel,start_protos,4},{net_kernel,start_protos,3},{net_kernel,init_node,2},{net_kernel,init,1},{gen_server,init_it,6},{proc_lib,init_p_do_apply,3}]}]}
{error_logger,{{2012,7,13},{14,8,56}},crash_report,[[{initial_call,{net_kernel,init,['Argument__1']}},{pid,<0.20.0>},{registered_name,[]},{error_info,{exit,{error,badarg},[{gen_server,init_it,6},{proc_lib,init_p_do_apply,3}]}},{ancestors,[net_sup,kernel_sup,<0.10.0>]},{messages,[]},{links,[#Port<0.216>,<0.17.0>]},{dictionary,[{longnames,true}]},{trap_exit,true},{status,running},{heap_size,233},{stack_size,24},{reductions,484}],[]]}
{error_logger,{{2012,7,13},{14,8,56}},supervisor_report,[{supervisor,{local,net_sup}},{errorContext,start_error},{reason,{'EXIT',nodistribution}},{offender,[{pid,undefined},{name,net_kernel},{mfargs,{net_kernel,start_link,[['riak@127.0.0.1',longnames]]}},{restart_type,permanent},{shutdown,2000},{child_type,worker}]}]}
{error_logger,{{2012,7,13},{14,8,56}},supervisor_report,[{supervisor,{local,kernel_sup}},{errorContext,start_error},{reason,shutdown},{offender,[{pid,undefined},{name,net_sup},{mfargs,{erl_distribution,start_link,[]}},{restart_type,permanent},{shutdown,infinity},{child_type,supervisor}]}]}
{error_logger,{{2012,7,13},{14,8,56}},std_info,[{application,kernel},{exited,{shutdown,{kernel,start,[normal,[]]}}},{type,permanent}]}
{"Kernel pid 
terminated",application_controller,"{application_start_failure,kernel,{shutdown,{kernel,start

Re: REST query returns JSON properties as strings?

2012-07-14 Thread Gary Thomas 
Thanks for the explanation Sean, will look closer at map/reduce.
I'm not sure how well it fits the Riak architecture, but when previously 
working with CouchDB+Lucene, the ability to pass "?include_docs=true" to 
retrieve the original JSON doc was mighty handy.

Best,
Gary




On Saturday, July 14, 2012 at 8:31 AM, Sean Cribbs wrote:

> Hi (what's your name?),
> 
> What a search query returns to you is the "index document", or the result of 
> transforming the input before adding it to the search indexes; this also 
> means that deeply nested fields will be flattened. If you want the original, 
> you can use MapReduce to retrieve the documents resulting from a search query.
> 
> On Fri, Jul 13, 2012 at 8:54 PM, 7zark7 <7za...@gmail.com 
> (mailto:7za...@gmail.com)> wrote:
> > I would like to store JSON documents in a Riak 1.1.x bucket (using 
> > bitcask), and would like to search for them via REST using JSON property 
> > values.
> > 
> > E.g. JSON docs with typed properties like this: 
> > 
> > {
> >"type": "whatever",
> >"owner": 123456,
> >"text": "Blah blah blah",
> >"enabled": true
> > }
> > 
> > However, the query response (/solr/mybucket/select?q=enabled:true&wt=json) 
> > returns each doc's properties as JSON strings, and not the original JSON 
> > type:
> > 
> > {"responseHeader":{ 
> > "status":0,
> > "QTime":1,
> > "params":{
> > "q":"enabled:true",
> > "q.op":"or",
> > "filter":"",
> > "wt":"json"
> > }
> > },"response":{
> > "numFound":1,
> > "start":0,
> > "maxScore":"0.353553",
> > "docs":[
> > {
> > "id":"Obdoy3XbJe3kdh1T2bfxJgISCCL",
> > "index":"rest",
> > "fields":{
> > "enabled": "true",
> > "owner": "123456",
> > "text": "Blah blah blah",
> > "type": "whatever"
> > },
> > "props":{}
> > }
> > ]
> > }}
> > 
> > 
> > This makes the results unusable for me, and I'm concerned I'd have to fall 
> > back to N calls back to retrieve each original doc, which is not feasible. 
> > 
> > 
> > I'm used to CouchDB+Lucene which behaves "correctly" to me.
> > 
> > Is there some way to do what I'd like?  E.g. another query method or option?
> > 
> > Thanks
> > 
> > 
> > 
> > ___
> > riak-users mailing list
> > riak-users@lists.basho.com (mailto:riak-users@lists.basho.com)
> > http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
> > 
> 
> 
> 
> -- 
> Sean Cribbs mailto:s...@basho.com)>
> Software Engineer
> Basho Technologies, Inc.
> http://basho.com/
> 

___
riak-users mailing list
riak-users@lists.basho.com
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com

Setting up Riak for highly concurrent load testing

2012-07-14 Thread Jeremy Ong 
Hi Basho and Riak Users,

I am in the process of evaluating Riak for commercial usage here at
Playmesh. Riak seems great and has a lot of features we like but I am
running into some difficulties in my initial testing.

I am attempting to hit the database at 100 rps on a single machine (aka one
node). In doing so, I run into a number of problems (tcp sockets closing
being the primary one). I am using the Erlang client and have tried talking
to Riak through a single client process and also by spawning a new
concurrent process for every request.

I suspect that Riak is throttling the number of allowable concurrent
connections. Also, I suspect that my understanding about how to properly
utilize the Erlang client is underwhelming at best.

Any suggestions on how to properly set up a local test bed for Riak and
communicating to Riak?

Thanks,
Jeremy

--
Jeremy Ong
Playmesh, Inc.
___
riak-users mailing list
riak-users@lists.basho.com
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com