Re: repair-2i stops with "bad argument in call to eleveldb:async_write"

2014-09-04 Thread Effenberg, Simon
Or could it be that the timeout is also used inside the automatic 2i AAE? Now I try to repair all partitions.. but this will take some time (2-4hours per partition with 256 partitions overall and probably only 3-4 partitions in parallel.. so ~10 days minimum). But still the question, why was it no

AW: repair-2i stops with "bad argument in call to eleveldb:async_write"

2014-09-03 Thread Effenberg, Simon
erg, Simon" Datum:03.09.2014 16:28 (GMT+01:00) An: bryan hunt Cc: riak-users@lists.basho.com Betreff: Re: repair-2i stops with "bad argument in call to eleveldb:async_write" I changed now in the code the timeout from 5 to 60mins on one node.. after 23mins the repair-2i was able to co

Re: repair-2i stops with "bad argument in call to eleveldb:async_write"

2014-09-03 Thread Effenberg, Simon
I changed now in the code the timeout from 5 to 60mins on one node.. after 23mins the repair-2i was able to continue: 2014-09-03 12:58:06.913 UTC [info] <0.10345.8>@riak_kv_2i_aae:repair_partition:257 Acquired lock on partition 548063113999088594326381812268606132370974703616 2014-09-03 12:58:06

Re: repair-2i stops with "bad argument in call to eleveldb:async_write"

2014-08-11 Thread Effenberg, Simon
Hi, any updates on this issue? I'm still able to search a range of 2i and I'm getting 3 results.. 0, 557 and 13853 :(.. I cannot rely on 2i right now nor can I repair it. Cheers Simon On Fri, Aug 08, 2014 at 07:12:58AM +, Effenberg, Simon wrote: > Hi Bryan, > > thanks for this. I tried it

Re: repair-2i stops with "bad argument in call to eleveldb:async_write"

2014-08-08 Thread Effenberg, Simon
Hi Bryan, thanks for this. I tried it but to be honest I cannot see any specific stuff in the logs (on the specific host). I attached the logfile from the specific node. If you think it is also/more important to look into the logfiles on the other nodes I can send them as well.. but a quick look

Re: repair-2i stops with "bad argument in call to eleveldb:async_write"

2014-08-08 Thread Effenberg, Simon
t;>>>> > >>>>>> -- node 5 --- > >>>>>> > >>>>>> 2014-07-30 06:16:09.191 UTC [info] > >> <0.15985.355>@riak_kv_2i_aae:next_partition:160 Finished 2i repair: > >>>>&g

Re: repair-2i stops with "bad argument in call to eleveldb:async_write"

2014-08-08 Thread Effenberg, Simon
160 Finished 2i repair: > >>>>>> Total partitions: 1 > >>>>>> Finished partitions: 1 > >>>>>> Speed: 100 > >>>>>> Total 2i items scanned: 0 > >>>>>> Total tree objects: 0

Re: repair-2i stops with "bad argument in call to eleveldb:async_write"

2014-08-06 Thread Engel Sanchez
rgument in call to eleveldb:async_get(#Ref<0.0.351.26505>, > <<>>, > <<131,104,2,109,0,0,0,20,99,111,110,118,101,114,115,97,116,105,111,110,95,115,101,99,114,101,116,...>>, > []) in eleveldb:get/3 line 143 in context child_terminated > >>>> >

Re: repair-2i stops with "bad argument in call to eleveldb:async_write"

2014-08-06 Thread bryan hunt
gt;>> 2014-07-30 06:16:09.192 UTC [error] <0.1598.0> Supervisor >> {<0.1598.0>,poolboy_sup} had child riak_core_vnode_worker started with >> {riak_core_vnode_worker,start_link,undefined} at <0.1601.0> exit with >> reason bad argument in call to eleveldb:async_get(#R

Re: repair-2i stops with "bad argument in call to eleveldb:async_write"

2014-08-06 Thread Effenberg, Simon
f<0.0.351.26505>, > <<>>, > > <<131,104,2,109,0,0,0,20,99,111,110,118,101,114,115,97,116,105,111,110,95,115,101,99,114,101,116,...>>, > []) in eleveldb:get/3 line 143 in context child_terminated > >>>> > >>>> ---

Re: repair-2i stops with "bad argument in call to eleveldb:async_write"

2014-08-04 Thread Damian Murphy
er started with >>>>> {riak_core_vnode_worker,start_link,undefined} at <0.1601.0> exit with >>>>> reason bad argument in call to eleveldb:async_get(#Ref<0.0.351.26505>, >>>>> <<>>, >>>>> <<131,104,2,109,0,

Re: repair-2i stops with "bad argument in call to eleveldb:async_write"

2014-08-01 Thread Russell Brown
4 UTC [info] >>>> <0.32042.379>@riak_kv_2i_aae:next_partition:160 Finished 2i repair: >>>> Total partitions: 1 >>>> Finished partitions: 1 >>>> Speed: 100 >>>> Total 2i items scanned: 0 >>>> Tot

Re: repair-2i stops with "bad argument in call to eleveldb:async_write"

2014-08-01 Thread Effenberg, Simon
<0.32042.379>@riak_kv_2i_aae:next_partition:160 Finished 2i repair: > > > Total partitions: 1 > > > Finished partitions: 1 > > >Speed: 100 > > >Total 2i items scanned: 0 > > >Total tree objects: 0 > > >

Re: repair-2i stops with "bad argument in call to eleveldb:async_write"

2014-07-30 Thread Effenberg, Simon
t;0.4086.0> gen_server <0.4086.0> > > terminated with reason: bad argument in call to > > eleveldb:async_get(#Ref<0.0.2698.198008>, <<>>, > > <<131,104,2,109,0,0,0,20,99,111,110,118,101,114,115,97,116,105,111,110,95,115,101,99,114,101,116,...>

Re: repair-2i stops with "bad argument in call to eleveldb:async_write"

2014-07-30 Thread Russell Brown
0,0,20,99,111,110,118,101,114,115,97,116,105,111,110,95,115,101,99,114,101,116,...>>, > []) in eleveldb:get/3 line 143 in gen_server:terminate/6 line 747 > 2014-07-30 06:16:09.154 UTC [error] <0.4085.0> Supervisor > {<0.4085.0>,poolboy_sup} had child riak_core_vnode_wo

Re: repair-2i stops with "bad argument in call to eleveldb:async_write"

2014-07-30 Thread Effenberg, Simon
20,99,111,110,118,101,114,115,97,116,105,111,110,95,115,101,99,114,101,116,...>>, []) in eleveldb:get/3 line 143 in context child_terminated On Wed, Jul 30, 2014 at 09:50:22AM +0100, Russell Brown wrote: > Hi Simon, > So the earlier “this is on wheezy, rest are on squeeze” thin

Re: repair-2i stops with "bad argument in call to eleveldb:async_write"

2014-07-30 Thread Russell Brown
014 07:49 (GMT+01:00) > An: bryan hunt > Cc: riak-users@lists.basho.com > Betreff: AW: repair-2i stops with "bad argument in call to > eleveldb:async_write" > > Hi, > > I tried it on two different nodes with one partition each. Both multiple > times before the

AW: repair-2i stops with "bad argument in call to eleveldb:async_write"

2014-07-29 Thread Effenberg, Simon
uot; Datum:30.07.2014 07:49 (GMT+01:00) An: bryan hunt Cc: riak-users@lists.basho.com Betreff: AW: repair-2i stops with "bad argument in call to eleveldb:async_write" Hi, I tried it on two different nodes with one partition each. Both multiple times before the upgrade and after the upg

AW: repair-2i stops with "bad argument in call to eleveldb:async_write"

2014-07-29 Thread Effenberg, Simon
this is not a disk corruption issue.. Simon Von Samsung Mobile gesendet Ursprüngliche Nachricht Von: bryan hunt Datum:29.07.2014 18:21 (GMT+01:00) An: "Effenberg, Simon" Cc: riak-users@lists.basho.com Betreff: Re: repair-2i stops with "bad argument in call to eleve

Re: repair-2i stops with "bad argument in call to eleveldb:async_write"

2014-07-29 Thread bryan hunt
Hi Simon, Does the problem persist if you run it again? Does it happen if you run it against any other partition? Best Regards, Bryan Bryan Hunt - Client Services Engineer - Basho Technologies Limited - Registered Office - 8 Lincoln’s Inn Fields London WC2A 3BP Reg 07970431 On 29 Jul 2014

Re: repair-2i stops with "bad argument in call to eleveldb:async_write"

2014-07-29 Thread bryan hunt
Sounds like disk corruption to me. Bryan Hunt - Client Services Engineer - Basho Technologies Limited - Registered Office - 8 Lincoln’s Inn Fields London WC2A 3BP Reg 07970431 On 29 Jul 2014, at 13:06, Effenberg, Simon wrote: > Said to say but the issue stays the same.. even after the upgrad

Re: repair-2i stops with "bad argument in call to eleveldb:async_write"

2014-07-29 Thread Effenberg, Simon
Said to say but the issue stays the same.. even after the upgrade to 1.4.10. Any ideas what is happening here? Cheers Simon On Tue, Jul 29, 2014 at 08:46:42AM +, Effenberg, Simon wrote: > Already started to prepare everything for it.. :) > > On Tue, Jul 29, 2014 at 09:43:22AM +0100, Guido M

Re: repair-2i stops with "bad argument in call to eleveldb:async_write"

2014-07-29 Thread Effenberg, Simon
Already started to prepare everything for it.. :) On Tue, Jul 29, 2014 at 09:43:22AM +0100, Guido Medina wrote: > Hi Simon, > > There are some (maybe related) Level DB fixes in 1.4.9 and 1.4.10, I don't > think there isn't any harm for you to do a rolling upgrade since nothing > major changed, ju

Re: repair-2i stops with "bad argument in call to eleveldb:async_write"

2014-07-29 Thread Guido Medina
Hi Simon, There are some (maybe related) Level DB fixes in 1.4.9 and 1.4.10, I don't think there isn't any harm for you to do a rolling upgrade since nothing major changed, just bug fixes, here is the release notes' link for reference: https://github.com/basho/riak/blob/1.4/RELEASE-NOTES.md

Re: repair-2i stops with "bad argument in call to eleveldb:async_write"

2014-07-29 Thread Effenberg, Simon
One thing I realized right now.. the new node is on wheezy whereas all other 12 nodes are running squeeze.. but shouldn't be a problem, right? On Tue, Jul 29, 2014 at 08:35:15AM +, Effenberg, Simon wrote: > Hi, > > we have some issues with 2i queries like that: > > seffenberg@kriak46-1:~$ wh

repair-2i stops with "bad argument in call to eleveldb:async_write"

2014-07-29 Thread Effenberg, Simon
Hi, we have some issues with 2i queries like that: seffenberg@kriak46-1:~$ while :; do curl -s localhost:8098/buckets/conversation/index/createdat_int/0/23182680 | ruby -rjson -e "o = JSON.parse(STDIN.read); puts o['keys'].size"; sleep 1; done 13853 13853 0 557 557 557 13853 0 ... So I trie