Hey all,
We were having the "space in the key" bug in our cluster so we went through
the whole dataset, backing it up to json file and removing the spaces in
the keys. Then we trashed our whole cluster and restart from scratch
reimporting the whole data. Everything worked like a charm for two weeks
but this weekend, not sure what happened but AAE died again.
I have two issues now:
1. AAE is trying to recreate an index that already exists and crashes with
an "Already exists" error ... I get this in my error log every 15 seconds
2. AAE crashes while iterating through entropy data with a request timeout
error every hour followed by tons of failed to index objects with request
timeout as well. Stack trace looks like this:
*[error] emulator Error in process <025931.7166> on node 'riak@IP' with
exit value: {function_clause,[{yz_entropy,iterate_entropy_data,[<<11
bytes>>,[{continuation,<<159
bytes>>},{limit,100},{partition,12}],#Fun<yz_index_hashtree.5.46188917,{error,{error,req_timeout}}],[{file,"src/yz_entropy.erl"},{line,44}]},{yz_index_hashtree,'-fold_keys/2-lc$^0/1-0-',3,[{file...
**(TRUNCATED)*
*[error] <0.1371.0>@yz_kv:index:215 failed to index object
{{<<"TYPE">>,<<"BUCKET">>},<<"KEY">>} with error {"Failed to index
docs",{error,req_timeout}} because ... **(REPEATED MULTIPLE TIME FOR
DIFFERENT KEYS)*
I tried clearing the yz anti-entropy tree and reinitialising the
yz_entropy_mgr with no luck. Anything I can do to fix this?
Oh FYI, I cannot insert data with spaces in the key anymore because we are
using a wrapper on top of riak that will prevent us to do so therefore my
issues are not related to this for sure.
These are some config changes that may be good to know for more context.
We added this to ibrowse.conf:
{dest, "localhost", 8093, 100, 1000, []}.
Jetty is set with minthread 80, acceptors 80, and is using the NIO
connector.
All our solr buckets have filterCache disabled with softcommits set to 10s
instead of 1.
Our riak.conf has background_manager turned on with AAE and handoff using
it.
Thanks,
Steve
_______________________________________________
riak-users mailing list
riak-users@lists.basho.com
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
