Hi Sanket,
Another thing I'd like to confirm - you have installed your compiled
.beam file on all Riak nodes and can confirm via "m(precommit)" that
the code is available?
--
Luke Bakken
Engineer
lbak...@basho.com
On Wed, May 11, 2016 at 12:12 PM, Sanket Agrawal
<sanket.agra...@gmail.com> wrote:
> Yep, I can always load the module in question in riak console fine. I did
> all my testing in riak console, before trying to turn on precommit hook.
> Here is the output for loading the module that you asked for - if sasl
> logging is not working, perhaps something is broken about commit hooks then:
>
>> $ ~/riak/riak1/bin/riak attach
>> (riak1@127.0.0.1)1> m(precommit).
>> Module precommit compiled: Date: May 11 2016, Time: 02.05
>> Compiler options: []
>> Object file: /home/ec2-user/riak/customcode/precommit.beam
>> Exports:
>> module_info/0
>> module_info/1
>> pre_all/1
>> pre_uauth/1
>> pre_uprofile/1
>> pre_uuid/1
>> ok
>
>
> On Wed, May 11, 2016 at 2:55 PM, Douglas Rohrer <droh...@basho.com> wrote:
>>
>> Huh - I get a huge amount of logging when I turn on sasl using
>> advanced.config - specifically, I have:
>>
>> {sasl,[{sasl_error_logger,{file, "/tmp/sasl-1.log"}}]}
>>
>> in my advanced.config, and for just a startup/shutdown cycle I get a
>> 191555 byte file.
>>
>> Just to confirm that you can, in fact, load the modules in question, what
>> happens if you do a `riak-admin attach` and do `m(precommit).` what do you
>> see?
>>
>> Doug
>>
>> On Wed, May 11, 2016 at 11:32 AM Sanket Agrawal <sanket.agra...@gmail.com>
>> wrote:
>>>
>>> Thanks, Doug. I have enabled sasl logging now through advanced.config
>>> though it doesn't seem to be creating any log yet.
>>>
>>> If this might help you folks with debugging precommit issue, what I have
>>> observed is that erl-reload command doesn't load the precommit modules for
>>> any of the three nodes (though precommit hook has been enabled on one of the
>>> buckets for testing).
>>>>
>>>> $ ~/riak/riak1/bin/riak-admin erl-reload
>>>> Module precommit not yet loaded, skipped.
>>>> Module rutils not yet loaded, skipped.
>>>
>>>
>>>
>>> On Wed, May 11, 2016 at 2:05 PM, Douglas Rohrer <droh...@basho.com>
>>> wrote:
>>>>
>>>> As to the SASL logging, unfortunately it's not "on by default" and the
>>>> setting in riak.conf, as you found out, doesn't work correctly. However,
>>>> you
>>>> can enable SASL via adding a setting to your advanced.config:
>>>>
>>>> {sasl,[{sasl_error_logger,tty}]} %% Enable TTY output for the SASL app
>>>> {sasl,[{sasl_error_logger,{file, "/path/to/log"}]} %% Enable SASL and
>>>> output to "/path/to/log" file
>>>>
>>>> We're evaluating if we shouldn't just remove the sasl setting from
>>>> riak.conf altogether, as you're the first person (that we know of) since
>>>> 2012 that has tried to turn it on and noticed this bug.
>>>>
>>>> Doug
>>>>
>>>> On Wed, May 11, 2016 at 10:14 AM Luke Bakken <lbak...@basho.com> wrote:
>>>>>
>>>>> Hi Sanket -
>>>>>
>>>>> I'd like to confirm some details. Is this a one-node cluster? Did you
>>>>> install an official package or build from source?
>>>>>
>>>>> Thanks -
>>>>> --
>>>>> Luke Bakken
>>>>> Engineer
>>>>> lbak...@basho.com
>>>>>
>>>>>
>>>>> On Tue, May 10, 2016 at 6:49 PM, Sanket Agrawal
>>>>> <sanket.agra...@gmail.com> wrote:
>>>>> > One more thing - I set up the hooks by bucket, not bucket type. The
>>>>> > documentation for 2.1.4 says that hooks are defined on the bucket
>>>>> > level.
>>>>> > Here is how I set up precommit hook (derived from "Riak Handbook"
>>>>> > p95):
>>>>> >
>>>>> > curl -X PUT localhost:8098/types/test_kv_wo/buckets/uuid_log/props -H
>>>>> > 'Content-Type: application/json' -d '{ "props": { "precommit":
>>>>> > [{"mod":
>>>>> > "precommit", "fun": "pre_uuid"}]}}' -v
>>>>> >
>>>>> >
>>>>> > On Tue, May 10, 2016 at 9:15 PM, Sanket Agrawal
>>>>> > <sanket.agra...@gmail.com>
>>>>> > wrote:
>>>>> >>
>>>>> >> I just set up a precommit hook function in dev environment (KV
>>>>> >> 2.1.4)
>>>>> >> which doesn't seem to be triggering off at all. The object is being
>>>>> >> stored
>>>>> >> in the bucket, but the precommit logic is not kicking off. I checked
>>>>> >> couple
>>>>> >> of things as listed below but came up with no error - so, it is a
>>>>> >> head-scratcher why precommit hook is not triggering:
>>>>> >>>
>>>>> >>> - Verify precommit is set in bucket properties - snippet from curl
>>>>> >>> query
>>>>> >>> for bucket props below:
>>>>> >>> "precommit":[{"mod":"precommit","fun":"pre_uuid"}]
>>>>> >>>
>>>>> >>> - check there is no error in logs
>>>>> >>>
>>>>> >>> - check riak-console for commit errors:
>>>>> >>> $ ./riak1/bin/riak-admin status|grep commit
>>>>> >>> postcommit_fail : 0
>>>>> >>> precommit_fail : 0
>>>>> >>>
>>>>> >>> - Run the precommit function manually on Riak console itself with a
>>>>> >>> riak
>>>>> >>> object (that the hook failed to trigger on), and verify it works
>>>>> >>
>>>>> >>
>>>>> >>
>>>>> >> Also, there is no sasl-error.log. "sasl = on" doesn't work in 2.1.4
>>>>> >> because it fails with bad_config error. So, I am assuming sasl
>>>>> >> logging is
>>>>> >> enabled by default.
>>>>> >>
>>>>> >> Here is what precommit function does:
>>>>> >> - For the object (an immutable log append of JSON), calculate the
>>>>> >> location
>>>>> >> of a LWW bucket, and update a easily calculated key with that JSON
>>>>> >> body. It
>>>>> >> works fine from Riak console itself. Code below - we call pre_uuid
>>>>> >> in
>>>>> >> precommit hook - both precommit.beam (where the function is) and
>>>>> >> rutils.beam
>>>>> >> have been copied to the relevant location as set in riak config, are
>>>>> >> accessible through Riak console and work fine if manually executed
>>>>> >> on an
>>>>> >> object:
>>>>> >>
>>>>> >>> %% Preprocess JSON, and copy to a LWW bucket type
>>>>> >>> preprocessJ(RObj,B,Choplen) ->
>>>>> >>> Bn = {rutils:calcBLWWType(RObj),B}, %%this returns the location
>>>>> >>> of LWW
>>>>> >>> bucket - works fine in riak console
>>>>> >>> %% We store uuid map in <username> key - we take out timestamp of
>>>>> >>> length 32 including "_"
>>>>> >>> K = riak_object:key(RObj),
>>>>> >>> Kn = binary:part(K,0,byte_size(K) - Choplen),
>>>>> >>> NObj =
>>>>> >>>
>>>>> >>> riak_object:new(Bn,Kn,riak_object:get_value(RObj),riak_object:get_metadata(RObj)),
>>>>> >>> {ok, C} = riak:local_client(),
>>>>> >>> case C:put(NObj) of
>>>>> >>> ok -> RObj;
>>>>> >>> _ -> {fail,<<"Error when trying to process in precommit
>>>>> >>> hook">>}
>>>>> >>> end.
>>>>> >>>
>>>>> >>> pre_uuid(RObj) -> preprocessJ(RObj,<<"uuid_latest">>,32).
>>>>> >>
>>>>> >>
>>>>> >> Below is a manual execution from riak console of precommit function
>>>>> >> -
>>>>> >> first we execute it to confirm it is returning the original object:
>>>>> >>>
>>>>> >>> (riak1@127.0.0.1)5> precommit:pre_uuid(O1).
>>>>> >>> {r_object,{<<"test_kv_wo">>,<<"uuid_log">>},
>>>>> >>> <<"ahmed_2016-05-10T20%3a47%3a47.346299Z">>,
>>>>> >>> [{r_content,{dict,3,16,16,8,80,48,
>>>>> >>>
>>>>> >>> {[],[],[],[],[],[],[],[],[],[],[],[],[],[],...},
>>>>> >>>
>>>>> >>> {{[],[],[],[],[],[],[],[],[],[],[[...]|...],[],...}}},
>>>>> >>>
>>>>> >>>
>>>>> >>> <<"{\"uname\":\"ahmed\",\"uuid\":\"df8c10e0-381d-5f65-bf43-cb8b4cb806fc\",\"timestamp\":\"2016-05-"...>>}],
>>>>> >>> [{<<0>>,{1,63630132467}}],
>>>>> >>> {dict,1,16,16,8,80,48,
>>>>> >>> {[],[],[],[],[],[],[],[],[],[],[],[],[],[],[],...},
>>>>> >>> {{[],[],[],[],[],[],[],[],[],[],[],[],[],...}}},
>>>>> >>> undefined}
>>>>> >>
>>>>> >>
>>>>> >> Now, we check if the object has been written to test_lww/uuid_latest
>>>>> >> bucket type:
>>>>> >>>
>>>>> >>> (riak1@127.0.0.1)6>
>>>>> >>> C:get({<<"test_lww">>,<<"uuid_latest">>},<<"ahmed">>).
>>>>> >>>
>>>>> >>> {ok,{r_object,{<<"hnm_fsm_lww">>,<<"uuid_latest">>},
>>>>> >>> <<"ahmed">>,
>>>>> >>> [{r_content,{dict,4,16,16,8,80,48,
>>>>> >>>
>>>>> >>> {[],[],[],[],[],[],[],[],[],[],[],[],...},
>>>>> >>>
>>>>> >>> {{[],[],[],[],[],[],[],[],[],[],...}}},
>>>>> >>>
>>>>> >>>
>>>>> >>> <<"{\"uname\":\"ahmed\",\"uuid\":\"df8c10e0-381d-5f65-bf43-cb8b4cb806fc\",\"timestamp\":\""...>>}],
>>>>> >>> [{<<153,190,230,200,210,126,212,127,0,0,156,65>>,
>>>>> >>> {1,63630148036}}],
>>>>> >>> {dict,1,16,16,8,80,48,
>>>>> >>> {[],[],[],[],[],[],[],[],[],[],[],[],[],...},
>>>>> >>> {{[],[],[],[],[],[],[],[],[],[],[],...}}},
>>>>> >>> undefined}}
>>>>> >>
>>>>> >>
>>>>> >> Will appreciate pointer on how to debug precommit hook.
>>>>> >
>>>>> >
>>>>> >
>>>>> > _______________________________________________
>>>>> > riak-users mailing list
>>>>> > riak-users@lists.basho.com
>>>>> > http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
>>>>> >
>>>>>
>>>>> _______________________________________________
>>>>> riak-users mailing list
>>>>> riak-users@lists.basho.com
>>>>> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
>>>
>>>
>
_______________________________________________
riak-users mailing list
riak-users@lists.basho.com
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
