Re: Riak and ec2 with vm.args - Node 'r...@ip-10-234-117-74.eu-west-1.compute.internal' not responding to pings.

2013-09-15 Thread David Montgomery
s some instructions >>> on getting things set up. In the docs, they recommend using the AWS >>> internal IP address instead of the FQDN. >>> >>> This advice is also repeated in the Riak wiki [2]. >>> >>> >>> [1]: http://media.amazonwebs

Re: Riak and ec2 with vm.args - Node 'r...@ip-10-234-117-74.eu-west-1.compute.internal' not responding to pings.

2013-09-15 Thread David Montgomery
MCITP: SQL Server 2008, MVP > Cloudera Certified Developer for Apache Hadoop > > > On Sun, Sep 15, 2013 at 1:06 AM, David Montgomery < > davidmontgom...@gmail.com> wrote: > >> Hi, >> >> I am having a very difficult time installing riak on ec2. >> >>

Riak and ec2 with vm.args - Node 'r...@ip-10-234-117-74.eu-west-1.compute.internal' not responding to pings.

2013-09-15 Thread David Montgomery
Hi, I am having a very difficult time installing riak on ec2. To start ...I cant get past using private ip address or private dns when setting vm.args -name riak@127.0.0.1 root@ip-10-234-117-74:/home/ubuntu# hostname --fqdn ip-10-234-117-74.eu-west-1.compute.internal riak start riak ping Node

Re: Does riak control work?

2013-09-14 Thread David Montgomery
_controller) ({application_start_failure,riak_core,{shutdown,{riak_core_app,start,[normal,[]]}}}) root@ip-10-234-223-183:/etc/riak# Will older versions of riak work? On Mon, Sep 9, 2013 at 6:17 PM, David Montgomery wrote: > I am using the lastest ubuntu deb package.

[no subject]

2013-09-09 Thread David Montgomery
Why do I get errors restarting riak when i use aws EIP's? I change the ip address per this doc. http://docs.basho.com/riak/1.2.1/cookbooks/Basic-Cluster-Setup/ I am using the latest version of riak for ubuntu 12.04 I change the 127.0.0.1 to the EIP. It should work. Yetriak will not work.

Does riak control work?

2013-09-09 Thread David Montgomery
I am using the lastest ubuntu deb package. https://localhost:8069/admin I get page not found The connection was reset The connection to the server was reset while the page was loading. The site could be temporarily unavailable or too busy. Try again in a few moments. If

MR error - How to make sense of error message in riak

2013-07-18 Thread David Montgomery
Hi, I am trying to run a MR script; WHat does the below error translate into in in english? Traceback (most recent call last): File "/home/ubuntu/workspace/ndkt-scraper/src/parsers/archive/riak_mapreduce_archive.py", line 188, in bucket = 'archive_temp' File "/home/ubuntu/workspace/ndkt

Re: Riak does not work on ec2

2013-06-21 Thread David Montgomery
Ok..yes..using a EIP worked. Thanks and will now up my understanding of NAT Thanks On Sat, Jun 22, 2013 at 12:46 PM, Andrew Thompson wrote: > By default AWS machines use NAT, they provide a temporary external IP > that is NATed through to the internal IP applied to the AWS instance. > You can

Re: Riak does not work on ec2

2013-06-21 Thread David Montgomery
m > > [0] > http://docs.basho.com/riak/latest/cookbooks/Performance-Tuning-AWS/#Dealing-with-IP-addresses > > > On Fri, Jun 21, 2013 at 4:55 AM, David Montgomery < > davidmontgom...@gmail.com> wrote: > >> Hi, >> >> I am trying to get riak to work

Riak does not work on ec2

2013-06-20 Thread David Montgomery
Hi, I am trying to get riak to work on ubuntu 12.04 and is proving not to be a friendly install on ec2. 1) I am using riak_1.3.1-1_amd64.deb and have libssl0.9.8 installed 2) I stop riak then run riak-admin cluster replace riak@127.0.0.1 r...@xxx.xxx.xxx.xxx riak-admin cluster plan riak-admin

Re: Riak will not work remote from ec2

2013-06-13 Thread David Montgomery
xited,{shutdown,{kernel,start,[normal,[]]}}},{type,permanent}]} {"Kernel pid terminated",application_controller,"{application_start_failure,kernel,{shutdown,{kernel,start,[normal,[]]}}}"} Crash dump was written to: /var/log/riak/erl_crash.dump Kernel pid terminated (application

Re: Riak will not work remote from ec2

2013-06-13 Thread David Montgomery
sing documentation? On Fri, Jun 14, 2013 at 9:02 AM, David Montgomery wrote: > Hi, > > Why will riak not work on ec2 with a single node? > > 1) I have my sec groups open for all ports. This should be a no brainier. > > 2) Using the python client

Riak will not work remote from ec2

2013-06-13 Thread David Montgomery
Hi, Why will riak not work on ec2 with a single node? 1) I have my sec groups open for all ports. This should be a no brainier. 2) Using the python client = riak.RiakClient(host=riak_host,port=8087,transport_class=riak.transports.pbc.RiakPbcTransport) where riak_host is the public ip address o

Re: Exception: {"phase":0,"error":"[timeout]

2013-01-31 Thread David Montgomery
Yeah...the update using the ubuntu package worked! Whew! scary See Ya On Wed, Jan 30, 2013 at 6:07 PM, David Montgomery wrote: > If I build from source will that solve the problem? Elsehow can i get > the data out so I can use hadoop? > > I am using ubuntu. Do I just

Re: Exception: {"phase":0,"error":"[timeout]

2013-01-30 Thread David Montgomery
Hi David, > > This is a known error that has been resolved in trunk, but is yet to be > released: > > https://github.com/basho/riak_kv/issues/290 > > Any word on a release date, Basho guys? > > > On 25 January 2013 12:52, David Montgomery wrote: > >> Hi, >&g

Re: Exception: {"phase":0,"error":"[timeout]

2013-01-24 Thread David Montgomery
k, but is yet to be > released: > > https://github.com/basho/riak_kv/issues/290 > > Any word on a release date, Basho guys? > > > On 25 January 2013 12:52, David Montgomery wrote: > >> Hi, >> >> Why will riak throw this type of exception when I have a tim

Exception: {"phase":0,"error":"[timeout]

2013-01-24 Thread David Montgomery
Hi, Why will riak throw this type of exception when I have a timeout of TIMEOUT=300? What does this error mean? How of now I cant get data out of a production system. 'Traceback (most recent call last): File "gg.py", line 324, in results = getDomainReportIndex(bucket=bucket,date_s

Re: Same MR query, different results every run...........

2013-01-07 Thread David Montgomery
tion only. Adding a reduce fixed it. > > I believe as the map fans out to multiple nodes, whichever node returns > data first is written to output and not collected by a reduce stage. > > Please correct me if I'm wrong. > > Thanks > > -- > Mridul > > >

Same MR query, different results every run...........

2013-01-05 Thread David Montgomery
Hi, Here is my my mapper... query.map(''' function(value, keyData, arg) { if(value.length == 0){ return []; }else{ var data = Riak.mapValuesJson(value)[0]; var obj = {}; if(data['campaign_id']=='%s'){ try{

Re: Keyfilters. I cant get date ranges to work using key filters

2012-12-24 Thread David Montgomery
23" // end of your range > } > > Not only is this simpler to express, but it also limits the range of > scans done in the backend, meaning your queries shouldn't take as long > to complete. > > On Mon, Dec 24, 2012 at 2:16 AM, David Montgomery > wrote:

Re: Keyfilters. I cant get date ranges to work using key filters

2012-12-23 Thread David Montgomery
Thanks, If I use a logical & I get no data. If I just use a greater than than it works. date_start = '2012122300' date_end = '2012122323' filters = key_filter.tokenize(":", filter_map['date']).greater_than_eq(date_start) & key_filter.tokenize(":", filter_map['date']).less_than_eq(date_end

Re: CRASH REPORT Process <<"TypeError: reduce of empty array with no initial value">>},{<<"source">>,<<"unknown">>}]}

2012-12-21 Thread David Montgomery
hen I get the error. > So...what I cant figure out is why I cant use the last filter. My key > looks like this. > 07712f3d-6661-44a7-89ff-4e4ed3b5f5a6:cid5989410021:agid7744464312:2012122205:SG > > uuid:campaign_id:aid:date:country where date is in MMDDHH format > > I just wa

Re: CRASH REPORT Process <<"TypeError: reduce of empty array with no initial value">>},{<<"source">>,<<"unknown">>}]}

2012-12-21 Thread David Montgomery
filter_map['date']).string_to_int().greater_than_eq(date_start) the query works and I get the data I want. If I include then I get the error. So...what I cant figure out is why I cant use the last filter. My key looks like this. 07712f3d-6661-44a7-89ff-4e4ed3b5f5a6:cid5989410021:agid7

CRASH REPORT Process <<"TypeError: reduce of empty array with no initial value">>},{<<"source">>,<<"unknown">>}]}

2012-12-20 Thread David Montgomery
2012-12-20 13:31:27.906 [error] <0.30794.1> CRASH REPORT Process <0.30794.1> with 0 neighbours exited with reason: {fitting_exited_abnormally,[{<<"lineno">>,3},{<<"message">>,<<"TypeError: reduce of empty array with no initial value">>},{<<"source">>,<<"unknown">>}]} in gen_fsm:terminate/7 line 611

where did my data go when I write to riak?

2012-12-18 Thread David Montgomery
Hi, I am using latest ubuntu release for 64. I am writing to riak and i get no errors. Then againgiven a key...,I also get no data. Where did it go? How does one debug riak when you write to riak..it looks like a success, get no data when get by key and get no errors. To start, I wrote to

"Failed to create ~p for mapred_queue_dir defaulting to %s: ~p" ["/data/riak/mr_queue", "/tmp/mr_queue", eacces]

2012-12-17 Thread David Montgomery
Hi, I cant get riak to work when changing default data directory to /data/riak. I am using the current realease for ubunt\u %% Platform-specific installation paths (substituted by rebar) {platform_bin_dir, "/usr/sbin"}, {platform_data_dir, "/d

Re: nebie with volumes and best practices with /var/lib/riak data directory

2012-12-12 Thread David Montgomery
> If you wanted to snapshot your riak data, it all lives in * > platform_data_dir* > * > * > * > * > Hope this helps, > > > Sean > @densone > > > On Wednesday, December 12, 2012 at 12:32 AM, David Montgomery wrote: > > Hi, > > I use datapipe f

nebie with volumes and best practices with /var/lib/riak data directory

2012-12-11 Thread David Montgomery
Hi, I use datapipe for by cloud provider. An I am rather new from a sysadmin perspective of having to attach volumes. I am using ubuntu 12.04 64. So...now that I can attach a 1T volume on machine bootby question is this. When I mount the new volume.should I mount /var/lib/riak? I see

Mal Reduce Exception: {"phase":0,"error":"[timeout]","input":"

2012-12-09 Thread David Montgomery
Hi, I am having an issues with getting data out of MR.. If I run on a few particular days I get the below error and returns almost immediate. On other days returns valid data. I am using a key filter to select by day when I noticed that a date range was failing. key_filter.starts_with('20

error with MR and someone had painted it blue

2012-12-08 Thread David Montgomery
I am trying to run a mr. I get the below error. Other than a 500 error...I dont see any other means to debug. What does the error mean? Traceback (most recent call last): File "/home/ubuntu/workspace/rtbopsConfig/rtbServers/rtbWorkerServer/mapred_nokeypy.py", line 69, in main() File "/

python map reduce and secondary indexes

2012-12-07 Thread David Montgomery
Hi, Given that map reduce is the primary way of getting data out of riak, and i use python api, I am hard pressed to find any simple examples. Not even on the officially supported riak python api. Below is how I add a record to riak: id = """%s:%s:%s:%s:%s""" % (str(uuid4()),campaig

Riak node is a member of two clusters?

2012-12-04 Thread David Montgomery
I have three nodes. I go to riak control on one machine I see nodes A B On anther node I see B C I though riak was supposed to prevent this from happening. ___ riak-users mailing list riak-users@lists.basho.com http://lists.basho.com/mailman/listinfo/r

CRASH REPORT Process <0.549.0> with 0 neighbours exited with reason:

2012-12-04 Thread David Montgomery
Hi, I am trying to add a new nodes. I get node un reachable when handing off. I restart the new node and dies quick. Below is the output of the error log. Sohow do I resolve? 2012-12-05 02:50:35.335 [error] <0.14969.101>@riak_core_handoff_sender:start_fold:210 ownership_handoff transfer

Map Reduce and long queries -

2012-10-14 Thread David Montgomery
Hi, Below is my code for running a map reduce in python. I have a six node cluster, 2 cores each with 4 gigs for ram. I am no load and about 3 Mill keys and using leveldb with riak 1.2. Doing the below is taking a terribly long time. Never finished and I dont even know how I can check if it i

map reduce time outs

2012-10-13 Thread David Montgomery
Hi, I am trying to run this map reduce job. I get a time out. Logic would dictate that I add a time out to the RiakClient but that flag is not there. How do I deal with timeouts with the python api? I have data...I just cant get it out. ile "/usr/local/lib/python2.7/dist-packages/riak-1.5.0-

Getting Stats into a python dict

2012-10-12 Thread David Montgomery
Hi, I see in the riak control I can see stats. https://111.111.11.111:8069/stats Does the python api support getting the stats into a dictionary? Thanks ___ riak-users mailing list riak-users@lists.basho.com http://lists.basho.com/mailman/listinfo/r

Fastest write to riak

2012-10-11 Thread David Montgomery
Hi, I am using the python riak api and upon stress testing I would have had a death spiral but my riak cluster was not tuned. I will now set the bucket N to 2 e.g. test_bucket.set_n_val(2) Now what is left is w and dw. I would like the option as least getting that is most 1 write occurred. Al

One node has 100% of the partitions

2012-10-06 Thread David Montgomery
Hi, I booted up a two node cluster. I got to riak control and I see that one node has 100% of the partions. I see both nodes in riak control. I am using 1.2. How do I resolve? Below is my code on how I join cmd = 'riak-admin cluster join riak@%s' % existing_riak[0]

Riak Control and issues with cluster

2012-09-02 Thread David Montgomery
Hi, I am using riak control now. I just booted up 5 nodes with no data. I use chef to boot up the cluster and when booted I ssh into the machine and run riak-admin cluster join riak@111.111.111.111 where 111.111.111.111 is a randomly chosen ip address in the group. I go to riak control and th

How Reduce RAM usage as much as possible

2012-09-01 Thread David Montgomery
Hi, >From my understanding keys are stored in RAM. For the interim, I will have 5 nodes with one core each and 1 gig of RAM. I am not doing any big MR jobs for a while and when do, off hours. How do I configure riak to be RAM friendly? Are there knobs I can tweak? Thanks Sent from my i

Failed: This node is already a member of a cluster

2012-08-31 Thread David Montgomery
Hi, I need top add a node to a cluster. I get this error root@dp-riak-hk:~# riak-admin join -f riak@103.4.11.11 Attempting to restart script through sudo -H -u riak Attempting to restart script through sudo -H -u riak Failed: This node is already a member of a cluster Well...that node is not

Re: how to do a key filter using the python api for map reduce

2012-08-30 Thread David Montgomery
I did not scroll down far enough in the api docs regarding python. Problem solved. Sent from my iPad On Aug 30, 2012, at 6:51 PM, David Montgomery wrote: > Hi, > > I am using the python api. > > I need to do a key filter kinda like this. > > > "inputs&q

how to do a key filter using the python api for map reduce

2012-08-30 Thread David Montgomery
Hi, I am using the python api. I need to do a key filter kinda like this. "inputs":{ "bucket":"impressions" "key_filters":[["tokenize", ":", 2],["starts_with", "2012"]] }, But from the python api docs...I have a query that looks like this. query = client.add('impressions') qu

riak.RiakError: 'this transport is not available (no protobuf)'

2012-08-28 Thread David Montgomery
What does this error me and how do I resolve? I am using the latest version of riak for ubunut and the riak python api. File "workerServer.py", line 57, in client = riak.RiakClient(host='riak.hk.test.com',port=8087,transport_class=riak.transports.pbc.RiakPbcTransport) File "/usr/local/l

Best practice for buckets and timestamps

2012-08-07 Thread David Montgomery
Hi, I have yet to do MR for querying data but I am assuming I can query across several buckets. This leads me to the question, can I have a formated date as part of a bucket and when it comes time to to MR then something like this mybucket:2012_11_10 mybucket:2012_11_11 mybucket:2012_11_12 For

Re: Cant remote write to riak

2012-07-26 Thread David Montgomery
2 at 11:59 AM, David Montgomery wrote: > As a note, from a remote machine, this command worked curl -v > http://111.222.333.444:8098/riak/test and returned the expected > output. > > > * About to connect() to 111.222.333.444 port 8098 (#0) > * Trying 111.222.333.44

Re: Cant remote write to riak

2012-07-26 Thread David Montgomery
n":"mapreduce_linkfun"},"n_val":3,"notfound_ok":true,"old_vclock":86400,"postcommit":[],"pr":0,"precommit":[],"pw":0,"r":"quorum","rw":"quorum","small_vclock":50,&qu

Cant remote write to riak

2012-07-26 Thread David Montgomery
Hi, I have one node installed on my cloud provider and tried to remote write using a python client but I get a connection failure. On the riak node: riak-admin status | grep ring_members ring_members : ['r...@xxx.xxx.xxx.xxx] service riak ping Attempting to restart script through sudo -u riak p

Trying to connect to riak from python client

2012-07-26 Thread David Montgomery
I have riak installed on my local machine. I am not trying to do a cluster simulation on my local machine, a simple node node locally is fine. when I do a service ping I get a pong. When I run the below python client it just hangs. Below is the code. So...rather confused on where to go at this

Riak, pip, and monit - How to

2012-07-26 Thread David Montgomery
Hi, I tried the below for testing to see if riak was running on a node but I get connection failed. check process riak matching "/usr/lib/riak/erts-5.8.5/bin/beam" group riak start program = "service riak start" if failed host localhost port 8098 protocol HTTP request "/" then al

Riak failed to start within 15 seconds,

2012-07-25 Thread David Montgomery
Hi, I am new to riak. I followed the instructions for a cluster setup at https://wiki.basho.com/Basic-Cluster-Setup.html. It did not work. I am using ubuntu64 I changed the ip in app.confing and vm.args and stoped the service. then I ran the below. riak-admin reip riak@127.0.0.1 r...@xxx.xx

error compiling cpp driver for riak

2012-01-26 Thread David Montgomery
I tried to compile the cpp driver code on gitbub and the below are the erros. I am on Ubuntu 64 using gcc 4.5. At the prompt I typed >scons So...what do I do? Thanks test@test:~/Downloads/riak-cpp$ scons scons: Reading SConscript files ... scons: done reading SConscript files. scons: Building