Re: Status and maturity of riak-cs

2017-07-31 Thread Toby Corkindale
Riak CS works, but has some long-standing issues that have never been resolved. It doesn't support AWSv4 auth signatures, so some client libraries won't talk to it at all. It's quite fragile in respect to its connections to Riak KV -- if KV is slow to start up, then Riak CS dies. If KV is restarted

Re: Bug: Stanchion 1.5 in repo along riak cs 2.0

2017-06-06 Thread Toby Corkindale
; wrote: >> > Kota, >> > >> > I think Toby is right, if you look at packagecloud.io, there is no >> reference >> > to stanchion 2, only 1.5. >> > >> > https://packagecloud.io/basho/riak-cs?filter=debs >> > >> > Not sure w

Re: Long delays when trying to recover from errors in Java client at startup

2017-05-22 Thread Toby Corkindale
instantly. On Mon, 22 May 2017 at 17:14 Magnus Kessler wrote: > On 22 May 2017 at 07:02, Toby Corkindale wrote: > >> Hi, >> I've been trying to make a JVM-based app have better error recovery when >> the Riak cluster is still in a starting-up state. >> I have a

Re: Long delays (8min) when trying to recover from errors in Java client at startup

2017-05-21 Thread Toby Corkindale
n bad, retrying... 28 tries remaining Node state: RUNNING . Then we hang for almost exactly eight minutes! Then things continue fine! It's that eight minutes of sleeping that I really want to fix -- as timeouts go it is just way too long! Thanks for any advice, Toby On Mon, 22 May 2017 at

Long delays when trying to recover from errors in Java client at startup

2017-05-21 Thread Toby Corkindale
Hi, I've been trying to make a JVM-based app have better error recovery when the Riak cluster is still in a starting-up state. I have a fairly naive wait-loop that tries to connect and list buckets, and if there's an exception, retry again after a short delay. However once the Riak cluster comes g

Re: 5-node low-cost, low-power Riak KV cluster -- this sound feasible?

2017-05-16 Thread Toby Corkindale
On Thu, 27 Apr 2017 at 13:02 Lloyd R. Prentice wrote: > Hello, > > I'd like to build a five-node low-cost, low-power Riak KV cluster. Use > case: learning Riak, development, testing, storage of personal data. > > -- Based on pure hand-waving, I've plugged numbers that seem more than > adequate f

Re: Error folding keys - incomplete_hint

2017-03-29 Thread Toby Corkindale
5 at 11:10 Toby Corkindale wrote: > Quick follow-up: As a bit of a hack, deleting all the .hint files prior to > each start-up does resolve the errors, and immediately results in a whole > lot of Bitcask merges happening. > But that doesn't strike me as a good long-term fix. >

Re: riak-cs fails to start after reimporting Docker container

2017-03-05 Thread Toby Corkindale
I tried quite hard to get Riak to work reliably in a Docker container, in a long-term-use kind of way. Riak would never shutdown cleanly, though, and so at startup there would always be lots of lock files left around that had to be deleted first. Riak is not well-behaved after a rough shutdown --

Re: Riak CS race condition at start-up (was: Riak-CS issues when Riak endpoint fails-over to new server)

2017-01-22 Thread Toby Corkindale
rvice riak_kv" command to ensure Riak is ready first: > > > http://docs.basho.com/riak/kv/2.2.0/using/admin/riak-admin/#wait-for-service > > -- > Luke Bakken > Engineer > lbak...@basho.com > > > On Thu, Jan 19, 2017 at 5:38 PM, Toby Corkindale wrote: > > Hi g

Riak CS race condition at start-up (was: Riak-CS issues when Riak endpoint fails-over to new server)

2017-01-19 Thread Toby Corkindale
[normal,[]]}} in application_master:init/4 line 133 2017-01-20 12:23:12.938 [info] <0.7.0> Application riak_cs exited with reason: {error_verifying_props,{riak_cs_app,start,[normal,[]]}} ``` On Fri, 6 Jan 2017 at 12:33 Toby Corkindale wrote: > Hi Shaun, > We've been running Ri

Re: Riak CS - admin keys changing

2017-01-15 Thread Toby Corkindale
-abilities determined by something set in the admin user's data in Riak? Thanks, Toby On Fri, 13 Jan 2017 at 16:38 Toby Corkindale wrote: > Thanks, Luke! > > On Fri, 13 Jan 2017 at 12:10 Luke Bakken wrote: > > Hi Toby, > > When you create the user, the data is

Re: Riak CS - admin keys changing

2017-01-12 Thread Toby Corkindale
Bakken Engineer lbak...@basho.com On Thu, Jan 12, 2017 at 3:47 PM, Toby Corkindale wrote: > Hi, > In Riak CS, the admin key and secret is in the config files for both CS and > Stanchion. > Is that the authoritative location for the secrets, or is the > initially-created admin user

Riak CS - admin keys changing

2017-01-12 Thread Toby Corkindale
Hi, In Riak CS, the admin key and secret is in the config files for both CS and Stanchion. Is that the authoritative location for the secrets, or is the initially-created admin user the source, and those just have to match? I tried to figure this out from the source code, but my Erlang really isn'

Re: Riak-CS issues when Riak endpoint fails-over to new server

2017-01-05 Thread Toby Corkindale
re people with highly-available Riak CS installations > haven't hit the same issues. > > As I mentioned, our customers go with the setup Magnus described. I can't > speak for setups like yours as I've not seen them in the wild. > > Kind Regards, > Shaun > > On

Re: Riak-CS issues when Riak endpoint fails-over to new server

2017-01-04 Thread Toby Corkindale
hat a CS instance will talk to a co-located KV node on the same > host. To achieve high availability, in CS deployments HAProxy is often > deployed in front of the CS nodes. Could you please let me know if this is > an option for your setup? > > Kind Regards, > > Magnus > > >

Re: Riak-CS issues when Riak endpoint fails-over to new server

2017-01-03 Thread Toby Corkindale
On Thu, 22 Dec 2016 at 16:48 Toby Corkindale wrote: > Hi, > We've been seeing some issues with Riak CS for a while in a specific > situation. Maybe you can advise if we're doing something wrong? > > Our setup has redundant haproxy instances in front of a cluster of riak >

Riak-CS issues when Riak endpoint fails-over to new server

2016-12-21 Thread Toby Corkindale
Hi, We've been seeing some issues with Riak CS for a while in a specific situation. Maybe you can advise if we're doing something wrong? Our setup has redundant haproxy instances in front of a cluster of riak nodes, for both HTTP and PBC. The haproxy instances share a floating IP address. Only one

Java client -- slow to shutdown

2016-11-23 Thread Toby Corkindale
Hi, I'm using the Java client via protocol buffers to Riak. (Actually I'm using it via Scala 2.11.8 on OpenJDK 8) After calling client.shutdown(), there is always a delay of 4 seconds before the app actually exits. Why is this, and what can I do about it? To demonstrate the issue, use these files

Riak CS, logging and admin secrets

2016-11-16 Thread Toby Corkindale
Hi, The other day I was concerned when I discovered the riak-cs console.log contained the following: ``` 2016-11-17 16:36:10.639 [warning] <0.155.0>@riak_cs_app:check_admin_creds:76 The admin user's secret is specified. Ignoring. 2016-11-17 16:36:10.680 [info] <0.155.0>@riak_cs_app:fetch_and_cache_

Re: Java client via Scala - StringBinIndex.Name not accessible

2015-12-16 Thread Toby Corkindale
xes().getIndex[StringBinIndex,StringBinIndex.Name](idx).add(contents) } val response = riak.execute(new StoreValue.Builder(obj).withLocation(loc).build) // .. } On Wed, 16 Dec 2015 at 15:10 Toby Corkindale wrote: > Hi, > I'm having a lot of trouble getting a particular par

Re: Java client via Scala - StringBinIndex.Name not accessible

2015-12-15 Thread Toby Corkindale
n Wed, 16 Dec 2015 at 15:34 Toby Corkindale wrote: > Hi Luke, > Thanks for the quick response.. That link does sound like it's describing > the problem. > I'll see if I can figure out a way to get around it :( > The rest of the code works lovely with thin wrappers around th

Re: Java client via Scala - StringBinIndex.Name not accessible

2015-12-15 Thread Toby Corkindale
gt; Not being familar with Java / Scala interop I hope that the discussion > and further links there can help you out. > > -- > Luke Bakken > Engineer > lbak...@basho.com > > > On Tue, Dec 15, 2015 at 8:11 PM, Toby Corkindale wrote: > > Hi, > > I'm havi

Java client via Scala - StringBinIndex.Name not accessible

2015-12-15 Thread Toby Corkindale
Hi, I'm having a lot of trouble getting a particular part of the Riak java client to work when called from Scala. There's a Java class which I'm not allowed to instantiate directly, but must instead call an embedded subclass's static constructor method. ie. import com.basho.riak.client.core.query

Re: Riak in Docker - Error folding keys - incomplete_hint

2015-10-22 Thread Toby Corkindale
Quick follow-up: As a bit of a hack, deleting all the .hint files prior to each start-up does resolve the errors, and immediately results in a whole lot of Bitcask merges happening. But that doesn't strike me as a good long-term fix. On Fri, 23 Oct 2015 at 10:52 Toby Corkindale wrote:

Re: Riak in Docker - Error folding keys - incomplete_hint

2015-10-22 Thread Toby Corkindale
ak > data directory is setup as a Docker volume? > > Other things that come to mind: > > - What OS is the Docker host running? > - What storage driver are you using for Docker? > - What file system is the Docker data directory using? > > -- > Hector > > > On Th

Re: Riak in Docker - Error folding keys - incomplete_hint

2015-10-21 Thread Toby Corkindale
12:23 Toby Corkindale wrote: > Hi, > I've been working on getting Riak to run inside Docker containers - in a > multi-machine cluster. (Previous work I've seen has only run Riak as a > cluster all on the same machine.) > I thought I had it cracked, although I tripped up on

Riak in Docker - Error folding keys - incomplete_hint

2015-10-20 Thread Toby Corkindale
Hi, I've been working on getting Riak to run inside Docker containers - in a multi-machine cluster. (Previous work I've seen has only run Riak as a cluster all on the same machine.) I thought I had it cracked, although I tripped up on the existing issue with Riak and lockfiles[1]. But the nodes ha

Re: Problem with Riak not thinking hostname is fully-qualified

2015-08-12 Thread Toby Corkindale
I found the problem. It was quite annoying. Essentially -- the systems were all valid, but Riak wouldn't recognise the fully-qualified hostnames until I rm -rf /var/lib/riak/* and restarted riak. On Thu, 13 Aug 2015 at 13:52 Toby Corkindale wrote: > I'm trying to set up another clu

Re: Problem with Riak not thinking hostname is fully-qualified

2015-08-12 Thread Toby Corkindale
I should have mentioned, the actual Riak.conf has nodename set to the FQDN. The FQDN resolves correctly in our DNS. On Thu, 13 Aug 2015 at 13:52 Toby Corkindale wrote: > I'm trying to set up another cluster, and I'm hitting problems with Riak > complaining that ** System runn

Problem with Riak not thinking hostname is fully-qualified

2015-08-12 Thread Toby Corkindale
I'm trying to set up another cluster, and I'm hitting problems with Riak complaining that ** System running to use fully qualified hostnames ** ** Hostname db04 is illegal ** However, as far as I can see, the host does have a fully-qualified hostname: root@db04:/# hostname db04 root@db04:/# hostn

Re: is it possible to use FQDN instead of IP in riak.conf for "listener.http.$name" field?

2015-07-27 Thread Toby Corkindale
Hi Roman, I just set the IP to 0.0.0.0 as then it'll bind to whatever IPs are available, even if they change. Obviously you want to make sure your firewall isn't allowing random internet addresses into the Riak ports, though. -Toby On Tue, 21 Jul 2015 at 06:48 ROMAN SHESTAKOV wrote: > > Hello,

Re: haproxy issues with protocol buffers on 2.x releases

2015-07-05 Thread Toby Corkindale
will be getting closed down all the time. My fix is to add these lines: timeout tunnel 7d timeout client-fin 30s But I'd appreciate your thoughts too. -Toby On Mon, 6 Jul 2015 at 13:13 Toby Corkindale wrote: > Hi Matt, > I've tested against both haproxy 1.4 and ha

Re: Riak-CS RAM usage

2015-07-05 Thread Toby Corkindale
Bitcask keeps all the *keys* in RAM. That's like an index in database terms. So even if you have 9Gb of data, you should only have a tiny fraction of that in keys, in memory. So you might have some other problem going on there? Riak uses quite a bit of memory for LevelDB buffers, by default, so m

Re: Bug: Stanchion 1.5 in repo along riak cs 2.0

2015-06-16 Thread Toby Corkindale
erence > > to stanchion 2, only 1.5. > > > > https://packagecloud.io/basho/riak-cs?filter=debs > > > > Not sure who can fix this, but its certainly not aligned with our > > recommendations. > > > > > > On Mon, Jun 8, 2015 at 9:04 PM, Toby Corkindal

Re: Bug: Stanchion 1.5 in repo along riak cs 2.0

2015-06-08 Thread Toby Corkindale
ing home directory `/var/lib/stanchion'. > > Processing triggers for ureadahead (0.100.0-16) ... > > # > > > On Fri, Jun 5, 2015 at 11:40 PM, Matthew Brender > wrote: > >> For others not finding it, the instructions show up here: >> >> https://pack

Re: Riak CS docs have incorrect packagecloud URL

2015-06-08 Thread Toby Corkindale
ted discussing it here [1]. I'd love for you to weigh in. > > > > [1] https://github.com/basho-labs/the-riak-community/issues/49 > > > > Thanks, > > Matt > > > > Matt Brender | Developer Advocacy Lead > > Basho Technologies > > t: @mjbrender >

Re: haproxy issues with protocol buffers on 2.x releases

2015-06-04 Thread Toby Corkindale
CS 1.5 and > 2.0, I think it should work. What's the version the load balancing > working stable? It depends of the reason why connection has been cut, > but I would recommend you restart just the CS node and recreate the > connection pool. > > On Thu, Jun 4, 2015 at 2:33 P

Re: Bug: Stanchion 1.5 in repo along riak cs 2.0

2015-06-04 Thread Toby Corkindale
ore, difference between Stanchion 2.0 and 1.5 is very small, it's > just about configuration file format, and several items. > > On Thu, Jun 4, 2015 at 1:45 PM, Toby Corkindale wrote: > > Hi, > > I think this is a bug? > > > > Riak CS 2.0.1 recommends Stanchion 2.0.0 to

haproxy issues with protocol buffers on 2.x releases

2015-06-03 Thread Toby Corkindale
Hi, I've been happily using haproxy in front of Riak and Riak CS 1.x in production for quite a while. I've been trying to bring up a new cluster based on riak/cs 2.0.x recently, as you've probably noticed from the flurry of emails to this list :) I'm discovering that if I have haproxy sitting bet

Bug: Stanchion 1.5 in repo along riak cs 2.0

2015-06-03 Thread Toby Corkindale
Hi, I think this is a bug? Riak CS 2.0.1 recommends Stanchion 2.0.0 to be installed. However if you follow the instructions to add the Riak CS repo, you get Riak CS 2.0.1, and Stanchion 1.5.0. Toby ___ riak-users mailing list riak-users@lists.basho.com

Riak CS docs have incorrect packagecloud URL

2015-06-02 Thread Toby Corkindale
Hi, Riak CS 2 moved to a new Ubuntu/Debian repository, at packagecloud.io. However attempts to use it as per the documentation result in the message "Incorrect username or token" and an HTTP status of 401. This is because the docs give the URL as similar to: deb https://packagecloud.io/basho/riak_

Riak CS 2 backend config confusion

2015-06-01 Thread Toby Corkindale
Hi I'm in the process of moving from a Riak 1.4.x w/CS installation, to a Riak CS 2 installation, and I'm confused by comments in the riak.conf compared to the documentation online for riak cs, regarding the backend. riak.conf contains the following lines: ## Specifies the storage engine used fo

Re: Ansible galaxy roles only support three-year-old Ubuntu 12.04

2015-05-28 Thread Toby Corkindale
ckets and security settings will fail on everything after the first run, because they already exist. ie. The commands aren't idempotent. Cheers Toby On Thu, 28 May 2015 at 15:41 Toby Corkindale wrote: > Hi guys, > I'm afraid I'm already wearing too many hats to pick up the

Re: Ansible galaxy roles only support three-year-old Ubuntu 12.04

2015-05-27 Thread Toby Corkindale
Hi guys, I'm afraid I'm already wearing too many hats to pick up the role of maintainer of the Ansible-riak repository. However, definitely happy to help out a little. I'll take a look at the dev-2.x branch and see what I have to change to make it run on Ubuntu 14.04. Cheers, Toby > > On 27 May

Ansible galaxy roles only support three-year-old Ubuntu 12.04

2015-05-26 Thread Toby Corkindale
Hi, Searching for riak roles on Ansible Galaxy brings up this one: https://galaxy.ansible.com/list#/roles/777 However this is listed as only supporting RHEL 6 and Ubuntu 12.04. Ubuntu 14.04 is the current long-term-support version, and what we're deploying stuff to; it'd be great if the Ansible r

Prometheus stats gathering for Riak? Riak CS?

2015-03-22 Thread Toby Corkindale
Hi, I wondered if anyone has written a stats gathering plugin for Prometheus? It doesn't look like it'd be too hard to do; but I'm still lazy enough to hope that someone else has done it first :) http://prometheus.io/ Toby ___ riak-users mailing list ri

Re: Riak CS expiration

2015-02-08 Thread Toby Corkindale
On 7 February 2015 at 06:21, Kota Uenishi wrote: > Bitcask doesn't have such interface like hooking anything. As of > internal data structure of Riak CS built on top of Riak, bitcask keys > don't correspond simply to leveldb key. Say, things like this cannot > happen: "all tombstones of blocks of

Re: RiakCS poor s3 upload speeds 2MB/s

2015-01-28 Thread Toby Corkindale
Your cluster will merge more data which should help with disk usage. > If your typical use is to create and delete objects frequently, this > will help. > > -- > Luke Bakken > Engineer > lbak...@basho.com > > On Wed, Jan 21, 2015 at 4:40 AM, Toby Corkindale wrote: &g

Re: RiakCS poor s3 upload speeds 2MB/s

2015-01-21 Thread Toby Corkindale
On 21 January 2015 at 15:22, Luke Bakken wrote: > Hi Toby - > > Are you using the stock bitcask configuration for merging? Hi Luke, Yes, pretty much. > On Tue, Jan 20, 2015 at 5:07 PM, Toby Corkindale wrote: >> Hi Kota, >> I had a bit of an off-list chat about this a w

Re: RiakCS poor s3 upload speeds 2MB/s

2015-01-20 Thread Toby Corkindale
nal guess not a research >>> founded conclusion). The team is (much) smaller then bash (obviously) but >>> they’re a very nice and responsive bunch. I ended up using it as a s3 >>> backend for Project-FiFo due to it’s performance characteristics. With >>> cu

Re: Using apache as a reverse proxy

2015-01-14 Thread Toby Corkindale
Just a thought, but have you tried to replicate the problem when using haproxy (with the example configs provided on basho's site)? That might let you discover if the problem really IS with Apache, or if it's something to do with your Riak config instead. On 15 January 2015 at 11:05, Fred Grim wr

Re: Rick CS MethodNotAllowed

2015-01-14 Thread Toby Corkindale
I'm going to take a punt that you're using the wrong domain name. Eg. you've told riak to use, say, s3.internal as the base, but you're actually sending queries to s3.example.com (or if you are using default settings on some tools, s3.amazonaws.com) If you're using a reverse proxy in front, check

Re: RiakCS poor s3 upload speeds 2MB/s

2014-11-26 Thread Toby Corkindale
et/leofs/ > [2] https://github.com/rlmcpherson/s3gof3r/tree/master/gof3r > --- > Cheers, > Heinz Nikolaus Gies > he...@licenser.net > > > > On Nov 25, 2014, at 6:08, Toby Corkindale wrote: > > Hi, > I wondered if you managed to significantly improve your Riak CS >

Re: RiakCS poor s3 upload speeds 2MB/s

2014-11-24 Thread Toby Corkindale
Hi, I wondered if you managed to significantly improve your Riak CS performance, or not? I just ask as we've been getting not-dissimilar performance out of Riak CS too (4-5 mbyte/sec max per client, on bare metal hardware), for quite a long time. (I swear it was faster originally, when there was a

Re: RiakCS poor s3 upload speeds 2MB/s

2014-11-10 Thread Toby Corkindale
I found that adjusting these RiakCS settings improved write speeds a little for us: {put_concurrency, 4}, {put_buffer_factor, 8}, As well as making sure the Riak vm.args included a high +zdbbl setting. But even so, Riak CS has been really quite slow compared to other

Re: RiakCS poor s3 upload speeds 2MB/s

2014-11-10 Thread Toby Corkindale
I found that adjusting these RiakCS settings improved write speeds a little for us: {put_concurrency, 4}, {put_buffer_factor, 8}, As well as making sure the Riak vm.args included a high +zdbbl setting. But even so, Riak CS has been really quite slow compared to other

Re: [ANN] Riak CS 1.5.1

2014-10-21 Thread Toby Corkindale
but maybe some users will forget) Just a thought, anyway. -Toby On 22 October 2014 00:43, Shunichi Shinohara wrote: > Yes. Riak CS 1.5.1 is tested with Riak 1.4.10. > > Shino > > On Tue, Oct 21, 2014 at 10:34 PM, Toby Corkindale wrote: >> My understanding was that Riak CS wa

Re: [ANN] Riak CS 1.5.1

2014-10-21 Thread Toby Corkindale
iak-cs/install. > Currently only Riak 2.0+ and Riak CS 1.5.1 are hosted there. I've opened an > issue[1] on the docs to get this updated for Riak CS > > [1] https://github.com/basho/basho_docs/issues/1394 > > On Mon, Oct 20, 2014 at 10:40 PM, Toby Corkindale wrote: >

Re: [ANN] Riak CS 1.5.1

2014-10-20 Thread Toby Corkindale
This release hasn't made it through to the Basho Ubuntu repositories yet; is that an oversight or by intent? for REL in precise trusty; do curl http://apt.basho.com/dists/$REL/main/binary-amd64/Packages |grep 1.5.1 done On 27 September 2014 14:56, Kota Uenishi wrote: > Hi all, > > Riak CS 1.5

Re: Brokenness specific to nginx and jets3t with Riak CS

2014-09-25 Thread Toby Corkindale
them to actual slashes before forwarding to Riak CS. Fix was to change proxy_pass http://upstreamservers/; to proxy_pass http://upstreamservers; -Toby On 25 September 2014 16:24, Toby Corkindale wrote: > Quite update to mention that by removing some extra (unneeded) custom > headers from the

Re: Brokenness specific to nginx and jets3t with Riak CS

2014-09-24 Thread Toby Corkindale
52-6d125bb7b27c/coreos_production_ami_hvm.txt On 25 September 2014 15:59, Toby Corkindale wrote: > Hi, > I've hit an issue that only seems to occur when using the Jets3t > client library and Nginx as a load balancer in front of Riak CS. > The issue is NOT present when using other S3 libraries, nor is it

Brokenness specific to nginx and jets3t with Riak CS

2014-09-24 Thread Toby Corkindale
Hi, I've hit an issue that only seems to occur when using the Jets3t client library and Nginx as a load balancer in front of Riak CS. The issue is NOT present when using other S3 libraries, nor is it present if I switch out Nginx for haproxy. Unfortunately, in this instance it is desirable to use

Re: Generic server memsup terminating Mountain Lion

2014-09-09 Thread Toby Corkindale
Are you trying to use Riak CS for file storage, or are you just using Riak and storing 20M against a single key? It's not clear from your email. I ask because if you're in the latter case, it's just not going to work -- I believe the maximum per key is around a single megabyte. On 10 September 20

Re: Riak CS: Undeletable broken buckets

2014-07-08 Thread Toby Corkindale
...]}, > Then create bucket as usual (e.g. s3cmd mb ...). > > These steps solves one of partial update patterns that Andrew mentioned. > > Thanks, > Shino > > On Mon, Jul 7, 2014 at 2:12 PM, Toby Corkindale wrote: >> Hi Andrew, >> Thanks for the details. >&g

Re: Riak CS: Undeletable broken buckets

2014-07-06 Thread Toby Corkindale
. We can probably get you a copy if you want. Just let me > know. And make sure when running in production that allow_mult = true. > > -Andrew > > > > On Sun, Jul 6, 2014 at 9:59 PM, Toby Corkindale wrote: >> >> Hi, >> At some point we've managed to cre

Riak CS: Undeletable broken buckets

2014-07-06 Thread Toby Corkindale
Hi, At some point we've managed to create a couple of buckets that don't work and can't be deleted (in a development/testing cluster, not production). They show up with both 's3cmd ls' or by querying the HTTP API for a user's buckets. However attempting to list files in the bucket, or removing the

Re: Slow write performance for Riak CS

2014-07-03 Thread Toby Corkindale
y, and applied things like filesystem and > scheduler options. > I haven't touched the networking sysctls since they came with a warning > that they should only be messed with if networking *was* the bottleneck. > > Given what I've mentioned so far, do you think it's likely

Re: CS: Query path for storage/access statistics appears wrong?

2014-07-02 Thread Toby Corkindale
the query parameters, in order for it to succeed though. I just assumed that was intentional as the AWS spec does indicate that too. Toby > On Wed, Jul 2, 2014 at 1:23 AM, Toby Corkindale wrote: >> Hi, >> Maybe I'm doing something wrong, but I find I can only access

Re: CS: Query path for storage/access statistics appears wrong?

2014-07-02 Thread Toby Corkindale
for this? > > Thanks, > > Kelly > > On July 1, 2014 at 11:25:00 PM, Toby Corkindale (t...@dryft.net) wrote: > > e=20140602T00Z&s=20140528T00Z -- Turning and turning in the widening gyre The falcon cannot hear the falconer Things fall apart;

CS: Query path for storage/access statistics appears wrong?

2014-07-01 Thread Toby Corkindale
Hi, Maybe I'm doing something wrong, but I find I can only access the Riak CS storage/access information per-user by using the s3-compatible path, not the REST API path -- even though I am connecting to the REST API port. ie. To get the statistics, I am querying http://SERVERNAME:8080/riak-cs/usag

Re: Understanding an intermittent mapreduce error

2014-06-16 Thread Toby Corkindale
sonify_bkeys,2}, {parse_request,1}, {jsonify_pipe_error,2}]], []} ok On 17 June 2014 11:19, Sean Cribbs wrote: > Sorry for the typo, replace 'rr' with 'rp' on the front of that. > > Sean Cribbs > >> On Jun 16, 2014, at 7:59 PM, Toby Corkindale wrote: >&g

riak-admin backup - Backup documentation out of date?

2014-06-16 Thread Toby Corkindale
Hi, The documentation here, apparently the latest, doesn't appear to mention the riak-admin backup command at all, nor is it mentioned on the command-line tools page for riak-admin. http://docs.basho.com/riak/latest/ops/running/backups/ http://docs.basho.com/riak/1.4.9/ops/running/tools/riak/ Is t

Re: Understanding an intermittent mapreduce error

2014-06-16 Thread Toby Corkindale
Toby On 16 June 2014 23:22, Sean Cribbs wrote: > Toby, > > Could you try this? > > 1. Run `riak attach` > 2. Type into the console: rr(rpc:multicall(riak_kv_mapred_json, module_info, > [exports])). > 3. Paste back to me what the result is > > Cheers, > > > On Mo

Understanding an intermittent mapreduce error

2014-06-16 Thread Toby Corkindale
Hi, I'm trying to understand why I'm getting an intermittent error on a particular map-reduce call to Riak 1.4.9. The query isn't particularly complicated, and filters mean it should only be running over ~100 keys, and returning <10 each time, so I don't think I should be hitting any stack or memor

Cost of many PBC connections held open?

2014-06-05 Thread Toby Corkindale
Hi, Just looking for a bit of guidance on good practice when using Riak via the PBC interface. To keep request latency low, it's good to keep the connection open between requests, rather than making a new connection for just a few requests and then dropping it again, right? In the JVM client, poo

Re: Cleaning up bucket after basho_bench run

2014-03-17 Thread Toby Corkindale
Hi Istvan, I posted about this same problem to the list quite a while ago. The problem is that the Erlang API allows arbitrary binary sequences, whereas the PBC (via Java) and HTTP APIs only handle valid Unicode strings. So basho_bench creates keys that can't be manipulated by anything except erlan

Riak CS - POST upload support

2014-03-17 Thread Toby Corkindale
Hi, I wondered if Basho is intending to add support for user POST uploads in Riak CS at any time soon? (ie. As per http://docs.aws.amazon.com/AmazonS3/latest/API/sigv4-UsingHTTPPOST.html ) Cheers, Toby ___ riak-users mailing list riak-users@lists.basho

Re: [ANN] Riak 1.4.8

2014-02-26 Thread Toby Corkindale
ould be back in > business once that the new trees are ready. > > Engel > > > On Tue, Feb 25, 2014 at 11:02 PM, Toby Corkindale wrote: >> >> Hi, >> After upgrading to 1.4.8 in staging, last week, we've been seeing a >> fairly constant extra amount o

Re: [ANN] Riak 1.4.8

2014-02-26 Thread Toby Corkindale
On 27 February 2014 02:04, John Caprice wrote: > Toby, > > Were you running with AAE disabled until 1.4.8 was released? If so, AAE is > likely repairing objects that needed to be read repaired during that time. It was still running for a fair while; the instructions to disable AAE only came thro

Re: [ANN] Riak 1.4.8

2014-02-25 Thread Toby Corkindale
Hi, After upgrading to 1.4.8 in staging, last week, we've been seeing a fairly constant extra amount of riak hits in the stats. Consistently 300-400 a minute. Can get lost in the much higher, varying levels during busy hours, but outside of that it's quite visible as a baseline. The logs show a lot

Re: RIAK 1.4.6 - Mass key deletion

2014-02-20 Thread Toby Corkindale
On 19 February 2014 03:18, Matthew Von-Maszewski wrote: > Riak 2.0 is coming. Hold your mass delete until then. The "bug" is within > Google's original leveldb architecture. Riak 2.0 sneaks around to get the > disk space freed. I'm interested to know what happens if someone deletes a lot of da

Re: Riak crashing with {error,{badmatch,{error,eexist}}} in bitcask:do_put/5 line 1232

2014-02-06 Thread Toby Corkindale
see a number > of 0 byte data files and 18 byte hintfiles with the same file ID > number. Stop the node, move those aside and you should be back in > business. find can be used to automate the process. Another 1.4 > release will be forthcoming to remedy the issue. > >

Riak crashing with {error,{badmatch,{error,eexist}}} in bitcask:do_put/5 line 1232

2014-02-06 Thread Toby Corkindale
Our test environment Riak cluster is looking pretty unhealthy at the moment, with quite a lot of crashes being reported. Is anyone able to advise what the cause is? >From error.log: [error] <0.13413.31> CRASH REPORT Process <0.13413.31> with 0 neighbours exited with reason: no match of right hand

Re: Changing app.config of Live Riak Cluster

2014-01-05 Thread Toby Corkindale
On 06/01/14 05:19, Praveen Baratam wrote: Currently its configured with "*multi_backend*" and two buckets as follows- [snip] Now we want to delete the "*cache*" bucket and add another bucket "*temp*" for another purpose. [snip] I am aware that I can do it using the HTTP Buckets API but can

Re: Keys that won't disappear from indexes

2013-11-06 Thread Toby Corkindale
ct:new(<<"test">>, <<"b">>, <<"c">>)). ok (perfdev@127.0.0.1)21> C:get_index(<<"test">>, Query). {ok,[<<"b">>]} This should get you a list of keys in erlang binary format, which will lo

Re: Keys that won't disappear from indexes

2013-11-05 Thread Toby Corkindale
Nov 5, 2013 at 4:17 PM, Toby Corkindale wrote: Hi Evan, These keys were originally created by basho-bench, using: {key_generator, {int_to_bin, {uniform_int, 1}}}. Of the 10k keys, it seems half could be removed, but not the other half. Now I've tried storing keys with the same key as th

Re: Keys that won't disappear from indexes

2013-11-05 Thread Toby Corkindale
down and a lot of manual effort), would be to write another value that would have the same index, then delete it, which should normally succeed. I'll ask around to see if there is anything that might work better. On Sun, Nov 3, 2013 at 7:40 PM, Toby Corkindale wrote: On 01/11/13 14:04, Tob

Re: Keys that won't disappear from indexes

2013-11-03 Thread Toby Corkindale
On 01/11/13 14:04, Toby Corkindale wrote: Hi, I have around 5000 keys which just won't die. No matter how many times I delete them, they still show up in the 2i $bucket=_ index. Actually attempting to retrieve the keys results in a not-found - even if I've requested that tombstones b

Keys that won't disappear from indexes

2013-10-31 Thread Toby Corkindale
Hi, I have around 5000 keys which just won't die. No matter how many times I delete them, they still show up in the 2i $bucket=_ index. Actually attempting to retreive the keys results in a not-found - even if I've requested that tombstones be returned. I'm interested to know what is going o

Re: Import big data to Riak

2013-10-31 Thread Toby Corkindale
On 30/10/13 02:59, Georgi Ivanov wrote: On Tuesday 29 October 2013 16:50:25 Rune Skou Larsen wrote: Den 29-10-2013 16:32, Georgi Ivanov skrev: Hi and thank you for the reply. My comment follow: Your tests are not close to what you are going to have in production My tests are exactly what we

Re: benchmark results on my $75/month 5 node cluster :)

2013-10-27 Thread Toby Corkindale
Thanks, it's interesting to hear of what performance other people are achieving with Riak. With a not-too-dissimilarly-configured[1] basho_bench test, we see throughput between 6000-7000 ops/sec. get/put latencies are under 2/4ms respectively for mean values, and under 6/12ms for 99th percent

Date implementation in Riak's java vm

2013-10-21 Thread Toby Corkindale
Hi, I've been having some trouble getting the javascript vm in Riak to operate correctly on Date objects. According to this, I should be able to create dates using an ISO8601 date string, or a simplified -MM-DD format. http://www.ecma-international.org/ecma-262/5.1/#sec-15.9.1.15 Howeve

Re: List buckets results in error: bad_utf8_character_code

2013-10-16 Thread Toby Corkindale
On 17/10/13 11:54, Toby Corkindale wrote: If I run curl -i "http://bld-riak/buckets?buckets=true"; I receive this error about a bad utf8 character code, and nothing else. This is Riak version 1.4.2-1 (from the official apt repo) HTTP/1.1 500 Internal Server Error Vary: Accept-Encod

List buckets results in error: bad_utf8_character_code

2013-10-16 Thread Toby Corkindale
If I run curl -i "http://bld-riak/buckets?buckets=true"; I receive this error about a bad utf8 character code, and nothing else. This is Riak version 1.4.2-1 (from the official apt repo) HTTP/1.1 500 Internal Server Error Vary: Accept-Encoding Server: MochiWeb/1.1 WebMachine/1.10.0 (never brea

Unit testing, Riak buckets

2013-10-13 Thread Toby Corkindale
Hi, I'd like to hear how other people are approaching the problem of cleaning Riak buckets up at the end of unit tests for their apps. The problem I have is that multiple tests may be run at once (by different developers or different Jenkins' jobs or even just a parallelised test suite) so I

Re: Invalid hintfiles

2013-09-29 Thread Toby Corkindale
d those we've experienced, so I'll look at getting haproxy used at some point. I've used haproxy in smaller test clusters and it's certainly felt like a better fit. Cheers, Toby On September 18, 2013 at 8:48:27 PM, Toby Corkindale (toby.corkind...@strategicdata.com.au) w

Re: Invoking Python script as part of a M/R job?

2013-09-25 Thread Toby Corkindale
On 26/09/13 14:33, jeffrey k eliasen wrote: I'm trying to do some image processing using OpenCV. Later I'll be doing some video processing as well. In a future project I will be using R to do deep analysis on some data I'm collecting. In all these cases, what I want to do is very simple with exte

Re: Debugging mapreduce

2013-09-24 Thread Toby Corkindale
On 25/09/13 11:20, Charl Matthee wrote: Hi, I am trying to run the following mapreduce query across my cluster: # curl -XPOST http://10.179.229.209:8098/mapred -H "Content-Type: application/json" -d '{"inputs":"tweets", "query":[{"map":{"language":"javascript", "source":"function(value, keyData

Re: Invalid hintfiles

2013-09-18 Thread Toby Corkindale
de of connections from the CS connection pool on startup. For one reason or another the resets don't come through and the hanging disconnected socket isn't discovered until the next send attempt. -Andrew On Wed, Sep 18, 2013 at 10:31 PM, Toby Corkindale mailto:toby.corkind...@strat

Re: Invalid hintfiles

2013-09-18 Thread Toby Corkindale
On 19/09/13 11:17, Luke Bakken wrote: The "error: disconnected" message is a good clue. If you can provide log files that may point to the cause. See below for logs from riak and riak cs, for a roughly five minute window during which I'd fiddled with the load balancer and DNS to try and isola

  1   2   >