Re: Questions about Riak Enterprise
On Tue, May 01, 2012 at 03:49:02PM -0400, Mark Rose wrote: > I've got some questions about Riak Enterprise I haven't been able to find > the answers to. Hi Mark, I'm the riak EDS 'maintainer'. Sorry I didn't reply earlier, I was travelling all week. > I understand that the open source version of Riak's replication is designed > for single data center usage only, but I'm unsure about how Riak Entreprise > handles replication. Specifically, I'm curious about locality and high > availability. > > Our setup is already running in multiple availability zones on EC2. We're > running Galera across the zones to provide both redundancy and a local copy > of the data to avoid the network latency of going to another zone. However, > Galera, as nice as it is, doesn't scale writes. We're going to be using > Riak to store a lot of information going forward, and may eventually move > our existing data to it as well. > > The only thing holding us back from going to multiple regions on Amazon is > our datastore. > > How well does Riak handle layered topologies, such as EC2? > > Is it possible to configure Riak Enterprise to store two copies of the data > in each EC2 region, ensuring that the two copies are in different zones > when there are more than one Riak servers in a zone? Current EDS replication is pretty simple, it will just try to (eventually) ensure that data on one cluster is mirrored on another. It won't forward reads and riak doesn't have anything like 'rack awareness', at least not yet. > When a query is run, is it run in one region only? Would Riak prefer copies > of the data in the local zone? Riak only queries the local cluster, yes. > > For what it's worth, our current datastore load is roughly half and half > writes and reads. We heavily cache reads with memcache (99%). We may drop > memcache if reads on Riak prove fast enough (thus avoiding the issues of > invalidating remote caches). Given the current limitations, you'd probably be best off with N clusters in different regions and/or zones. Don't try to span a single cluster across a zone, or even worse, a region. Then hook them together with replication. There's also some fun with NAT on EC2, but it can be made to work. Let me know if that helps, Andrew ___ riak-users mailing list riak-users@lists.basho.com http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Re: Questions about Riak Enterprise
On Tue, May 08, 2012 at 10:59:19AM -0400, Mark Rose wrote: > > Approximately how fast is "eventually"? Is it based on the network > bandwidth/latency between the clusters or is there additional latency? > Basically, would I expect more or less than a second if the two clusters > were 250 ms away? Depends on the write speed and the bandwidth between clusters, EDS will try to immediately forward a write but there's no gurantees on latency. If the write doesn't make it, a periodic 'fullsync' should make it happen. > Also, how does EDS work with regards to writes to the same key in both > clusters? Does it use a second level of vector clocks or is the vector > clock shared? I imagine last-one-wins wouldn't work in such a setup. Is it > standard practice to send all the writes to a single cluster? A write on both sides is a conflict, plain and simple. You'll either get a sibling or get stuck with last-write-wins. Obviously last-write-wins isn't ideal so you'd probably want to think about resolving siblings yourself. > Does the approximately 1 ms of latency between av zones affect Riak's > performance that much? If the latency is *guranteed* to be that low, then you should be ok, although I'm not sure how the networking works across zones. If the latency can do crazy things in outage conditions, you'll stand a decent chance of screwing the cluster. A downed node is better than a really, really slow one. > We were planning to run across av zones for fault tolerance, just beefing > up single nodes for the moment until rack awareness is available. So the > recommended solution is to use EDS to accomplish this? I'm not sure what you're describing here. > We currently using "role based" host names. On boot, the Riak machine sets > a CNAME to its public DNS name. > > riak1.us-east-1a.example.net -> ec2-203-0-113-239.compute-1.amazonaws.com > riak1.us-east-1b.example.net -> ec2-198-51-100-13.compute-1.amazonaws.com > riak1.us-east-1c.example.net -> ec2-192-0-2-144.compute-1.amazonaws.com > > and the cluster members would be r...@riak1.us-east-1a.example.net, > r...@riak1.us-east-1a.example.net, and r...@riak1.us-east-1c.example.net. > > The names resolve to either the private or public IP of the machine > depending on whether the machine is in the same region or not. We haven't > tried running across regions yet... so I gather this might confuse > Riak/Erlang? Actually, its worse than that because of some legacy behaviour. EDS wants to know the bind IP, not a hostname, and it will exchange node IPs with the other side of the connection, so internal IPs can 'leak' to the other cluster and cause connection problems. There is a workaround for this, and I do plan to address it. Andrew ___ riak-users mailing list riak-users@lists.basho.com http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Re: Questions about Riak Enterprise
On Thu, May 10, 2012 at 11:09:36AM -0700, Ahmed Bashir wrote: > Hey Andrew, > > Can you elaborate on how EDS replication does this mirroring? Does > each vnode have the ability to connect to the other cluster, or is > there a coordinator that sends data to the other cluster, etc? There is a coordinator. Andrew ___ riak-users mailing list riak-users@lists.basho.com http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Re: Questions about Riak Enterprise
On Wed, May 09, 2012 at 04:35:06PM +, Elias Levy wrote: > That's problematic. We will be rolling out an EDS implementation in the > near future. One cluster will be in metal, but we are likely to place the > other on EC2, and we'd use elastic IPs and split horizon DNS. > > When is this expected to be fixed? Its on my shortlist. I'm not sure it'll make 1.2 though, given my current workload. We should be able to provide a workaround that will simply cause a delay in connecting the clusters but not have any other harmful effects. Andrew ___ riak-users mailing list riak-users@lists.basho.com http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Re: beam.smp idle cpu usage.
On Thu, May 24, 2012 at 08:28:26PM +1000, hankin wrote: > I installed riak v1.1.2 via the 64bit deb package a few days ago. > > I have done almost nothing with it, there is only one server. But beam.smp > has already used up 15 mins of CPU according to 'top'. > When I watch 'top', beam.smp seems to be using up 0.01 second of the cpu > every second. This doesn't seem to happen with other databases. Riak does several things periodically, including gossip the ring, checking for handoff and checking for fallback vnodes with data in them. Basicially this is riak trying to keep your data safe. Maybe other databases don't do this, but they also may not have the same data resiliancy that Riak does. Andrew ___ riak-users mailing list riak-users@lists.basho.com http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Re: upgration from riak 0.14.2 to riak 1.2
On Fri, Aug 17, 2012 at 07:25:57PM +0530, raghwani sohil wrote: > Hi , > > I have riak 0.14.2 install on my server . I have five node riak cluster > and i want to upgrade it to riak 1.2 . So i want information about what all > things i need to take care of while upgrading it to riak 1.2 . > We only support upgrading from one of the last two major releases, in this case 1.0 and 1.1. If you want to go from 0.14.2 you should jump to 1.1 and from there to 1.2. The release notes contain some notes on upgrading for each release. Andrew ___ riak-users mailing list riak-users@lists.basho.com http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Re: Node cannot join
On Tue, Aug 21, 2012 at 05:00:31PM +0100, Daniel Iwan wrote: > Thanks Ryan! > > I'll give it a try. Where I can find details about it, parameters options > etc.? Just type riak-admin cluster and it will print the help message. Andrew ___ riak-users mailing list riak-users@lists.basho.com http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
[ANN] Lager 1.2.1 and backend API changes happening on master
I just tagged lager 1.2.1, the last 'stable' release for a while. I know I never actually *announced* 1.2.0 (or maybe even 1.0.0 for that matter) but this one is more important as it is probably the tag you want to track in your applications for a while. After tagging 1.2.1 I merged a branch to master that breaks the backend API that lager backends had been using, specifically the way log messages are sent to the backends. Any 3rd party backends (including lager_syslog) will currently be broken if you use them against master instead of 1.2.1. They will either need have their lager dep pinned to 1.2.1 or updated to reflect the new API. I apologize for the inconvienience, but it was necessary to allow some future feature development. Andrew ___ riak-users mailing list riak-users@lists.basho.com http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Re: Efficiency of RIAK
Are you taking into account that the default N value for riak is 3, so every put you do is done 3 times in riak, but only once in standalone leveldb? Andrew ___ riak-users mailing list riak-users@lists.basho.com http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Re: Efficiency of RIAK
My bad, you're setting n_val to 1, missed that the first time. Also bear in mind, however, that riak has more work to do around a PUT because it has to read the old vector clock for a key (if any) and increment it, while the eldb basho_bench driver just does a PUT with no reads/vclock magic. Andrew ___ riak-users mailing list riak-users@lists.basho.com http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Re: log configuration not effective.
Look under 'lager' in app.config for the logfile paths. Andrew ___ riak-users mailing list riak-users@lists.basho.com http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Re: Riak packaging for RPM-based distros
To amplify Jared's reply a little... The packaging choices we have made are with good reason. I'll try to counter the most important points in your email below: - Erlang VM upgrade. Erlang VM, used by Riak installed, can be upgraded only with Riak code; This is actually what we want. Each Riak release is tested with a specific Erlang release, and on occasion we've shipped a patched VM to work around bugs that have not yet been fixed in the upstream Erlang distribution. There's also been cases where particular Erlang releases have security bugs (eg. SSL validation code not being called, without error). Additionally, if Riak did force an exact Erlang version, that could hold up other Erlang packages that use new Erlang VM features. At one point Homebrew for OSX was having a lot of trouble because they could not upgrade their Erlang package recipe without breaking Riak, but other Homebrew packages would break *without* the Erlang update. Erlang itself can sometimes introduce backwards and (forwards incompatible) changes (eg. removing system stats, adding arguments that error on older version, etc). - integration with 3d party Erlang code. I mean filesystem hierarchy usage Riak explicitly discourages you running other Erlang code on a Riak node. We recommend that you stand up a separate Erlang node and run your custom code there. The exception to this rule is custom map/reduce code, which has a well defined way of loading 3rd party Erlang code. - this "release" uses its own location for Erlang applications; Again, by design. We sometimes ship with our own forks of common Erlang libraries (folsom, poolboy, protobuffs, etc) which may be incompatible with upstream, or just include some fixes that we were not able to merge upstream by the release window. If Riak was forced to use, for example, the 'system' folsom, users could see performance degradation or Riak crashes as a result. Also recall that Erlang lacks a decent way to require a certain version of an application, unlike something like rubygems. - package each Riak-related Erlang application in separate RPM/DEB etc; This is a LOT of work. We only have a single release manager and cutting a riak release is already a non-trivial task. Having to create RPMs for every single library that composes riak would only add complexity. Again, this would also make it hard for ALL Erlang applications, because of the inability to 'require' a certain version of an Erlang application. For example, if 2 Erlang applications were installed, but one depended on lager 1.2 and one on lager 2.0 (which are incompatible versions), and lager was installed to /usr/lib/erlang/lib (or wherever), one application would be unusuable. Erlang's 3rd party libraries are, in general, in quite a bit of flux, and this kind of system wide dependency management would be a nightmare. Also, remember that we don't only support RPM and DEB based systems, we'd have to create 30+ packages for Solaris, FreeBSD, SmartOS, etc as well. So the added complexity is: NumberOfApplications * NumberOfSupportedPlatforms Which, in my quick estimation, is something like 150+ packages. Andrew ___ riak-users mailing list riak-users@lists.basho.com http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Re: Lager issue
On Thu, Jan 10, 2013 at 01:11:18PM -0500, John Kemp wrote:
> Hi,
>
> I don't usually have issues with lager, but in logging for my application, I
> feel like I've done all the right things:
>
> i) Put lager_transform in my rebar config:
>
> {erl_opts, [debug_info, warnings_as_errors, {parse_transform,
> lager_transform}]}.
>
> ii) Lager is started when my app starts, and I know by asking at the console:
>
> (lilith@127.0.0.1)3> application:which_applications().
> [{lilith,[],"1"},
> {lager,"Erlang logging framework","0.9.4"},
> {crypto,"CRYPTO version 2","2.2"},
> {sasl,"SASL CXC 138 11","2.2.1"},
> {stdlib,"ERTS CXC 138 10","1.18.3"},
> {kernel,"ERTS CXC 138 10","2.15.3"}]
>
> Then:
>
> (lilith@127.0.0.1)4> application:start(lager).
> {error,{already_started,lager}}
>
> And yet...
>
> (lilith@127.0.0.1)5> lager:info("foo ~s", [bar]).
> ** exception error: undefined function lager:info/2
> (lilith@127.0.0.1)6> lager:error("foo ~s", [bar]).
> ** exception error: undefined function lager:error/2
> (lilith@127.0.0.1)7> lager:warning("foo ~s", [bar]).
> ** exception error: undefined function lager:warning/2
> (lilith@127.0.0.1)8> lager:info("Hello!").
> ** exception error: undefined function lager:info/1
>
> Can anyone tell me why lager continues to give me 'undef'?
>
Lager doesn't actually define lager:info/3, the parse transform walks
the AST of the module and replaces any calls to
lager:(debug|info|notice|...) with some different code. So, since you're
calling lager from the shell, the parse transform isn't involved and you
get undef. If you want to log from the shell, you can use use
lager:log(info, self(), "foo ~s", [bar]), or whatever.
Andrew
___
riak-users mailing list
riak-users@lists.basho.com
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Re: Understanding read_repairs
On Fri, Feb 22, 2013 at 11:56:31AM -0700, Jared Morrow wrote: >- stop each node and restart them one at a time You can actually just drop a file in basho-patches and do a l(module) at the riak shell (via riak attach). You can then do the code:which trick to check it worked. You should be aware of the rules of when erlang switches to the 'new' code, as the new code may not immediately take effect. Andrew ___ riak-users mailing list riak-users@lists.basho.com http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Re: Error interpretation
Riak does not log crashed pre/post commit hooks by default, this is to
avoid a bad hook or object from denial of servicing the cluster in
production.
If you want to debug this, you can use a lager trace at the riak
console like this:
lager:trace_file("/tmp/put_fsm.log", [{module, riak_kv_put_fsm}], debug).
Then do whatever operation is failing and look at the /tmp/put_fsm.log
file. When you want to disable the trace simply do:
lager:clear_all_traces().
Andrew
___
riak-users mailing list
riak-users@lists.basho.com
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
[ANN] Lager 2.0.0RC2
So, I announced lager 2.0.0rc1 during my Erlang factory talk ( http://www.youtube.com/watch?v=8BNpOHFvg_Q ), but forgot to announce it to the relevant mailing lists. So I'm announcing RC2 here. What is lager? Lager (as in the beer) is a logging framework for Erlang. Its purpose is to provide a more traditional way to perform logging in an erlang application that plays nicely with traditional UNIX logging tools like logrotate and syslog. https://github.com/basho/lager Changes since lager 1.2.2: * Metadata is passed to lager backends (breaks compatability with old * backends) * Switch from Mochiglobal to ETS for global 'config' storage * Add 'application' to default trace metadata, captured at compile time * Support for 'pretty printing' Cowboy (0.8+) and WebMachine errors * Record pretty printing with lager:pr/2 * Colored console support (requires R16, thanks DeadZen) * Overload protection from error_logger storms * Many performance improvements (often more than 2x as fast) * Many minor bug fixes Changes since lager 2.0.0rc1 * Support for logging unicode to files * New tracing engine, based on goldrush (thanks DeadZen) * Updated documentation for 2.0 Please give it a try, and let me know if you have any problems. Lager 2.0.0 final will ship with Riak 1.4. Andrew ___ riak-users mailing list riak-users@lists.basho.com http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Re: Unable to compile Riak from source (Ubuntu precise 32bit)
Ubuntu does some braindead splitting up of the erlang standard library (so the standard library is no longer guranteed to be 'standard'). You'll probably need to install erlang-syntax-tools or something. Andrew ___ riak-users mailing list riak-users@lists.basho.com http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Re: Riak does not work on ec2
By default AWS machines use NAT, they provide a temporary external IP that is NATed through to the internal IP applied to the AWS instance. You can also permanantly assign an 'elastic IP address' if you want a particular instance to always be reachable on a specific IP. http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/using-instance-addressing.html When you put that public IP in the riak config file, it makes no sense to Riak because that IP isn't applied to any interface it knows about. When you put the private IP in, it works, but you can't access it. This is probably because AWS is dropping inbound traffic by default: http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/using-network-security.html#default-security-group This is not a Riak problem, it is an issue with configuring AWS. Andrew ___ riak-users mailing list riak-users@lists.basho.com http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Re: Migration from memcachedb to riak
On Wed, Jul 10, 2013 at 08:19:23AM -0700, Howard Chu wrote: > If you only need a pure key/value store, you should consider > memcacheDB using LMDB as its backing store. It's far faster than > memcacheDB using BerkeleyDB. > http://symas.com/mdb/memcache/ > > I doubt LevelDB accessed through any interpreted language will be > anywhere near its performance either, though I haven't tested. (Is > there a LevelDB backend for modular memcache yet?) Except that the comparison was not memcache vs leveldb, it was memcache vs Riak. Yes, Riak does impose some overhead, but on the other hand, I don't see anything about scaling LMDB to more nodes as your dataset grows, which was one of the goals listed in the original email. Also, if you think Erlang is an intepreted language, you probably need to look again. Andrew ___ riak-users mailing list riak-users@lists.basho.com http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Re: Quickly deleting + recreating item in Riak deletes new item
See this old post: http://lists.basho.com/pipermail/riak-users_lists.basho.com/2011-June/004601.html Effectively, you should be doing a GET before ANY PUT, and using the deletedvclock option if you're using PB or checking for the X-Riak-VClock header on the 404 response. If you get back a tombstone, you should be doing the new PUT with that tombstone so you don't accidentally create a sibling with that tombstone. Alternatively, turn off allow_mult=false, so you can resolve the siblings yourself. Andrew ___ riak-users mailing list riak-users@lists.basho.com http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Re: /usr/lib/riak/erts-5.9.1/bin/epmd -daemon
On Wed, Jul 17, 2013 at 11:23:03AM -0400, Patrick Durusau wrote: > -BEGIN PGP SIGNED MESSAGE- > Hash: SHA1 > > Greetings! > > I am at the start of configuring a multi-node development environment > with Riak 1.4 on Ubuntu 12.04. Riak was installed using apt-get. > > I stopped the one node test of Riak with sudo riak stop but > ps -ef | grep erlang shows: > > /usr/lib/riak/erts-5.9.1/bin/epdm -daemon (owner of the process is riak) > > after I ran the stop command. > > Is this expected behaviour? > > Should I use kill to stop the daemon? > No, you can leave epmd alone, it is started the first time a distibuted erlang node is started, but will survive that node's death. It is used to advertise what ports distributed erlang is using on a machine, so erlang nodes can communicate. Andrew ___ riak-users mailing list riak-users@lists.basho.com http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Re: riak 1.4.0 upgrade failed
On Fri, Jul 19, 2013 at 09:53:02AM -0700, kzhang wrote: > Thanks! > > install went through. > # sudo rpm -Uvh riak-1.4.0-1.el6.x86_64.rpm > Preparing...### > > [100%] >1:riak warning: /etc/riak/app.config creat > > ed as /etc/riak/app.config.rpmnew > warning: /etc/riak/vm.args created as /etc/riak/vm.args.rpmnew > ### [100%] > > but failed on start. > # riak start > riak failed to start within 15 seconds, > see the output of 'riak console' for more information. > If you want to wait longer, set the environment variable > WAIT_FOR_ERLANG to the number of seconds to wait. > It looks like riak is still running as well, can you kill it and try to start it again? Andrew ___ riak-users mailing list riak-users@lists.basho.com http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Re: Dangling keys/objects after a batch of sequential inserts (for going on 3 days)
You should be doing a GET before the PUT with the deletedvclock option. Here is an example I used to originally test (and fix) this bug: https://gist.github.com/Vagabond/965376#file-delete_bug-erl-L48 Sadly, because of backwards compatability issues, it was not possible to default deletedvclock to be 'on'. Perhaps enough time has passed to revisit that now. All the client libraries should understand the message now. Andrew ___ riak-users mailing list riak-users@lists.basho.com http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Re: Riak not binding to anything but localhost
Are you trying to apply an external IP to riak, which may actually be NATted in from outside? This is very common on AWS: http://lists.basho.com/pipermail/riak-users_lists.basho.com/2013-June/012318.html Andrew ___ riak-users mailing list riak-users@lists.basho.com http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Re: Riak not binding to anything but localhost
So what does ifconfig -a (or whatever the heck the modern linux equivalent is) report for the local interfaces? Is the IP you're trying to assign present? Andrew ___ riak-users mailing list riak-users@lists.basho.com http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Re: Riak not binding to anything but localhost
Ok, so why, in your gist, is the HTTP IP set to [[IP]], at first I thought you were anonymizing IPs, but I just noticed that you set the PB IP correctly? Andrew ___ riak-users mailing list riak-users@lists.basho.com http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Re: Riak not binding to anything but localhost
On Wed, Jul 24, 2013 at 05:16:03PM +0100, John Le Drew wrote: > Sorry, that was just a mistake. I will correct it, my vm.args and > app.config were generated from a shell script and that was replaces > out, when I made my gist I copied the contents of the templates rather > than a generated file by mistake. > But the error.log snippet mentioned the same [[IP]] thing: https://gist.github.com/antz29/f159b7e5400af22dacd8#file-error-log Andrew ___ riak-users mailing list riak-users@lists.basho.com http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
RFC: Adding security to Riak
As an effort to get some community feedback (and maybe even some contributions) and to give people a heads up, I'd like to present my RFC on adding security to Riak as part of the next major release: https://github.com/basho/riak/issues/355 Feel free to provide any input, preferably via github, but here is OK too. Andrew ___ riak-users mailing list riak-users@lists.basho.com http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Re: Data Replica algortihm
On Mon, Aug 19, 2013 at 04:03:25PM -0300, João Machado wrote:
> Hi all,
>
> Anyone knows how the algorithm responsible for choosing where the data
> replica is stored works?
>
> Is possible to customise (or configure) it to ensure that the data will be
> in different physical nodes (or even physical racks) like Cassandra does?
Riak writes the replicas to the vnode to which the {bucket, key} hashes
and the following N-1 vnodes.
Riak uses target_n (which defaults to 4) to determine what the largest N
expected is, it then tries to lay out the vnodes that no
sequence of vnodes are on the same physical nodes. Here's an example 4
node cluster with the default target_n of 4:
(dev1@127.0.0.1)5> riak_core_ring:pretty_print(element(2,
riak_core_ring_manager:get_my_ring()), [legend]).
Nodes
Node a: 16 ( 25.0%) dev1@127.0.0.1
Node b: 16 ( 25.0%) dev2@127.0.0.1
Node c: 16 ( 25.0%) dev3@127.0.0.1
Node d: 16 ( 25.0%) dev4@127.0.0.1
Ring
abcd|abcd|abcd|abcd|abcd|abcd|abcd|abcd|abcd|abcd|abcd|abcd|abcd|abcd|abcd|abcd|
ok
As you can see, there are no sequences where any 2 vnodes in a 4 vnode
sequence are on the same machine. Now look at what happens when # of
nodes < target_n:
Nodes
Node a: 22 ( 34.4%) dev2@127.0.0.1
Node b: 21 ( 32.8%) dev3@127.0.0.1
Node c: 21 ( 32.8%) dev4@127.0.0.1
Ring =
abca|bcab|cabc|abca|bcab|cabc|abca|bcab|cabc|abca|bcab|cabc|abca|bcab|cabc|abca|
Oops, now we have nodes appearing twice in the list of every 4 vnodes.
This is bad because it means that if you loose one node, and N=4, you
have a chance to loose 2 of your replicas, instead of one.
Rack awareness is something that is being worked on, but we don't have
it just yet, but as long as you have more than target_n nodes, Riak will
try pretty hard to balance your replicas across physical machines. This
is also why we don't recommend riak clusters smaller than 5 nodes
(satisfy target_n and one more machine so you can tolerate 2/5 machines
being down).
Andrew
___
riak-users mailing list
riak-users@lists.basho.com
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Re: Missing keys in bucket?
Tombstones are still riak objects. Do the 404 responses you get back have an X-Riak-Vclock header? Andrew ___ riak-users mailing list riak-users@lists.basho.com http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Re: Connection attempt from disallowed node
On Fri, Oct 11, 2013 at 07:32:28PM +0530, raghwani sohil wrote: > Hi , > > I have three node riak(0.14.2) cluster on production(all nodes running on > RHEL OS). > > I have one CentOS system on which one riak node(0.14.2) is setup using RHEL > binary. > > When I m trying add CentOS riak node to existing cluster then I am getting > > error *Node r...@a.b.c.xxx is not reachable! *and in sasl-error.log file > it's showing > > *Connection attempt from disallowed node 'riak@**a.d.c.YYY**'* > * > * > can any one help on the same? Do the cookies from the vm.args file line up for all nodes? Andrew ___ riak-users mailing list riak-users@lists.basho.com http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Re: Riak occasionally not storing objects
On Wed, Oct 23, 2013 at 02:36:11PM -0400, David Greenstein wrote: > > Hello All, > > I have a Riak cluster with 5 nodes on AWS m1.large instances, all using EBS > deadline scheduling. The cluster is accessed via node.js and the riak-js > library using the HTTP protocol. I’ve tried it with and without connection > pooling. All software has been updated to the latest versions. > > Occasionally, riak seems to not store an object I try to save. I have run > tcpdump on the node receiving the request to ensure it is receiving the http > packets with the correct JSON from the client. When the issue occurs the node > is in fact receiving the request with the correct JSON. > > Has anyone else experienced an issue like this? Any hints on how to debug the > situation would be greatly appreciated. What are your read and write quorums, and does the write report that it suceeded? Andrew ___ riak-users mailing list riak-users@lists.basho.com http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Re: Riak occasionally not storing objects
On Wed, Oct 23, 2013 at 04:57:12PM -0400, David Greenstein wrote: > > One other thing that might be worth mentioning here is the writes I’m > mentioning are actually updates to existing objects. The object exists, an > attempt to write an update for the object appears to be received by a node, > but the object maintains it’s original value. > > Any help you can provide to debug this would be greatly appreciated. thank > you! Ok, that changes things a lot. You're running with allow_mult=false, so Riak will automatically resolve siblings for you if you create them. Are you doing 'get before put' for your updates? That is, doing the PUT with the vclock of the old value, or are you doing 'new' puts with a blank vector clock? Andrew ___ riak-users mailing list riak-users@lists.basho.com http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Re: Riak 2.0 Yokozuna performance
On Wed, Mar 19, 2014 at 04:15:44PM +0800, Andrey Anpilogov wrote: > Hi, > > I've been playing with Riak 2.0 and found some strange performance drop > with new Search system. > Run two Riak nodes on E3 machines with 32GB RAM and SSD drives. Here's your first problem. The default riak config is not designed to run on a 2 node cluster (I believe the recommendation is 5). If you do want to do this, make sure your N (and target_n) are set to 2. That should at least prevent you doing 2 writes to each machine. Andrew ___ riak-users mailing list riak-users@lists.basho.com http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Re: erlang_v8
Speaking as a Basho engineer, not Basho engineering, I'd much rather see the ability to run M/R 'workers' in whatever language you want and just have Riak M/R submit map and reduce jobs to them. I don't know if/when that will ever happen, though. Andrew ___ riak-users mailing list riak-users@lists.basho.com http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Re: Erlang R14B02
On Mon, Apr 04, 2011 at 12:54:05PM -0600, Jason J. W. Williams wrote: > Any word on R14B02 support in Riak? Anxious to get it running again in > Homebrew. > > -J > It doesn't build out of the box if you use latest git? I thought I'd fixed the R12B02 issues. Andrew ___ riak-users mailing list riak-users@lists.basho.com http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Re: Riak pb client build problem from tip
On Wed, Apr 06, 2011 at 01:50:24PM -0700, Mike Oxford wrote:
> Riak's Erlang pb client rebar.config was updated to use protobuffs 0.6 but
> the protobuffs repo itself still only has 0.5.1
>
>{expected,"0.6.*"},
>{has,"0.5.1"}}}.
>
> Just a heads-up for whomever is coordinating those internally. :)
>
I just double-checked with a fresh checkout and it worked. Try deleting
your deps directory and runing make again. Rebar isn't great at updating
dependancies it seems.
Andrew
___
riak-users mailing list
riak-users@lists.basho.com
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Re: PBC API: Generating key for an object
On Fri, Apr 08, 2011 at 02:15:39AM -0700, Ishwar wrote: > Riak folks, > > I see that storing an object without a key is supported in the HTTP > APIs(http://wiki.basho.com/REST-API.html#Store-a-new-object-without-a-key), > but not? in the PBC APIs(http://wiki.basho.com/PBC-API.html) or the python > client(http://basho.github.com/riak-python-client/). > > Is there a reason why this is so? > Its implemented in git as of yesterday: https://github.com/basho/riak_kv/pull/66 https://github.com/basho/riak-erlang-client/pull/20 It will be in the next release. Andrew ___ riak-users mailing list riak-users@lists.basho.com http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Re: "Failed to compact" in RiakSearch
On Wed, Apr 13, 2011 at 05:06:58PM +0200, Morten Siebuhr wrote: > Any ideas about what's going on? > > (We're running on a four-node cluster with NFS-based storage.) > I just have to know.. why is NFS involved? Andrew ___ riak-users mailing list riak-users@lists.basho.com http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Re: building on FreeBSD?
I actually have a couple pull requests open to add support for all the BSDs for riak. You're welcome to contribute to the effort. https://github.com/basho/erlang_js/pull/11 https://github.com/basho/skerl/pull/5 Piotr actually has a better fix for skerl on OpenBSD here https://github.com/basho/skerl/pull/6 We do need to solve the make/gmake issue in erlang_js (its hardcoded to make in rebar.config). I was thinking a little wrapper script that would use 'gmake' if available and fall back on 'make' if its not. Andrew ___ riak-users mailing list riak-users@lists.basho.com http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Re: building on FreeBSD?
On Thu, Apr 14, 2011 at 09:34:26AM -0700, David Leimbach wrote: > 1. GNUMakefile instead of Makefile This would work if we added a regular Makefile that invoked GNU make because GNU make looks for GNUMakefile first. > 2. Allow overriding the configuration Rebar doesn't support this, to my knowledge. > 3. Platform specific include files for configuration based on uname. Not sure how this would work. > I've only just started bumping into this problem, so I'm sure you guys are a > bit ahead of me :-). I think you're about as far in as I am at this point, so not really. > My biggest concern is getting the changes upstream so a port is not > necessary. Or at least make it so a port is trivial to create without a bunch of custom patching. I'm going to spend 5 minutes and try the GNUMakefile trick and see if it works. Andrew ___ riak-users mailing list riak-users@lists.basho.com http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Re: building on FreeBSD?
It works! https://github.com/basho/erlang_js/commit/25d9edfbe1956ece87ed20454b1e2ed53ee3881d This makes erlang_js run the right make flavor on both BSD and linux. Thanks a lot for the suggestion, I think this neatly solves the problem. Andrew ___ riak-users mailing list riak-users@lists.basho.com http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Re: building on FreeBSD?
On Thu, Apr 14, 2011 at 10:44:53AM -0700, David Leimbach wrote: > That's great! Does it still work on Linux too though? :-) I'm having one > of those interrupt heavy days, and all the context switching is making it > difficult to focus on this task :-) > Yes it still works on linux (this was the whole point of the exercise). Andrew ___ riak-users mailing list riak-users@lists.basho.com http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Re: building on FreeBSD?
On Thu, Apr 14, 2011 at 01:29:49PM -0700, David Leimbach wrote: > I've checked out the branch for the last riak release, made a few > modifications to the downloaded rebar.conf files for riak_kv, and one > adjustment to the erlang_js/ebin/.app file (forget whole path), and it looks > like I now have a tarball I can build on FreeBSD, and run some very basic > tests against. > > Pretty cool :-) > What did you have to change in the rebar.conf and in the .app file? Andrew ___ riak-users mailing list riak-users@lists.basho.com http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Re: building on FreeBSD?
On Thu, Apr 14, 2011 at 02:09:35PM -0700, David Leimbach wrote: > And now it builds. Oh yeah, just switching to my branch. I thought you had to patch something. > > To get all the bash scripts to run I hunted down all the instances of: > #!/bin/bash > > and made them > > #!/usr/bin/env bash > > I just don't agree with making symlinks on my system. I also think that if > these scripts could be run with POSIX sh it'd be better to make them > #!/bin/sh in the long run. They actually do run under sh (from what I've tested). I think if we're sure they work even on solaris' sh then we should switch them over, there's no reason to have a bash dependancy. Andrew ___ riak-users mailing list riak-users@lists.basho.com http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Re: Build from source following tutorial
On Fri, Apr 15, 2011 at 09:26:18AM -0700, Shane Adams wrote: > ==> riak_kv (compile) > src/riak_client.erl:278: call to put/2 will call erlang:put/2; not put/2 in > this module (add an explicit module name to the call to avoid this error) > src/riak_client.erl:309: call to put/2 will call erlang:put/2; not put/2 in > this module (add an explicit module name to the call to avoid this error) > src/riak_client.erl:320: call to put/2 will call erlang:put/2; not put/2 in > this module (add an explicit module name to the call to avoid this error) > src/riak_client.erl:332: call to put/2 will call erlang:put/2; not put/2 in > this module (add an explicit module name to the call to avoid this error) > src/riak_client.erl:345: call to put/2 will call erlang:put/2; not put/2 in > this module (add an explicit module name to the call to avoid this error) > make: *** [rel] Error 1 > > > Get this on Debian. ?Didn't find anything on google > What erlang release is this? Have you tried building erlang from source? Andrew ___ riak-users mailing list riak-users@lists.basho.com http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Re: Build from source following tutorial
On Fri, Apr 15, 2011 at 10:14:44AM -0700, Shane Adams wrote: > Hey Andrew - > > Thank you for replying. > > > Erlang R13B04 (erts-5.7.5) [source] [rq:1] [async-threads:0] [hipe] > [kernel-poll:false] > > Eshell V5.7.5 ?(abort with ^G) > 1> > > I tried building from the source but got a compilation error so installed via > apt-get. ? > I suspect debian is doing its usual thing of screwing with packages and turning on some option that isn't standard. What was the compilation error you had? I'd be interested to know if this happens with R13B04 from source (so we can truly pin the blame on debian). Andrew ___ riak-users mailing list riak-users@lists.basho.com http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Re: Build from source following tutorial
On Fri, Apr 15, 2011 at 03:57:43PM -0700, Shane Adams wrote: > I followed the instructions on the website which indicates that I should be > able to do this > > http://wiki.basho.com/Installing-Riak-from-Source.html > > > Perhaps there is a stable/release branch? > I will fix this in the next couple days. Are you building from the source package or from git? The source package *should* work. Andrew ___ riak-users mailing list riak-users@lists.basho.com http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Re: When's the PBC API going to be brought up to snuff?
On Wed, Apr 06, 2011 at 02:12:30PM -0700, Bryan O'Sullivan wrote:
> Many thanks, Jon. Please bear in mind that the examples I extended were just
> the first two I could think of - as you know, the REST API is considerably
> more subtle and feature-rich than PBC. That said, the combination of
> conditional PUT and GET and bucket properties would probably hit 70% of the
> common cases I find myself wanting.
So I'm the lucky fellow assigned to try to implement some of the
requests in this thread. I have some changes in the
az259-enhance-pbc-api branches on riak_kv and on the riak-erlang client:
https://github.com/basho/riak-erlang-client/commits/az259-enhance-pbc-api
https://github.com/basho/riak_kv/commits/az259-enhance-pbc-api
Changes include conditional get [{if_modified, VClock}], conditional put
[if_none_match], [if_not_modified] and a way to get the metadata of an
object without its value; [head].
Let me know if this addresses some of the pain points and if there's
anything else missing. Bucket properties are one we already know about.
Andrew
___
riak-users mailing list
riak-users@lists.basho.com
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Re: When's the PBC API going to be brought up to snuff?
On Thu, Apr 21, 2011 at 01:36:47PM -0700, Bryan O'Sullivan wrote:
> On Wed, Apr 20, 2011 at 2:28 PM, Andrew Thompson wrote:
>
> > Changes include conditional get [{if_modified, VClock}], conditional put
> > [if_none_match], [if_not_modified] and a way to get the metadata of an
> > object without its value; [head].
> >
>
> It's a good start, thanks. One particular case of conditional PUT that I
> would find very useful is "put only if this bucket/key does not already
> exist", which I can achieve using HTTP via "If-None-Match: *", and now via
> the new if_none_match on RpbPutReq. It seems weird to transplant the
> relevant HTTP header names to the protobuf specifications of RpbPutReq and
> RpbGetReq, though, as those names make no sense outside of a HTTP context,
> and they don't mirror the HTTP semantics anyway. If if_none_match was
> if_missing, for instance, that would reduce the cognitive dissonance.
I can certainly rename them, I wasn't sure if it was better to try to
approximate the REST API or not. I'm happy to make that change and
consider any other suggestions now before the API gets merged into
master and we have to start considering backwards compatability.
if_none_match and head seem like the ones that make less sense outside
the concept of the REST API, I think if_[not]_modified is fairly clear.
Thoughts?
Andrew
___
riak-users mailing list
riak-users@lists.basho.com
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Re: When's the PBC API going to be brought up to snuff?
Oh, and does it make sense to have some approximation of the HEAD option on put, like return_body but returning the object sans the value so you can get the metadata without round-tripping the value itself? Andrew ___ riak-users mailing list riak-users@lists.basho.com http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Re: When's the PBC API going to be brought up to snuff?
On Thu, Apr 21, 2011 at 04:07:44PM -0500, Jon Brisbin wrote: > Yeah, this would be good to put a value and get the vector clock back in a > single operation for future updates... > Just pushed. This will also let you check if siblings were created because it returns the full riak object, just with the values blanked. Andrew ___ riak-users mailing list riak-users@lists.basho.com http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Re: not working server generated object keys in riak-erlang-client
On Sat, May 07, 2011 at 03:47:02PM +0300, Dmitry Rozhkov wrote: > Hi! > > I've just tried to put an object without a key as described in the docs, but > this feature doesn't seem to work (sasl-error.log attached): > I'm assuming both the client and server are running latest from git, right? The server needs to be updated to support this as well. This feature is not present in any stable release at the moment. Andrew ___ riak-users mailing list riak-users@lists.basho.com http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Re: Question: Object Not Saved After Save/Delete/Save
On Fri, Jun 03, 2011 at 02:00:23PM -0700, Aphyr wrote:
> Riak can't use the vclock for conflict resolution on a fresh object,
> i.e. one without a vclock. Deletes are writes. You should use get or
> reload before writing to help Riak sequence your writes correctly.
>
> On top of this, Riak has some weirdness around very quick sequences
> of deletes/writes due, IIRC, to deletes not being tagged with a
> vector clock. I... think... this will be addressed in an upcoming
> release.
>
Ah, my favorite bug. This is indeed mostly solved on master by exposing
the vclocks for tombstones so they can be cleanly overwritten by the new
object rather than merged into a frankenobject that has the metadata of
the tombstone but the value of the new object (which is then subject to
real deletion).
So the fix for the issue was to add a new type of return value for a get
that finds a tombstone, {error, {deleted, Vclock}} instead of always
returning {error, notfound} on a true notfound or when a tombstone is
encountered.
An example of how to safely delete for all 3 APIs can be found at
https://gist.github.com/965376
Note the new deletedvclock option. The REST API will always use this
option on gets and will return a X-Riak-Vclock header along with any 404
that is actually a tombstone.
Some further reading can be found here:
https://issues.basho.com/show_bug.cgi?id=260
https://issues.basho.com/show_bug.cgi?id=555
So, long story short; either upgrade to master and use the deletedvclock
option or avoid doing rapid put/delete/put cycles.
Andrew
___
riak-users mailing list
riak-users@lists.basho.com
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Re: Question: Object Not Saved After Save/Delete/Save
On Fri, Jun 03, 2011 at 06:08:06PM -0400, Keith Bennett wrote:
> Aphyr & Andrew -
>
> Thanks for your responses. I'm trying to wrap my head around the issues you
> raised, and I must confess it's difficult. Anyway, some questions for you...
>
> On Jun 3, 2011, at 5:12 PM, Andrew Thompson wrote:
>
> > On Fri, Jun 03, 2011 at 02:00:23PM -0700, Aphyr wrote:
> >> Riak can't use the vclock for conflict resolution on a fresh object,
> >> i.e. one without a vclock. Deletes are writes. You should use get or
> >> reload before writing to help Riak sequence your writes correctly.
>
> If the caller doesn't have a handle to the RObject in the Ruby framework, or
> the metadata of an HTTP response, but is only accessing the data by bucket
> and key values, is there any way to use reload? And if the object has been
> deleted is there any way to use get?
>
> >>
> >> On top of this, Riak has some weirdness around very quick sequences
> >> of deletes/writes due, IIRC, to deletes not being tagged with a
> >> vector clock. I... think... this will be addressed in an upcoming
> >> release.
>
> I tried inserting a 20 second pause between each read/write, but there was no
> change in the behavior. Should it be longer than that?
This is scary, it should not be possible to trigger this behaviour with
such a large gap. Even 10ms was enough to prevent it in my testing.
>
> >>
> >
> > Ah, my favorite bug. This is indeed mostly solved on master by exposing
> > the vclocks for tombstones so they can be cleanly overwritten by the new
> > object rather than merged into a frankenobject that has the metadata of
> > the tombstone but the value of the new object (which is then subject to
> > real deletion).
> >
> > So the fix for the issue was to add a new type of return value for a get
> > that finds a tombstone, {error, {deleted, Vclock}} instead of always
> > returning {error, notfound} on a true notfound or when a tombstone is
> > encountered.
> >
>
> That sounds like exactly what I need. Can I simulate that fix in my Ruby
> code? Any pointers about that? Any chance we could get that into the Ruby
> client? I'd be willing to put a little effort into that, but I don't really
> understand the issues well enough yet.
I suspect we will want to update the clients to support this, but I
don't know of a specific plan to do so yet. I don't know the details of
the ruby client enough (or at all) to advise on making changes to it.
>
> > An example of how to safely delete for all 3 APIs can be found at
> >
> > https://gist.github.com/965376
> >
> > Note the new deletedvclock option. The REST API will always use this
> > option on gets and will return a X-Riak-Vclock header along with any 404
> > that is actually a tombstone.
> >
> > Some further reading can be found here:
> >
> > https://issues.basho.com/show_bug.cgi?id=260
> > https://issues.basho.com/show_bug.cgi?id=555
> >
> > So, long story short; either upgrade to master and use the deletedvclock
> > option or avoid doing rapid put/delete/put cycles.
>
> You're suggesting I use mercurial to pull down the HEAD and use that, right?
>
git, not mercurial. I don't think we maintain the bitbucket repos
anymore, everything has moved to github.
Andrew
___
riak-users mailing list
riak-users@lists.basho.com
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Re: Question for Riak Developer Advocates
On Thu, Jun 09, 2011 at 08:01:24PM -0700, Ben Tilly wrote: > It sounds like you understood perfectly. > > Basically we are running a cluster of machines that are busy doing > lots of stuff. We wanted to use Riak to keep configuration > information about those machines and the stuff they were doing. So > Riak would be running on machines whose primary job is something else. > A critical use case for us is to figure out what needs to be done on > which other machines after one of the machines goes down. Therefore > having the potential to have our data unavailable during a failover > because of the failover kills the benefit that we wanted from a high > availability system. I think you're confusing 2 issues. The time when a *new* node is joining a cluster or an established node is leaving the cluster *VS* when a node in the cluster is unavailable. In the first case, when you are changing the cluster membership (not the list of available nodes) you will currently get notfounds as the partitions claims are changed. In the second case (which should be the more common one if you aren't constantly adding/removing nodes from the cluster) you will NOT receive notfounds (assuming your n_val is > 1 and your r value is less than the n_val). When you do a get on the key which is stored on the downed node riak will create a fallback vnode for the missing vnode, do a GET on that key (and it should get N-1 replies) and do a read-repair on the newly spun up fallback vnode. You should not get notfounds in this case unless R==N or N=1 or you've had multiple nodes fail since you last fetched this key (if the cluster is large enough this should be unlikely). If you want to increase your robustness further, you can increase N and reduce R. Git HEAD also has a new GET parameter called notfound_ok, which instructs riak to treat notfound responses as valid responses (counting towards R) instead as an error. Let me know if that clarifies things at all. Andrew ___ riak-users mailing list riak-users@lists.basho.com http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Re: Problem with deleting keys
On Thu, Jun 16, 2011 at 08:40:45AM -0700, Greg Nelson wrote:
> Well, it is kind of Riak specific. An implementation that treated DELETEs
> like PUTs (tombstones w/ vector clocks for ordering), then this would not be
> an issue, right? When no primary nodes are down, the tombstones can be
> physically deleted on the backend. A logical delete could never reappear if
> that were how it worked.
>
> Is this essentially what is on the current master branch (not yet released)?
>
Yes, this is essentially how its supposed to work on master. A tombstone
is put and then an async get is fired off and if the async get finds all
the primary nodes in the preflist are up, it does the delete. If not,
the next time the key is fetched, it does the same check again and will
do the delete then if the downed node is up.
Some issues still remain with this however, specifically how to override
tombstones (since notfounds do not include a vector clock). I've also
added a 'deletedvclock' GET option to change the return behaviour for
when a tombstone is found to instead return {deleted, VClock} so you can
safely override a tombstone, instead of triggering a merge (or creating
siblings).
This isn't perfect and we're discussing better solutions, but it does
make things significantly better.
Andrew
___
riak-users mailing list
riak-users@lists.basho.com
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Re: Problem with deleting keys
On Thu, Jun 16, 2011 at 06:09:37PM +0200, Kresten Krab Thorup wrote:
>
> On Jun 16, 2011, at 5:53 PM, Andrew Thompson wrote:
>
> > I've also
> > added a 'deletedvclock' GET option to change the return behaviour for
> > when a tombstone is found to instead return {deleted, VClock} so you can
> > safely override a tombstone, instead of triggering a merge (or creating
> > siblings).
>
> On master ... Is this reflected in the REST API, so that e.g. when you get a
> 404 you may also get a VClock header if the object was previously deleted,
> but Riak hasn't been able to effectuate that because a primary is down?
Yes, the REST interface will *always* send a X-Riak-Vclock header if it
encounters a tombstone and 404s. This was the only API that I could
change to make this new option always on without breaking compatability.
Andrew
___
riak-users mailing list
riak-users@lists.basho.com
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Re: Problem with deleting keys
On Thu, Jun 16, 2011 at 09:19:51AM -0700, Greg Nelson wrote: > "A tombstone > is put and then an async get is fired off and if the async get finds all > the primary nodes in the preflist are up, it does the delete. If not, > the next time the key is fetched, it does the same check again and will > do the delete then if the downed node is up." > > What if the key is never fetched again after the node is up? Does this mean > one would have to do GETs on all keys deleted while the node was down after > the node is back online, in order to reclaim that space? As I said, we know this is not the ideal solution. We have a proposal internally that should solve this issue by not relying on GETs to trigger actually erasing the key. If this really concerns you, you could implement something like postgres's VACUUM by doing a keylist and doing a fetch on all keys in a slow background process (while you know all the nodes are up) to force the issue. Andrew ___ riak-users mailing list riak-users@lists.basho.com http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Re: Problem with deleting keys
On Thu, Jun 16, 2011 at 09:24:28AM -0700, David Leimbach wrote: > Riak doesn't have tombstones (as far as I know) so, you have to make sure > all your nodes are up to do a delete. This, to me, seems like a misfeature. > Please read my other responses in this thread, riak absolutely *does* have tombstones. And if you don't have all the primary nodes for a key up at the time of a delete, you'll write tombstones they just won't be able to trigger an actual removal of the key. Now, that said. 0.14 has several bugs in how delete works and so the behaviour is probably going to be unpredictable. To be perfectly honest, I recommend avoiding deletes where possible on 0.14. As you suggested, implementing tombstones at the application layer might be a better strategy until the next major riak release. Unfortunately, the changes were too extensive to be backported into the 0.14 branch. Andrew ___ riak-users mailing list riak-users@lists.basho.com http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Re: Problem with deleting keys
On Thu, Jun 23, 2011 at 09:59:38AM +0200, Runar Jordahl wrote: > As this discussion shows, deleting objects in an eventual consistency > database is complicated. The concept of multiple APIs to do a delete, > and the possibility of having objects reappear, can scare new > customers away from Riak. > > I understand delete is complicated, but Basho should make it as simple > as possible to understand. They should provide a single, simple, > well-documented delete API, and document how delete happens. > > I find it unnatural to use the PUT operation to do a delete. I > understand the API compatibility issues, but for new users this can be > quite confusing: ?So you are telling me there are two APIs I can use > to delete an object? Why!?? In my mind, changing the DELETE API would > be far better. After all, Riak is still not legacy software. > > A wiki page describing how delete is done in a Riak system should be > created. The wiki page should describe how delete happens internally: > Which API to use (PUT or DELETE), tombstone concepts, etc. It should > describe current bugs, and future plans. In addition the page should > describe how the client app should handle deletes: Can you configure > tombstone lifetime? Pros/cons of using own tombstones, when can > deleted objects reappear, and how to detect and handle that. > I agree that we should do a writeup on deletes, past present and future. I'll try to put something together in the next few weeks. Andrew ___ riak-users mailing list riak-users@lists.basho.com http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Re: Riak 0.14.* and Erlang 14B03
On Wed, Jun 29, 2011 at 05:23:36PM -0400, Gregory Haskins wrote: > On 6/29/11 1:19 PM, Jared Morrow wrote: > > Greg, > > > > At this time there are no plans for a 0.14.3 release. > > Just to be clear: are there any plans to cut a release with the 14B03 > fixes in it anytime soon (perhaps 0.15, etc..I don't care if it is > literally another 0.14 point release per se). FWIW, I am dead in the > water w.r.t. Riak until this is fixed. > The prebuilt riak binaries shouldn't use the system erlang install, the binary includes its own copy of the erlang VM. The R14B03 incompatability will manifest as a compile time issue too, not a runtime one. I suspect something else is wrong. Andrew ___ riak-users mailing list riak-users@lists.basho.com http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Re: With Ruby Client, How to Test for Server Availability or Clear All Data?
On Thu, Jun 30, 2011 at 06:43:40PM -0400, Keith Bennett wrote: > My problem is that it takes a little time after running riak start or stop > for the operation to complete, and so I have to insert a sleep afterwards so > that the next step in the code is not performed prematurely. Is there a way > using the Ruby Riak client to know for sure whether or not riak is ready for > action? I could loop every fraction of a second, or, if there were a blocking > call, that would be even better. > I actually patched 'riak start' to block until riak starts in this pull request (that slipped through the cracks on getting merged). https://github.com/basho/riak/pull/26 'riak stop' already blocks, for the record. I'll also get that pull request merged into master. Andrew ___ riak-users mailing list riak-users@lists.basho.com http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Re: CentOS 5.6 and Riak
On Fri, Jul 01, 2011 at 12:33:42PM -0400, Jeff Kirkell wrote: > Has anyone been able to get Riak running on CentOS 5.6? I installed Erlang > 14B03 from source, have tried compiling from github, downloading the source > tar, and even tried to install from the RHEL and Fedora RPM package and each > have failed. > Some mention of the errors you're seeing in any of those cases would help in figuring out what the problem(s) might be. Andrew ___ riak-users mailing list riak-users@lists.basho.com http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Re: riak_err starts writing to stdout/stderr
On Tue, Jul 19, 2011 at 10:42:47AM -0700, Anthony Molinaro wrote: > Ahh, yeah that seems to be the case. I will have to investigate further > and simply trying to turn off the error_logger adding > > -kernel error_logger silent > > does not seem to turn things off, and invoking > > error_logger:tty(false) > > seems to turn off riak_err logging as well, seems overly complicated > just to redirect all error logging to a file which can be rotated > regularly :( > Stay tuned. We at Basho have something that should go public this week that should help. Andrew ___ riak-users mailing list riak-users@lists.basho.com http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Re: Erlang PBC won't generate key
Unless you're running erlang-client from git master, and riak from git master, you won't be able to generate keys via the PBC interface. Andrew ___ riak-users mailing list riak-users@lists.basho.com http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
riak-admin script not working for me
Hi All, This one has probably been answered many times before and I apologize in advance... I have built and installed my first riak instance and along the way discovered two things: 1. Build with Erlang version <= R14B01 2. This one is a bit different Execution Environment OS = Ubuntu 10.04 LTS ERL = Erlang R14B01 (erts-5.8.2) [source] [smp:4:4] [rq:4] [async-threads:0] [hipe] [kernel-poll:false] Riak = riak-0.14.0 The Problem === I cant execute raik-admin commands because ~/build/riak-0.14.0/rel/riak/erts-5.8.2/bin/escript ~/build/riak-0.14.0/rel/riak/erts-5.8.2/bin/nodetool -name riak@127.0.0.1 -setcookie riak ping is returning ... "Running Erlang pong" and as a result the "$RES" != "pong" is returning positive ie "Node is not running!" ... if [ "$RES" != "pong" ]; then echo "Node is not running!" exit 1 fi ... and it is running. Has this been observed before? Was there a solution that I can use? How do we suppress the "Running Erlang ~n"? Thanks Andrew ___ riak-users mailing list riak-users@lists.basho.com http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Re: riak-admin script not working for me
Hi All, Please ignore me. Just remove the first entry in your .erlang file (an exercise in one of my "welcome to erlang texts"). Sorry, will be back with more challenging questions soon :P Regards Andrew On Sun, Aug 7, 2011 at 9:21 PM, Andrew Thompson wrote: > Hi All, > > This one has probably been answered many times before and I apologize > in advance... > > I have built and installed my first riak instance and along the way > discovered two things: > > 1. Build with Erlang version <= R14B01 > 2. This one is a bit different > > Execution Environment > > OS = Ubuntu 10.04 LTS > ERL = Erlang R14B01 (erts-5.8.2) [source] [smp:4:4] [rq:4] > [async-threads:0] [hipe] [kernel-poll:false] > Riak = riak-0.14.0 > > The Problem > === > I cant execute raik-admin commands because > > ~/build/riak-0.14.0/rel/riak/erts-5.8.2/bin/escript > ~/build/riak-0.14.0/rel/riak/erts-5.8.2/bin/nodetool -name > riak@127.0.0.1 -setcookie riak ping > > is returning ... > > "Running Erlang > pong" > > and as a result the "$RES" != "pong" is returning positive ie "Node is > not running!" ... > > if [ "$RES" != "pong" ]; then > echo "Node is not running!" > exit 1 > fi > > ... and it is running. > > Has this been observed before? Was there a solution that I can use? > How do we suppress the "Running Erlang ~n"? > > Thanks > > Andrew > ___ riak-users mailing list riak-users@lists.basho.com http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Re: Anyone running Riak on FreeBSD?
On Wed, Aug 10, 2011 at 04:26:08PM -0700, Mark Phillips wrote: > > Emails need a wiki feature ;-) > > > > If no-one else steps forward, I'll attempt a FreeBSD build this weekend. > > FreeBSD used to be my primary server OS (until Ubuntu Server won me over), > > so I've got some familiarity there. > > If you use the adt-bsd-support branch for erlang_js, skerl and eleveldb, riak will build with no issues. To run it however, you need to edit the shebang paths for the scripts in rel/files to not point to /bin/bash before generating a release (or install bash). The scripts don't have any bashisms in them though, they work just fine under /bin/sh. Andrew ___ riak-users mailing list riak-users@lists.basho.com http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Re: Running riak on SunOS
On Thu, Aug 11, 2011 at 01:07:27PM +1000, Robert Leftwich wrote: > Hard to cover every possibility. > The completely silent failure mode made it hard to get traction, > spitting out some sort of error would be helpful, although reliably > detecting this category of problem is potentially non-trivial :-/ > Actually, the next release will print an error to the 'console' if the log files are unwriteable and not just crash. However, that requires you run 'riak console', not 'riak start'. Andrew ___ riak-users mailing list riak-users@lists.basho.com http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Re: Building Riak Search on OS X
On Tue, Aug 23, 2011 at 09:02:30PM -0700, Jeremiah Peschka wrote: > Has anyone managed to build Riak Search 0.14.2 on OS X? When I try with the > version from homebrew (or pulling the riak_search-0.14.2 tag from github) I > get the following error: > > Compiling src/riakclient.proto > WARNING: option deprecated > Config option 'fail_on_warning' has been deprecated > in favor of 'warnings_as_errors'. > 'fail_on_warning' will be removed once OTP R14B03 is released. > > Compiled src/riakc_obj.erl > Compiled src/riakc_pb.erl > src/riakc_pb_socket.erl:62: type ip_address() undefined > make: *** [compile] Error 1 > > When I attempt to follow the wiki directions for getting started with the > pre-built packages, I download a tiny file that's not in a valid .tar.gz > format and that's all she wrote. > > Thoughts? Experiences? I'm pretty sure that's caused by the wrong version of Erlang. You can correct the error by prefixing the ip_address type with inet: to give you inet:ip_address() but you'll likely run into NIF compilation errors later on in the build. Andrew ___ riak-users mailing list riak-users@lists.basho.com http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Re: Lager question...
On Sun, Aug 28, 2011 at 10:56:58AM -0700, Bryan Hughes wrote: > Hi Everyone, > > Wasnt sure where to post this question, so sorry if this is the wrong place. > > We have Riak as our persistence and have decided to switch to Lager > as our logging framework for the entire system - good stuff!!! So > far things have been going well. Unfortunately I am encountering > unexpected results from the same code running on Erlang R14B02 > (erts-5.8.3) on a MacBook Pro and a 32bit Ubuntu 11 instance. Both > Erlangs were downloaded from source and compiled with the same > flags. > > The issue is with the formatted results of a float using ~w. > > On the Linux instance: > > (herd@127.0.0.1)4> lager:log(info, self(), "~w", [1.0]). > 22:33:32.329 [info] 1.e+00 > ok > > On the Mac: > > (herd@127.0.0.1)2> lager:log(info, self(), "~w", [1.0]). > 22:43:22.208 [info] 1.0 > ok > > ~p formats correctly, but it also prettifies the output which is not > desirable for logs that may be parsed later. > Interestingly, I see the same result on both my 64 bit macbook and my 64 bit linux installs. I get the 18:49:23.358 [info] 1.e+00 style print. I will look into the issue. Are the git commit ids for lager the same on both machines? Andrew ___ riak-users mailing list riak-users@lists.basho.com http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Re: Lager question...
I just pushed a commit that should resolve this. Andrew ___ riak-users mailing list riak-users@lists.basho.com http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Re: Keys aren't deleted
On Wed, Aug 31, 2011 at 05:11:16PM -0400, Tony Bussieres wrote:
> Hi,
>
> We have a cluster of two nodes.
>
> We add keys to a bucket :
> curl -v -X PUT -d '{"prop":"val"}' -H "Content-Type: application/json"
> http://riak/store/jdTestBucket/key1?w=1
> curl -v -X PUT -d '{"prop":"val"}' -H "Content-Type: application/json"
> http://riak/store/jdTestBucket/key2?w=1
> curl -v -X PUT -d '{"prop":"val"}' -H "Content-Type: application/json"
> http://riak/store/jdTestBucket/key3?w=1
> curl -v -X PUT -d '{"prop":"val"}' -H "Content-Type: application/json"
> http://riak/store/jdTestBucket/key4?w=1
>
> We shut down one node.
>
> We delete the keys:
> curl -v -X DELETE http://riak/store/jdTestBucket/key1?w=1
> curl -v -X DELETE http://riak/store/jdTestBucket/key2?w=1
> curl -v -X DELETE http://riak/store/jdTestBucket/key3?w=1
> curl -v -X DELETE http://riak/store/jdTestBucket/key4?w=1
>
>
> We list the keys :
> curl -i http://riak/store/jdTestBucket?keys=true\&props=false
>
> All keys are listed.
>
> Is it normal?
>
Yes, keys are only deleted when all the values are tombstones and all
the nodes are up. In theory bringing the other node back up and
re-reading the keys will read-repair the keys on the node that was down
to be tombstones and then hopefully they'll be removed.
However, deletes are flaky on 0.14. Things should be a little better in
1.0.
Andrew
___
riak-users mailing list
riak-users@lists.basho.com
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Re: Keys aren't deleted
On Wed, Aug 31, 2011 at 05:35:09PM -0400, Jonathan Langevin wrote: > What if a node were permanently removed for some odd reason? You could never > delete those keys in that situation?* > Then you should remove that node from the cluster and do a read-repair. That should make the tombstones go away. Andrew ___ riak-users mailing list riak-users@lists.basho.com http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Re: Riak Search Precommit Hook in 1.0
On Tue, Sep 20, 2011 at 10:53:28AM -0700, Greg Pascale wrote: > Hi Jon, > > Thanks for getting back to me. > > I'm doing full-text indexing, but I don't need to keep the text around once > my object has been indexed, so I wrote a second precommit hook to remove the > text from the object. This reduces the size of my objects drastically (since > text tends to be the largest part) and has given us substantial perf gains in > our tests. > I just pushed a fix for this. It should be present in the next 1.0 prerelease. https://github.com/basho/riak_search/pull/87 Andrew ___ riak-users mailing list riak-users@lists.basho.com http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Re: Lager question - how do I turn off truncation?
On Mon, Oct 24, 2011 at 03:30:25PM -0700, Bryan Hughes wrote: > Hello, > > We have switched over to lager but have a problem with it truncating > large records similar to io:fwrite() ~P - how do I turn this off, or > switch it to ~p? > > Thanks so much for your help. Probably something simple that I am missing. > So, this is technically a 'feature', in that lager protects you from printing massive terms and out-of-memory killing your node. However, the limit at which lager kicks in and starts limiting the message size is hardcoded right now to 4096 bytes. I can easily make that configurable, so you can bump it up if needed, you'll just have to be careful you don't push it too high. Andrew ___ riak-users mailing list riak-users@lists.basho.com http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Re: Lager error_logger formatting issue?
On Sat, Nov 05, 2011 at 03:17:13PM -0700, Bryan Hughes wrote: > Hello, > > I did an update on my Lager deps and am now experiencing some > formatting issues with error_log:error_report(Report). Here is my > code in my mochiweb loop: > Good catch. This was a side-effect of rewriting lager's output routines to be ~4 times faster (and much closer to the behaviour of io_lib:format). I've pushed a fix and a new test to master: https://github.com/basho/lager/commit/16fb55890c97d0a0eb5a95646903f39ed1e9ceb2 There may be a couple other issues like this lurking, let me know if you find any more. Riak is still using the 0.9.4 tag of lager (and will continue to do so until the 1.1 release), so master, which I hope to tag as 1.0 soon, isn't seeing quite as much scrutiny right now. Andrew ___ riak-users mailing list riak-users@lists.basho.com http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Re: Lager and Scribe
On Thu, Nov 17, 2011 at 04:52:58PM +0100, Bip Thelin wrote:
> Hi,
>
> I've been thinking about using lager to push logs to our Big Data Hadoop
> backend for later analysis using Scribe. Looking at the lager_syslog backend
> it looks fairly simple to create such a backend. But thrift works kind of
> different and some implementation questions occur when going through the
> lager code.
>
> The way thrift works is that you create a connection:
> {ok, Client} = thrift_client_util:new()
>
> then you can use the Client to do subsequent calls like
> {Client2, _Result} = thrift_client:call(Client, ...)
> {Client3, _Result} = thrift_client:call(Client2, ...)
> [...]
>
> The problem is that you get a new Client descriptor every time and as I've
> understood it you shouldn't use the same Client twice but instead use the new
> one with each call. One could of course create a client and do the call
> within each handle_event() and pay that penalty hit but there must be a
> smarter way. I was going through the lager_file_backend and saw that you use
> a FD that get's passed around, is that a good approach to use something
> similar for a Scribe backend?
>
It should be fine to create the client and just store the current
descriptor in the backend's state. Creating a whole new client on every
message seems like a bad idea.
Andrew
___
riak-users mailing list
riak-users@lists.basho.com
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Re: Lager AMQP backend
On Thu, Nov 17, 2011 at 02:05:58PM -0600, Jon Brisbin wrote: > I pushed to my Github a Lager backend for sending log messages to RabbitMQ > via AMQP: > > https://github.com/jbrisbin/lager_amqp_backend > > It uses a re-connecting connection pool for sending messages, so it's pretty > fast and will automatically recover if RabbitMQ goes down (but it does *not*, > at the moment, internally queue log messages if it can't connect to the > broker). > > The idea is to aggregate logging from riak_core applications, but you should > be able to use it in Riak/DB as well. > Many thanks, it looks good. I've added a link to it from lager's README. Andrew ___ riak-users mailing list riak-users@lists.basho.com http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Re: [lager] RFC on refactorings for a Graylog2 backend
On Mon, Nov 28, 2011 at 12:47:17PM -0500, Jason Wagner wrote:
> This is a very developer oriented question about Lager that I'm posting
> here because I didn't find a dedicated list.
Yes, it doesn't have a list. I'm not sure if it is worth rectifying that
yet. Here should be fine for now.
> I'm working on a Graylog2 backend for Lager. I'm sitting on a very basic
> plugin that does little more than post log messages to a graylog2 system
> that I'll push to github after I do some integration testing tonight.
>
> However, there's a next step with the Graylog2 integration that would
> really leverage the power of both tools-- pushing Lager's trace attributes
> into Graylog2 user fields to really leverage the analytics that the newest
> versions of Graylog2 provide. This would allow both pre-log trace
> filtering and after the fact trace analysis through Graylog2.
>
> This would be a major change to the Lager internals and I wanted to solicit
> opinions on the changes.
>
> *Proposed changes
> **Pass through all trace attributes*
> The backends need to receive all trace attributes, including default
> attributes such as line, file, function, pid, etc. This would allow proper
> population of the graylog2 fields.
>
> I would change this by removing the lager_transform:transform_statement's
> call of lager_util:check_traces so that all Traces got passed through. I
> don't believe this would impact any backends except via sending them more
> payload in the event, but please correct me if I'm wrong.
I've actually been considering doing this as well, for similar reasons.
I don't see a problem with doing it.
> *Refactor message formatting*
> The formatting needs to be extracted and externalized from lager:log and
> lager:log_dest. This is probably desirable, anyway, since it would be a
> very small step from this to have completely orthogonal format/sink
> separation, allowing for a user compromise between speed and flexibility in
> their log file formats.
>
> I would create a behavior lager_message_formatter with one function that
> takes the trace attributes, the message, and returns an iolist for output.
> The current one would simply be:
>
> format(Config,Trace,Format,Args) ->
> Pid=proplists:get_value(pid,Trace),
> % etc for Module, Function, Line, Level
> [["[", atom_to_list(Level), "] "],
>io_lib:format("~p@~p:~p:~p ", [Pid, Module, Function, Line]),
>safe_format_chop(Format, Args, 4096)].
>
> The backends would make this call rather than the lager:log or
> lager:log_dest. This would allow for a configuration parameter on the
> backends to set which formatter they use, with a reasonable default, and
> different formats for each backend configuration.
I think that might be overkill, but I suppose it's acceptable.
>
> *Concerns*
> One drawback is that it significantly increases the size of the events,
> especially if there is concern about run-away tuples being sent. I'm not
> sure how this would impact the performance in the long haul.
>
> The formatting change also pushes some of the workload from the user
> process to the logging process. This can be an advantage if the workloads
> aren't taxing the backend, but could penalize even simple logging
> statements if the backend gets swamped and can't format/process fast
> enough. The potentially large messages have impact this as well.
Technically, this won't happen because lager uses sync_notify, which
means the log call blocks until all the backends have handled it.
However, this *does* mean that long log formatting calls in the
gen_event will probably cause logging calls to block *more*.
What about doing the format string formatting in the calling process,
*once*, and then just sending along the rest of the data (timestamp,
callsite information, trace attributes, etc) with the output from doing
the string formatting? That should be the meat of the formatting anyway,
and I'd like to make sure we only do it once (we actually do it twice at
the moment, if a trace matches).
> There are a couple ways to mitigate this-- additional checks on the log
> statement, separate format processes from write processes and pool the
> formatters, and other ideas that I wouldn't propose until I actually see
> the bottleneck.
>
> *Request For Comments*
> Are these changes desirable in the direction of lager?
>
> I've seen performance based changes in the recent history. Is performance
> paramount over flexibility? How do you currently measure performance? Are
> any scripts,scenarios,etc available?
>
> Any historical lessons that I might be unaware in the vicinity of things
> I'd be changing?
I think overall this is the path Lager is going to have to tread.
I'd rather balance Lager in the direction of flexibility and power over
performance, but egregious bottlenecks shouldn't be tolerated if there's
something that can be done to fix them. Lager is really about *safely*
logging messages, making them easy to re
Re: Erlang Port Configuration
On Wed, Nov 30, 2011 at 01:50:48PM -0600, Paul Gross wrote: > I'm setting up a new cluster with an iptables firewall, and I'm > trying to figure out which ports to open. It looks like I need 4369, > 8099, 8098 (http), and a configurable range. Can I configure the > range to be a single port, or is there a minimum number of required > ports? What determines how many ports I need? > http://wiki.basho.com/Network-Security-and-Firewall-Configurations.html Andrew ___ riak-users mailing list riak-users@lists.basho.com http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Re: Erlang Port Configuration
On Wed, Nov 30, 2011 at 02:09:45PM -0600, Paul Gross wrote: > >>I'm setting up a new cluster with an iptables firewall, and I'm > >>trying to figure out which ports to open. It looks like I need 4369, > >>8099, 8098 (http), and a configurable range. Can I configure the > >>range to be a single port, or is there a minimum number of required > >>ports? What determines how many ports I need? > >> > >http://wiki.basho.com/Network-Security-and-Firewall-Configurations.html > I saw that page, and how it references a configurable range, but I > didn't see anything about the requirements of that range. Did I miss > something? > A range of 1 might work, but I'd go for a range of 5 or so, just to be safe. Andrew ___ riak-users mailing list riak-users@lists.basho.com http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Re: Erlang Port Configuration
On Wed, Nov 30, 2011 at 10:12:24PM -0600, Paul Gross wrote: > >A range of 1 might work, but I'd go for a range of 5 or so, just to be > >safe. > Thanks for the guideline. Can you give me some insight on when riak > would use more than one port? > The most likely scenario is if riak crashes, and by the time you restart the node the OS hasn't released that port. There may be other situations, but I can't think of any. Andrew ___ riak-users mailing list riak-users@lists.basho.com http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Re: repost: bucket key listing
I think if you disable legacy keylisting, we'll just break a limb now. Seriously though, keylists are at least non-blocking in 1.0, so the vnodes aren't prevented from doing any reads/writes while the keylist is running. However, remember that a keylist of any kind that isn't on a single vnode (the way riak does keylists internally is all per vnode) requires a traversal of ALL the keyspace, even if your bucket only has 10 things in it. Andrew ___ riak-users mailing list riak-users@lists.basho.com http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Re: repost: bucket key listing
On Thu, Dec 01, 2011 at 08:38:15PM -0500, Jeremy Raymond wrote: > So if I do have a bucket that I regularly want a listing of the keys for (or > really that I want to access all of the objects) what is the recommend > procedure? Put the same 2i index on each object in the bucket and then do an > index query on it? That's certainly one way, yes. You could also maintain your own index, or use something like redis to hold it (and just use a keylist if redis restarts). > Is doing a mapred operation specifying a bucket as the > input (instead of a list of key/value pairs) as bad as listing the keys in > the bucket? Yes, it is, a key-filter is also just as bad. Andrew ___ riak-users mailing list riak-users@lists.basho.com http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Re: RiakEDS - one way replication?
On Tue, Dec 06, 2011 at 10:49:40AM -0500, Phil Stanhope wrote: > What are some examples of use cases for one-way replication? > A read-only secondary cluster where you want to run expensive queries without affecting the production cluster. Andrew ___ riak-users mailing list riak-users@lists.basho.com http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Re: Riak & FreeBSD
On Mon, Dec 12, 2011 at 07:00:57PM +, Tim Ariyeh wrote: > Has there been any development in this area? I saw this: > > http://lists.basho.com/pipermail/riak-users_lists.basho.com/2011-August/005228.html > > But I wondered if any clear "official" or "least horrible" path existed now. > > I mean, it only seems logical that Riak run on the flavor of Unix preferred > by literally 100% of people without debate or contention. > I'm still valiantly trying to get all my BSD patches merged, which add support for Free/Net/Open/DragonFlyBSD to riak. Several have already been merged and will appear in 1.1, but there's outstanding branches against eleveldb and the riak repo itself that still aren't merged: https://github.com/basho/eleveldb/pull/4 https://github.com/basho/riak/pull/90 The eleveldb change is blocked pending google accepting the upstream change at http://code.google.com/p/leveldb/issues/detail?id=59 so feel free to make some noise in that bugreport. However, I think it's unlikely that riak 1.1 will build out of the box on FreeBSD. Andrew ___ riak-users mailing list riak-users@lists.basho.com http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Re: Riak fails to start on ubuntu 11.10
The actual error was about riak being unable to write to /tmp/riak, rebooting the machine may have simply reset the permissions on /tmp/riak (had you previously run riak as a different user?) Andrew ___ riak-users mailing list riak-users@lists.basho.com http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Re: start_clean processes?
On Tue, Dec 27, 2011 at 12:18:23PM +0800, Denis Barishev wrote: > Hello! > > I've faced a problem recently. There are strange processes like > ...start_clean which use 100% of CPU. My riak version is 0.14.2. > Do you know what is it and what can be done with it? > Can you provide more information? When do you see this happen, and how are you determing the process is called start_clean? Start_clean is a boot script used in releases, it should only be important when a node is booting. Andrew ___ riak-users mailing list riak-users@lists.basho.com http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Re: "riak_err" crashes during startup of my app
On Tue, Jan 03, 2012 at 04:48:11PM +0100, Zabrane Mickael wrote: > Any chance that someone from Basho's team help me undertsand my problem (see > below)? > What possible reason would you have to use *both* lager and riak_err on the same project? Lager provides most, if not all, of riak_err's featureset. Andrew ___ riak-users mailing list riak-users@lists.basho.com http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Re: "riak_err" crashes during startup of my app
On Tue, Jan 03, 2012 at 05:37:40PM +0100, Zabrane Mickael wrote: > . riak_err prevents/minimizes SASL error dumps on the VM. > . lager is a logger. > Lager actually borrows a bunch of code from riak_err to provide riak_err's functionality (and actually fixes at least one issue that riak_err had). They're not 100% compatible, it's true (you can't truncate SASL logs AND have them printed to the console with lager), but you don't need riak_err to simply guard against large SASL log messages. Lager and riak_err both install error_logger handlers, and are probably stepping on each other's toes when they do it, triggering that error. Andrew ___ riak-users mailing list riak-users@lists.basho.com http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Re: Absolute consistency
Thomas, I just replicated your setup (at least for the PR gets) and you can indeed violate PR/PW when you pause a node on a VM. The reason this happens is that riak's check for PR/PW simply looks at the ring's preflist for a partition and checks that the required number of partitions for that preflist are marked as primaries. Now, when you pause a VM you interrupt any TCP connections that node has open, just like if you unplugged the network cable, but not like if the OS shut down or riak itself crashed. In those cases a FIN packet is sent so that the other erlang nodes notice that their persistant connections to that node have been reset, they will then reassign ownership of the partitions owned by that downed node and PR/PW will start to fail. However, since FIN packets are not generated when you pause the VM, it takes a few moments for the erlang network heartbeat stuff to notice that the node is down, so the preflists aren't recalculated. This is the window where you see the mysterious behaviour. Now, this is arguably a bug, although fixing it might be challenging. I've filed https://issues.basho.com/show_bug.cgi?id=1318 to track this. I don't have a workaround that I can think of offhand, unfortunately. Andrew ___ riak-users mailing list riak-users@lists.basho.com http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Re: Riak Control installed when upgrading to 1.0.2?
Riak control will be part of the 1.1 series, it is not present in 1.0.2 or any of the 1.0 releases. Andrew ___ riak-users mailing list riak-users@lists.basho.com http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Re: Riak Recap for January 6 - 10
On Wed, Jan 11, 2012 at 10:07:57AM -0800, Mark Phillips wrote: > 1) Q - How can I check which version of Riak I'm running? > > A - > http://serverfault.com/questions/348530/how-can-i-check-which-version-of-riak-i-am-running Newer riak packages also include the 'riak version' command: riak (1.0.3rc5 2012-01-10) RedHat x86_64 Andrew ___ riak-users mailing list riak-users@lists.basho.com http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Re: No message in sasl-error.log for wrong post commit erlang hook
On Fri, Feb 03, 2012 at 11:44:49PM +0200, Alin Popa wrote: > Ryan, > > I would say that having those informations into the log files, especially > the reason why it failed, is very useful, so, would you say it would be > helpful if I'm going to open a pull request which will increase the log > level usage from debug to error for pre/post commit hooks execution errors > (as it seems to be pretty straight forward) ? > The reason we put it at the debug level is to avoid logging flood when you break a postcommit hook in production and all your postcommits start to fail. If you're doing thousands of writes per second, that would obviously be bad. Andrew ___ riak-users mailing list riak-users@lists.basho.com http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Re: A bout source compile error in freebsd 9.0
On Wed, Feb 08, 2012 at 11:12:27AM +0800, 郎咸武 wrote: > Hey riak-users, > There is a compile error in the following. I using freebsd. Eriang version > is R14B03 . Riak version is 1.0.2. > Please give me some idea. Thank everyone. > I don't understand why this is failing, what is the version of gmake on your machine? erlang_js should build fine on FreeBSD. Andrew ___ riak-users mailing list riak-users@lists.basho.com http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Re: Windows
On Mon, Feb 20, 2012 at 05:19:09AM -0800, Jeremiah Peschka wrote: > Bitcask and LevelDB are pluggable back ends for Riak. Bitcask is the default, > and LevekDB is needed right now for secondary indexes. > > Windows was once POSIX compliant for about 15 minutes, then Microsoft changed > their mind. What this means is that you need to write wrappers around how > Windows handles file access. Windows also uses a completely different > networking API. Since Riak relies heavily on both network and I/O, porting it > to Windows is problematic. Actually, the BSD sockets layer is pretty close to the standard on Windows, and the Erlang VM hides those differences from the code anyway (and we don't do any socket programming in any of the NIFs, to my knowledge). I agree that the busted file API would be very problematic, though. Andrew ___ riak-users mailing list riak-users@lists.basho.com http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Re: riak install error: R15B does not match required regex R13B04|R14
R15 is blacklisted for a reason, we don't support it yet. We haven't changed all our NIFs to export the new version stuff R15 needs and there's a couple other things that need some work too. I suggest you either use R14B or install riak from a package. Andrew ___ riak-users mailing list riak-users@lists.basho.com http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Re: building riak on FreeBSD9
There's also an adt-bsd-support branch for deps/eleveldb you didn't use, which would probably have solved this problem for both FreeBSD and OpenBSD. Andrew ___ riak-users mailing list riak-users@lists.basho.com http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
