problem building latest
Following the steps here: http://wiki.basho.com/Installing-on-RHEL-and-CentOS.html NOTE: I did install and build erlang14B02 (not 13B04). No problems until doing the make rel for riak 0.14: Compiling c_src/ebloom_nifs.cpp c_src/ebloom_nifs.cpp: In function ‘ERL_NIF_TERM ebloom_new_filter(ErlNifEnv*, int, const ERL_NIF_TERM*)’: c_src/ebloom_nifs.cpp:85: error: ‘enif_alloc_resource_compat’ was not declared in this scope c_src/ebloom_nifs.cpp:90: error: ‘enif_release_resource_compat’ was not declared in this scope c_src/ebloom_nifs.cpp: In function ‘ERL_NIF_TERM ebloom_serialize(ErlNifEnv*, int, const ERL_NIF_TERM*)’: c_src/ebloom_nifs.cpp:247: error: ‘enif_alloc_binary_compat’ was not declared in this scope c_src/ebloom_nifs.cpp: In function ‘ERL_NIF_TERM ebloom_deserialize(ErlNifEnv*, int, const ERL_NIF_TERM*)’: c_src/ebloom_nifs.cpp:263: error: ‘enif_alloc_resource_compat’ was not declared in this scope c_src/ebloom_nifs.cpp:266: error: ‘enif_release_resource_compat’ was not declared in this scope c_src/ebloom_nifs.cpp: In function ‘int on_load(ErlNifEnv*, void**, ERL_NIF_TERM)’: c_src/ebloom_nifs.cpp:287: error: ‘enif_open_resource_type_compat’ was not declared in this scope ERROR: $CXX -c $CXXFLAGS $DRV_CFLAGS c_src/ebloom_nifs.cpp -o c_src/ebloom_nifs.o failed with error: 1 make: *** [rel] Error 1 Any thoughts on what to do to get past this? ___ riak-users mailing list riak-users@lists.basho.com http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Re: problem building on CentOS 64bit
Thanks Sean. I did what you suggested. Pulled and built a new rebar. Copied that into /usr/local/bin. Checked that it would take precedence over the one that is packaged in the RIAK repo. The one in the root of RIAK is "rebar version: 2 date 20110310_184739 2e1b4da". The newly built one is: "rebar version: 20110708_181908 vcs: git 5e04e7d" Unfortunately, the makefile in the root of RIAK explicitly calls the pre-packaged rebar. I get the same build errors. So I replaced the prepackaged rebar with the newly built one ... and a clean build works fine with no makefile changes required. -phil On Fri, Jul 8, 2011 at 2:08 PM, Sean Cribbs wrote: > Phil, > > This is a question best sent to the riak-users list. Recent changes in the > codebase necessitate an upgrade to rebar. In your source checkout, download > rebar from here: https://github.com/downloads/basho/rebar/rebar ... then > re-attempt the build. > > On Fri, Jul 8, 2011 at 2:01 PM, Phil Stanhope wrote: > >> Hi all, >> >> Environment = CentOS 64bit 5.6 >> >> I've got working 13B04 (from source) + .14.2 ... no problems there. >> >> I've got a working 14B02 (from source) ... no problems there. >> >> I've cloned RIAK from github today. Have two problems while building: >> >> erlang_js ... need to modify Makefile to explicitly set --enable-64bit >> eleveldb ... need to modify leveldb/Makefile to explicitly set -fPIC >> >> NOTE: When I pull the individual repos directly, they build fine. It's >> only when pulling RIAK and rebar has done the get_deps process that I've got >> the problem. >> >> It seems that there are commits in either of the dependent repositories >> for these fixes. >> >> I just don't know how rebar does it's dependency magic (yet) to understand >> what commit it's pulling from github. >> >> -phil >> > > > > -- > Sean Cribbs > Developer Advocate > Basho Technologies, Inc. > http://www.basho.com/ > > ___ riak-users mailing list riak-users@lists.basho.com http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Re: LevelDB driver
Can you share some LevelDB config examples?
I've built latest from github and know that I've got the elevedb
driver/wrapper.
make devrel is producing a {storage_backend, riak_kv_bitcask_backend} ... is
it as simple as changing this to riak_kv_eleveldb_backend?
Perhaps I missed it ... but did you guys also share the sample data that you
used for the benchmark and instructions about how to attempt recreation of
it?
For example, the blog post mentioned "commodity" server. But I don't recall
whether a mention was made of a stock 3 or 4 node localhost cluster, where
all nodes hit by the benchmark, etc.
This would be excellent information to make available to the community.
-phil
On Tue, Jul 12, 2011 at 11:16 AM, David Smith wrote:
> On Mon, Jul 11, 2011 at 4:24 PM, Will Moss wrote:
>
> > I know you guys are busy working on various new features, is this going
> to
> > make it into the next production release? Is there a date for that? Is
> there
> > somewhere to look at your road map so I don't have to spam the list?
>
> Hi Will,
>
> Yes, the current plan is to have LevelDB a default part of the next
> production release.
>
> We're still hashing out the timing of the next release, but it will
> _probably_ be sometime in the next 3 months (or so...).
>
> D.
>
> --
> Dave Smith
> Director, Engineering
> Basho Technologies, Inc.
> diz...@basho.com
>
> ___
> riak-users mailing list
> riak-users@lists.basho.com
> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
>
___
riak-users mailing list
riak-users@lists.basho.com
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Re: Python transports
I like the idea of RiakTransport as you describe it. It opens the door to other potential underlying transports and isolating a client from knowledge of those (websockets, tcp, zeromq messaging come to mind). I'm not suggesting that RIAK will ever support these transports by this comment, however. I also agree with the need to have RiakRingAwareTransport as an additional layer that *might* be used by a client. There may be valid reasons why a client might want to force particular traffic onto a subset of the ring (e.g. M/R config, Search Config, forcing read/write traffic onto different nodes, etc). Again, I'm not suggesting that using a subset of the ring for particular operations is the best practice. But it may be necessary to do so in order to validate and do certain types of testing to prove or disprove certain access patterns. -phil On Thu, Sep 8, 2011 at 7:31 AM, Greg Stein wrote: > Hey, all... > > After a couple comments on my recent work, and some archaeology on the > Python/Riak work over the past month... I've realized that I might > have a very different view of RiakTransport compared to what I'm > seeing in the current work. I figured it best to bring that to the > forefront and discuss: > > In my view, RiakTransport is used by RiakClient (and others) to "talk > to the Riak server". > > Some of the current work, and some proposed pull requests, seem to > take the position that a RiakTransport is "one connection to a server, > and the client should manage those". > > Needless to say, I'm in favor of my own position :-) ... I think it is > best to transfer *all* responsibility for talking to the server(s) to > the transport layer. I really don't think the client/bucket/object > layers should know anything about talking to the server(s). I'd like > to see the transport layer be told about all server(s) available, and > then it Just Works. > > I'm still a newbie with all this code, and need to keep plugging away > at the higher levels of functionality and compensation for problems. > I'd like to build up some code that contacts *one* given server, asks > for all of the ring servers, and then opens connections to those > servers. And then, it should (automatically) maintain client > connections based on what is happening with the Riak cluster. The > current (proposed) code manages connections to N servers, but has no > automatic add/remove based on changes in the cluster status. I think > this happens at a layer *just* above the actual transport. ie. > something tracks the changes in the ring status and its servers, and > transmits those changes into the transport layer, which alters its > communication with that cluster (regardless of whether that > communication is via HTTP or protobuf). > > > Okee doke. That's the end of my brain dump and future thoughts on the > transport and communication layer. I'd really like some feedback, > review, and thoughts. > > Thanks! > -g > > ___ > riak-users mailing list > riak-users@lists.basho.com > http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com > ___ riak-users mailing list riak-users@lists.basho.com http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Re: configurable prefix for consistent hashing?
Tread carefully here ... by forcing localilty ... you will sacrifice high availability by algorithmically creating a bias and a single point of failure in the cluster. Cassandra made this easier to do ... and then there were many questions on the user list about: "why is my cluster is unbalanced?" 'why is one of my nodes is falling over when the others are not?" etc... On Thu, Nov 10, 2011 at 4:25 AM, Elias Levy wrote: > On Wed, Nov 9, 2011 at 12:00 PM, wrote: > >> From: Nate Lawson >> >> >> We have been looking into ways to cluster keys to benefit from the >> LevelDB backend's prefix compression. If we were doing a batch of lookups >> and the keys from a given document could be grouped on a partition, they >> could be read with less disk IO. However, consistent hashing in Riak >> currently spreads keys randomly, based on SHA-1. >> >> What I think we could really use is a per-bucket configurable prefix >> length for consistent hashing. We would then create keys of the form >> "DocumentID:" and tell Riak to only hash DocumentID to get the >> partition. This way, keys from the same document would be clustered on the >> same partition. >> >> Is this on your roadmap already? It seems like it wouldn't require too >> many changes to Riak. >> > > If you look at the bucket properties by fetching them, you'll notice that > there is already an entry for specifying a key hashing function. Its not > documented, but you can read the Erlang code for the existing function > (pretty much just a call to SHA1), and create your own to do whatever > partitioning you want. > > > > ___ > riak-users mailing list > riak-users@lists.basho.com > http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com > > ___ riak-users mailing list riak-users@lists.basho.com http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Re: RiakEDS - one way replication?
What are some examples of use cases for one-way replication? On Tue, Dec 6, 2011 at 10:24 AM, David Smith wrote: > On Tue, Dec 6, 2011 at 8:19 AM, Kresten Krab Thorup > wrote: > > Hi folks, > > > > Quick question - can Riak EDS do one-way replication, or does it always > have to be two-way? > > Yup, it can be one-way. > > D. > > -- > Dave Smith > Director, Engineering > Basho Technologies, Inc. > diz...@basho.com > > ___ > riak-users mailing list > riak-users@lists.basho.com > http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com > ___ riak-users mailing list riak-users@lists.basho.com http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
