Google Summer of Code for Lasp

[Christopher Meiklejohn]

There's only a few days left to apply, but we're looking for more
students to come be part of the Google Summer of Code project for
Lasp.

In terms of proposals, I'm happy to help any eager students who are
interested in submitting something write a proposal.

For ideas, we've got a lot of things that are interesting happening:

* I'd like to see someone bring SSL/TLS support to our partisan
library -- partisan is a highly scalable work-in-progress replacement
for distributed Erlang for edge and mobile computing.

* We've got a newish work-in-progress report out on our Loquat system
-- recently presented at Erlang Factory 2016 in SF -- and we've love
to have students help us build out a first functioning prototype of
the system and evaluate it.

* Documentation: help us write documentation about how to use Lasp
from Erlang and Elixir.

* Finally, we'd love to see some students work on building real
applications in Lasp: we've got several ideas for applications, but
haven't had time to implement them yet.  (Ideally, we'd love to see
these example applications in Elixir and Erlang!)

Last year was super fun: not only did we have great success with two
student projects, one of them resulted in a draft paper and
presentation at the AGERE! workshop located at ACM's SPLASH
conference.

[Working on open source, on a research project, where your results
will possibly contribute to a conference paper and you as co-author,
not bad!]

Please reach out, and I'm looking forward to your submissions!

Thanks,
Christopher Meiklejohn

___
riak-users mailing list
riak-users@lists.basho.com
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com

Re: RiakTS Query Question

[Andrei Zavada]

Hi Joe,

First of all, updates in Riak TS are not safe. There are write-once
optimizations which take shortcuts in the logic that ensures your
writes are consistent. In the simplest scenario of a single client
doing the writes at one and the same node of a cluster that completes
the writing faster than the client issues a new request, and no
partitions occur, data will be successfully overwritten. If any of the
above conditions are not met, you may see old values in subsequent
reads.

That said, there is no difference, in terms of amount of data going
from coordinator to the leveldb nodes, between initial (clean) writes
and overwrites. If you do a deletion before writing, however, the full
record will be read (but not delivered from the eleveldb node to the
coordinator). The latter issue was fixed in
https://github.com/basho/riak_kv/pull/1630 and should be gone in 1.6.

Regards,
Andrei

On Thu, Mar 30, 2017 at 5:06 PM, Joe Olson  wrote:
>>So no, the 1M blobs will be read from the leveldb backend, and they will
>> hog the bandwidth between storage and coordinator nodes.
>
>
> Ok that is what I thought. Thanks for the info.
>
>
> One more question...what is the penalty for doing an in place update (i.e.
> an insert of the same exact keys) of the BLOB in the same situation? I have
> a new blob, and I want to overwrite the BLOB field with a new one. Obviously
> I'll have to pay to shuffle the new blob around the cluster. Am I paying the
> same as it were a clean, new insert? Do I get to re-use the existing
> (already sorted in its correct place) record, and just update the object
> component? Or when I do a duplicate insert, am I paying the price for a
> delete + insert?
>
>
> Thanks again!
>
> 
> From: Andrei Zavada 
> Sent: Wednesday, March 29, 2017 3:14:58 PM
> To: Alexander Sicular
> Cc: Joe Olson; riak-users@lists.basho.com
> Subject: Re: RiakTS Query Question
>
> Joe,
>
> TS records in a given table all have the same structure and are stored
> and retrieved as single objects (in Riak KV sense); the backend cannot
> introspect them and only extract some fields.
>
> Full records are read from backend and these are delivered, in chunks,
> to the coordinator node (the node your client has connected to). On
> receipt, columns not appearing in the SELECT clause are dropped; thus,
> only the relevant columns are kept while the coordinator collects the
> remaining chunks. From there, records are shipped to the client (via
> temp tables if the query had an ORDER BY or LIMIT clause).
>
> So no, the 1M blobs will be read from the leveldb backend, and they
> will hog the bandwidth between storage and coordinator nodes. They
> will not appear in the traffic going from the coordinator to the
> client, though.
>
> Andrei
>
> On Wed, Mar 29, 2017 at 10:17 PM, Alexander Sicular 
> wrote:
>> I'm not 100% certain but I do not believe that is the case. Part of the
>> reason for structured data is efficient retrieval. I believe the data is
>> read but only the data selected leaves the leveldb backend, unselected
>> data
>> never leaves leveldb so there's no overhead when passing data from level
>> to
>> Riak or on the network.
>>
>> I defer to the engineering folks working on TS tho.
>>
>> -Alexander
>>
>> @siculars
>> http://siculars.posthaven.com
>>
>> Sent from my iRotaryPhone
>>
>> On Mar 29, 2017, at 15:08, Joe Olson  wrote:
>>
>> Suppose I have the following table in RiakTS:
>>
>>
>> CREATE TABLE T1 (
>>
>> idVARCHAR NOT NULL,
>>
>> eventtime TIMESTAMP NOT NULL,
>>
>> field2 SINT64,
>>
>> data   BLOB NOT NULL,
>>
>> primary key (id, QUANTUM(eventtime, 365, 'd')),id)
>>
>> )
>>
>>
>> Assume the BLOB field is close to the max size for a RiakTS BLOB value
>> (~1MB)
>>
>>
>> Suppose I want to execute the following query:
>>
>>
>> Select id, eventtime, field2 from T1 where ((id = ID1) and (eventtime >=
>> T1
>> and eventtime < T2))
>>
>>
>> I only want the SINT64 field, not the 1MB BLOB.
>>
>>
>> Am I paying for the bandwidth for the RiakTS cluster to pass around
>> (internally) the 1MB BLOB field just to get the SINT64 field?
>>
>>
>> If so, is there a way to avoid this, besides creating a second table
>> without
>> the BLOB field?
>>
>> ___
>> riak-users mailing list
>> riak-users@lists.basho.com
>> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
>>
>>
>> ___
>> riak-users mailing list
>> riak-users@lists.basho.com
>> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
>>

___
riak-users mailing list
riak-users@lists.basho.com
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com

Problem with Solr Search

[Nguyễn Văn Nghĩa Em]

Dear All,

I use Riak for my company's production, this is my environment:
- Server: cluster of 5 Riak nodes.
- Each node have the same hardware resource: c4.2xlarge amazon instance
- Riak version: 2.1.1
- Backend: Bitcask
- Search enabled via yokozuna
- Bucket type setting:

> young_vclock: 20
> w: quorum
> small_vclock: 50
> rw: quorum
> r: quorum
> pw: 0
> precommit: []
> pr: 0
> postcommit: []
> old_vclock: 86400
> notfound_ok: true
> n_val: 3
> linkfun: {modfun,riak_kv_wm_link_walker,mapreduce_linkfun}
> last_write_wins: false
> dw: quorum
> dvv_enabled: true
> chash_keyfun: {riak_core_util,chash_std_keyfun}
> big_vclock: 50
> basic_quorum: false
> backend: <<"bitcask_mult">>
> allow_mult: true
> datatype: map
> active: true
>

I have met some problems:
1. Create search index with error: "Can not create search index on all
nodes". This error appear after 30 seconds timeout. I opened the index
folder in /var/lib/riak/yz, I found that there are some folder without
core.properties file. I don't know whether it is normal or not.

2. When I make a searching load test with Riak Benchmark tool with about
1000 Request Per second. I see about 5-10% error which relate to "Failed to
search". But when I make a load test with Solr HTTP API, the throuput is
about 2000 Request Per Second without any error. It may be the problem in
Riak config. I hope someone can show me why this error happend


Thanks for any help.
___
riak-users mailing list
riak-users@lists.basho.com
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com