Riak as Binary File Store

[Praveen Baratam]

Hello Everybody!

I have read abundantly over the web that Riak is very well suited to store
and retrieve small binary objects such as images, docs, etc.

In our scenario we are planning to use Riak to store uploads to our portal
which is a Social Network. Uploads are mostly images with maximum size of 2
MB and typical size ranges between few KBs to few 100 KBs.

Does this usage pattern fit Riak? What are the caveats if any?

Thank you!
___
riak-users mailing list
riak-users@lists.basho.com
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com

Re: Riak as Binary File Store

[Praveen Baratam]

Thank you everybody for the feedback.

I am planning to write binary data to Riak from the application server
which handles upload. On the read side I am planning to use Apache with
mod_proxy and mod_rewrite to directly fetch from Riak and serve the files
without any other application server / scripting overhead.

I feel this is the least overhead path to using Riak as a file store.

Comments please.

On Thu, May 31, 2012 at 2:48 PM, Praveen Baratam
wrote:

> Thank you everybody for the feedback.
>
> I am planning to write binary data to Riak from the application server
> which handles upload. On the read side I am planning to use Apache with
> mod_proxy and mod_rewrite to directly fetch from Riak and serve the files
> without any other application server / scripting overhead.
>
> I feel this is the least overhead path to using Riak as a file store.
>
> Comments please.
>
>
> On Tue, May 29, 2012 at 10:28 PM, Mark Phillips  wrote:
>
>>
>>
>> On Tue, May 29, 2012 at 12:51 PM, Vlad Gorodetsky  wrote:
>>
>>> I've read somewhere here on the mailing list that storing blobs that
>>> are more than 50KB isn't recommended.
>>> Is that correct? If so, is it something specific to storage backend?
>>>
>>>
>> Riak can probably handle objects up to about 10MBs. That said, depending
>> on your hardware and network, the functional limit is probably smaller than
>> that. Also, keep in mind n_val when writing large values. (i.e., 5MB with
>> an n_val of 3 = 15MB across the wire).
>>
>> Mark
>>
>>
>>> ~Vlad
>>>
>>> On Tue, May 29, 2012 at 3:51 PM, Alvaro Videla 
>>> wrote:
>>> > Like this perhaps: https://github.com/johnthethird/riak-fuse*cough* 
>>> > *cough*
>>> >
>>> >
>>> > On Tue, May 29, 2012 at 2:49 PM, Shuhao Wu  wrote:
>>> >>
>>> >> It'll be interesting if you can write a filesystem on top of Riak.
>>> >>
>>> >> That would be a cool project to see on github :P
>>> >>
>>> >> Shuhao
>>> >>
>>> >>
>>> >> On Tue, May 29, 2012 at 8:36 AM, Praveen Baratam
>>> >>  wrote:
>>> >>>
>>> >>> Hello Everybody!
>>> >>>
>>> >>> I have read abundantly over the web that Riak is very well suited to
>>> >>> store and retrieve small binary objects such as images, docs, etc.
>>> >>>
>>> >>> In our scenario we are planning to use Riak to store uploads to our
>>> >>> portal which is a Social Network. Uploads are mostly images with
>>> maximum
>>> >>> size of 2 MB and typical size ranges between few KBs to few 100 KBs.
>>> >>>
>>> >>> Does this usage pattern fit Riak? What are the caveats if any?
>>> >>>
>>> >>> Thank you!
>>> >>>
>>> >>> ___
>>> >>> riak-users mailing list
>>> >>> riak-users@lists.basho.com
>>> >>> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
>>> >>>
>>> >>
>>> >>
>>> >> ___
>>> >> riak-users mailing list
>>> >> riak-users@lists.basho.com
>>> >> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
>>> >>
>>> >
>>> >
>>> > ___
>>> > riak-users mailing list
>>> > riak-users@lists.basho.com
>>> > http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
>>> >
>>>
>>> ___
>>> riak-users mailing list
>>> riak-users@lists.basho.com
>>> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
>>>
>>
>>
>> ___
>> riak-users mailing list
>> riak-users@lists.basho.com
>> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
>>
>>
>
___
riak-users mailing list
riak-users@lists.basho.com
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com

Riak app.config

[Praveen Baratam]

Hello!

I Installed Riak using the deb package and found that the default
app.config does not have the necessary stubs for settings such as
default_bucket_properties, ring_creating_size, etc. Can some one upload a
complete sample app.config.

Thank you.
___
riak-users mailing list
riak-users@lists.basho.com
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com

404 Error in Live Cluster

[Praveen Baratam]

Hello Everybody,

We are currently alpha testing out our Webapp which uses Riak for storing
binary files (< 2 MB) with n_val as 2. We currently have 3 nodes in the
cluster and all files are stored in the same bucket. We are using Bitcask
engine.

Sometimes we see 404 error cropping up when accessing these keys. This
happens only once in a day or so and all existing keys are accessible most
of the time.  We are only trying to access the keys that we have put in
Riak so there is no doubt of their existence.

Can a network split cause such an error? Is there anything we can do to
avoid such corner cases.

Thank you

Praveen
___
riak-users mailing list
riak-users@lists.basho.com
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com

Re: 404 Error in Live Cluster

[Praveen Baratam]

Here are some more details about the cluster.

{ring_creation_size, 1024},

{default_bucket_props, [
 {n_val, 2},
 {r, 1},
 {w, 1},
 {allow_mult, false},
 {last_write_wins, false},
 {precommit, []},
 {postcommit, []},
 {chash_keyfun, {riak_core_util, chash_std_keyfun}},
 {linkfun, {modfun, riak_kv_wm_link_walker, mapreduce_linkfun}}
]},

{storage_backend, riak_kv_multi_backend},
{multi_backend_default, <<"store">>},
{multi_backend, [
{<<"store">>, riak_kv_bitcask_backend, [
   {data_root, "/var/lib/riak/bitcask/store"}
]},
{<<"cache">>, riak_kv_bitcask_backend, [
   {data_root, "/var/lib/riak/bitcask/cache"},
   {expiry_secs, 21600}
]}
]},

On using Riaknostic I am seeing these errors.

[warning] The following preflists do not satisfy the n_val:
[[{973382926425464638881751031164555683013033197568,'riak@178.238.237.241'},.
a big list similar to this...]

Please help. We are stuck.

Praveen



On Mon, Sep 17, 2012 at 4:18 AM, Praveen Baratam
wrote:

> Hello Everybody,
>
> We are currently alpha testing out our Webapp which uses Riak for storing
> binary files (< 2 MB) with n_val as 2. We currently have 3 nodes in the
> cluster and all files are stored in the same bucket. We are using Bitcask
> engine.
>
> Sometimes we see 404 error cropping up when accessing these keys. This
> happens only once in a day or so and all existing keys are accessible most
> of the time.  We are only trying to access the keys that we have put in
> Riak so there is no doubt of their existence.
>
> Can a network split cause such an error? Is there anything we can do to
> avoid such corner cases.
>
> Thank you
>
> Praveen
>
___
riak-users mailing list
riak-users@lists.basho.com
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com

Re: 404 Error in Live Cluster

[Praveen Baratam]

The frequency of error is now more common. Upto 1 failed request in 10.
This is breaking everything.

On Mon, Sep 17, 2012 at 3:04 PM, Praveen Baratam
wrote:

> Here are some more details about the cluster.
>
> {ring_creation_size, 1024},
>
> {default_bucket_props, [
>  {n_val, 2},
>  {r, 1},
>  {w, 1},
>  {allow_mult, false},
>  {last_write_wins, false},
>  {precommit, []},
>  {postcommit, []},
>  {chash_keyfun, {riak_core_util, chash_std_keyfun}},
>  {linkfun, {modfun, riak_kv_wm_link_walker, mapreduce_linkfun}}
> ]},
>
> {storage_backend, riak_kv_multi_backend},
> {multi_backend_default, <<"store">>},
> {multi_backend, [
> {<<"store">>, riak_kv_bitcask_backend, [
>{data_root, "/var/lib/riak/bitcask/store"}
> ]},
> {<<"cache">>, riak_kv_bitcask_backend, [
>{data_root, "/var/lib/riak/bitcask/cache"},
> {expiry_secs, 21600}
> ]}
> ]},
>
> On using Riaknostic I am seeing these errors.
>
> [warning] The following preflists do not satisfy the n_val:
> [[{973382926425464638881751031164555683013033197568,'riak@178.238.237.241'},.
> a big list similar to this...]
>
> Please help. We are stuck.
>
> Praveen
>
>
>
> On Mon, Sep 17, 2012 at 4:18 AM, Praveen Baratam <
> praveen.bara...@gmail.com> wrote:
>
>> Hello Everybody,
>>
>> We are currently alpha testing out our Webapp which uses Riak for storing
>> binary files (< 2 MB) with n_val as 2. We currently have 3 nodes in the
>> cluster and all files are stored in the same bucket. We are using Bitcask
>> engine.
>>
>> Sometimes we see 404 error cropping up when accessing these keys. This
>> happens only once in a day or so and all existing keys are accessible most
>> of the time.  We are only trying to access the keys that we have put in
>> Riak so there is no doubt of their existence.
>>
>> Can a network split cause such an error? Is there anything we can do to
>> avoid such corner cases.
>>
>> Thank you
>>
>> Praveen
>>
>
>
___
riak-users mailing list
riak-users@lists.basho.com
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com

Changing app.config of Live Riak Cluster

[Praveen Baratam]

Hello Everybody,

We are using Riak to store small files (<2MB) in our stack.

Currently its configured with "*multi_backend*" and two buckets as follows-


{storage_backend, riak_kv_multi_backend},
{multi_backend_default, <<"store">>},
{multi_backend, [
{<<"store">>, riak_kv_bitcask_backend, [
   {data_root, "/var/lib/riak/bitcask/store"}
]},
{<<"cache">>, riak_kv_bitcask_backend, [
   {data_root, "/var/lib/riak/bitcask/cache"},
   {expiry_secs, 21600}
]}
]},
-

Now we want to delete the "*cache*" bucket and add another bucket "*temp*"
for another purpose.

--

   {<<"temp">>, riak_kv_bitcask_backend, [
   {data_root, "/var/lib/riak/bitcask/temp"},
   {expiry_secs, 604800}
]}
-

I am aware that I can do it using the HTTP Buckets API but can I shut down
all nodes, modify the app.config on all nodes and restart the cluster
safely with out losing any data?

Thanks in advance.

Praveen
___
riak-users mailing list
riak-users@lists.basho.com
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com

Riak bind to hostname instead of IP Address!!

[Praveen Baratam]

Hello All,

I found it on the FAQ that using hostnames instead of IP addresses will
incur a overhead! Will it be a significant overhead? I believe that if a
hostname is queried during startup, the same will be used through the life
of the VM or in an optimized case till the DNS TTL expires!!

Best,

Praveen Baratam

about.me <http://about.me/praveen.baratam>
ᐧ
___
riak-users mailing list
riak-users@lists.basho.com
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com

Re: Riak bind to hostname instead of IP Address!!

[Praveen Baratam]

Just to clarify... I was referring to...

Is it legal to bind Riak to a hostname instead of IP?


Yes, it's legal but it will incur the overhead of the lookup. If you're
talking about the HTTP/PBC interfaces, it's best to use IP addresses, but
for the node name, it's totally fine to use a hostname.
Will it be a significant overhead? I believe the it will just be a function
call after the initial lookup rather than a network call.

Praveen
ᐧ

On Wed, Apr 29, 2015 at 12:40 AM, Praveen Baratam  wrote:

> Hello All,
>
> I found it on the FAQ that using hostnames instead of IP addresses will
> incur a overhead! Will it be a significant overhead? I believe that if a
> hostname is queried during startup, the same will be used through the life
> of the VM or in an optimized case till the DNS TTL expires!!
>
> Best,
>
> Praveen Baratam
>
> about.me <http://about.me/praveen.baratam>
> ᐧ
>



-- 
Dr. Praveen Baratam

about.me <http://about.me/praveen.baratam>
___
riak-users mailing list
riak-users@lists.basho.com
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com

Loopback Address as Handoff IP/Interface

[Praveen Baratam]

Hello Everybody,

I am trying to setup a single node Riak cluster and want to use a loopback
address (127.0.0.x) for the Handoff IP to keep the instance private and
invisible to others.

But the Riak node fails to start and throws an error -  "handoff.ip
invalid, must be a valid IP address"

Is it illegal to use loopback interface for Handoff IP? Came across some JIRA
issues  but couldn't find a
solution with out having to build packages from source.

Any workarounds? I am trying to use Riak 1.4.12 on Ubuntu 14.04!

Any advice in this regard will be greatly helpful.

Thank you.

Praveen

about.me 
ᐧ
___
riak-users mailing list
riak-users@lists.basho.com
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com

Riak 1.4.12 - Missing keys...

[Praveen Baratam]

Hello everyone,

I have setup a Riak test node in a VPS with n = 1, r = 1 and w =1, Bitcask
engine and AAE turned off.. loaded some 8k blobs into it and everything was
fine...

Today, after three weeks, I noticed that a few of those 8K blobs are
missing - not found...

I also see a lot of invalid HintFile error in the console.log

Can anybody explain why this is happening?

Thanks in advance.

Best,

Praveen Baratam

about.me <http://about.me/praveen.baratam>
ᐧ
___
riak-users mailing list
riak-users@lists.basho.com
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com