Brian,
Yes I read about the hack somewhere in your documentation. My understanding
is that the link walking operation will cease to work via HTTP after links
to a node grow beyond a particular number. This happens because a HTTP
header is used to send link related data and there are limits around how
much data a header can hold.
I was interested in exploring if the PB client can overcome this
limitation. The use case I am interested in is this - If a single node
links to 10k nodes, how much time would it take a link walker to visit all
nodes via the PB client ? What sort of client would you recommend for
something like this ?
I know that linking is only meant to be used as a lightweight feature.
Would 10k links to a node be considered lightweight ?
Thanks
Deepak
On Tue, May 29, 2012 at 9:33 PM, Brian Roach <ro...@basho.com> wrote:
> Deepak -
>
> I'll take a look at it this week, but more than likely it's a bug.
>
> Link walking is a REST only operation as far as Riak’s interfaces are
> concerned. Link Walking in the protocol buffers Java client is a hack that
> issues two m/r jobs to the protocol buffers interface (the first constructs
> the inputs to the second by walking the links, the second returns the data).
>
> Thanks,
> Brian Roach
>
> On May 27, 2012, at 7:19 AM, Deepak Balasubramanyam wrote:
>
> > This looks like a bug. The code to walk links via a HTTP client works
> perfectly. The same code fails when the PB client is used. The POJO
> attached in this email reproduces the problem.
> >
> > I searched the email archives and existing issues and found no trace of
> this problem. Please run the POJO by swapping the clients returned from the
> getClient() method to reproduce the problem. I can create a bug report once
> someone from the dev team confirms this really is a bug.
> >
> > Riak client pom:
> > <dependency>
> > <groupId>com.basho.riak</groupId>
> > <artifactId>riak-client</artifactId>
> > <version>1.0.5</version>
> > </dependency>
> >
> > Riak server version - 1.1.2. Built from source.
> > 4 nodes running on 1 machine. OS - Linux mint.
> >
> > On Sun, May 27, 2012 at 10:05 AM, Deepak Balasubramanyam <
> deepak.b...@gmail.com> wrote:
> > Hi,
> >
> > I have a cluster that contains 2 buckets. A bucket named 'usersMine'
> contains the key 'user2', which is linked to several keys (about 10) under
> a bucket named userPreferences. The relationship exists under the name
> 'myPref'. A user and a preference have String values.
> >
> > I can successfully traverse the link over HTTP using the following URL -
> >
> > curl -v localhost:8091/riak/usersMine/user2/_,myPref,1
> >
> > ------------------------ ------------------------
> ------------------------
> > > User-Agent: curl/7.21.6 (i686-pc-linux-gnu) libcurl/7.21.6
> OpenSSL/1.0.0e zlib/1.2.3.4 libidn/1.22 librtmp/2.3
> > > Host: localhost:8091
> > > Accept: */*
> > >
> > < HTTP/1.1 200 OK
> > < Server: MochiWeb/1.1 WebMachine/1.9.0 (someone had painted it blue)
> > < Expires: Sun, 27 May 2012 04:12:39 GMT
> > < Date: Sun, 27 May 2012 04:02:39 GMT
> > < Content-Type: multipart/mixed; boundary=IYGfKNqjGdco9ddfyjRP1Utzfi2
> > < Content-Length: 3271
> > <
> >
> > --IYGfKNqjGdco9ddfyjRP1Utzfi2
> > Content-Type: multipart/mixed; boundary=3YVES0x2tFnUDOdTzfn1OGS6uMt
> >
> > --3YVES0x2tFnUDOdTzfn1OGS6uMt
> > X-Riak-Vclock: a85hYGBgzGDKBVIcMRuuc/nvy7mSwZTImMfKUKpodpIvCwA=
> > Location: /riak/userPreferences/preference3004
> > Content-Type: text/plain; charset=UTF-8
> > Link: </riak/userPreferences>; rel="up"
> > Etag: 5GucnGSk4TjQc8BO1eNLyI
> > Last-Modified: Sun, 27 May 2012 03:54:29 GMT
> >
> > junk
> >
> > << ------------------------ Truncated ------------------------ >>
> > junk
> > --3YVES0x2tFnUDOdTzfn1OGS6uMt--
> >
> > --IYGfKNqjGdco9ddfyjRP1Utzfi2--
> > ------------------------ ------------------------
> ------------------------
> >
> > However when I use the java client to walk the link, I get a
> ClassCastException.
> >
> > ------------------------ ------------------------
> ------------------------
> > Java code:
> > ------------------------ ------------------------
> ------------------------
> > private void getAllLinks()
> > {
> > String user="user2";
> > IRiakClient riakClient = null;
> > try
> > {
> > long past = System.currentTimeMillis();
> > riakClient = RiakFactory.pbcClient("localhost",8081);
> > Bucket userBucket =
> riakClient.fetchBucket("usersMine").execute();
> > DefaultRiakObject user1 =(DefaultRiakObject)
> userBucket.fetch(user).execute();
> > List links = user1.getLinks();
> > System.out.println(links.size());
> > WalkResult execute =
> riakClient.walk(user1).addStep("userPreferences", "myPref",true).execute();
> > Iterator iterator = execute.iterator();
> > while(iterator.hasNext())
> > {
> > Object next = iterator.next();
> > System.out.println(next);
> > }
> > long now = System.currentTimeMillis();
> > System.out.println("Retrieval in " + (now-past) + " ms");
> > }
> > catch (Exception e)
> > {
> > e.printStackTrace();
> > }
> > finally
> > {
> > if(riakClient != null)
> > {
> > riakClient.shutdown();
> > }
> > }
> > }
> >
> > ------------------------ ------------------------
> ------------------------
> > Stack:
> > ------------------------ ------------------------
> ------------------------
> > java.lang.ClassCastException: java.lang.String cannot be cast to
> java.util.List
> > at
> com.basho.riak.client.raw.pbc.PBClientAdapter.linkWalkSecondPhase(PBClientAdapter.java:380)
> > at
> com.basho.riak.client.raw.pbc.PBClientAdapter.linkWalk(PBClientAdapter.java:325)
> > at com.basho.riak.client.query.LinkWalk.execute(LinkWalk.java:63)
> > at
> com.chatterbox.persistence.riak.RiakTest.getAllLinks(RiakTest.java:81)
> > at com.chatterbox.persistence.riak.RiakTest.main(RiakTest.java:25)
> > ------------------------ ------------------------
> ------------------------
> >
> > At line 380 on PBClientAdapter, an enhanced for loop attempts to iterate
> over a Collection<List<String>> while the actual value is a List<String>.
> This causes the exception. The value of the variable 'step' at the time of
> the exception is [userPreferences, preference3000, myPref].
> >
> > Am i doing something wrong ? The java code retrieves links in a fashion
> that is similar to the HTTP retrieval, so I am puzzled by the ClassCast.
> >
> > Thanks
> > Deepak
> >
> > <RiakTest.java>_______________________________________________
> > riak-users mailing list
> > riak-users@lists.basho.com
> > http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
>
>
_______________________________________________
riak-users mailing list
riak-users@lists.basho.com
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
