Hi, what's the advantage of cursors compared to key-based pagination? The latter at least allows for paginating backwards from any point. Why don't cursors just build on the same principle?
Bye, Waldemar Kornewald On Feb 16, 5:46 pm, "Nick Johnson (Google)" <[email protected]> wrote: > Hi Andy, > > 2010/2/16 Andy Freeman <[email protected]> > > > > Furthermore, it > > > seems highly probable that as things are, many people will obliviously > > > write public webapps that take a raw cursor as a parameter. This > > > could be the new SQL injection attack. > > > Can you comment a bit more on the security issues? > > > AFAIK, cursors can not be used to write anything. The cursor still > > has to match the query with its parameters, so I don't see how they > > can synthesize a cursor to see anything that they haven't already seen > > (replay) or that they'd see by requesting more and more pages (skip > > ahead). > > I was mistaken when I stated that they shouldn't be sent to the user in the > clear. As you point out, in order to use a cursor, you still have to > reconstruct the original query, so a user could not modify a cursor to cause > you to display records they should not have access to. > > > The cursor may, as part of its "is this the right query" content, > > reveal something about the query. > > > Hmm - the latter seems somewhat serious. It isn't data modification, > > but it is a data reveal. > > This is true, though I wouldn't personally consider it a serious issue. That > is up to you, naturally. > > > > > What information can someone extract from a production cursor? Does > > it contain the parameters (bad) or signatures (okay if someone can't > > derive one parameter given the other parameters). > > It contains the complete key of the next record to be returned, along with > some extra information about the query. Feel free to experiment and see for > yourself, of course. :) > > -Nick Johnson > > > > > -andy > > > On Feb 9, 9:02 am, Jeff Schnitzer <[email protected]> wrote: > > > Still, a slightly modified version of the original request does not > > > seem unreasonable. He would have to formulate his URLs something like > > > this: > > > > myblog.com/comments/?q=the&first=1234 > > > > or maybe: > > > > myblog.com/comments/?q=the&after=1234 > > > > I could see this being really useful, since encrypting (or worse, > > > storing on the server) the cursor is pretty painful. Furthermore, it > > > seems highly probable that as things are, many people will obliviously > > > write public webapps that take a raw cursor as a parameter. This > > > could be the new SQL injection attack. > > > > Jeff > > > > 2010/2/9 Alkis Evlogimenos ('Αλκης Ευλογημένος) <[email protected]>:> > > If the cursor had to skip entries by using an offset, its performance would > > > > depend on the size of the offset. This is what the current > > Query.fetch() api > > > > is doing when you give it an offset. A cursor is a pointer to the entry > > from > > > > which the next query will start. It has no notion of offset. > > > > On Tue, Feb 9, 2010 at 4:07 PM, Nickolas Daskalou <[email protected]> > > wrote: > > > > >> Does the production cursor string contain information about the app > > id, > > > >> kind, any filter()s or order()s, and (more importantly) some sort of > > > >> numerical value that indicates how many records the next query should > > > >> "skip"? If so, and if we could extract this information (and then use > > it > > > >> again to the reconstruct the cursor), that would make for much > > cleaner, > > > >> safer and intuitive URLs than including the entire cursor string (or > > some > > > >> sort of encrypted/encoded cursor string replacement). > > > > >> 2010/2/10 Nick Johnson (Google) <[email protected]> > > > > >>> Hi Nickolas, > > > > >>> 2010/2/9 Nickolas Daskalou <[email protected]> > > > > >>>> I'd want to do this so that I could include parts of the cursor > > (such as > > > >>>> the offset) into a URL without including other parts (eg. the model > > kind and > > > >>>> filters). I could then reconstruct the cursor on the server side > > based on > > > >>>> what was passed into the URL. > > > > >>> The offset argument you're talking about is specific to the > > > >>> dev_appserver's implementation of cursors. In production, offsets are > > not > > > >>> used, so this won't work. > > > >>> -Nick Johnson > > > > >>>> For example, if I was searching for blog comments that contained the > > > >>>> word "the" (with the default order being the creation time, > > descending), the > > > >>>> URL might look like this: > > > > >>>> myblog.com/comments/?q=the > > > > >>>> With model: > > > > >>>> class Comment(db.Model): > > > >>>> .... > > > >>>> created_at = db.DateTimeProperty(auto_now_add=True) > > > >>>> words = db.StringListProperty() # A list of all the words in a > > comment > > > >>>> (forget about exploding indexes for now) > > > >>>> ... > > > > >>>> The query object for this URL might look something like: > > > > >>>> .... > > > >>>> q = > > > Comment.all().filter('words',self.request.get('q')).order('-created_at') > > > >>>> .... > > > > >>>> To get to the 1001st comment, it'd be good if the URL looked > > something > > > >>>> like this: > > > > >>>> myblog.com/comments/?q=the&skip=1000 > > > > >>>> instead of: > > > > >>>> myblog.com/comments/?q=the&cursor=[something ugly] > > > > >>>> so that when the request comes in, I can do this: > > > > >>>> .... > > > >>>> q = > > > Comment.all().filter('words',self.request.get('q')).order('-created_at') > > > >>>> cursor_template = q.cursor_template() > > > >>>> cursor = > > > db.Cursor.from_template(cursor_template,offset=int(self.request.get('skip') > > )) > > > >>>> .... > > > >>>> (or something along these lines) > > > > >>>> Does that make sense? > > > > >>>> On 10 February 2010 01:03, Nick Johnson (Google) > > > >>>> <[email protected]> wrote: > > > > >>>>> Hi Nickolas, > > > > >>>>> 2010/2/9 Nickolas Daskalou <[email protected]> > > > > >>>>>> Will we be able to construct our own cursors much the same way > > that we > > > >>>>>> are able to construct our own Datastore keys (Key.from_path())? > > > > >>>>> No, not practically speaking. > > > > >>>>>> Also along the same lines, will we be able to "deconstruct" a > > cursor > > > >>>>>> to get its components (offset, start_inclusive etc.), as we can > > now do with > > > >>>>>> keys (key.name(), key.id(), key.kind() etc.)? > > > > >>>>> While you could do this, there's no guarantees that it'll work (or > > > >>>>> continue to work), as you'd be digging into internal implementation > > details. > > > >>>>> Why do you want to do this? > > > >>>>> -Nick Johnson > > > > >>>>>> 2010/2/9 Nick Johnson (Google) <[email protected]> > > > > >>>>>>> 2010/2/9 Stephen <[email protected]> > > > > >>>>>>>> I'm asking if it's wise to store it as a query parameter > > embedded in > > > >>>>>>>> a > > > >>>>>>>> web page. > > > > >>>>>>> You're right that it's unwise. Depending on how you construct > > your > > > >>>>>>> query, a user could potentially modify the cursor they send to > > you to return > > > >>>>>>> results from any query your datastore is capable of performing, > > which could > > > >>>>>>> result in you revealing information to the user that they > > shouldn't know. > > > >>>>>>> You should either store the cursor on the server-side, or encrypt > > it before > > > >>>>>>> sending it to the client. > > > >>>>>>> I was going to mention something about this in my post, but it > > > >>>>>>> slipped my mind. > > > >>>>>>> -Nick Johnson > > > > >>>>>>>> On Feb 9, 12:26 am, "Ikai L (Google)" <[email protected]> > > wrote: > > > >>>>>>>> > A cursor serializes to a Base64 encoded String, so you can > > store > > > >>>>>>>> > it anywhere > > > >>>>>>>> > you want to store strings: Memcached, Datastore, etc. You can > > even > > > >>>>>>>> > pass it > > > >>>>>>>> > as an URL parameter to task queues. > > > > >>>>>>>> > 2010/2/8 Stephen <[email protected]> > > > > >>>>>>>> > > Ah right, Nick's blog does say start_key and not offset. My > > bad. > > > > >>>>>>>> > > Maybe there will be warnings in the upcoming documentation, > > but > > > >>>>>>>> > > my > > > >>>>>>>> > > first instinct was to embed the serialised cursor in the > > HTML as > > > >>>>>>>> > > the > > > >>>>>>>> > > 'next' link. But that doesn't look like a good idea as > > Nick's > > > >>>>>>>> > > decoded > > > >>>>>>>> > > query shows what's embedded: > > > > >>>>>>>> > > PrimaryScan { > > > >>>>>>>> > > start_key: "shell\000TestModel\000foo\000\232bar\000\200" > > > >>>>>>>> > > start_inclusive: true > > > >>>>>>>> > > } > > > >>>>>>>> > > keys_only: false > > > > >>>>>>>> > > First, you may or may not want to leak this info. Second, > > could > > > >>>>>>>> > > this > > > >>>>>>>> > > be altered on the client to change the query in any way > > that's > > > >>>>>>>> > > undesirable? > > > > >>>>>>>> > > Once you have a cursor, where do you store it so you can use > > it > > > >>>>>>>> > > again? > > > > >>>>>>>> > > On Feb 8, 10:17 pm, "Ikai L (Google)" <[email protected]> > > wrote: > > > >>>>>>>> > > > I got beaten to this answer. No, there is no traversal to > > get > > > >>>>>>>> > > > to the > > > >>>>>>>> > > offset. > > > > >>>>>>>> > > > BigTable has an underlying mechanism for range queries on > > > >>>>>>>> > > > keys. Indexes > > > >>>>>>>> > > are > > > >>>>>>>> > > > essentially a key comprised of a concatenation of > > application > > > >>>>>>>> > > > ID, entity > > > >>>>>>>> > > > type, column, value. When a filter operation is performed, > > the > > > >>>>>>>> > > > datastore > > > >>>>>>>> > > > looks for a range matching this criteria, returning the > > set of > > > >>>>>>>> > > > keys. A > > > >>>>>>>> > > > cursor also adds the datastore key of the entity so it is > > > >>>>>>>> > > > possible to > > > >>>>>>>> > > > serialize where to begin the query. This is actually a bit > > > >>>>>>>> > > > awkward to > > > >>>>>>>> > > > explain without visuals. You can watch Ryan Barrett's talk > > > >>>>>>>> > > > here: > > > > >>>>>>>> > > >http://www.youtube.com/watch?v=tx5gdoNpcZM > > > > >>>>>>>> > > > Hopefully, we'll be able to post an article at some point > > in > > > >>>>>>>> > > > the future > > > >>>>>>>> > > > explaining how cursors work. > > > > >>>>>>>> > > > 2010/2/8 Alkis Evlogimenos ('Αλκης Ευλογημένος) > > > >>>>>>>> > > > <[email protected]> > > > > >>>>>>>> > > > > There is no offset. The protocol buffer stores a > > start_key... > > read more » -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to [email protected]. To unsubscribe from this group, send email to [email protected]. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
