Brandom, please cut message tails since it not readable :) http://groups.google.com/group/google-appengine/browse_thread/thread/986e19cd7366a655
Example: We aren't real bright here. We didn't do much with tasks until recently J We knew we should but to make our code work on multiple platforms we had avoided it. Now we are starting to branch our code so that functions detect which platform they are on and to things better. So we will look in to using your logic. Brandon Wirtz BlackWaterOps: President / Lead Mercenary Description: http://www.linkedin.com/img/signature/bg_slate_385x42.jpg Work: 510-992-6548 Toll Free: 866-400-4536 IM: drak...@gmail.com (Google Talk) Skype: drakegreene YouTube: <http://www.youtube.com/blackwateropsdotcom> BlackWaterOpsDotCom <http://www.blackwaterops.com/> BlackWater Ops <http://www.cloudonastring.com/> Cloud On A String Mastermind Group From: google-appengine@googlegroups.com [mailto:google-appengine@googlegroups.com] On Behalf Of Robert Kluin Sent: Wednesday, February 01, 2012 10:41 PM To: google-appengine@googlegroups.com Subject: Re: [google-appengine] App Engine SDK 1.6.2 is out! Ouch. By the way, a faster pattern looks like this: while there's stuff to delete: do a keys only query (using a cursor to continue) insert one or more tasks to delete those keys insert a task and break from the while to continue if there are still more to delete (to prevent deadline errors) You can speed that up further by sharding the key space in the keys only query, which will yield better parallelism. Robert On Thu, Feb 2, 2012 at 01:18, Brandon Wirtz <drak...@digerat.com> wrote: The version I have goes like this. Yourapp.appspot.com/psuedodroptable/Name Then the code logic says For all batches of maximum size delete async Next batch We ran it the person hitting it's Chrome crashed an hour later (unrelated) they came back in the morning Got the "Chrome has crashed would you like to restore?" "Yes why note" and they deleted all the data we had spent 18 hours putting in to the system. well not all the data, since we were still loading it, so new data wasn't deleted cause it wasn't in the batch J Brandon Wirtz BlackWaterOps: President / Lead Mercenary Description: http://www.linkedin.com/img/signature/bg_slate_385x42.jpg Work: 510-992-6548 Toll Free: 866-400-4536 IM: drak...@gmail.com (Google Talk) Skype: drakegreene YouTube: <http://www.youtube.com/blackwateropsdotcom> BlackWaterOpsDotCom <http://www.blackwaterops.com/> BlackWater Ops <http://www.cloudonastring.com/> Cloud On A String Mastermind Group From: google-appengine@googlegroups.com [mailto:google-appengine@googlegroups.com] On Behalf Of Robert Kluin Sent: Wednesday, February 01, 2012 10:11 PM To: google-appengine@googlegroups.com Subject: Re: [google-appengine] App Engine SDK 1.6.2 is out! WTF, a confirmation? Coward. I'd also like to see a magic "reset" button, especially one that was fast and didn't cost a fortune. Would sure be nice for testing purposes. Robert On Thu, Feb 2, 2012 at 01:07, Brandon Wirtz <drak...@digerat.com> wrote: Yeah, we have a something, I don't like it. I need to password it, and ad a confirmation, and a bunch of stuff. Plus, we have a use case where we actually want a Flush every things NOW button. That the SQL version of our code does really well. And we can't quite find an analog for in GQL Brandon Wirtz BlackWaterOps: President / Lead Mercenary Description: http://www.linkedin.com/img/signature/bg_slate_385x42.jpg Work: 510-992-6548 Toll Free: 866-400-4536 IM: drak...@gmail.com (Google Talk) Skype: drakegreene YouTube: <http://www.youtube.com/blackwateropsdotcom> BlackWaterOpsDotCom <http://www.blackwaterops.com/> BlackWater Ops <http://www.cloudonastring.com/> Cloud On A String Mastermind Group From: google-appengine@googlegroups.com [mailto:google-appengine@googlegroups.com] On Behalf Of Robert Kluin Sent: Wednesday, February 01, 2012 9:38 PM To: google-appengine@googlegroups.com Subject: Re: [google-appengine] App Engine SDK 1.6.2 is out! I wrote a simple task-based script that can delete a few tens of millions of entities within a few hours. So I'd suggest just writing something. Robert On Thu, Feb 2, 2012 at 00:27, Brandon Wirtz <drak...@digerat.com> wrote: It can take days. Brandon Wirtz BlackWaterOps: President / Lead Mercenary Description: http://www.linkedin.com/img/signature/bg_slate_385x42.jpg Work: 510-992-6548 Toll Free: 866-400-4536 IM: drak...@gmail.com (Google Talk) Skype: drakegreene YouTube: <http://www.youtube.com/blackwateropsdotcom> BlackWaterOpsDotCom <http://www.blackwaterops.com/> BlackWater Ops <http://www.cloudonastring.com/> Cloud On A String Mastermind Group From: google-appengine@googlegroups.com [mailto:google-appengine@googlegroups.com] On Behalf Of Timofey Koolin Sent: Wednesday, February 01, 2012 9:14 PM To: google-appengine@googlegroups.com Subject: Re: [google-appengine] App Engine SDK 1.6.2 is out! You can delete all items of any kind from datastore admin. 2012/2/2 Brandon Wirtz <drak...@digerat.com> I want Drop Table. Can I restore empty table? Brandon Wirtz BlackWaterOps: President / Lead Mercenary Description: http://www.linkedin.com/img/signature/bg_slate_385x42.jpg Work: 510-992-6548 Toll Free: 866-400-4536 IM: drak...@gmail.com (Google Talk) Skype: drakegreene YouTube: <http://www.youtube.com/blackwateropsdotcom> BlackWaterOpsDotCom <http://www.blackwaterops.com/> BlackWater Ops <http://www.cloudonastring.com/> Cloud On A String Mastermind Group From: google-appengine@googlegroups.com [mailto:google-appengine@googlegroups.com] On Behalf Of Chris Collins Sent: Wednesday, February 01, 2012 6:09 PM To: google-appengine@googlegroups.com Subject: Re: [google-appengine] App Engine SDK 1.6.2 is out! Namespace backups would be pure awesomeness. Or some way to automate the backups. It seems you have to backup each entity manually at the moment? On Wed, Feb 1, 2012 at 10:35 AM, WGuerlich <w.guerl...@gmail.com> wrote: The backup feature is great! But having backups not only by entity kind but also by namespace would be uber-awesome. -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To view this discussion on the web visit https://groups.google.com/d/msg/google-appengine/-/yd8HVGZPREsJ. To post to this group, send email to google- appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com <mailto:google-appengine%2bunsubscr...@googlegroups.com> . For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google- appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google- appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com <mailto:google-appengine%2bunsubscr...@googlegroups.com> . For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- Blog: www.rekby.ru -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google- appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google- appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com <mailto:google-appengine%2bunsubscr...@googlegroups.com> . For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google- appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google- appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com <mailto:google-appengine%2bunsubscr...@googlegroups.com> . For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google- appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google- appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com <mailto:google-appengine%2bunsubscr...@googlegroups.com> . For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google- appeng...@googlegroups.com. To ... read more » image001.jpg < 1K View Download image002.jpg < 1K View Download On Feb 2, 7:57 am, "Brandon Wirtz" <drak...@digerat.com> wrote: > We aren't real bright here. We didn't do much with tasks until recently J > We knew we should but to make our code work on multiple platforms we had > avoided it. Now we are starting to branch our code so that functions detect > which platform they are on and to things better. So we will look in to > using your logic. > > Brandon Wirtz > BlackWaterOps: President / Lead Mercenary > Description:http://www.linkedin.com/img/signature/bg_slate_385x42.jpg > > Work: 510-992-6548 > Toll Free: 866-400-4536 > IM: drak...@gmail.com (Google Talk) > Skype: drakegreene > YouTube: <http://www.youtube.com/blackwateropsdotcom> BlackWaterOpsDotCom > > <http://www.blackwaterops.com/> BlackWater Ops > > <http://www.cloudonastring.com/> Cloud On A String Mastermind Group > > From: google-appengine@googlegroups.com > [mailto:google-appengine@googlegroups.com] On Behalf Of Robert Kluin > Sent: Wednesday, February 01, 2012 10:41 PM > To: google-appengine@googlegroups.com > Subject: Re: [google-appengine] App Engine SDK 1.6.2 is out! > > Ouch. > > By the way, a faster pattern looks like this: > > while there's stuff to delete: > do a keys only query (using a cursor to continue) > insert one or more tasks to delete those keys > insert a task and break from the while to continue if there are still more > to delete (to prevent deadline errors) > > You can speed that up further by sharding the key space in the keys only > query, which will yield better parallelism. > > Robert > > On Thu, Feb 2, 2012 at 01:18, Brandon Wirtz <drak...@digerat.com> wrote: > > The version I have goes like this. > > Yourapp.appspot.com/psuedodroptable/Name > > Then the code logic says > For all batches of maximum size > delete async > Next batch > > We ran it the person hitting it's Chrome crashed an hour later (unrelated) > they came back in the morning Got the "Chrome has crashed would you like to > restore?" "Yes why note" and they deleted all the data we had spent 18 > hours putting in to the system. well not all the data, since we were still > loading it, so new data wasn't deleted cause it wasn't in the batch J > > Brandon Wirtz > BlackWaterOps: President / Lead Mercenary > Description:http://www.linkedin.com/img/signature/bg_slate_385x42.jpg > > Work: 510-992-6548 > Toll Free: 866-400-4536 > IM: drak...@gmail.com (Google Talk) > Skype: drakegreene > YouTube: <http://www.youtube.com/blackwateropsdotcom> BlackWaterOpsDotCom > <http://www.blackwaterops.com/> BlackWater Ops > > <http://www.cloudonastring.com/> Cloud On A String Mastermind Group > > From: google-appengine@googlegroups.com > [mailto:google-appengine@googlegroups.com] On Behalf Of Robert Kluin > Sent: Wednesday, February 01, 2012 10:11 PM > > To: google-appengine@googlegroups.com > Subject: Re: [google-appengine] App Engine SDK 1.6.2 is out! > > WTF, a confirmation? Coward. > > I'd also like to see a magic "reset" button, especially one that was fast > and didn't cost a fortune. Would sure be nice for testing purposes. > > Robert > > On Thu, Feb 2, 2012 at 01:07, Brandon Wirtz <drak...@digerat.com> wrote: > > Yeah, we have a something, I don't like it. I need to password it, and ad a > confirmation, and a bunch of stuff. Plus, we have a use case where we > actually want a Flush every things NOW button. That the SQL version of our > code does really well. And we can't quite find an analog for in GQL > > Brandon Wirtz > BlackWaterOps: President / Lead Mercenary > Description:http://www.linkedin.com/img/signature/bg_slate_385x42.jpg > > Work: 510-992-6548 > Toll Free: 866-400-4536 > IM: drak...@gmail.com (Google Talk) > Skype: drakegreene > YouTube: <http://www.youtube.com/blackwateropsdotcom> BlackWaterOpsDotCom > <http://www.blackwaterops.com/> BlackWater Ops > > <http://www.cloudonastring.com/> Cloud On A String Mastermind Group > > From: google-appengine@googlegroups.com > [mailto:google-appengine@googlegroups.com] On Behalf Of Robert Kluin > Sent: Wednesday, February 01, 2012 9:38 PM > > To: google-appengine@googlegroups.com > Subject: Re: [google-appengine] App Engine SDK 1.6.2 is out! > > I wrote a simple task-based script that can delete a few tens of millions of > entities within a few hours. So I'd suggest just writing something. > > Robert > > On Thu, Feb 2, 2012 at 00:27, Brandon Wirtz <drak...@digerat.com> wrote: > > It can take days. > > Brandon Wirtz > BlackWaterOps: President / Lead Mercenary > Description:http://www.linkedin.com/img/signature/bg_slate_385x42.jpg > > Work: 510-992-6548 > Toll Free: 866-400-4536 > IM: drak...@gmail.com (Google Talk) > Skype: drakegreene > YouTube: <http://www.youtube.com/blackwateropsdotcom> BlackWaterOpsDotCom > <http://www.blackwaterops.com/> BlackWater Ops > > <http://www.cloudonastring.com/> Cloud On A String Mastermind Group > > From: google-appengine@googlegroups.com > [mailto:google-appengine@googlegroups.com] On Behalf Of Timofey Koolin > Sent: Wednesday, February 01, 2012 9:14 PM > > To: google-appengine@googlegroups.com > Subject: Re: [google-appengine] App Engine SDK 1.6.2 is out! > > You can delete all items of any kind from datastore admin. > 2012/2/2 Brandon Wirtz <drak...@digerat.com> > I want Drop Table. > > Can I restore empty table? > > Brandon Wirtz > BlackWaterOps: President / Lead Mercenary > Description:http://www.linkedin.com/img/signature/bg_slate_385x42.jpg > > Work: 510-992-6548 > Toll Free: 866-400-4536 > IM: drak...@gmail.com (Google Talk) > Skype: drakegreene > YouTube: <http://www.youtube.com/blackwateropsdotcom> BlackWaterOpsDotCom > <http://www.blackwaterops.com/> BlackWater Ops > > <http://www.cloudonastring.com/> Cloud On A String Mastermind Group > > From: google-appengine@googlegroups.com > [mailto:google-appengine@googlegroups.com] On Behalf Of Chris Collins > Sent: Wednesday, February 01, 2012 6:09 PM > To: google-appengine@googlegroups.com > Subject: Re: [google-appengine] App Engine SDK 1.6.2 is out! > > Namespace backups would be pure awesomeness. Or some way to automate the > backups. It seems you have to backup each entity manually at the moment?On > Wed, Feb 1, 2012 at 10:35 AM, WGuerlich <w.guerl...@gmail.com> wrote: > > The backup feature is great! But having backups not only by entity kind but > also by namespace would be uber-awesome. > -- > You received this message because you are subscribed to the Google Groups > "Google App Engine" group. > To view this discussion on the web > visithttps://groups.google.com/d/msg/google-appengine/-/yd8HVGZPREsJ. > > To post to this group, send email to google-appengine@googlegroups.com. > To unsubscribe from this group, send email to > google-appengine+unsubscr...@googlegroups.com > <mailto:google-appengine%2bunsubscr...@googlegroups.com> . > For more options, visit this group > athttp://groups.google.com/group/google-appengine?hl=en. > > -- > You received this message because you are subscribed to the Google Groups > "Google App Engine" group. > To post to this group, send email to google-appengine@googlegroups.com. > To unsubscribe from this group, send email to > google-appengine+unsubscr...@googlegroups.com. > For more options, visit this group > athttp://groups.google.com/group/google-appengine?hl=en. > -- > You received this message because you are subscribed to the Google Groups > "Google App Engine" group. > To post to this group, send email to google-appengine@googlegroups.com. > To unsubscribe from this group, send email to > google-appengine+unsubscr...@googlegroups.com > <mailto:google-appengine%2bunsubscr...@googlegroups.com> . > For more options, visit this group > athttp://groups.google.com/group/google-appengine?hl=en. > > -- > Blog:www.rekby.ru > -- > You received this message because you are subscribed to the Google Groups > "Google App Engine" group. > To post to this group, send email to google-appengine@googlegroups.com. > To unsubscribe from this group, send email to > google-appengine+unsubscr...@googlegroups.com. > For more options, visit this group > athttp://groups.google.com/group/google-appengine?hl=en. > -- > You received this message because you are subscribed to the Google Groups > "Google App Engine" group. > To post to this group, send email to google-appengine@googlegroups.com. > To unsubscribe from this group, send email to > google-appengine+unsubscr...@googlegroups.com > <mailto:google-appengine%2bunsubscr...@googlegroups.com> . > For more options, visit this group > athttp://groups.google.com/group/google-appengine?hl=en. > > -- > You received this message because you are subscribed to the Google Groups > "Google App Engine" group. > To post to this group, send email to google-appengine@googlegroups.com. > To unsubscribe from this group, send email to > google-appengine+unsubscr...@googlegroups.com. > For more options, visit this group > athttp://groups.google.com/group/google-appengine?hl=en. > -- > You received this message because you are subscribed to the Google Groups > "Google App Engine" group. > To post to this group, send email to google-appengine@googlegroups.com. > To unsubscribe from this group, send email to > google-appengine+unsubscr...@googlegroups.com > <mailto:google-appengine%2bunsubscr...@googlegroups.com> . > For more options, visit this group > athttp://groups.google.com/group/google-appengine?hl=en. > > -- > You received this message because you are subscribed to the Google Groups > "Google App Engine" group. > To post to this group, send email to google-appengine@googlegroups.com. > To unsubscribe from this group, send email to > google-appengine+unsubscr...@googlegroups.com. > For more options, visit this group > athttp://groups.google.com/group/google-appengine?hl=en. > -- > You received this message because you are subscribed to the Google Groups > "Google App Engine" group. > To post to this group, send email to google-appengine@googlegroups.com. > To unsubscribe from this group, send email to > google-appengine+unsubscr...@googlegroups.com > <mailto:google-appengine%2bunsubscr...@googlegroups.com> . > For more options, visit this group > athttp://groups.google.com/group/google-appengine?hl=en. > > -- > You received this message because you are subscribed to the Google Groups > "Google App Engine" group. > To post to this group, send email to google-appengine@googlegroups.com. > To ... > > read more » > > image001.jpg > < 1KViewDownload > > image002.jpg > < 1KViewDownload -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
