The 100 email scenario is where Ikai's first solution works best. You can put flags in memcache for the first 80, so you know not to send them again. This is a pretty good solution to this problem - so if you *want* to send multiple emails per task, then using Ikai's solution doesn't really leave you any worse off - your problem is still with the one email that was sent despite a DeadlineExceeded exception being thrown. You do have the slightly possibility of a memcache eviction between tasks, but this seems highly unlikely to be due to LRU, with other reasons such as memcache crashing, hardware failure or planned outage being your main problems.
On Apr 15, 8:13 am, Ubaldo Huerta <[email protected]> wrote: > So far every time this exception is thrown the email is actually sent. > I've seen it at least 10 times so far, ironically mainly to gmail > addresses. Pretty much I take my chances and raise > deferred.PermanentTaskFailure and close my eyes on the problem. BBCing > and then processing the email back in order to actually find out if > email was sent is an option but seems like to much work. > > This flakiness made me realize that I had to send one email per task. > Imagine the situation when you send 100 emails in a task and at > recipient 80th the email exception is thrown. When the task gets > retried, the first 80 recipients will get a duplicate email, well > assuming second time around the task executes without hitch. > > Ideally, the bug should be fixed, mail.send should only raise > exceptions when email is not sent. > > On Apr 14, 10:44 pm, hawkett <[email protected]> wrote: > > > > > Sounds like it would reliably set the flag, but probably not soon > > enough (at least in some situations) to beat the task queue retry.... > > > On Apr 14, 7:14 pm, "Ikai L (Google)" <[email protected]> wrote: > > > > I've seen another solution where you BCC your application and use that to > > > mark the flag as successful or not. > > > > On Wed, Apr 14, 2010 at 6:19 AM, hawkett <[email protected]> wrote: > > > > It seems the author is saying that he only sends one email per task, > > > > so the only 'email sent' flag to stick in memcache is the one that is > > > > occurring when the exception is raised. I'm not sure that solution > > > > will work - how can he know if an email has been sent successfully > > > > when the mail API is throwing an exception? He would need a list of > > > > the exceptions that occur *despite* the email being successfully > > > > sent. Seems a bit shaky to me. As the author says, this seems like a > > > > GAE bug - if GAE is going to raise the DeadlineExceededError on the > > > > mail API, then the mail should not be sent - either it succeeds within > > > > the deadline or it fails to meet the deadline - not both. I can see > > > > how this could be tricky to guarantee though.... > > > > > On Apr 12, 11:15 pm, "Ikai L (Google)" <[email protected]> wrote: > > > > > A possible solution would be to store emails where you have already > > > > > sent > > > > a > > > > > message to in Memcache. If the tasks get retried, you check for the > > > > presence > > > > > of a key and email hash. If it exists, you don't send the message. If > > > > > it > > > > > does not, you send the message and insert into Memcache. > > > > > > On Tue, Apr 6, 2010 at 3:33 PM, Ubaldo Huerta <[email protected]> > > > > > wrote: > > > > > > I have a task that sends a single email which works well, but every > > > > > > now and then (every 100 emails or so, my guess), the tasks get's > > > > > > retried because of an exception is thrown while sending the email > > > > > > message. The problem is that the email message is actually sent even > > > > > > though the exception below is thrown. So, naturally, I'm lossing > > > > > > face > > > > > > sending duplicate emails. Is there a way around this before the bug > > > > > > gets fixed? :-( > > > > > > > File "/base/python_lib/versions/1/google/appengine/api/mail.py", > > > > > > line > > > > > > 742, in send > > > > > > make_sync_call('mail', self._API_CALL, message, response) > > > > > > File "/base/python_lib/versions/1/google/appengine/api/ > > > > > > apiproxy_stub_map.py", line 78, in MakeSyncCall > > > > > > return apiproxy.MakeSyncCall(service, call, request, response) > > > > > > File "/base/python_lib/versions/1/google/appengine/api/ > > > > > > apiproxy_stub_map.py", line 278, in MakeSyncCall > > > > > > rpc.CheckSuccess() > > > > > > File "/base/python_lib/versions/1/google/appengine/api/ > > > > > > apiproxy_rpc.py", line 126, in CheckSuccess > > > > > > raise self.exception > > > > > > DeadlineExceededError: The API call mail.Send() took too long to > > > > > > respond and was cancelled. > > > > > > > -- > > > > > > You received this message because you are subscribed to the Google > > > > Groups > > > > > > "Google App Engine" group. > > > > > > To post to this group, send email to > > > > > > [email protected] > > > > . > > > > > > To unsubscribe from this group, send email to > > > > > > [email protected]<google-appengine%2Bunsubscrib > > > > > > [email protected]><google-appengine%2Bunsubscrib > > > > [email protected]> > > > > > > . > > > > > > For more options, visit this group at > > > > > >http://groups.google.com/group/google-appengine?hl=en. > > > > > > -- > > > > > Ikai Lan > > > > > Developer Programs Engineer, Google App Enginehttp:// > > > > googleappengine.blogspot.com|http://twitter.com/app_engine > > > > > -- > > > > You received this message because you are subscribed to the Google > > > > Groups > > > > "Google App Engine" group. > > > > To post to this group, send email to [email protected]. > > > > To unsubscribe from this group, send email to > > > > [email protected]<google-appengine%2Bunsubscrib > > > > [email protected]> > > > > . > > > > For more options, visit this group at > > > >http://groups.google.com/group/google-appengine?hl=en. > > > > -- > > > Ikai Lan > > > Developer Programs Engineer, Google App > > > Enginehttp://googleappengine.blogspot.com|http://twitter.com/app_engine -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to [email protected]. To unsubscribe from this group, send email to [email protected]. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
