I didn't deeply consider your problem, but the aspect of doing a repetitive task at unrelated time intervals reminds me of how the Unix kernel handles the alarm() system call for multiple, unrelated processes. For you, it will take at most one thread to handle any number of users.
1. Maintain a first-in-first-out queue, sorted by time. 2. An item in the queue has identifying info about a user, and (*this is the key fact*) a measure of time relative to the queue item before it. 3. When a new user appears, create a queue entry for that user at the appropriate spot in the time-relative queue. That might always be at the end, but if different users get the email fetched at different intervals, the insertion spot might be in the interior of the queue. 4. When a user logs out, or by whatever criteria you chose, scan the list and remove any queue entry that refers to that user. The API for other code to call the queue-management thread is only two things: - Create an entry for this user, at this repetition interval. - Remove the entry this user. The rest of the code for the queue is internal implementation details.This is the action of the queue-management thread: 1. At start-time, create this thread. It is idle to start with. 2. When a user logs in, other code will send a signal to the thread. When a user logs out, likewise send a signal to the queue-management thread. 3. The thread calculates how long to sleep before waking up and fetching email for the entry at the head of the queue. 4. When the thread wakes up, it does the work for the item at the head of the queue. If that work takes a long time, or has complex handling of errors during the work, you might spawn a thread to do the work outside of the queue-management thread. 5. Remove that queue-head item. Re-insert it at the place in the queue for its next execution on behalf of this user. That will likely be at the end, but might be in the interior of the queue. 6. Loop back to #3. On Monday, July 2, 2018 at 9:36:13 AM UTC-4, Vineeth Sagar wrote: > > Problem: > We are developing a email-CRM. The task at hand is to fetch emails every N > minutes or hours. I have received a codebase written by another guy who > left the company,my job is to re-write/re-factor. The guy who previously > wrote the code started a thread whenever a user logged and he put a > time.sleep() inside it, There's no limit on number of threads that can be > initialized, if 100 users log in, we will have 100 active threads, he used > locks so that at any given time only one thread can be executed. > > Solution: > It's messy in so many ways(mentioned above) and I think writing django > middleware would be appropriate for this task. So here's what I plan to do, > using a Django cache I store a Python datetime object and a flag to see if > there is any other request that's already retrieving the mails, I don't > want multiple requests to retrieving mails at once. Is this a better idea? > Are there any other ways to do this particular task? > > One more doubt I have is I want the user to know that the mails are being > retrieved so he doesn't refresh the page passive-aggressively. How can I do > this? I want the user to know it's happening and in the background > middleware should be executed, I have to keep retrieving mails and once > it's completed I'll execute the view_func parameter received via the > process_request hook. > > I am a junior at the company(Start up),I am fairly new to django, I keep > reading the documentation and find new things daily,after picking my mind > for a day this to me seems best, but please if you have any suggestion let > me know. Please help me out. Thanks. > > -- You received this message because you are subscribed to the Google Groups "Django users" group. To unsubscribe from this group and stop receiving emails from it, send an email to django-users+unsubscr...@googlegroups.com. To post to this group, send email to django-users@googlegroups.com. Visit this group at https://groups.google.com/group/django-users. To view this discussion on the web visit https://groups.google.com/d/msgid/django-users/f0f061c8-a04f-445a-aa16-210cfd9e693d%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
