Thanks Brian and Mark A combination may well help.
I am not for this project needing ALL but have been using UNSEEN. The mailboxes we are referencing have around 100-1200 emails coming in from O2 every morning at 4am. These then get processed at 5am to update their databases. I will have to first find the FIRST UNSEEN UID and then run a process to get the UIDs of the next 100 (and repeat) till they run out I guess. Once I’ve built the list of UIDs I can carry on as normal processing each one in turn. Finding the first unseen UID though seems to be the next issue I’m going to have to overcome. A never ending stream of workarounds. Sean Cole Pi Digital Productions Ltd eMail Ts & Cs > On 23 Mar 2020, at 16:48, Mark Waddingham via use-livecode > <use-livecode@lists.runrev.com> wrote: > > On 2020-03-23 15:21, Pi Digital via use-livecode wrote: >> Thanks Mark, your input here is appreciated. >> This reply in that forum wasn’t helpful, was it >>> > Known bug #90 was reported in 2014. However it still occurs in 2019. Does >>> > anybody know how to overcome this situation? >>> Yes: by fixing the code! >> In the thread it talks of pagination (as do you) but doesn’t give an >> example of how to do it. How would we implement this in LC? If it is >> not giving us the correct count how do we know how many pages we will >> have to allow for and so on. > > That's where I got the reference to pagination from - as I said it's not a > particularly useful post there. I suspect the person replying assumed the > person asking the question had a detailed understanding of IMAP queries (my > knowledge of interacting with IMAP servers is more than it was earlier on, > but still not really sufficient to give any detailed advice). > > The tsNet support for IMAP comes entirely from cURL - so it works pretty much > the same as the 'curl' shell command. > > I found this with a quick search which might help: > > <https://gist.github.com/akpoff/53ac391037ae2f2d376214eac4a23634> > > The queries listed there at least appear to give ways to query the count of > things in a mailbox. > > This page also looks like it might be helpful: > > <https://nbsoftsolutions.com/blog/introduction-to-imap> > > Also this has some more info on using CURL to talk to IMAP: > > > <https://debian-administration.org/article/726/Performing_IMAP_queries_via_curl> > > Having pondered this a bit, I'm not sure that even if cURL guys fixed the > above bug it would actually make a difference to what you need to do to have > 100% correct code. The method of fetching all UIDs of messages in a mailbox > at once is completely unscalable - imagine an INBOX with 100,000s of messages > for example over a slow/flaky connection; or an INBOX which has a lot of > traffic and changes exceptionally frequently; or sits on a heavily loaded > server (you can imagine that no server would like to be polled for complete > message lists all the time by lots of clients). > > The IMAP protocol appears to be designed to be treated as something to use to > synchronize local and remote state. Indeed, there is quite an extensive > description of how to 'synchronize with an IMAP server' in an RFC: > > <https://tools.ietf.org/html/rfc4549#page-18> > > Indeed, it states in one place: > > <quote> > The following is an example of the first FETCH: > > C: A011 UID fetch 131:* (FLAGS BODYSTRUCTURE INTERNALDATE > RFC822.SIZE) > > Note 1: The first FETCH may result in the server's sending a huge > volume of data. A smart disconnected client should use message > ranges (see also Section 3.2.1.2 of [RFC2683]), so that the user is > able to execute a different operation between fetching information > for a group of new messages. > </quote> > > The unique id used to perform the sync operation is the UID; you can fetch > ranges of those as they are 'guaranteed' to always increase and never be > re-used. (The only caveat is the UIDVALIDITY, which is basically a > cache-generation number - i.e. if the UIDVALIDITY changes you have to dump > your local cache and start from scratch as it means the meaning of individual > UIDs has changed). > > Not sure how much the above helps... > > Warmest Regards, > > Mark. > > P.S. I've been chatting to Seb about the issue (he reported the bug > originally) - he has confirmed that the problem affects Kognition's IMAP code > too (that component is still in development at this stage which is why it > hadn't been noticed until I asked him to check). I had hoped the approach > there might have been slightly different and so provided a viable workaround, > but it would appear that it is not the case :( > > -- > Mark Waddingham ~ m...@livecode.com ~ http://www.livecode.com/ > LiveCode: Everyone can create apps > > _______________________________________________ > use-livecode mailing list > use-livecode@lists.runrev.com > Please visit this url to subscribe, unsubscribe and manage your subscription > preferences: > http://lists.runrev.com/mailman/listinfo/use-livecode _______________________________________________ use-livecode mailing list use-livecode@lists.runrev.com Please visit this url to subscribe, unsubscribe and manage your subscription preferences: http://lists.runrev.com/mailman/listinfo/use-livecode
