Like several here, I fetch email from my ISP mail spool(s) and file
messages locally. If my laptop's been offline overnight there can be
hundreds of messages to fetch when I wake it up. On a satellite link
(geostationary) with a ping time of over 600ms this can be many minutes
of tedium.
The raw bandwidth is fine and my filing process is pretty expeditious;
the root cause of that tedium is network latency and the synchronous
behaviour of getmail. Its cycle is like this:
- RETRieve the message, collect and save
- DELEte the message
- repeat for all the messages, then QUIT to commit the deletes
Each of steps 1 and 2 above incurs over a second just in network
latency. That scales up over hundreds of messages.
My new tool streams the fetches: it issues RETRs for every message up
front at maximum network speed - fully buffered and with no waits. A
parallel worker thread collects the messages as they come in at full
speed (the upstream server likely also gets to fully buffer); it issues
DELEtes as each message is saved, also fully buffered.
The code's here:
https://hg.sr.ht/~cameron-simpson/css/browse/lib/python/cs/pop3.py?rev=tip
The cs.pop3 modules on PyPI here:
https://pypi.org/project/cs.pop3/
and can be installed with:
pip install cs.pop3
which also provides a "pop3" command in your Python environment.
Typical use is:
pop3 dl mylo...@mail.cskk.id.au ~/var/mail/spool
specifying my internet mail spool and the local Maildir to receive the
messages.
Cheers,
Cameron Simpson <c...@cskk.id.au>
If you cannot, in the long run, tell everyone what you have been doing,
your doing has been worthless. - Erwin Schrodinger
