Re: URLs screwed in the mail body

[Francis Moreau]

Kyle,

On Sat, Mar 8, 2008 at 6:15 PM, Kyle Wheeler <[EMAIL PROTECTED]> wrote:
> On Saturday, March  8 at 02:10 PM, quoth Francis Moreau:
>
> > Actually it would be better to fix the source of the problem instead
>  > of trying to find a workaround... But I don't know where these URLs
>  > get splitted at first. Perhaps you could enlight me ?
>
>  Well, the SMTP RFC specifies a recommended maximum line length of 78
>  characters, and a hard limit of 998. If a URL is longer than 78
>  characters, many email clients will split the URL.
>
>  One way that it's sometimes done, for example by Apple's email client,
>  is as a format=flowed email with delsp=yes. What that does is allows
>  them to indicate (with a single space at the end of the broken line)
>  that the next line should be appended without a space separating them.
>

My last example of screwed URL has been sent by outlook. format=flowed
was not set.

So I guess there's nothing in the email header that could indicate that
the email has been reformatted to have a maximum length set to 78.

In this case, your approach seems correct: we have no other way to parse
the email body and try to detect splitted email. If we found some, then we
need to resplice them.

>
>  >>  macro pager \cb "tr -d ' \r\n' | urlview"
>  >>
>  >>  The second option seems to be the best... but it has the problem that
>  >>  it may concatenate urls that shouldn't be concatenated (For example,
>  >>  imagine the sentence "Go to http://www.google.com/ and tell me what
>  >>  you think" - the url would become
>  >>  http://www.google.com/andtellmewhatyouthink
>  >
>  > Why in this case spaces would be deleted ?
>
>  That's what `tr -d` does: it deletes all the characters you specify.
>  In that macro, I specified three characters: a space, a carriage
>  return, and a newline. The reason I specified a space is because of
>  Apple's format=flowed trick.

Sorry I didn't see the space caracter. My font is too small on that laptop
and it's hard to see it.

>
>  What I wish we could do is use mutt to decode format=flowed and
>  quoted-printable without having it feed things to the standard mailcap
>  program when pipe_decode is set.
>

Well, it may be not enough to rely on format=flowed thing only, as I said
before.

>
>  >>  Perhaps it's better to do this:
>  >>
>  >>  3. Pipe it through lynx to extract the urls before piping it to
>  >> urlview, like so:
>  >>
>  >> macro pager \cb "lynx --force-html --dump | 
> urlview"
>  >>
>  >
>  > Yes. It would be nice to apply this macro for html emails only. For
>  > text emails just do the usual/fast thing.
>  >
>  > Thank you for your usefull feedbacks.
>
>  Yeah... unfortunately, that macro (I've been trying it all yesterday)
>  doesn't *quite* work on all emails. Lynx sometimes gets confused by
>  the message headers, I think.
>

But doesn't pipe_decode, when set,  remove them ?

>  I just put together a perl script that would do the trick instead of
>  lynx. It's attached.
>

Sorry but I don't speak perl, so I can't comment. I'm wondering if
sed + a good regexp could do the job instead.

Thanks
-- 
Francis

Re: Mutt + LDAP + multiple email addresses

[Rado S]

=- Roland Hill wrote on Sun  9.Mar'08 at 17:14:41 +1300 -=

> I am using mutt to query my LDAP server using the
> mutt_ldap_query.pl program.
> As is it works fine, except, as designed, it will only retrieve
> the first email address.
> Has anyone modified it to retrieve more than one email address for
> an entry?

See '?' help when you enter 'query' mode.
See also manual.txt about "bindings".

-- 
© Rado S. -- You must provide YOUR effort for your goal!
EVERY effort counts: at least to show your attitude.
You're responsible for ALL you do: you get what you give.

problems with Maildir, procmail, getmail

[Jamie Griffin]

I've been struggling for ages trying to set up mutt on my linux
machine. 

I've specified the Maildir mailboxes in muttrc
set up procmail
set up getmail to pop my email

but when i open mutt it wants to use /var/mail/user

i created this using:

sudo touch /var/mail/user
sudo chown user:users /var/mail/user
sudo chmod 600/var/mail/user

This stopped mutt complaining it wasn't there but now when i pop my
email using getmail, i'm lots of error messages returned from the shell
with procmail. its saying the problem is with /var/mail/user.lock ??? 

none of the messages are being filtered into the right folders as a
result... i don't know what to do next. Any ideas?

Jamie

Re: problems with Maildir, procmail, getmail

[Michael Kjorling]

On 9 Mar 2008 14:34 +, by [EMAIL PROTECTED] (Jamie Griffin):
> but when i open mutt it wants to use /var/mail/user

Set mutt's $spoolfile setting to wherever your inbox is.

-- 
Michael Kjörling .. [EMAIL PROTECTED] .. http://michael.kjorling.se
* . No bird soars too high if he soars with his own wings . *
* ENCRYPTED email preferred -- OpenPGP key ID: 0x 758F8749 BDE9ADA6 *
* ASCII Ribbon Campaign: Against HTML mail, proprietary attachments *



pgp3xXU8YoBax.pgp
Description: PGP signature

Re: problems with Maildir, procmail, getmail

[Jamie Griffin]

I did try that, but mutt returned a message saying /home/user/mail/inbox
is not a mailbox. 

Would that have caused the error messages i got after runnning getmail?
On Sun, 2008-03-09 at 14:44 +, Michael Kjorling wrote:
> On 9 Mar 2008 14:34 +, by [EMAIL PROTECTED] (Jamie Griffin):
> > but when i open mutt it wants to use /var/mail/user
> 
> Set mutt's $spoolfile setting to wherever your inbox is.
> 

Re: problems with Maildir, procmail, getmail

[Steve S]

On Sun, Mar 09, 2008 at 02:34:23PM +, Jamie Griffin wrote:
> I've been struggling for ages trying to set up mutt on my linux
> machine. 
> 
> I've specified the Maildir mailboxes in muttrc
> set up procmail
> set up getmail to pop my email

> 
> but when i open mutt it wants to use /var/mail/user
> 
> i created this using:
> 
> sudo touch /var/mail/user
> sudo chown user:users /var/mail/user
> sudo chmod 600/var/mail/user
> 
> This stopped mutt complaining it wasn't there but now when i pop my
> email using getmail, i'm lots of error messages returned from the shell
> with procmail. its saying the problem is with /var/mail/user.lock ??? 
> 
> none of the messages are being filtered into the right folders as a
> result... i don't know what to do next. Any ideas?
> 
> Jamie

Hmm never seen errors like these, but I'm using pretty much your setup. 

muttrc:

set mbox_type=Maildir
set folder=~/Maildir
set spoolfile=~/Maildir
set trash=+trash
set record=+sent
set postponed=+postponed
mailboxes +spam +inbox +sent ...
 
.procmailrc starts with:

MAILDIR=$HOME/Maildir/

The MDA part of my getmailrc files (I use `getmail -r  -r 
...` to fetch mail from several POP3 accounts):

[destination]
type = MDA_external
arguments = ("-f", "%(sender)")
path = /usr/bin/procmail


HTH

cheers,
steve

Re: Mailing lists and subscribe

[Alain Bench]

Hello Michelle,

 On Monday, March 3, 2008 at 20:48:27 +0100, Michelle Konzack wrote:

>| unlists *
>| unsubscribe *
>| `for file in ~/Mail/lists/* ; do name=$(basename $file) ; \
>|  echo -e "lists $name\nsubscribe $name" ; done`

This doesn't work:

 - the backtick expansion picks the first line only
 - subscribe alone suffices to declare a list as lists + subscribe
 - in Mutt 1.4 "unlists *" alone suffices to erase all previous lists
and subscribes


Bye!Alain.
-- 
Mutt compressed folders tip for stable archive timestamp:
| open-hook \\.gz$ "gzip -cd '%f' > '%t' ; ret=$? ; touch --no-create 
--reference='%f' '%t' ; exit \$ret"
| close-hook \\.gz$ "gzip -c '%t' > '%f' ; ret=$? ; touch --no-create 
--reference='%t' '%f' ; exit \$ret"
| append-hook \\.gz$ "gzip -c '%t' >> '%f'"

mutt -y (was: USE_SMTP flag)

[Alain Bench]

Hello Michael, Amit,

 On Saturday, March 8, 2008 at 16:19:06 +, Michael Kjörling wrote:

> On 8 Mar 2008 16:45 +0200, by [EMAIL PROTECTED] (Amit Finkler):
>> through "mutt -y" I see both of them
> Try c?.

Since Mutt 1.5.12, the system-wide /etc/Muttrc should define a macro
to switch back and forth from anywhere to the mailboxes browser on press
of the [y] key. Amit: If yours doesn't, add this to your ~/.muttrc:

| # show the incoming mailboxes list (just like "mutt -y") and back when 
pressing "y"
| macro index,pager y "?" "show incoming 
mailboxes list"
| bind browser y exit

This [y] quick toggle can also be seen as a quite convenient
substitute to the sidebar patch.


Bye!Alain.
-- 
How to Report Bugs Effectively
http://www.chiark.greenend.org.uk/~sgtatham/bugs.html>

Re: Mailing lists and subscribe

[Alain Bench]

Hello Michael,

 On Sunday, March 2, 2008 at 21:18:37 +, Michael Kjörling wrote:

> `lists' and `subscribe' takes an address regexp

Regexps came in Mutt 1.5.6. The OP uses Mutt 1.4.2.2i, where lists
and subscribe patterns were simple strings matched against the beginning
of the address (ie. implicit ^patterns).


Bye!Alain.
-- 
set honor_followup_to=yes in muttrc is the default value, and makes your
list replies go where the original author wanted them to go: Only to the
list, or with a private copy.

Re: URLs screwed in the mail body

[Francis Moreau]

Kyle,

On Sun, Mar 9, 2008 at 1:58 PM, Francis Moreau <[EMAIL PROTECTED]> wrote:
>  On Sat, Mar 8, 2008 at 6:15 PM, Kyle Wheeler <[EMAIL PROTECTED]> wrote:
>  > On Saturday, March  8 at 02:10 PM, quoth Francis Moreau:
>  >
>  > > Actually it would be better to fix the source of the problem instead
>  >  > of trying to find a workaround... But I don't know where these URLs
>  >  > get splitted at first. Perhaps you could enlight me ?
>  >
>  >  Well, the SMTP RFC specifies a recommended maximum line length of 78
>  >  characters, and a hard limit of 998. If a URL is longer than 78
>  >  characters, many email clients will split the URL.
>  >
>  >  One way that it's sometimes done, for example by Apple's email client,
>  >  is as a format=flowed email with delsp=yes. What that does is allows
>  >  them to indicate (with a single space at the end of the broken line)
>  >  that the next line should be appended without a space separating them.
>  >
>
>  My last example of screwed URL has been sent by outlook. format=flowed
>  was not set.
>  fix the common case once
we agree on it.
>  So I guess there's nothing in the email header that could indicate that
>  the email has been reformatted to have a maximum length set to 78.
>
>  In this case, your approach seems correct: we have no other way to parse
>  the email body and try to detect splitted email. If we found some, then we
>  need to resplice them.
>

OK, I though more about it and I think the situation is quite bad ;)

Here are the different use cases I can think of:

"""
this is an example of an URL not splitted http://www.splitted.ex/foo
and the next word following the URL is not part of it.

this is an example of an URL not splitted http://www.splitted.ex/foo
but with a trailing space and the next work is not part of the URL.

this is an example of a splitted URL http://www.splitted.ex/foo
/bar/file.html. There's no trailing whitespace.

this is an example of a splitted URL http://www.splitted.ex/foo
/bar/file.html but with a trailing white space.

this is an example of a splitted URL http://www.splitted.ex/foo/
file.html with no trailing white space. This time there's no way
to detect that the next word following the URL is part of it because
it is not a path.

this is an example of a splitted URL http://www.splitted.ex/foo/
file.html with a trailing white space. This time there's no way
to detect that the next word following the URL is part of it.
"""

Looking at them, I can't think of a script that could fix the URL
when needed.

Probably the best we could do is to write a script with 2 modes:
normal and agressive.

The normal mode would fix the easiest and common case
whereas the agressive mode would fix the special cases but
break the common case. We could bind each mode to different
shortcuts.

What do you think ?
-- 
Francis

Re: problems with Maildir, procmail, getmail

[Jamie Griffin]

I think i might have some problems with Maildir:

I've got the Maildir directory, with the require cur.tmp,new sub directories.

Then i've got IN-inbox (spoolfile) and its sub-directires inside ~/Maildir.
I've then got 3 other directories for mailing list mailboxes set up in the same 
way. 
as well as 'Drafts' 'Sent'

Having ~/Maildir/cur/new/tmp in this way, and then having other maildirs, is 
this the correct way to set them up? this is the output from ls -al :

drwx-- 11 jamie jamie 4096 2008-03-09 21:52 .
drwxr-xr-x 39 jamie jamie 4096 2008-03-09 22:36 ..
drwx--  2 jamie jamie 4096 2008-03-09 15:25 cur
drwx--  5 jamie jamie 4096 2008-03-09 21:52 Drafts
drwx--  5 jamie jamie 4096 2008-03-09 15:26 IN-inbox
drwx--  5 jamie jamie 4096 2008-03-09 15:26 IN-mutt-dev
drwx--  5 jamie jamie 4096 2008-03-09 15:26 IN-mutt-users
drwx--  5 jamie jamie 4096 2008-03-09 15:27 IN-ubuntu-users
drwx--  2 jamie jamie 4096 2008-03-09 15:25 new
drwx--  5 jamie jamie 4096 2008-03-09 15:57 Sent
drwx--  2 jamie jamie 4096 2008-03-09 15:25 tmp

in case it helps, i know it's a basic thing but i'd appreciate it if someone 
could provide any tips.

Jamie 

On Sun, Mar 09, 2008 at 05:10:15PM +0100, Steve S wrote:
> On Sun, Mar 09, 2008 at 02:34:23PM +, Jamie Griffin wrote:
> > I've been struggling for ages trying to set up mutt on my linux
> > machine. 
> > 
> > I've specified the Maildir mailboxes in muttrc
> > set up procmail
> > set up getmail to pop my email
> 
> > 
> > but when i open mutt it wants to use /var/mail/user
> > 
> > i created this using:
> > 
> > sudo touch /var/mail/user
> > sudo chown user:users /var/mail/user
> > sudo chmod 600/var/mail/user
> > 
> > This stopped mutt complaining it wasn't there but now when i pop my
> > email using getmail, i'm lots of error messages returned from the shell
> > with procmail. its saying the problem is with /var/mail/user.lock ??? 
> > 
> > none of the messages are being filtered into the right folders as a
> > result... i don't know what to do next. Any ideas?
> > 
> > Jamie
> 
> Hmm never seen errors like these, but I'm using pretty much your setup. 
> 
> muttrc:
> 
> set mbox_type=Maildir
> set folder=~/Maildir
> set spoolfile=~/Maildir
> set trash=+trash
> set record=+sent
> set postponed=+postponed
> mailboxes +spam +inbox +sent ...
>  
> .procmailrc starts with:
> 
> MAILDIR=$HOME/Maildir/
> 
> The MDA part of my getmailrc files (I use `getmail -r  -r 
> ...` to fetch mail from several POP3 accounts):
> 
> [destination]
> type = MDA_external
> arguments = ("-f", "%(sender)")
> path = /usr/bin/procmail
> 
> 
> HTH
> 
> cheers,
> steve

Re: Saving changed mime types?

[Jörg Sommer]

Hello Alain,

Alain Bench <[EMAIL PROTECTED]> wrote:
>  On Thursday, March 6, 2008 at 10:27:07 +, Jörg Sommer wrote:
>
>> If you get a broken message that says it is iso-8859-1, but is cp1252.
>> You don't want to edit the MIME header every time, but save it to the
>> mailbox after the first time.
>
> This quite common case has a way better automagic on-the-fly
> solution, safely keeping the original mail unmodified:
>
>| charset-hook ^iso-8859-1$ cp1252

Works! I don't understand why, because I read the manual as charset-hook
(re)defines iso-8859-1, but it works.

Bye, Jörg.
-- 
Fazit: „Schießen Sie nicht mit dem Geiger!“

Re: problems with Maildir, procmail, getmail

[Steve S]

On Sun, Mar 09, 2008 at 10:53:55PM +, Jamie Griffin wrote:
> I think i might have some problems with Maildir:

Did you check the Wiki http://wiki.mutt.org/?MuttFaq/Maildir? That helped me to
set up everything.

> I've got the Maildir directory, with the require cur.tmp,new sub directories.

You souldn't need cur, new, tmp in ~/Maildir but only in your mailboxes
(IN-inbox, ...).

> Then i've got IN-inbox (spoolfile) and its sub-directires inside ~/Maildir.

As I said, you should

set spoolfile=~/Maildir

> drwx-- 11 jamie jamie 4096 2008-03-09 21:52 .
> drwxr-xr-x 39 jamie jamie 4096 2008-03-09 22:36 ..
> drwx--  2 jamie jamie 4096 2008-03-09 15:25 cur
> drwx--  5 jamie jamie 4096 2008-03-09 21:52 Drafts
> drwx--  5 jamie jamie 4096 2008-03-09 15:26 IN-inbox
> drwx--  5 jamie jamie 4096 2008-03-09 15:26 IN-mutt-dev
> drwx--  5 jamie jamie 4096 2008-03-09 15:26 IN-mutt-users
> drwx--  5 jamie jamie 4096 2008-03-09 15:27 IN-ubuntu-users
> drwx--  2 jamie jamie 4096 2008-03-09 15:25 new
> drwx--  5 jamie jamie 4096 2008-03-09 15:57 Sent
> drwx--  2 jamie jamie 4096 2008-03-09 15:25 tmp

cheers,
steve

Re: problems with Maildir, procmail, getmail

[Jānis Rūcis]

Hi Jamie,

On Sun, Mar 09, 2008 at 10:53:55PM +, Jamie Griffin wrote:
> I've got the Maildir directory, with the require cur.tmp,new sub directories.

You only need the {cur,tmp,new} subdirectories for actual mailboxes.  If
you don't intend to use "~/Maildir" directly as a mailbox, there's no
need for those directories.

> Then i've got IN-inbox (spoolfile) and its sub-directires inside ~/Maildir.
> I've then got 3 other directories for mailing list mailboxes set up in the 
> same way. 
> as well as 'Drafts' 'Sent'
> 
> Having ~/Maildir/cur/new/tmp in this way, and then having other maildirs, is 
> this the correct way to set them up? this is the output from ls -al :
> 
[...]
> drwx--  5 jamie jamie 4096 2008-03-09 15:26 IN-inbox
> drwx--  5 jamie jamie 4096 2008-03-09 15:26 IN-mutt-dev
> drwx--  5 jamie jamie 4096 2008-03-09 15:26 IN-mutt-users
> drwx--  5 jamie jamie 4096 2008-03-09 15:27 IN-ubuntu-users

procmail creates new mailboxes automatically, you just need to tell it
to to use the Maildir format.  Here's what
/usr/share/doc/procmail/README.Maildir says on my system:

This version of procmail supports Maildir folders.

To make procmail to deliver into a Maildir folder, just append
a slash (/) to the name of the maildir folder in your ~/.procmailrc file.
For example, the following rule:

:0
* [EMAIL PROTECTED]
debian-user/

will deliver all mail from the debian-user mailing list to the Maildir
folder "debian-user".

Re: URLs screwed in the mail body

[Kyle Wheeler]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

On Sunday, March  9 at 01:58 PM, quoth Francis Moreau:
> My last example of screwed URL has been sent by outlook. 
> format=flowed was not set.

Ahh, yeah, then you're screwed.

>>  Yeah... unfortunately, that macro (I've been trying it all 
>>  yesterday) doesn't *quite* work on all emails. Lynx sometimes gets 
>>  confused by the message headers, I think.
>>
>
> But doesn't pipe_decode, when set,  remove them ?

True, but if pipe_decode is set, mutt will feed the message through 
(in the case of html emails) your mailcap before piping it to where 
you want. Piping pre-rendered html text to lynx won't achieve 
anything.

I use, for example, w3m to render auto_view html emails. w3m does a 
great job, but doesn't collect all the URLs into a list at the end of 
the email (like lynx or links would do). Piping the output of that 
(which is what happens with pipe_decode set) through urlview usually 
results in urlview not finding any URLs, for obvious reasons. If I 
have pipe_decode unset, however, mutt doesn't even decode 
quoted-printable encoding, so the URLs can be really badly mangled --- 
and unfortunately, even lynx can't help with that.

A better solution would be to have pipe_decode set and then use lynx 
for doing auto_view html rendering. But that doesn't address URLs that 
have been mangled with format=flowed wrapping (and, in my personal 
opinion, w3m does a better job of rendering html).

>>  I just put together a perl script that would do the trick instead of
>>  lynx. It's attached.
>
> Sorry but I don't speak perl, so I can't comment. I'm wondering if 
> sed + a good regexp could do the job instead.

It would have to be a pretty complicated sed script. One of the big 
benefits of my perl script there is that it handles all the possible 
quoted-printable encoding and even possible Base64 encoding---which 
would be pretty hard to do with a sed script. The other problem with a 
sed script is that it can't detect the difference between 
format=flowed emails and non-flowed emails (or the difference between 
delsp=yes and delsp=no). My perl script, on the other hand, can. Which 
is not to suggest that it's perfect... but, for an afternoon's worth 
of hacking, I think it does a pretty good job fixing all the problems 
I've had so far. :)

~Kyle
- -- 
Holding on to anger, resentment and hurt only gives you tense muscles, 
a headache and a sore jaw from clenching your teeth. Forgiveness gives 
you back the laughter and the lightness in your life.
 -- Joan Lunden
-BEGIN PGP SIGNATURE-
Comment: Thank you for using encryption!

iD8DBQFH1Jr+BkIOoMqOI14RAlBFAKDth9Ew7mKyLpLX7wlK3fumLxGF2gCcDBn9
alIPiKfEecQ4M1YNIZqjouk=
=zjxK
-END PGP SIGNATURE-

Re: URLs screwed in the mail body

[Kyle Wheeler]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

On Sunday, March  9 at 07:37 PM, quoth Francis Moreau:
>OK, I though more about it and I think the situation is quite bad ;)
>
>Here are the different use cases I can think of:

You forget, labeling is part of the issue. There's a difference 
between how the URL should be handled based on whether the message has 
been labeled as format=flowed or not, and whether its been labeled as 
delsp=yes or not. All your use-cases here are just for plain-text.

>Probably the best we could do is to write a script with 2 modes:
>normal and agressive.
>
>The normal mode would fix the easiest and common case
>whereas the agressive mode would fix the special cases but
>break the common case. We could bind each mode to different
>shortcuts.
>
>What do you think ?

For handling *broken* emails (such as the one you were sent), I think 
that you're right, and that's the best you can do. If the email was 
correctly encoded, I think it becomes much easier to "do the right 
thing".

~Kyle
- -- 
My definition of a free society is a society where it is safe to be 
unpopular.
  -- Adlai E. Stevenson Jr.
-BEGIN PGP SIGNATURE-
Comment: Thank you for using encryption!

iD8DBQFH1JwtBkIOoMqOI14RAhH7AJ4opTR7ggqrIhcFAnDhHpcIOcOQxACfVdRZ
LsrmO3+onuYIfTMoFiPQ1+g=
=kd8/
-END PGP SIGNATURE-

Re: Saving changed mime types?

[Kyle Wheeler]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

On Sunday, March  9 at 10:04 PM, quoth Jörg Sommer:
>>| charset-hook ^iso-8859-1$ cp1252
>
>Works! I don't understand why, because I read the manual as charset-hook
>(re)defines iso-8859-1, but it works.

FYI it works beccause cp1252 is a superset of iso-8859-1 (in other 
words, all valid characters in iso-8859-1 are the same characters in 
cp1252, but cp1252 has a few extra ones).

~Kyle
- -- 
However many holy words you read, However many you speak, what good 
will they do you If you do not act on upon them?
  -- Buddha
-BEGIN PGP SIGNATURE-
Comment: Thank you for using encryption!

iD8DBQFH1Jx/BkIOoMqOI14RAssMAJ9PyeSNRq8p8xCv9gUreXcV7CUXnQCg8GDC
HurcmBpELZrRvNfbswZohe8=
=ZPEe
-END PGP SIGNATURE-