On Tue, 2010-10-19 at 17:28 +0100, Timo Sirainen wrote:
> > But .. I suppose it could be possible to use a combination of attachment-id
> > + mailbox GUID + message UID number. I'll have to think about it..
>
> Implemented.
Oh, forgot to say: Now attachments won't get lost if you replace hard
l
On Sat, 2010-09-25 at 13:34 +0100, Timo Sirainen wrote:
> But .. I suppose it could be possible to use a combination of attachment-id +
> mailbox GUID + message UID number. I'll have to think about it..
Implemented. Although saving messages now requires an extra rename(). It
should be possible t
On 25.9.2010, at 11.50, William Blunn wrote:
>> I can't think of any other reasonable way to handle this though, so unless
>> someon has some great ideas, I think the solution is to simply
>> add enough warnings that message store shouldn't be accessed directly. Maybe
>> add some import/export c
On 25.9.2010, at 9.49, Daniel L. Miller wrote:
> On 9/24/2010 11:57 AM, Timo Sirainen wrote:
>> Just a note to myself and whoever else cares, should be added to wiki
>> once it has its own page about this:
>>
>> With single-dbox messages can be copied with hard linking. This means
>> that there c
On 24/09/2010 19:57, Timo Sirainen wrote:
Just a note to myself and whoever else cares, should be added to wiki once it
has its own page about this:
I have been meaning to say we should have a wiki page about this.
With single-dbox messages can be copied with hard linking. This means tha
On 9/24/2010 11:57 AM, Timo Sirainen wrote:
Just a note to myself and whoever else cares, should be added to wiki
once it has its own page about this:
With single-dbox messages can be copied with hard linking. This means
that there can be multiple files that point to the same attachment file.
T
On 2010-09-24 2:57 PM, Timo Sirainen wrote:
> Maybe add some import/export commands to doveadm which can be used to
> add a bunch of mails to storage without doing it directly on filesystem.
+1
I've always just used cp -rp to restore mail, so yeah, if that would
break SiS, I agree there should be
On 17.9.2010, at 2.10, Daniel L. Miller wrote:
> Something I wasn't quite clear about previously - does the SIS provide
> single-instance only within each account, or is it server wide?
Server-wide. It would be pretty pointless otherwise.
> The examples given for mail_attachment_dir sometimes l
On 9/16/2010 11:20 AM, Timo Sirainen wrote:
On Tue, 2010-08-31 at 20:16 +0100, Timo Sirainen wrote:
Can you append some "trivial" information from the data file to the hash
in generating the file name to help ensure uniqueness? Like filesize,
I guess size could be there at least optionally, I
On Tue, 2010-08-31 at 20:16 +0100, Timo Sirainen wrote:
> > Can you append some "trivial" information from the data file to the hash
> > in generating the file name to help ensure uniqueness? Like filesize,
>
> I guess size could be there at least optionally, I'm not sure about as
> default.
N
On Fri, 2010-08-27 at 09:34 -0700, Daniel L. Miller wrote:
> On 8/24/2010 4:35 PM, Timo Sirainen wrote:
> > On 24.8.2010, at 23.16, Ed W wrote:
> >
> >> At the moment I would claim that you are just automatically generating a
> >> very complicated filename. If you never trust your hash then you m
On Fri, 2010-08-27 at 09:41 -0700, Daniel L. Miller wrote:
> On 8/24/2010 4:19 PM, Timo Sirainen wrote:
> > It depends on your configuration.. The attachment directory is a setting. I
> > was thinking that it it would typically be the same for all users, so if
> > you have two filesystems, you'd
On 8/24/2010 4:19 PM, Timo Sirainen wrote:
It depends on your configuration.. The attachment directory is a setting. I was
thinking that it it would typically be the same for all users, so if you have
two filesystems, you'd need to decide which one will have the /attachments
directory.
Dunn
On 8/24/2010 4:35 PM, Timo Sirainen wrote:
On 24.8.2010, at 23.16, Ed W wrote:
At the moment I would claim that you are just automatically generating a very
complicated filename. If you never trust your hash then you might as well
instead simply use one of the existing GUID algorithms, if y
Some interesting reading on SHA256 checksum
http://blogs.sun.com/bonwick/entry/zfs_dedup
http://blogs.sun.com/darren/entry/improving_zfs_dedup_performance_via
On 24.8.2010, at 23.16, Ed W wrote:
> At the moment I would claim that you are just automatically generating a very
> complicated filename. If you never trust your hash then you might as well
> instead simply use one of the existing GUID algorithms, if you trust your
> hash then you use that.
On 24.8.2010, at 23.16, Ed W wrote:
> On 24/08/2010 16:48, Timo Sirainen wrote:
>>
>> Current implementation checks how many hard links are left for the hash
>> while deleting it. If it's deleting the last reference then the final
>> hashes/hash file is also deleted.
>
> I sense an interesting r
On 24.8.2010, at 22.53, Mike Abbott wrote:
>> attachments/ha/sh/hash-guid, which is a hard link to:
>> attachments/ha/sh/hashes/hash
>
> If I store, say, all students' mail directories on one file system, and all
> teachers' mail on another, will your SIS store one copy of each attachment,
> or
On 24/08/2010 16:48, Timo Sirainen wrote:
Current implementation checks how many hard links are left for the hash
while deleting it. If it's deleting the last reference then the final
hashes/hash file is also deleted.
I sense an interesting race possibility here?
The hash is already a full
> attachments/ha/sh/hash-guid, which is a hard link to:
> attachments/ha/sh/hashes/hash
If I store, say, all students' mail directories on one file system, and all
teachers' mail on another, will your SIS store one copy of each attachment, or
one copy per file system?
On Tue, 2010-08-24 at 17:43 +0100, Timo Sirainen wrote:
> On Tue, 2010-08-24 at 11:36 -0500, Mike Abbott wrote:
> > > Current implementation checks how many hard links are left for the hash
> >
> > I haven't followed all the details and speculation, but this talk of
> > hard-linking makes me wond
On Tue, 2010-08-24 at 11:36 -0500, Mike Abbott wrote:
> > Current implementation checks how many hard links are left for the hash
>
> I haven't followed all the details and speculation, but this talk of
> hard-linking makes me wonder how SIS will work with mail stores spread across
> multiple fi
> Current implementation checks how many hard links are left for the hash
I haven't followed all the details and speculation, but this talk of
hard-linking makes me wonder how SIS will work with mail stores spread across
multiple file systems.
On Tue, 2010-08-24 at 15:48 +0100, Ed W wrote:
> Hi, thanks for responding
>
> > If you have a hash 351641b73feb7cf7e87e5a8c3ca9a37d7b21e525, you can see
> > if it exists with:
> >
> > ls /attachments/35/16/hashes/351641b73feb7cf7e87e5a8c3ca9a37d7b21e525
>
> This would be great for a bunch of use
Hi, thanks for responding
If you have a hash 351641b73feb7cf7e87e5a8c3ca9a37d7b21e525, you can see
if it exists with:
ls /attachments/35/16/hashes/351641b73feb7cf7e87e5a8c3ca9a37d7b21e525
This would be great for a bunch of uses, if the hash were unique and
determinable from just some other
On Tue, 2010-08-24 at 13:42 +0100, Ed W wrote:
> Hi
>
> > The idea is to have dbox and mdbox support saving attachments (or MIME
> > parts in general) to separate files, which with some magic gives a
> > possibility to do single instance attachment storage. Comments welcome.
>
> This is a really
Hi
The idea is to have dbox and mdbox support saving attachments (or MIME
parts in general) to separate files, which with some magic gives a
possibility to do single instance attachment storage. Comments welcome.
This is a really interesting idea. I have previously given it some
thought. M
On Mon, 2010-07-19 at 17:29 +0100, William Blunn wrote:
> > 2) If base64 attachment is in a standardized form that can be 100%
> reliably converted back to its original form, it could be stored
> decoded and then encoded back to original on the fly.
This is now done: http://hg.dovecot.org/dovecot
Damon Atkins wrote:
Just keep in mine Signed Emails, the email contents needs to be
present back to the client as it came, so the client says the email
has not been altered.
Timo will presumably correct me if I am wrong: I believe it is already a
design goal that messages going in to the mail
Just keep in mine Signed Emails, the email contents needs to be
present back to the client as it came, so the client says the email has
not been altered.
On Mon, 2010-07-19 at 16:24 +0100, Timo Sirainen wrote:
> Code status
> ---
>
> Initial version of the attachment reading/writing code is already done
> and works (lacks some error handling and probably performance
> optimizations). The SIS plugin code is also started and should be
> worki
Timo Sirainen wrote:
I was thinking it would be nice to be able to compress attachments after
they've already been delivered. Like maybe keep the attachments decoded for a
few weeks and then compress them. Similar to how some people do it with
Maildir. This can't work without a small header, o
On Mon, 2010-07-19 at 19:49 +0100, William Blunn wrote:
> > I thought about that also, but it would require calculating and using a
> > hash of the decoded message (but not the compressed message). Could get
> > complex.
> >
>
> BTW I am not attempting to suggest a complete system for de-duplic
Timo Sirainen wrote:
On Mon, 2010-07-19 at 18:30 +0100, William Blunn wrote:
Consider storing the recovery filter stack in the dbox metadata rather
than the attachment file.
This has a couple of upshots:
1. If one person receives a message with an attachment which is encoded
with base64 a
Timo Sirainen wrote:
BTW. I was thinking about using "number of characters per base64 line" rather than
"number of cells". I don't think it's required that line ends with a complete cell.
You're right.
Looking at RFC2045, whitespace aside, the entire stream must be an
integer quantity of 4 c
On Mon, 2010-07-19 at 18:30 +0100, William Blunn wrote:
> Consider storing the recovery filter stack in the dbox metadata rather
> than the attachment file.
>
> This has a couple of upshots:
>
> 1. If one person receives a message with an attachment which is encoded
> with base64 at say 19 cell
Timo Sirainen wrote:
X1442 2742784 94/b2/01f34a9def84372a440d7a103a159ac6c9fd752b
2744378 27423 27/c8/a1dccc34d0aaa40e413b449a18810f600b4ae77b
So the format is:
"X" 1*( )
...
Extra features
--
The attachment files begin with an extensible header. This allows a
couple of extr
Timo Sirainen wrote:
Now that v2.0.0 is only waiting for people to report bugs (and me to figure out
how to fix them), I've finally had time to start doing what I actually came
here (Portugal Telecom/SAPO) to do. :)
The idea is to have dbox and mdbox support saving attachments (or MIME parts i
On Mon, 2010-07-19 at 09:01 -0700, Daniel L. Miller wrote:
> > The idea is to have dbox and mdbox support saving attachments (or MIME
> > parts in general) to separate files, which with some magic gives a
> > possibility to do single instance attachment storage. Comments welcome.
> >
> >
> YAAA
On 7/19/2010 8:24 AM, Timo Sirainen wrote:
Now that v2.0.0 is only waiting for people to report bugs (and me to
figure out how to fix them), I've finally had time to start doing what I
actually came here (Portugal Telecom/SAPO) to do. :)
The idea is to have dbox and mdbox support saving attachme
40 matches
Mail list logo
