Re: The future of SIS

2024-08-16 Thread Mingye Wang via dovecot
Pedro Ribeiro wrote: Ooops, we are using SIS, guess the solution for a similar optimization will be a native deduplicated filesystem. A non-deduplicated filesystem is fine considering the current hash-based folder structure. Just: 1. Switch to a hash with no known collision method (i.e. not

Re: [ext] Re: The future of SIS

2023-11-02 Thread Ralf Hildebrandt via dovecot
* Pedro Ribeiro via dovecot : > Hello everyone! > I'm reviving the topic just to add that after reconstructing our storage with > SIS disabled the occupied space increased from 5.3TB to 9.6TB, almost > doubling! > It's a feature promoting storage efficiency, I think it demands some > ponderation t

Re: The future of SIS

2023-11-01 Thread Pedro Ribeiro via dovecot
Hello everyone! I'm reviving the topic just to add that after reconstructing our storage with SIS disabled the occupied space increased from 5.3TB to 9.6TB, almost doubling! It's a feature promoting storage efficiency, I think it demands some ponderation the advantages of keeping or improving the m

Re: The future of SIS

2023-10-23 Thread Aki Tuomi via dovecot
> On 17/10/2023 03:26 EEST Jan Bramkamp wrote: > > > On 16.10.23 13:17, Pedro Ribeiro via dovecot wrote: > > Hello to everyone! > > Ooops, we are using SIS, guess the solution for a similar optimization will > > be > > a native deduplicated filesystem. > > A block level de-duplicating files

Re: The future of SIS

2023-10-19 Thread Jan Bramkamp
On 16.10.23 13:17, Pedro Ribeiro via dovecot wrote: Hello to everyone! Ooops, we are using SIS, guess the solution for a similar optimization will be a native deduplicated filesystem. A block level de-duplicating filesystem can only deduplicate data that is aligned to block boundaries. E-mail

Re: The future of SIS

2023-10-18 Thread Oscar del Rio
On 2023-10-18 3:35 a.m., Marc wrote: Dovecot has this option to store attachments separately not? So I am not sure this is then still a problem. Interesting. How do you tell dovecot to do that ? I thought I read about something like this, mail_location =

Re: The future of SIS

2023-10-18 Thread Jean-Daniel Dupas
> Le 18 oct. 2023 à 09:35, Marc a écrit : > >> Dovecot has this option to store attachments separately not? So I am >> not sure this is then still a problem. >> >> >> >> Interesting. How do you tell dovecot to do that ? >> > > I thought I read about something like this, > > mail_loca

RE: The future of SIS

2023-10-18 Thread Marc
> Dovecot has this option to store attachments separately not? So I am > not sure this is then still a problem. > > > > Interesting. How do you tell dovecot to do that ? > I thought I read about something like this, mail_location = ATTACHMENTS=/attachment but now you have made me

Re: The future of SIS

2023-10-17 Thread Jean-Daniel Dupas
Le 17 oct. 2023 à 16:34, Marc a écrit : The problem is a bit what everyone understands as s3. I associate this indeed also with an http endpoint on object storage. But the ceph plugin skips this http and talks directly to object store. I don't think you wou

Re: The future of SIS

2023-10-17 Thread Laura Smith via dovecot
--- Original Message --- On Tuesday, October 17th, 2023 at 15:27, Filip Hanes via dovecot wrote: > Other S3 implementation is Minio on top of any posix filesystem - you can > choose which fills your needs. Minio is great in general, the only thing I would say it its a little bit wei

RE: The future of SIS

2023-10-17 Thread Marc
> > > > The problem is a bit what everyone understands as s3. I associate > this indeed also with an http endpoint on object storage. But the ceph > plugin skips this http and talks directly to object store. I don't think > you would like to operate on this http level. If I look at this pa

Re: The future of SIS

2023-10-17 Thread Jean-Daniel Dupas
Le 17 oct. 2023 à 13:12, Marc a écrit : Is s3 not to slow for this? I think the clue is in the name "s3- compatible". Clearly calling out to "real" (AWS) S3 would be a non-starter.

RE: The future of SIS

2023-10-17 Thread Marc
> > 17.10.2023 12:22, Filip Hanes via dovecot пишет: > > S3-compatible storage is very good for multi-server installations where > you need redundancy, availability. S3 is basically HTTP server so you can > code your own logic on stored emails, balancers, caches, deduplication, > compression, encr

Re: The future of SIS

2023-10-17 Thread Dmitry Melekhov
17.10.2023 12:22, Filip Hanes via dovecot пишет: S3-compatible storage is very good for multi-server installations where you need redundancy, availability. S3 is basically HTTP server so you can code your own logic on stored emails, balancers, caches, deduplication, compression,

RE: The future of SIS

2023-10-17 Thread Marc
> > > > > > If you are using Ubuntu, OpenZFS is readily available, and support > deduplication natively. > > > I thought nobody sane actually used ZFS dedup because it eats RAM for > breakfast, lunch and dinner ? > What an interesting and informing reading lately!! Thanks everyone!! __

RE: The future of SIS

2023-10-17 Thread Marc
> >>> > Is s3 not to slow for this? > > >>> I think the clue is in the name "s3-compatible". > >>> > >>> Clearly calling out to "real" (AWS) S3 would be a non-starter. > >>> > >>> But a local installation of something like CEPH, MinIO or whatever on > the > >>> same LAN ? I'd think that s

Re: The future of SIS

2023-10-17 Thread Laura Smith via dovecot
--- Original Message --- On Tuesday, October 17th, 2023 at 06:46, Jean-Daniel Dupas wrote: > > If you are using Ubuntu, OpenZFS is readily available, and support > deduplication natively. I thought nobody sane actually used ZFS dedup because it eats RAM for breakfast, lunch and d

Re: The future of SIS

2023-10-17 Thread Filip Hanes via dovecot
> Day 16. 10. 2023 21:30, Emmanuel Fusté wrote: > > Le 16/10/2023 à 19:44, Marc a écrit : >>> Is s3 not to slow for this? >>> I think the clue is in the name "s3-compatible". >>> >>> Clearly calling out to "real" (AWS) S3 would be a non-starter. >>> >>> But a local installation of

Re: The future of SIS

2023-10-17 Thread Filip Hanes via dovecot
> Day 17. 10. 2023 7:46, Jean-Daniel Dupas wrote: > > If you are using Ubuntu, OpenZFS is readily available, and support > deduplication natively. > Else it is also available on other platforms, but may require more setup. Filesystems does not have deduplication effective for emails. They most

Re: The future of SIS

2023-10-16 Thread Jean-Daniel Dupas
> Le 16 oct. 2023 à 15:51, Marc a écrit : > >>> Hello to everyone! >>> Ooops, we are using SIS, guess the solution for a similar optimization >> will be >>> a native deduplicated filesystem. >> >> did you really mean deduplicated or distributed? >> > > I think this duduplicating. Storage sys

Re: The future of SIS

2023-10-16 Thread Emmanuel Fusté
Le 16/10/2023 à 19:44, Marc a écrit : Is s3 not to slow for this? I think the clue is in the name "s3-compatible". Clearly calling out to "real" (AWS) S3 would be a non-starter. But a local installation of something like CEPH, MinIO or whatever on the same LAN ? I'd think that should be wor

RE: The future of SIS

2023-10-16 Thread Marc
> > > > Is s3 not to slow for this? > > > > I think the clue is in the name "s3-compatible". > > Clearly calling out to "real" (AWS) S3 would be a non-starter. > > But a local installation of something like CEPH, MinIO or whatever on the > same LAN ? I'd think that should be workable, no ?

RE: The future of SIS

2023-10-16 Thread Laura Smith via dovecot
> Is s3 not to slow for this? > I think the clue is in the name "s3-compatible". Clearly calling out to "real" (AWS) S3 would be a non-starter. But a local installation of something like CEPH, MinIO or whatever on the same LAN ? I'd think that should be workable, no ? ___

RE: The future of SIS

2023-10-16 Thread Laura Smith via dovecot
> > Interesting, nice they use this rust, I am curious how they define this > scaling. What I don't get is why are they messing with smtp. I always get a > bad feeling when a company is trying to do everything. Good they are using rust and even better they've had an independent security audi

Re: The future of SIS

2023-10-16 Thread Laura Smith via dovecot
> > Well, so Laura is absolutely right ... > > > "Things like dsync will be GONE in the community version." > > That's not right, dsync is still there. Replicator is not, so dsync can't be > triggered automatically by dovecot after changes to the mailbox Well, to be fair : 1. I said what I

RE: The future of SIS

2023-10-16 Thread Marc
> >> > >> What is being removed is the replicator plugin (that used dsync). > That's what is being referred to in the video. Replicator hasn't been > actively maintained for years now so this was dead code anyway. > > > > > > Well, so Laura is absolutely right ... > > "Things like dsync will

RE: The future of SIS

2023-10-16 Thread Marc
> > > > If that is the case, well then I have to find another way to keep mails > in sync between 2 mailservers. Hope the community will find a new solution! > > > > I have been keeping one eye on Stalwart (https://stalw.art/) for a while > now. > > I haven't tested it as yet, but I'm very much t

Re: The future of SIS

2023-10-16 Thread Oscar del Rio
On 2023-10-16 2:30 a.m., Michael Slusarz via dovecot wrote: To answer the OP: sis is also being removed and should not be used by any new installation. Code remains to read data written by the old plug-in so that these installations don't require a migration between 2.3 and 2.4

Re: The future of SIS

2023-10-16 Thread Laura Smith via dovecot
> > If that is the case, well then I have to find another way to keep mails in > sync between 2 mailservers. Hope the community will find a new solution! > I have been keeping one eye on Stalwart (https://stalw.art/) for a while now. I haven't tested it as yet, but I'm very much tempted to g

RE: The future of SIS

2023-10-16 Thread Marc
> > Hello to everyone! > > Ooops, we are using SIS, guess the solution for a similar optimization > will be > > a native deduplicated filesystem. > > did you really mean deduplicated or distributed? > I think this duduplicating. Storage systems are offering such solutions. I think ceph has some

Re: The future of SIS

2023-10-16 Thread Bjoern Franke via dovecot
Hi, Hello to everyone! Ooops, we are using SIS, guess the solution for a similar optimization will be a native deduplicated filesystem. did you really mean deduplicated or distributed? Regards Bjoern ___ dovecot mailing list -- dovecot@dovecot.org

RE: The future of SIS

2023-10-16 Thread Chris Candreva
On Mon, 16 Oct 2023, Marc wrote: > Is this feature really useful? I can imagine if you are twitter or ig and > everyone is posting the same video this could be usefull. Are there any stats > on this available, so you know what to expect implementing deduplication. In an office where people insi

RE: The future of SIS

2023-10-16 Thread Marc
> Ooops, we are using SIS, guess the solution for a similar optimization > will be a native deduplicated filesystem. > Is this feature really useful? I can imagine if you are twitter or ig and everyone is posting the same video this could be usefull. Are there any stats on this available, so yo

Re: The future of SIS

2023-10-16 Thread Pedro Ribeiro via dovecot
Hello to everyone! Ooops, we are using SIS, guess the solution for a similar optimization will be a native deduplicated filesystem. For server synchronization (non "realtime") we are using "imapsync" ( https:// imapsync.lamiral.info/ ) regards! On 16/10/23 08:11, Taavi Ansper via dovecot wrote:

Re: The future of SIS

2023-10-16 Thread Taavi Ansper via dovecot
Hi So in my 99-dsync.conf This would not work in newer releases? service replicator { unix_listener replicator-doveadm { mode = 0666 } } plugin { mail_replica = tcp:example.com:12345 } If that is the case, well then I have to find another way to keep mails in sync between 2 mailser

Re: The future of SIS

2023-10-16 Thread Dmitry Melekhov
16.10.2023 10:30, Michael Slusarz via dovecot пишет: Aki is correct and is consistent with what I said in the video, although I could have phrased my explanation better. "dsync" refers to the tool/utility (part of doveadm) that does mail synchronization between a source account to a destinati

Re: The future of SIS

2023-10-15 Thread Michael Slusarz via dovecot
Aki is correct and is consistent with what I said in the video, although I could have phrased my explanation better. "dsync" refers to the tool/utility (part of doveadm) that does mail synchronization between a source account to a destination account. As Aki said, this is not going anywhere.

Re: The future of SIS

2023-10-15 Thread Gedalya via dovecot
On 10/14/23 03:26, Laura Smith via dovecot wrote: > FUD ? > > I knew someone would accuse me of that which is why I linked to the video > from the horse's mouth, I transcribe what the speaker said: > > "there will be an open source version, but that open source version will be > maintained for s

Re: The future of SIS

2023-10-13 Thread Laura Smith via dovecot
FUD ? I knew someone would accuse me of that which is why I linked to the video from the horse's mouth, I transcribe what the speaker said: "there will be an open source version, but that open source version will be maintained for single server use only. we are actually taking out anything any

Re: The future of SIS

2023-10-13 Thread Aki Tuomi via dovecot
Dear Laura, please don't spread FUD that you made up. Dsync is not going anywhere, and we are not close-sourcing Dovecot Core. There is not a trove of code going into Dovecot 3.0 that "never sees the daylight". Thank you, Aki > On 13/10/2023 21:10 EEST Laura Smith via dovecot wrote: > > > T

Re: The future of SIS

2023-10-13 Thread Laura Smith via dovecot
TL;DR If you are a Dovecot Community user, don't waste your time reading the Dovecot Pro release notes. To expand: I think you have to understand that lots of things that are going into Dovecot 3 (Pro) will never see the light of day in the community edition. In addition, Dovecot have publicly