Paul Tomblin via Mailman-users writes:
> I backup the full server at a time
> I’ve been running Mailman 3 on it since January.
> The current disk use of my entire server is around 35 Gb.
> The borg backup is somewhere around 600 Gb.
OK, if you just did tar backups (uncompressed!) you should see
(7 + 4 + 6) x 35GB = 595 GB, no? So this is just nuts.
BLUF: I think you have a borg vs. DB problem, not a borg vs. Mailman
problem. You should ask them, because they're the ones who understand
their algorithms and software. I would imagine that (1) it's possible
to tune borg to keep its own overhead reasonable even if the use case
is outside of its primary desigh, and (2) the borg devs have very
likely had experience with "database backups bloat quickly". They can
tell you how to configure PostgreSQL to be more borg-friendly, or
vice-versa, or to configure borg around some other backup method for
PostgreSQL.
A few thoughts since I already thought them:
By "my entire server" do you mean the /var partition, or everything
the OS can access? If the latter, about 20GB of that is just
operating system and server software, no? Which presumably doesn't
change very often in a 6-month period? I'd guess you're looking at
most 15GB of potentially variable data, so 255GB in uncompressed
tarballs. Even if the 35GB is just /var, I just don't see how you
could have a Mailman installation generating 500GB of new backup in 5
months unless the traffic is truly gargantuan.
One possibility for a fraction of that bloat is that you have the
prototype ("write-only to maildir") archiver enabled. In that case
$var_dir/archives/prototype could be collecting GB fairly quickly.
Since it's write-only, there's no loss to disabling it and deleting
it. But it's append-only so should deduplicate well, and unless mail
traffic is exploding it should pretty quickly stabilize to contribute
a constant amount to a stable archive size -- it wouldn't explain your
600-lb gorilla. I could be wrong if you have enough traffic, so check
that.
If you want to continue the conversation here I'd need du data on the
mailman installation (especially Mailman's $var_dir), on Mailman's
$log_dir (which is $var_dir/logs by default but can be configured
elsewhere, often /var/log/mailman3), and on the PostgreSQL database
(usually /var/lib/postgresql/$version/main) Mailman is using, but I
really don't think I can help much. Also how big your Mailman 2
archives were, if you kept them.
> The old borg backup when I was using Mailman 2 was less than 100 Gb
That sounds right given the schedule of "keeps".
Steve
--
GNU Mailman consultant (installation, migration, customization)
Sirius Open Source https://www.siriusopensource.com/
Software systems consulting in Europe, North America, and Japan
_______________________________________________
Mailman-users mailing list -- [email protected]
To unsubscribe send an email to [email protected]
https://lists.mailman3.org/mailman3/lists/mailman-users.mailman3.org/
Archived at:
https://lists.mailman3.org/archives/list/[email protected]/message/G64M36PG2QGMAZLQB52BSQCDBIK7OXHT/
This message sent to [email protected]
