Tobias Diekershoff writes:
> On 22.11.24 07:59, Tobias Diekershoff wrote:
> > so far we could not find the reason behind the performance
> > problem.
Since you're already running two allegedly identical systems, have you
tried hitting the experimental one with the "do what the developers
recommend" hammer? That is, blow away the Debian packages in favor of
a vanilla installation from source in a virtual environment (and
without --system-site-packages!)? It's impossible to be sure what's
going on in a Debian package environment because not only Mailman, but
any of the scores of packages that Mailman 3 depends on may have been
patched or reconfigured by Debian.
> To rule out virtualization problems
Do you mean a VM, which should not cause a problem, or a container,
which is known to be tricky to configure? (Note, I've not heard of it
causing the kinds of problems you describe.)
> in our setup, we tried to migrate MM3 to a bare metal server in the
> data center. After importing the database for MM3 and Hyperkitty to
> the new postgresql server MM3 wcomplained about
> > MultipleResultsFound: Multiple rows were found when one or none
> > was required [in the listarchiver table]
I've seen that in a migration from a Mailman 2 site to Mailman 3.
Once I removed the duplicate rows in the archiver table, it stopped.
I don't know why it happened, but it was a small number of lists (4 or
5 out of 20k), and at most one extra per (list, archiver) pair. I
suspect it carried over from the Mailman 2 instance. I no longer have
access to that system to confirm, sadly.
> However the duplicates are added back to the listarchiver table for
> all archivers (mhonarc, hyperkitty, mail-archive and prototype)
> even though only hyperkitty is activated.
By "duplicates are added back" do you mean something like this:
mailman=> select * from listarchiver;
id | mailing_list_id | name | _is_enabled
----+-----------------+--------------+-------------
1 | 1 | mhonarc | f
2 | 1 | mail-archive | f
3 | 1 | prototype | t
4 | 1 | hyperkitty | t
5 | 2 | mhonarc | f
6 | 2 | mail-archive | f
7 | 2 | prototype | t
8 | 2 | hyperkitty | t
where each list has one of each archiver whether or not that archiver
is active? This is correct behavior, as long as the archiving/*.py
and mailman_hyperkitty.py files are importable by Mailman core. Or by
"duplicates are added back" do you mean this?
mailman=> select * from listarchiver;
id | mailing_list_id | name | _is_enabled
----+-----------------+--------------+-------------
1 | 1 | mhonarc | f
2 | 1 | mhonarc | f
3 | 1 | mail-archive | f
4 | 1 | mail-archive | f
5 | 1 | prototype | t
6 | 1 | prototype | t
7 | 1 | hyperkitty | t
8 | 1 | hyperkitty | t
9 | 2 | mhonarc | f
10 | 2 | mhonarc | f
11 | 2 | mail-archive | f
12 | 2 | mail-archive | f
13 | 2 | prototype | t
14 | 2 | prototype | t
15 | 2 | hyperkitty | t
16 | 2 | hyperkitty | t
(don't take the id -> (list_id, archiver) mapping too seriously). In
the multiple instances of an archiver per list case, which are enabled?
> We tried to add a constrain on the table to only allow one entry
> per list and archiver, but that caused the runner to fail and the
> message to be shunt.
>
> > Uncaught runner exception: (raised as a result of Query-invoked
> > autoflush; consider using a session.no_autoflush block if this
> > flush is occurring prematurely) (psycopg2.errors.UniqueViolation)
> > duplicate key value violates unique constraint "workaround"
> > DETAIL: Key (mailing_list_id, name)=(98, mhonarc) already
> > exists.
This doesn't make any sense. That appears to be the listarchiver
table. As far as I can tell in a quick grep, the only place that is
mutated is in mailman/models/mailinglist.py:643 (line number is
current HEAD), which happens whenever the list checks for its active
archivers. It's initialized lazily for each list in that place.
@public
@implementer(IListArchiverSet)
class ListArchiverSet:
@dbconnection
def __init__(self, store, mailing_list):
self._mailing_list = mailing_list
system_archivers = {}
for archiver in config.archivers:
system_archivers[archiver.name] = archiver
# Add any system enabled archivers which aren't already associated
# with the mailing list.
for archiver_name in system_archivers:
exists = store.query(ListArchiver).filter(
ListArchiver.mailing_list == mailing_list,
ListArchiver.name == archiver_name).one_or_none()
if exists is None:
store.add(ListArchiver(mailing_list, archiver_name,
system_archivers[archiver_name]))
As you can see, the .add is protected by "if exists". That code has
not changed since v3.3.8, which is what is in Bookworm according to
"apt list mailman3".
If that is so in your installation, I'm pretty sure the problem is
somewhere between the SQLAlchemy API and the RDBMS, not in Mailman
itself. And if that's true, that might explain the weirdness in
archiving as well.
What version of PostgreSQL are you using? of SQLAlchemy? Are both
the original system with performance problems and the new one
accessing the same PostgreSQL server, or is there are separate
PostgreSQL instance for each?
Steve
_______________________________________________
Mailman-users mailing list -- mailman-users@mailman3.org
To unsubscribe send an email to mailman-users-le...@mailman3.org
https://lists.mailman3.org/mailman3/lists/mailman-users.mailman3.org/
Archived at:
https://lists.mailman3.org/archives/list/mailman-users@mailman3.org/message/YCZPWEA75Y2AQZOVUTDQ5CX6O6O2KMGP/
This message sent to arch...@mail-archive.com
