All,
Due to Xtrabackup requiring a read lock for part of the backup process,
there will be an additional outage at the beginning of the maintenance
window so we can complete a final backup before the cluster upgrade. We
will do our best to minimize the outage time.
Please let us know if you have any questions or concerns.
On 4/18/22 2:41 PM, Ethan Hampton wrote:
All,
We had an outage of the mysql1-vip cluster from 20:18 to 21:04 UTC
today (Apr 18th) related to the final backup and preparation for this
upgrade. Xrabackup issued a "FLUSH TABLES WITH READ LOCK" for the
final stage of the backup process. When combined with a long running
query, this exhausted the MySQL connection pool and resulted in a
read/write outage.
We will take steps to mitigate this for the upgrade on the 20th.
Please let us know if you have any questions or concerns.
Best-
On 4/13/22 1:53 PM, Ethan Hampton wrote:
All,
We will be upgrading one of our MySQL clusters (mysql1-vip
(mysql1/mysql2)) next Wednesday, April 20th, 2022 between 17:00-20:00
UTC (10am-1pm PDT).
We are upgrading MySQL (Percona) from 5.5 to 5.6. This is the first
step towards managing the cluster under Chef and upgrading to
MySQL(Percona)5.7 at a future date. Expect more maintenance on this
cluster in the near future.
This will result in up to 20 minutes of downtime in the middle of the
window as we fail-over the cluster. We estimate that the actual
downtime will be far less than that bound.
We have tested this upgrade in a separate environment and believe
that it will have minimal impact to projects on the cluster. We will
make a full backup of the cluster before the upgrade work begins.
*If you are using the vip1 cluster, please make sure you are using
the mysql1-vip.osuosl.org address instead of mysql1/2.osuosl.org
directly or additional outages will occur.*
If you have any questions or concerns, please let us know. We'll be
in #osuosl on Libera Chat in case any one needs to contact us directly.
Thanks!
--
Ethan Hampton
Student Systems Engineer
Oregon State University | Open Source Lab
_______________________________________________
Hosting mailing list
host...@osuosl.org
https://lists.osuosl.org/mailman/listinfo/hosting
