All,
We had an outage of the mysql1-vip cluster from 20:18 to 21:04 UTC today
(Apr 18th) related to the final backup and preparation for this upgrade.
Xrabackup issued a "FLUSH TABLES WITH READ LOCK" for the final stage of
the backup process. When combined with a long running query, this
exhausted the MySQL connection pool and resulted in a read/write outage.
We will take steps to mitigate this for the upgrade on the 20th. Please
let us know if you have any questions or concerns.
Best-
--
Ethan Hampton
Student Systems Engineer
Oregon State University | Open Source Lab
On 4/13/22 1:53 PM, Ethan Hampton wrote:
All,
We will be upgrading one of our MySQL clusters (mysql1-vip
(mysql1/mysql2)) next Wednesday, April 20th, 2022 between 17:00-20:00
UTC (10am-1pm PDT).
We are upgrading MySQL (Percona) from 5.5 to 5.6. This is the first
step towards managing the cluster under Chef and upgrading to
MySQL(Percona)5.7 at a future date. Expect more maintenance on this
cluster in the near future.
This will result in up to 20 minutes of downtime in the middle of the
window as we fail-over the cluster. We estimate that the actual
downtime will be far less than that bound.
We have tested this upgrade in a separate environment and believe that
it will have minimal impact to projects on the cluster. We will make a
full backup of the cluster before the upgrade work begins.
*If you are using the vip1 cluster, please make sure you are using the
mysql1-vip.osuosl.org address instead of mysql1/2.osuosl.org directly
or additional outages will occur.*
If you have any questions or concerns, please let us know. We'll be in
#osuosl on Libera Chat in case any one needs to contact us directly.
Thanks!
--
Ethan Hampton
Student Systems Engineer
Oregon State University | Open Source Lab
_______________________________________________
Hosting mailing list
host...@osuosl.org
https://lists.osuosl.org/mailman/listinfo/hosting
