[Cloud] [Cloud-announce] [Toolforge] max_user_connections = 20 set for ToolsDB

2019-02-25 Thread Bryan Davis
During the flurry of activity we had recently in diagnosing and fixing problems with the shared ToolsDB MariaDB service [0], we made a configuration change to place a hard limit on the maximum number of simultaneous connections permitted for each user account [1][2]. The current limit is set at 20

Re: [Cloud] Recovering from issues with NFS

2019-02-25 Thread Giovanni Tirloni
It seems things are starting to calm down a little bit. The disk space issue is resolved for now and we've fixed the issues reported so far on #wikimedia-cloud We're keeping an eye on it for the new few hours. Thanks for bearing with us. -- Giovanni Tirloni Operations Engineer Wikimedia Fou

[Cloud] Recovering from issues with NFS

2019-02-25 Thread Giovanni Tirloni
Hello, Unfortunately, we had some disk space issues in our NFS server that required the NFS process to be restarted. This created all sorts of issues in Toolforge and Cloud VPS which we are still recovering from. If your tool is not working and wasn't restarted automatically, please try to