[Cloud] Re: [Cloud-announce] small toolsdb outage, just now

2022-12-07 Thread Francesco Negri
reboot. Apologies for the inconvenience. Francesco -- Francesco Negri (he/him) -- IRC: dhinus Site Reliability Engineer, Cloud Services team Wikimedia Foundation ___ Cloud mailing list -- cloud@lists.wikimedia.org List information: https://lists.wikimed

[Cloud] Re: Key errors on toolsdb database

2022-12-09 Thread Francesco Negri
gerrit.wikimedia.org/r/c/operations/puppet/+/837490/ [3] https://gerrit.wikimedia.org/r/c/operations/puppet/+/769025/ [4] https://phabricator.wikimedia.org/T301949 -- Francesco Negri (he/him) -- IRC: dhinus Site Reliability Engineer, Cloud Services team Wikimedia Foundation

[Cloud] ToolsDB: expect some turbulence

2023-01-16 Thread Francesco Negri
database. This work is tracked at https://phabricator.wikimedia.org/T301949 [1] https://wikitech.wikimedia.org/wiki/Help:Toolforge/Database -- Francesco Negri (he/him) -- IRC: dhinus Site Reliability Engineer, Cloud Services team Wikimedia Foundation

[Cloud] [Cloud-announce] Short downtime (expected: 2 mins) for ToolsDB

2023-02-20 Thread Francesco Negri
://phabricator.wikimedia.org/T301949 -- Francesco Negri (he/him) -- IRC: dhinus Site Reliability Engineer, Cloud Services team Wikimedia Foundation ___ Cloud-announce mailing list -- cloud-annou...@lists.wikimedia.org List information: https://lists.wikimedia.org/postorius

[Cloud] [Cloud-announce] Re: Short downtime (expected: 2 mins) for ToolsDB

2023-02-20 Thread Francesco Negri
On Mon, Feb 20, 2023 at 11:15 AM Francesco Negri wrote: > As part of the ToolsDB migration work [1], in about 1 hour from now I > will stop ToolsDB for a very short time (I expect the downtime to last > approximately 2 minutes). The total downtime (read-only time + full downtime) wa

[Cloud] [Cloud-announce] Re: Short downtime (expected: 2 mins) for ToolsDB

2023-03-06 Thread Francesco Negri
task https://phabricator.wikimedia.org/T301949 Thanks for your patience. -- Francesco Negri (he/him) -- IRC: dhinus Site Reliability Engineer, Cloud Services team Wikimedia Foundation ___ Cloud-announce mailing list -- cloud-annou...@lists.wikimedia.org

[Cloud] [Cloud-announce] Re: (Another) toolforge outage coming Thursday

2023-04-06 Thread Francesco Negri
st -- cloud-annou...@lists.wikimedia.org > List information: > https://lists.wikimedia.org/postorius/lists/cloud-announce.lists.wikimedia.org/ -- Francesco Negri (he/him) -- IRC: dhinus Site Reliability Engineer, Cloud Services team Wikimedia Foundation ___ Clo

[Cloud] [Cloud-announce] Re: (Another) toolforge outage coming Thursday

2023-04-06 Thread Francesco Negri
://phabricator.wikimedia.org/T333471 or ask in #wikimedia-cloud. Thanks! On Thu, Apr 6, 2023 at 6:07 PM Francesco Negri wrote: > > Reminder that this is starting in 1 hour. I will send another email > when this is done. > > Thanks for your patience. > > On Mon, Apr 3, 2023 at

[Cloud] [Cloud-announce] Cloud VPS maintenance: OpenStack upgrade

2023-11-30 Thread Francesco Negri
IRC (#wikimedia-cloud-admin). [1] https://releases.openstack.org/antelope/ -- Francesco Negri (he/him) -- IRC: dhinus Site Reliability Engineer, Cloud Services team Wikimedia Foundation ___ Cloud-announce mailing list -- cloud-annou

[Cloud] [Cloud-announce] Re: Cloud VPS maintenance: OpenStack upgrade

2023-11-30 Thread Francesco Negri
This upgrade is completed and everything is looking fine. Please report any unexpected behavior through the usual communication channels. [1] Thanks, Francesco [1] https://wikitech.wikimedia.org/wiki/Help:Cloud_Services_communication On Thu, Nov 30, 2023 at 11:12 AM Francesco Negri wrote

[Cloud] [Cloud-announce] Cloud VPS Network Outage

2023-12-01 Thread Francesco Negri
for your patience. -- Francesco Negri (he/him) -- IRC: dhinus Site Reliability Engineer, Cloud Services team Wikimedia Foundation ___ Cloud-announce mailing list -- cloud-annou...@lists.wikimedia.org List information: https://lists.wikimedia.org

[Cloud] [Cloud-announce] Re: Cloud VPS Network Outage

2023-12-01 Thread Francesco Negri
Most services have now recovered, but we are still working to stabilize things. More details will be added to the Phab task: https://phabricator.wikimedia.org/T352539 On Fri, Dec 1, 2023 at 3:56 PM Francesco Negri wrote: > > We are experiencing networking issues on Cloud VPS, which

[Cloud] Re: Fwd: Installing elasticsearch on a VPS instance?

2023-12-18 Thread Francesco Negri
nto further issues! Thanks, Francesco [1] https://wikitech.wikimedia.org/wiki/Wikitech:Cloud_Services_Terms_of_use -- Francesco Negri (he/him) -- IRC: dhinus Site Reliability Engineer, Cloud Services team Wikimedia Foundation ___ Cloud mai

[Cloud] Re: Installing elasticsearch on a VPS instance?

2023-12-19 Thread Francesco Negri
p 1073741824 bytes for committing reserved memory. Increasing the server total memory from 1 to 2 GB should fix it, just pick a different "instance flavor" in Cloud VPS. -- Francesco Negri (he/him) -- IRC: dhinus Site Reliability Engineer, Cloud Se

[Cloud] Re: Expired certificate?

2023-12-20 Thread Francesco Negri
SSL certificate, while traffic from the web proxy to your Cloud VPS instance does not use SSL at all. This means you need to disable SSL in OpenSearch, and let the proxy handle it. [1] https://wikitech.wikimedia.org/wiki/Help:Using_a_web_proxy_to_reach_Cloud_VPS_servers_from_the_internet -- F

[Cloud] [Cloud-announce] Toolforge Redis timeout for idle connections

2024-05-21 Thread Francesco Negri
-28_WMCS_Toolforge_Redis_refusing_connections -- Francesco Negri (he/him) -- IRC: dhinus Site Reliability Engineer, Cloud Services team Wikimedia Foundation ___ Cloud-announce mailing list -- cloud-annou...@lists.wikimedia.org List information: https

[Cloud] [Cloud-announce] Re: Toolforge Redis timeout for idle connections

2024-06-03 Thread Francesco Negri
This change went live today, one week later than planned. If you notice any issues with Toolforge Redis, please let us know. On Tue, May 21, 2024 at 5:05 PM Francesco Negri wrote: > > Starting next week, May 27th, connections to Toolforge Redis [0] that > remain idle for more than 1

[Cloud] [Cloud-announce] Access to ToolsDB public dbs from Quarry and Superset

2024-06-13 Thread Francesco Negri
kitech.wikimedia.org/wiki/Help:Toolforge/Database#Privileges_on_the_database [2] https://meta.wikimedia.org/wiki/Research:Quarry [3] https://superset.wmcloud.org/ -- Francesco Negri (he/him) -- IRC: dhinus Site Reliability Engineer, Cloud Services team Wikimedia Foundation

[Cloud] Re: Tunnel to Trove instance?

2024-06-24 Thread Francesco Negri
nformation: > > >>> https://lists.wikimedia.org/postorius/lists/cloud.lists.wikimedia.org/ > > >>> > > >> > > >> > > >> -- > > >> Daniel Zahn > > >> Site Reliability Engineer > > >> ___

[Cloud] [Cloud-announce] Re: Access to ToolsDB public dbs from Quarry and Superset

2024-07-02 Thread Francesco Negri
On Thu, Jun 13, 2024 at 4:59 PM Francesco Negri wrote: > == This change is planned to go live on Monday, July 1st. == This change is now live, and ToolsDB's "public" databases (the ones with a name ending with "_p") can be queried from Quarry. In Superset, databases

[Cloud] Re: [Cloud-announce] Re: Access to ToolsDB public dbs from Quarry and Superset

2024-07-23 Thread Francesco Negri
/T357624 Hope this helps, Francesco On Tue, Jul 23, 2024 at 3:23 PM Sam Wilson wrote: > > On 3/7/24 00:48, Francesco Negri wrote: > > On Thu, Jun 13, 2024 at 4:59 PM Francesco Negri > > wrote: > >> == This change is planned to go live on Monday, July 1st. == >

[Cloud] Re: [Cloud-announce] Re: Access to ToolsDB public dbs from Quarry and Superset

2024-07-25 Thread Francesco Negri
st a matter of waiting. > > > On 23/7/24 22:14, Francesco Negri wrote: > > Hi Sam, > > was that database created today? > > > > Quarry is connecting to the replica host, which currently has a > > replication lag of 13 hours. It should catch up in a day or two. > >

[Cloud] Re: [Cloud-announce] Re: Access to ToolsDB public dbs from Quarry and Superset

2024-07-29 Thread Francesco Negri
/query/11263 -- Francesco Negri (he/him) -- IRC: dhinus Site Reliability Engineer, Cloud Services team Wikimedia Foundation ___ Cloud mailing list -- cloud@lists.wikimedia.org List information: https://lists.wikimedia.org/postorius

[Cloud] [Cloud-announce] Re: Access to ToolsDB public dbs from Quarry and Superset

2024-09-02 Thread Francesco Negri
On Tue, Jul 2, 2024 at 6:48 PM Francesco Negri wrote: > In Superset, databases must unfortunately be added individually This has been solved [1], and all ToolsDB public databases (the ones ending with "_p") can be queried from both Quarry and Superset. > The last request in

[Cloud] [Cloud-announce] Temporary accounts are coming, check if your tool may be affected

2024-10-30 Thread Francesco Negri
iawiki.org/wiki/Trust_and_Safety_Product/Temporary_Accounts/Updates [2] https://www.mediawiki.org/wiki/Trust_and_Safety_Product/Temporary_Accounts/For_developers#How_should_I_update_my_code? [3] https://phabricator.wikimedia.org/T378516 -- Francesco Negri (he/him) -- IRC: dhinus Site Reliability Engin

[Cloud] [Cloud-announce] [wiki replicas] Expected replication lag on s8 (wikidatawiki)

2024-11-12 Thread Francesco Negri
nce work. [0] https://phabricator.wikimedia.org/T367856 [1] https://noc.wikimedia.org/db.php#tabs-s8 -- Francesco Negri (he/him) -- IRC: dhinus Site Reliability Engineer, Cloud Services team Wikimedia Foundation ___ Cloud-announce mailing

[Cloud] [Cloud-announce] ToolsDB is getting upgraded to MariaDB 10.6

2024-11-19 Thread Francesco Negri
orge/Admin/ToolsDB#Failing_over_to_the_replica_host [2] https://mariadb.com/kb/en/upgrading-from-mariadb-10-4-to-mariadb-10-5/ [3] https://mariadb.com/kb/en/upgrading-from-mariadb-10-5-to-mariadb-10-6/ -- Francesco Negri (he/him) -- IRC: dhinus Site Reliability Engineer, Cloud Services team

[Cloud] [Cloud-announce] Re: ToolsDB is getting upgraded to MariaDB 10.6

2024-11-25 Thread Francesco Negri
On Tue, Nov 19, 2024 at 4:29 PM Francesco Negri wrote: > Next Monday, November 25th, at around 13:00 UTC, ToolsDB will be > upgraded from MariaDB v10.4.29 to MariaDB v10.6.19. [0] This upgrade is happening today starting at 13:00 UTC (in one hour from now). -- Francesco Negri (he/him)

[Cloud] Re: Prometheus

2024-11-14 Thread Francesco Negri
ere a test instance that I can use so I don't start throwing garbage at the > production server? > > ___ > Cloud mailing list -- cloud@lists.wikimedia.org > List information: > https://lists.wikimedia.org/postorius/lists/cloud.li

[Cloud] [Cloud-announce] Re: ToolsDB is getting upgraded to MariaDB 10.6

2024-11-25 Thread Francesco Negri
The upgrade is now complete! It took a bit longer than expected, because of some unforeseen DNS issues that meant for about 1 hour ToolsDB was not reachable. I will write a more detailed report in the upgrade task. [0] [0] T352206 -- Francesco Negri (he/him) -- IRC: dhinus Site Reliability

[Cloud] [Cloud-announce] ToolsDB maintenance on Thursday, Feb 6

2025-02-03 Thread Francesco Negri
/ZOCPVXX6BKLC76OHMIQW26YLBCKEBTGQ/ -- Francesco Negri (he/him) -- IRC: dhinus Site Reliability Engineer, Cloud Services team Wikimedia Foundation ___ Cloud-announce mailing list -- cloud-annou...@lists.wikimedia.org List information: https://lists.wikimedia.org

[Cloud] Re: python 3.11?

2025-01-22 Thread Francesco Negri
.11 -- the tf- prefix isn't used any more and the alias only exists for the older images 03:57 https://wikitech.wikimedia.org/wiki/Module:Toolforge_images/data.json is the onwiki info about the images -- Francesco Negri (he/him) -- IRC: dhinus Site Reliability Engineer, Cloud Services

[Cloud] [Cloud-announce] Share your feedback on the "Toolforge UI" initiative

2025-01-14 Thread Francesco Negri
team is in the process of collecting feedback from the Toolforge community to help shape the solution to their needs." Read more and share your thoughts about Toolforge UI at this link: https://wikitech.wikimedia.org/wiki/Wikimedia_Cloud_Services_team/EnhancementProposals/Toolforge_UI -- F

[Cloud] Re: LLM services

2025-01-10 Thread Francesco Negri
> A phabricator ticket would also be useful, to help inform future actions. > Would > you mind creating one? We have https://phabricator.wikimedia.org/T336905 that seems to cover the same topic. -- Francesco Negri (he/him) -- IRC: dhinus Site Reliability Engineer, Cloud Services team

[Cloud] [Cloud-announce] Re: ToolsDB maintenance on Monday, February 24

2025-02-24 Thread Francesco Negri
]. [0] https://phabricator.wikimedia.org/T385885 On Thu, Feb 20, 2025 at 1:21 PM Francesco Negri wrote: > On Monday, February 24th at 14:00 UTC, I am going to upgrade MariaDB to > the latest patch version 10.6.20. [0] > > This will cause a brief downtime for ToolsDB (just a couple of minu

[Cloud] [Cloud-announce] ToolsDB maintenance on Monday, February 24

2025-02-20 Thread Francesco Negri
On Monday, February 24th at 14:00 UTC, I am going to upgrade MariaDB to the latest patch version 10.6.20. [0] This will cause a brief downtime for ToolsDB (just a couple of minutes). [0] https://phabricator.wikimedia.org/T385885 -- Francesco Negri (he/him) -- IRC: dhinus Site Reliability

[Cloud] Re: Problems login into horzion

2025-03-11 Thread Francesco Negri
t; List information: > https://lists.wikimedia.org/postorius/lists/cloud.lists.wikimedia.org/ > -- Francesco Negri (he/him) -- IRC: dhinus Site Reliability Engineer, Cloud Services team Wikimedia Foundation ___ Cloud mailing list -- cloud@lists.wiki

[Cloud] Re: Network maintenance operation on 2025-03-27 12:30 UTC

2025-03-27 Thread Francesco Negri
hanks for your patience while we fix this issue. -- Francesco Negri (he/him) -- IRC: dhinus Site Reliability Engineer, Cloud Services team Wikimedia Foundation ___ Cloud mailing list -- cloud@lists.wikimedia.org List information: https://lists.wikime