Re: [Cloud] CloudVPS maintenance on Wednesday 2019-10-16 (round 2 of cloudvirt reboots)
On 10/9/19 1:45 PM, Arturo Borrero Gonzalez wrote: > Hello! > > Next Wednesday 2019-10-16 at 09:00 UTC we will be doing another maintenance > operation on some of our cloudvirts servers (the hypervisor servers) that > involves rebooting both the physical servers and the virtual machines running > on > them. > The reasons is that we ned to update the running linux kernel version they > have. > > In this window we will reboot 4 hypervisors: > * cloudvirt1028 > * cloudvirt1029 > * cloudvirt1030 > > The procedure will be to reboot a server, wait for it to come back online > (could > take up to 5 minutes) and wait for all the VMs to come back online. Then move > to > the next server. > > Toolforge users may see their tools and webservices briefly disrupted due to > several components of the Toolforge infrastructure being rebooted in this > operation. > Remainder, this is happening today in about 10 minutes! regards -- Arturo Borrero Gonzalez SRE / Wikimedia Cloud Services Wikimedia Foundation ___ Wikimedia Cloud Services mailing list Cloud@lists.wikimedia.org (formerly lab...@lists.wikimedia.org) https://lists.wikimedia.org/mailman/listinfo/cloud
Re: [Cloud] CloudVPS maintenance on Wednesday 2019-10-16 (round 2 of cloudvirt reboots)
Hi, you said 4 servers but also you said cloudvirt1028, cloudvirt1029 and cloudvirt1030. Where is fourth? Best regards, Zoran. ___ Wikimedia Cloud Services mailing list Cloud@lists.wikimedia.org (formerly lab...@lists.wikimedia.org) https://lists.wikimedia.org/mailman/listinfo/cloud
[Cloud] CloudVPS maintenance on Wednesday 2019-10-23 (round 3 of cloudvirt reboots)
Hello! Next Wednesday 2019-10-23 at 09:00 UTC we will be doing another maintenance operation on some of our cloudvirts servers (the hypervisor servers) that involves rebooting both the physical servers and the virtual machines running on them. The reasons is that we ned to update the running linux kernel version they have. In this window we will reboot 3 hypervisors: * cloudvirt1014 * cloudvirt1025 * cloudvirt1026 The procedure will be to reboot a server, wait for it to come back online (could take up to 5 minutes) and wait for all the VMs to come back online. Then move to the next server. Toolforge users may see their tools and webservices briefly disrupted due to several components of the Toolforge infrastructure being rebooted in this operation. If nothing changes (reallocated or new virtual machine, etc) this is the list of affected VM instances in each hypervisor: * cloudvirt1014: VM: globalcu-app-01 PROJECT: globalcu VM: xtools-dev05 PROJECT: xtools VM: ores-web-05 PROJECT: ores VM: roebling PROJECT: wikispore VM: ores-web-04 PROJECT: ores VM: canary1014-01 PROJECT: testlabs VM: cloudinfra-db02 PROJECT: cloudinfra VM: Krypton PROJECT: codereview VM: dashiki-staging-01 PROJECT: dashiki VM: discovery-production-02 PROJECT: shiny-r VM: logparse01 PROJECT: security-tools VM: tools-sgegrid-shadow PROJECT: tools VM: maps-tiles1 PROJECT: maps VM: wikimetrics-01 PROJECT: wikimetrics VM: tools-sgecron-01 PROJECT: tools VM: wikitextexp-base-1002 PROJECT: wikitextexp VM: kask-client PROJECT: services VM: clm-web-01 PROJECT: community-labs-monitoring VM: accounts-appserver4 PROJECT: account-creation-assistance VM: taxonbota PROJECT: dwl VM: tofawiki02 PROJECT: fa-wp VM: federated-commons PROJECT: wikidata-federation VM: puppetmaster PROJECT: thumbor VM: lizenzhinweisgenerator PROJECT: lizenzhinweisgenerator VM: hound-puppet-02 PROJECT: hound VM: packagist-mirror1 PROJECT: packagist-mirror VM: deployment-elastic06 PROJECT: deployment-prep VM: deployment-changeprop PROJECT: deployment-prep VM: deployment-restbase02 PROJECT: deployment-prep VM: deployment-imagescaler01 PROJECT: deployment-prep VM: deployment-zookeeper02 PROJECT: deployment-prep VM: deployment-kafka-jumbo-1 PROJECT: deployment-prep VM: deployment-memc07 PROJECT: deployment-prep VM: deployment-eventlog05 PROJECT: deployment-prep VM: deployment-cpjobqueue PROJECT: deployment-prep VM: deployment-mediawiki-07 PROJECT: deployment-prep VM: deployment-chromium01 PROJECT: deployment-prep VM: deployment-puppetdb02 PROJECT: deployment-prep VM: deployment-puppetmaster03 PROJECT: deployment-prep VM: deployment-cache-text05 PROJECT: deployment-prep VM: gfg01 PROJECT: video VM: encoding01 PROJECT: video VM: whgi PROJECT: wikidumpparse VM: wikilabels PROJECT: wmf-research-tools VM: gitservices PROJECT: getstarted VM: design-research-methods PROJECT: design VM: wikilabels-backups PROJECT: wikilabels VM: wikilabels-02 PROJECT: wikilabels VM: dumpgrepper PROJECT: visualeditor VM: compiler1001 PROJECT: puppet-diffs VM: af-puppetdb02 PROJECT: automation-framework VM: clickmodel PROJECT: search VM: tool PROJECT: recommendation-api VM: missing-sections PROJECT: recommendation-api VM: ores-lb-03 PROJECT: ores VM: puppet-ema-2 PROJECT: puppet VM: matrix-synapse-01 PROJECT: matrix VM: wikibrain-embeddings-01 PROJECT: wikibrain VM: qube-node2 PROJECT: k8splay VM: captcha-tf-43 PROJECT: privpol-captcha VM: k4-2 PROJECT: analytics * cloudvirt1025: VM: integration-agent-docker-1006 PROJECT: integration VM: striker-deploy04 PROJECT: striker VM: rec-wiki-2 PROJECT: recommendation-api VM: deployment-ms-fe03 PROJECT: deployment-prep VM: deployment-poolcounter05 PROJECT: deployment-prep VM: deployment-ms-be05 PROJECT: deployment-prep VM: readers-web-stephen PROJECT: reading-web-staging VM: traffic-upload-stretch PROJECT: traffic VM: traffic-recdns-anycast PROJECT: traffic VM: deployment-maps05 PROJECT: deployment-prep VM: gerrit-sizzle PROJECT: security-tools VM: tools-sgewebgrid-generic-0901 PROJECT: tools VM: shinken-puppetmaster-01 PROJECT: shinken VM: osmit-due PROJECT: osmit VM: deployment-acme-chief03 PROJECT: deployment-prep VM: meza-cindy PROJECT: pluggableauth VM: accounts-db4 PROJECT: account-creation-assistance VM: krenair-clientpackages-py3-jessie PROJECT: testlabs VM: deployment-sessionstore01 PROJECT: deployment-prep VM: paws-worker-04 PROJECT: paws VM: paws-ext-lb-02 PROJECT: paws VM: paws-int-lb-01 PROJECT: paws VM: paws-master-03 PROJECT: paws VM: paws-master-01 PROJECT: paws VM: language-readership PROJECT: language VM: wmde-wikidiff2-patched-stretch PROJECT: wikidiff2-wmde-dev VM: tools-sgebastion-08 PROJECT: tools VM: compiler1002 PROJECT: puppet-diffs VM: phragile-db PROJECT: phragile VM: cloud-puppetmaster-01 PROJECT: cloudinfra VM: chicotest-cappy01 PROJECT: chicotestproject VM: visualeditor-prototype2 PROJECT: visualeditor VM: programs-and-events-dashboard PROJECT: globaleducation VM: osmit-uno PROJECT: osmit VM: tools-sgewebgrid-lighttpd-0904 PROJE
Re: [Cloud] CloudVPS maintenance on Wednesday 2019-10-16 (round 2 of cloudvirt reboots)
On 10/16/19 12:42 PM, Zoran Dori wrote: > Hi, > you said 4 servers but also you said cloudvirt1028, cloudvirt1029 and > cloudvirt1030. Where is fourth? > That's a typo. Sorry for that. We are rebooting *3* cloudvirts. Good catch! :-P regards. -- Arturo Borrero Gonzalez SRE / Wikimedia Cloud Services Wikimedia Foundation ___ Wikimedia Cloud Services mailing list Cloud@lists.wikimedia.org (formerly lab...@lists.wikimedia.org) https://lists.wikimedia.org/mailman/listinfo/cloud
Re: [Cloud] CloudVPS maintenance on Wednesday 2019-10-16 (round 2 of cloudvirt reboots)
No problem, thanks! sre, 16. okt 2019. 14:39 Arturo Borrero Gonzalez je napisao/la: > On 10/16/19 12:42 PM, Zoran Dori wrote: > > Hi, > > you said 4 servers but also you said cloudvirt1028, cloudvirt1029 and > > cloudvirt1030. Where is fourth? > > > > That's a typo. Sorry for that. We are rebooting *3* cloudvirts. > > Good catch! :-P > > regards. > > -- > Arturo Borrero Gonzalez > SRE / Wikimedia Cloud Services > Wikimedia Foundation > ___ Wikimedia Cloud Services mailing list Cloud@lists.wikimedia.org (formerly lab...@lists.wikimedia.org) https://lists.wikimedia.org/mailman/listinfo/cloud
Re: [Cloud] CloudVPS maintenance on Wednesday 2019-10-23 (round 3 of cloudvirt reboots)
Follow-up: We just discovered cloudvirt1014 doesn't require reboot, so this operation is only for cloudvirt1025 and cloudvirt1026. regards. On 10/16/19 1:10 PM, Arturo Borrero Gonzalez wrote: > Hello! > > Next Wednesday 2019-10-23 at 09:00 UTC we will be doing another maintenance > operation on some of our cloudvirts servers (the hypervisor servers) that > involves rebooting both the physical servers and the virtual machines running > on > them. > The reasons is that we ned to update the running linux kernel version they > have. > > In this window we will reboot 2 hypervisors: > * cloudvirt1025 > * cloudvirt1026 > > The procedure will be to reboot a server, wait for it to come back online > (could > take up to 5 minutes) and wait for all the VMs to come back online. Then move > to > the next server. > > Toolforge users may see their tools and webservices briefly disrupted due to > several components of the Toolforge infrastructure being rebooted in this > operation. > > If nothing changes (reallocated or new virtual machine, etc) this is the list > of > affected VM instances in each hypervisor: > > * cloudvirt1025: > > VM: integration-agent-docker-1006 PROJECT: integration > VM: striker-deploy04 PROJECT: striker > VM: rec-wiki-2 PROJECT: recommendation-api > VM: deployment-ms-fe03 PROJECT: deployment-prep > VM: deployment-poolcounter05 PROJECT: deployment-prep > VM: deployment-ms-be05 PROJECT: deployment-prep > VM: readers-web-stephen PROJECT: reading-web-staging > VM: traffic-upload-stretch PROJECT: traffic > VM: traffic-recdns-anycast PROJECT: traffic > VM: deployment-maps05 PROJECT: deployment-prep > VM: gerrit-sizzle PROJECT: security-tools > VM: tools-sgewebgrid-generic-0901 PROJECT: tools > VM: shinken-puppetmaster-01 PROJECT: shinken > VM: osmit-due PROJECT: osmit > VM: deployment-acme-chief03 PROJECT: deployment-prep > VM: meza-cindy PROJECT: pluggableauth > VM: accounts-db4 PROJECT: account-creation-assistance > VM: krenair-clientpackages-py3-jessie PROJECT: testlabs > VM: deployment-sessionstore01 PROJECT: deployment-prep > VM: paws-worker-04 PROJECT: paws > VM: paws-ext-lb-02 PROJECT: paws > VM: paws-int-lb-01 PROJECT: paws > VM: paws-master-03 PROJECT: paws > VM: paws-master-01 PROJECT: paws > VM: language-readership PROJECT: language > VM: wmde-wikidiff2-patched-stretch PROJECT: wikidiff2-wmde-dev > VM: tools-sgebastion-08 PROJECT: tools > VM: compiler1002 PROJECT: puppet-diffs > VM: phragile-db PROJECT: phragile > VM: cloud-puppetmaster-01 PROJECT: cloudinfra > VM: chicotest-cappy01 PROJECT: chicotestproject > VM: visualeditor-prototype2 PROJECT: visualeditor > VM: programs-and-events-dashboard PROJECT: globaleducation > VM: osmit-uno PROJECT: osmit > VM: tools-sgewebgrid-lighttpd-0904 PROJECT: tools > VM: canary1025-01 PROJECT: testlabs > VM: mathosphere PROJECT: math > VM: social-tools3 PROJECT: social-tools > VM: togetherjs PROJECT: visualeditor > VM: language-mleb-legacy PROJECT: language > VM: women-in-red PROJECT: globaleducation > VM: ntp-01 PROJECT: cloudinfra > VM: mc-clusterA-1 PROJECT: test-twemproxy > VM: wikifarm PROJECT: pluggableauth > VM: login-test PROJECT: catgraph > VM: puppenmeister PROJECT: planet > > * cloudvirt1026: > > VM: integration-agent-docker-1016 PROJECT: integration > VM: wikidata-new-wbterm PROJECT: wikidata-dev > VM: incubator-test PROJECT: incubator > VM: cloudinfra-internal-puppetmaster01 PROJECT: cloudinfra > VM: cloudinfra-db01 PROJECT: cloudinfra > VM: tools-checker-03 PROJECT: tools > VM: tools-static-13 PROJECT: tools > VM: wp1 PROJECT: mwoffliner > VM: pk8s PROJECT: planet > VM: arturo-k8s-test-4-1 PROJECT: openstack > VM: banner PROJECT: wikidumpparse > VM: packager01 PROJECT: packaging > VM: tools-package-builder-02 PROJECT: tools > VM: canary1026-02 PROJECT: testlabs > VM: security-checker1 PROJECT: packagist-mirror > VM: logstack02 PROJECT: security-tools > VM: logstack01 PROJECT: security-tools > VM: mediawiki2latex-large PROJECT: collection-alt-renderer > VM: tools-sge-services-03 PROJECT: tools > VM: tools-sgewebgrid-lighttpd-0928 PROJECT: tools > VM: tools-sgewebgrid-lighttpd-0921 PROJECT: tools > VM: tools-sgewebgrid-generic-0903 PROJECT: tools > VM: tools-sgeexec-0938 PROJECT: tools > VM: tools-sgeexec-0936 PROJECT: tools > VM: tools-sgeexec-0935 PROJECT: tools > VM: tools-sgeexec-0919 PROJECT: tools > VM: tools-sgeexec-0917 PROJECT: tools > VM: tools-sgeexec-0916 PROJECT: tools > VM: tools-sgeexec-0915 PROJECT: tools > VM: tools-sgeexec-0914 PROJECT: tools > VM: tools-paws-worker-1010 PROJECT: tools > VM: tools-paws-worker-1019 PROJECT: tools > VM: openstack-puppetmaster-01 PROJECT: openstack > VM: web1 PROJECT: graphql > VM: etytree-b PROJECT: etytree > VM: canary1026-01 PROJECT: testlabs > VM: db-instance PROJECT: videowiki > VM: tools-sgeexec-0906 PROJECT: tools > VM: mwoffliner5 PROJECT: mwoffliner > > regards > -- Arturo Borrero Gonzalez SRE / Wikimedia Cloud Services Wikimedia Foundation __
