Re: [osuosl-openpower] OSU OpenPOWER Survey

2017-07-28 Thread Lance Albertson
Thank you all who have sent responses to this survey. If you haven't filled out the survey yet, I please encourage you to do so soon. I will start reaching out to projects directly whom haven't responded in a few weeks. Thanks- On Thu, Jun 29, 2017 at 10:28 AM, Lance Albertson wro

[Hosting] OSL Crowdfunding - Student Sponsorship Campaign

2017-07-28 Thread Lance Albertson
them to open source conferences in the coming year. Please contribute and pass the word around to your colleagues and friends if possible! [1] https://create.osufoundation.org/project/6976 -- Lance Albertson Director Oregon State University | Open Source Lab

[Hosting] {ns1, ns2}.osuosl.org RAID firmware upgrade Aug 8, 2017 10:30AM PDT (1730 UTC)

2017-08-09 Thread Lance Albertson
gn [3] has 16 days left! Thanks- [1] https://bugs.centos.org/view.php?id=13184 [2] https://bugs.centos.org/view.php?id=13184#c29499 [3] http://osl.io/crowdfunding -- Lance Albertson Director Oregon State University | Open Source Lab ___ Hosting mailing

[Hosting] Fwd: Planned maintenance: Internet2 circuit migration 2017-08-14 and 2017-08-15

2017-08-09 Thread Lance Albertson
or concerns: email: n...@nero.net phone: 541.346.6376 -- Stephen Fromm Network for Education and Research in Oregon ___ systems mailing list syst...@osuosl.org https://lists.osuosl.org/mailman/listinfo/systems -- Lance Albertson Director Oregon State

Re: [Hosting] {ns1, ns2}.osuosl.org RAID firmware upgrade Aug 10, 2017 10:30AM PDT (1730 UTC)

2017-08-09 Thread Lance Albertson
Oops, I just realized I had the wrong date in the subject. The correct date is August 10th at 10:30am. On Wed, Aug 9, 2017 at 9:45 AM, Lance Albertson wrote: > All, > > I need to perform a RAID firmware upgrade on both ns1 and ns2 tomorrow at > 10:30AM. This upgrade is required t

Re: [Hosting] Planned maintenance: Internet2 circuit migration 2017-08-14 and 2017-08-15

2017-08-09 Thread Lance Albertson
ew minutes.* *Might be a slight blip if someone had an active data transfer that transitions to a different path across the Internet. But I don't suspect you would see any hard outages from this event, and if you did, it would very short (a few seconds)."* On Wed, Aug 9, 2017 at 9:50 AM, La

Re: [Hosting] {ns1, ns2}.osuosl.org RAID firmware upgrade Aug 10, 2017 10:30AM PDT (1730 UTC)

2017-08-10 Thread Lance Albertson
it properly. Thanks- On Wed, Aug 9, 2017 at 10:00 AM, Lance Albertson wrote: > Oops, I just realized I had the wrong date in the subject. The correct > date is August 10th at 10:30am. > > On Wed, Aug 9, 2017 at 9:45 AM, Lance Albertson wrote: > >> All, >> >>

[Hosting] Fwd: Emergency maintenance: OGIG route-servers 2017-08-14 17:00-18:00

2017-08-14 Thread Lance Albertson
...@osuosl.org https://lists.osuosl.org/mailman/listinfo/systems -- Lance Albertson Director Oregon State University | Open Source Lab ___ Hosting mailing list host...@osuosl.org https://lists.osuosl.org/mailman/listinfo/hosting

[Hosting] Fwd: Planned Maintenance: North-bound transit provider, 2018-08-24 00:00-06:00

2017-08-24 Thread Lance Albertson
> > Please contact the NERO NOC if you have questions or concerns: > > email: n...@nero.net > phone: 541.346.6376 > ___ systems mailing list syst...@osuosl.org https://lists.osuosl.org/mailman/listinfo/systems -- Lance Albertson Dir

[Hosting] Fwd: Planned Maintenance: North-bound transit provider, 2017-08-25 00:00-06:00

2017-08-24 Thread Lance Albertson
-- Stephen Fromm Network for Education and Research in Oregon ___ systems mailing list syst...@osuosl.org https://lists.osuosl.org/mailman/listinfo/systems -- Lance Albertson Director Oregon State University | Open Source Lab ___

[Hosting] Fwd: Planned maintenance: Oregon Gigapop to Internet2 LOSA, 2017-08-29 00:00-06:00

2017-08-28 Thread Lance Albertson
ailman/listinfo/systems -- Lance Albertson Director Oregon State University | Open Source Lab ___ Hosting mailing list host...@osuosl.org https://lists.osuosl.org/mailman/listinfo/hosting

[Hosting] Fwd: Emergency maintenance: Seattle Internet Exchange, 2017-08-30 08:00-16:30

2017-08-29 Thread Lance Albertson
suosl.org https://lists.osuosl.org/mailman/listinfo/systems -- Lance Albertson Director Oregon State University | Open Source Lab ___ Hosting mailing list host...@osuosl.org https://lists.osuosl.org/mailman/listinfo/hosting

[Hosting] Planned NERO maintenance notifications moving forward

2017-09-07 Thread Lance Albertson
rg/pages/maintenance/526712bf10cd4e587b14/59b1910a331fbea4050004f9 -- Lance Albertson Director Oregon State University | Open Source Lab ___ Hosting mailing list host...@osuosl.org https://lists.osuosl.org/mailman/listinfo/hosting

[Hosting] Fwd: Unplanned outage: Internet2 Seattle outage, 2017-09-09 20:06-20:11

2017-09-09 Thread Lance Albertson
FYI We had a few alerts on our end during that window so I'm sure some of you may have noticed something as well. Thanks -- Lance Albertson Director Oregon State University | Open Source Lab -- Forwarded message -- From: "Stephen Fromm" Date: Sep 9, 2017

[osuosl-openpower] OpenPOWER OpenStack Software Upgrade - Wed, Oct 4th, 10am-1pm PDT (1700-2000 UTC)

2017-09-29 Thread Lance Albertson
the upgrade happens however that may happen during the outage window. I will send an update once the upgrade is completed. If you have any questions or concerns please let me know. Thanks- -- Lance Albertson Director Oregon State University | Open Source Lab

Re: [osuosl-openpower] OpenPOWER OpenStack Software Upgrade - Wed, Oct 4th, 10am-1pm PDT (1700-2000 UTC)

2017-10-04 Thread Lance Albertson
Just a reminder that this upgrade will be happening in just under an hour and a half. Wish me luck! Thanks- On Fri, Sep 29, 2017 at 2:44 PM, Lance Albertson wrote: > We will be performing a software upgrade on our OpenPOWER OpenStack > cluster upgrading it from the Mitaka release to the

Re: [osuosl-openpower] OpenPOWER OpenStack Software Upgrade - Wed, Oct 4th, 10am-1pm PDT (1700-2000 UTC)

2017-10-04 Thread Lance Albertson
e, please email powerdev-requ...@osuosl.org with what you're experiencing. We'll be working on upgrading to the Ocata in a few months to try and keep up to date with the releases. Thanks! On Wed, Oct 4, 2017 at 8:38 AM, Lance Albertson wrote: > Just a reminder that this upgrade will

Re: [osuosl-openpower] OpenPOWER OpenStack Software Upgrade - Wed, Oct 4th, 10am-1pm PDT (1700-2000 UTC)

2017-10-09 Thread Lance Albertson
On Wed, Oct 4, 2017 at 10:53 AM, Lance Albertson wrote: > The upgrade is complete and everything seems to be running as expected. > One thing to know if you're deploying using the Horizon web panel. The > default setting for selecting the source will result in an error. I'm

[osuosl-openpower] openpower2 reboot

2017-10-13 Thread Lance Albertson
ces aren't back online within the next 10-15 min, please send us an email to powerdev-requ...@osuosl.org Sorry for the issues! Thanks- -- Lance Albertson Director Oregon State University | Open Source Lab ___ openpower mailing list openpo...@o

[osuosl-openpower] OpenStack Heat (Orchestration) service now enabled

2017-10-26 Thread Lance Albertson
anks- ​[1] https://docs.openstack.org/heat/latest/ [2] https://wiki.openstack.org/wiki/Heat -- Lance Albertson Director Oregon State University | Open Source Lab ___ openpower mailing list openpo...@osuosl.org https://lists.osuosl.org/mailman/listinfo/openpower

[Hosting] Ganeti Internal Node Rebuild - Nov 30, 2017 3:00pm PST (Nov 30 2300 UTC)

2017-11-28 Thread Lance Albertson
is rebuild goes and will send another email out for that outage. If you have any questions or concerns please let us know. Projects affected: None specifically -- Lance Albertson Director Oregon State University | Open Source Lab ___ Hosting ma

[Hosting] Ganeti Production Rebuild - Dec 11-15 & 18-19, 2017

2017-12-06 Thread Lance Albertson
teries on these nodes. If you have any questions or concerns please let us know. Projects affected: All hosted VMs on our production Ganeti cluster. ​[1]​ https://goo.gl/QEQsyu -- Lance Albertson Director Oregon State University | Open Source Lab ___

Re: [Hosting] Ganeti Production Rebuild - Dec 11-15 & 18-19, 2017

2017-12-06 Thread Lance Albertson
- 11:00 AM PST (1700 - 1900 UTC) - gprod2: 12/15/2017 9:00AM - 12:00 PM PST (1700 - 2000 UTC) *- gprod1: 12/18/2017 1:00PM - 3:00 PM PST (2100 - 2300 UTC)* *- gprod8: 12/19/2017 9:00AM - 11:00 AM PST (1700 - 1900 UTC)* On Wed, Dec 6, 2017 at 11:36 AM, Lance Albertson wrote: > Service(s) affec

[osuosl-openpower] EMERGENCY MAINTENANCE: openpower2 kernel upgrade and reboot -- 2PM PST TODAY (2200 UTC)

2017-12-08 Thread Lance Albertson
well, we'll schedule additional downtime for the other hosts, but give more lead time. ​I apologize for not giving you more notice but this​ is needed to ensure we can try and work around the current issues we're having on it that's impacting several projects. Thanks- -- Lance Alber

[Hosting] Fwd: Unplanned outage: Corvallis Hub 2017-12-14 04:46-04:50

2017-12-14 Thread Lance Albertson
__ systems mailing list syst...@osuosl.org https://lists.osuosl.org/mailman/listinfo/systems -- Lance Albertson Director Oregon State University | Open Source Lab ___ Hosting mailing list host...@osuosl.org https://lists.osuosl.org/mailman/listinfo/hosting

[osuosl-openpower] Hypervisor reboot (openpower2)

2018-01-16 Thread Lance Albertson
We seem to be having issues with stuck instances on the openpower2 hypervisor. Unfortunately, the only resolution is to reboot the machine. I will be doing that later this morning (likely in the next 30-45min). Sorry for the short notice. Thanks! -- Lance Albertson Director Oregon State

[Hosting] Fwd: Unplanned outage: North-bound transit provider down, 2018-02-22 11:57

2018-02-22 Thread Lance Albertson
email: n...@nero.net phone: 541.346.6376 -- Stephen Fromm Network for Education and Research in Oregon ___ systems mailing list syst...@osuosl.org https://lists.osuosl.org/mailman/listinfo/systems -- Lance Albertson Director Oregon State Univers

[Hosting] Fwd: Unplanned outage: North-bound transit provider down, 2018-02-22 11:57

2018-02-22 Thread Lance Albertson
d Research in Oregon ___ systems mailing list syst...@osuosl.org https://lists.osuosl.org/mailman/listinfo/systems -- Lance Albertson Director Oregon State University | Open Source Lab ___ Hosting mailing lis

[Hosting] Fwd: Planned maintenance: Internet2 and Oregon Gigapop maintenance, 2018-03-02 00:00-01:00

2018-03-02 Thread Lance Albertson
n and Research in Oregon ___ systems mailing list syst...@osuosl.org https://lists.osuosl.org/mailman/listinfo/systems -- Lance Albertson Director Oregon State University | Open Source Lab ___ Hosting mailing list host...@osuosl.org https://

[osuosl-openpower] MAINTENANCE: OpenStack cluster server moves: Mar 8, 9 & 12

2018-03-06 Thread Lance Albertson
migrate them over to the new Ceph cluster. I'll send a separate announcement about that once we're ready for that. If you have any questions or concerns please let me know directly via email or IRC. Thanks! -- Lance Albertson Director Oregon State University | Open Source Lab __

Re: [osuosl-openpower] MAINTENANCE: OpenStack cluster server moves: Mar 8, 9 & 12

2018-03-08 Thread Lance Albertson
2:36 PM, Lance Albertson wrote: > Service(s) affected: > > ​All VMs hosted on the OpenPOWER OpenStack cluster will be offline for > approximately 2-4 hours during each server move. In addition, any VMs which > have block storage attached to the affected nodes will have an outag

Re: [osuosl-openpower] MAINTENANCE: OpenStack cluster server moves: Mar 8, 9 & 12

2018-03-08 Thread Lance Albertson
The move for openpower2 has been completed. Sorry it took a little longer than planned. Please let me know if any VMs are still unreachable. On Thu, Mar 8, 2018 at 11:15 AM, Lance Albertson wrote: > The move for openpower1 has been completed and all VMs should be booting > up or already

Re: [osuosl-openpower] MAINTENANCE: OpenStack cluster server moves: Mar 8, 9 & 12

2018-03-09 Thread Lance Albertson
The move for openpower3 has been completed. Please let me know if any VMs are still unreachable. On Thu, Mar 8, 2018 at 5:25 PM, Lance Albertson wrote: > The move for openpower2 has been completed. Sorry it took a little longer > than planned. Please let me know if any VMs are

Re: [osuosl-openpower] MAINTENANCE: OpenStack cluster server moves: Mar 8, 9 & 12

2018-03-09 Thread Lance Albertson
The move for openpower5 has been completed. Please let me know if any VMs are still unreachable. On Fri, Mar 9, 2018 at 10:03 AM, Lance Albertson wrote: > The move for openpower3 has been completed. Please let me know if any VMs > are still unreachable. > > > On Thu, Mar 8,

Re: [osuosl-openpower] MAINTENANCE: OpenStack cluster server moves: Mar 8, 9 & 12

2018-03-12 Thread Lance Albertson
The move for openpower6 has hit an hardware issue and I'm needing to call IBM support to try and resolve it. I don't have any ETA on when this server will come back online at the moment. On Fri, Mar 9, 2018 at 3:18 PM, Lance Albertson wrote: > The move for openpower5 has been comp

Re: [osuosl-openpower] MAINTENANCE: OpenStack cluster server moves: Mar 8, 9 & 12

2018-03-12 Thread Lance Albertson
It looks like we need to replace the back plane on the server. I'm hoping the part will arrive tomorrow so we can get the system back online then. I'll send another update once I have something new. On Mon, Mar 12, 2018 at 2:52 PM, Lance Albertson wrote: > The move for openpowe

[osuosl-openpower] openpower6 status (was MAINTENANCE: OpenStack cluster server moves: Mar 8, 9 & 12)

2018-03-13 Thread Lance Albertson
fixed. I'll send another update once I have something. Thanks for your patience. On Mon, Mar 12, 2018 at 4:32 PM, Lance Albertson wrote: > It looks like we need to replace the back plane on the server. I'm hoping > the part will arrive tomorrow so we can get the system back onl

Re: [osuosl-openpower] openpower6 status (was MAINTENANCE: OpenStack cluster server moves: Mar 8, 9 & 12)

2018-03-13 Thread Lance Albertson
I got word from the IBM tech that the part should be in tomorrow morning and he should be on-site by mid morning. I'm hoping to have the system back online sometime after lunch or at least by the end of the day tomorrow. Thanks- On Tue, Mar 13, 2018 at 10:18 AM, Lance Albertson wrote: >

Re: [osuosl-openpower] openpower6 status (was MAINTENANCE: OpenStack cluster server moves: Mar 8, 9 & 12)

2018-03-14 Thread Lance Albertson
​VMs should be back online that are on openpower6. The IBM tech was able to replace the back plane and fixed the boot issue we were having. Please let me know if you're still having issues.​ On Tue, Mar 13, 2018 at 2:44 PM, Lance Albertson wrote: > I got word from the IBM tech that

[osuosl-openpower] MAINTENANCE: OpenStack cluster Ceph migration: April 9, 2018 (possibly additional days)

2018-03-29 Thread Lance Albertson
ee to look at this gist [1] I made for myself to keep track of all the commands. If you have any questions or concerns please let me know. Thanks! [1] https://gist.github.com/ramereth/5e11018570f8cd8aa7e707643a4bbf4b -- Lance Albertson Direc

Re: [osuosl-openpower] MAINTENANCE: OpenStack cluster Ceph migration: April 9, 2018 (possibly additional days)

2018-04-10 Thread Lance Albertson
On Thu, Mar 29, 2018 at 4:46 PM, Lance Albertson wrote: > Service(s) affected: > > ​All VMs hosted on the OpenPOWER OpenStack cluster will be offline for > approximately 5 minutes to 2 hours during each VM migration to Ceph. The > outages will only occur when we take a VM down for

[Hosting] ftp.osuosl.org maintenance - May 17, 2018 9am PDT (May 17 1600 UTC)

2018-05-14 Thread Lance Albertson
and all of the data currently on the system will remain intact. We have already taken this server out of our DNS rotation however other projects may be directly pointing to this server which we have no control over. We'll be doing the same reinstall on the other two nodes next week. Tha

[Hosting] Issues with mysql clusters

2018-05-15 Thread Lance Albertson
We just had some issues with several of our mysql clusters related to max connections. It seems to be resolved now however we're taking a closer look into seeing what caused the issue in the first place. Thanks- -- Lance Albertson Director Oregon State University | Open Sourc

Re: [Hosting] ftp.osuosl.org maintenance - May 17, 2018 9am PDT (May 17 1600 UTC)

2018-05-17 Thread Lance Albertson
s- On Mon, May 14, 2018 at 2:59 PM, Lance Albertson wrote: > Service(s) affected: ftp.osuosl.org > > One of the three servers (ftp-osl) which serve ftp.osuosl.org will be > offline during an OS reinstall. This particular machine acts as a syncing > master for the other two servers

[osuosl-openpower] OS issues with openpower3

2018-05-21 Thread Lance Albertson
. Apologies for any issues this may have caused. I think this problem has been resolved now. Thanks- -- Lance Albertson Director Oregon State University | Open Source Lab ___ openpower mailing list openpo...@osuosl.org https://lists.osuosl.org/mailman/listinfo

[osuosl-openpower] Issues with one of the hypervisors (openpower2)

2018-05-24 Thread Lance Albertson
One of the POWER8 hypervisor machines just went down and I'm attempting to bring that machine back online. I'm not sure what caused the original issue but it might be hardware related. It should be starting VMs on it very soon. Thanks- -- Lance Albertson Director Oregon State Univers

[Hosting] ns2.osuosl.org maintenance - June 11, 2018 10am PDT (June 11, 2018 1500 UTC)

2018-06-08 Thread Lance Albertson
l hour. If you want to reduce the effects of this outage, you can add the following to your /etc/resolv.conf (assuming your system is new enough): options rotate timeout:2 Please let me know if you have any questions or concerns. Thanks! -- Lance Albertson Director Oregon State University |

[Hosting] Unplanned outage: Hypervisor issue with gprod1 on primary Ganeti cluster

2018-06-13 Thread Lance Albertson
gprod1). I will look into bringing gprod1 back online later today. Thank you and sorry for the outages this caused. -- Lance Albertson Director Oregon State University | Open Source Lab ___ Hosting mailing list host...@osuosl.org https://lists.osuosl.org

[Hosting] ftp-osl storage upgrade (full rebuild required) - Jun 18, 2018 9:30AM PDT (Jun 18 1630 UTC)

2018-06-14 Thread Lance Albertson
with the local data centers to get them installed. Projects affected: Any project using our FTP cluster as a master syncing point -- Lance Albertson Director Oregon State University | Open Source Lab ___ Hosting mailing list host...@osuosl.org https://lists.osuosl.org/mailman/listinfo/hosting

Re: [Hosting] ftp-osl storage upgrade (full rebuild required) - Jun 18, 2018 9:30AM PDT (Jun 18 1630 UTC)

2018-06-14 Thread Lance Albertson
18 at 10:00 AM, Lance Albertson wrote: > Service(s) affected: ftp.osuosl.org > > During the outage, the master syncing node for our FTP cluster (ftp-osl) > will be offline which means any updates to our software mirrors will be > delayed. > > Outage Window: > Start: Mon,

Re: [Hosting] ftp-osl storage upgrade (full rebuild required) - Jun 18, 2018 9:30AM PDT (Jun 18 1630 UTC)

2018-06-18 Thread Lance Albertson
kely as good as before since it's serving the content via Ceph. I'll send another update tomorrow when I'm ready to switch back over to local storage. Please let me know if you notice any issues. Thanks- On Thu, Jun 14, 2018 at 3:52 PM, Lance Albertson wrote: > I had a few questions

Re: [Hosting] ftp-osl storage upgrade (full rebuild required) - Jun 18, 2018 9:30AM PDT (Jun 18 1630 UTC)

2018-06-19 Thread Lance Albertson
k over. Thanks- On Mon, Jun 18, 2018 at 3:49 PM, Lance Albertson wrote: > I just wanted to send you all an update on where we're at in the process. > > As of right now, ftp-osl is back online and serving it's content from the > the Ceph volume. I've gone ahead

Re: [Hosting] ftp-osl storage upgrade (full rebuild required) - Jun 18, 2018 9:30AM PDT (Jun 18 1630 UTC)

2018-06-22 Thread Lance Albertson
y last for about 10 minutes while the machine reboots. This does not affect anything pointed at ftp.osuosl.org, only ftp-osl (which is out of rotation). Thanks- On Tue, Jun 19, 2018 at 8:57 AM, Lance Albertson wrote: > It's taking longer than I expected to sync the data back to the loca

Re: [Hosting] ftp-osl storage upgrade (full rebuild required) - Jun 18, 2018 9:30AM PDT (Jun 18 1630 UTC)

2018-06-22 Thread Lance Albertson
This has been completed and I've put ftp-osl back into rotation! Thanks for your patience. On Fri, Jun 22, 2018 at 12:44 PM, Lance Albertson wrote: > The sync has been completed and I will be switching this over to the local > drives at 1:30PM PDT (2030 UTC) today. I'm going to

[osuosl-openpower] Unplanned outage (openpower1)

2018-06-26 Thread Lance Albertson
tomorrow but I wanted to let everyone know what happened in case you noticed. Thanks- -- Lance Albertson Director Oregon State University | Open Source Lab ___ openpower mailing list openpo...@osuosl.org https://lists.osuosl.org/mailman/listinfo/openpower

[Hosting] Partial datacenter outage this morning

2018-06-30 Thread Lance Albertson
on IRC at #osuosl. Thanks- -- Lance Albertson Director Oregon State University | Open Source Lab ___ Hosting mailing list host...@osuosl.org https://lists.osuosl.org/mailman/listinfo/hosting

Re: [Hosting] Partial datacenter outage this morning

2018-07-02 Thread Lance Albertson
both units. On Sat, Jun 30, 2018 at 9:21 AM, Lance Albertson wrote: > All, > > It seems as though we had some kind of a power event at approximately > 6:21AM PDT (13:21 UTC). that affected some (but not all) of our hosts. At > this point I'm not entirely sure what happen

Re: [Hosting] Partial datacenter outage this morning

2018-07-02 Thread Lance Albertson
until the generator was able to spin up and provide power. While we do have these units tested on a regular basis there is no way to predict when a battery is going give up the fight. The battery has been replaced and the unit returned to normal operation. On Mon, Jul 2, 2018 at 10:29 AM, Lance Albert

[Hosting] ftp-nyc out of rotation

2018-07-10 Thread Lance Albertson
ed. Please remove ftp-nyc's public IPs (64.50.233.100 & 2600:3404:200:237::2) from any DNS rotation you might be using until we get this resolved. I have already moved it from ftp.osuosl.org and rsync.osuosl.org. Thanks- -- Lance Albertson Director Oregon State Universit

[Hosting] Fwd: Unplanned outage: Corvallis POP, 2018-07-12 08:27-08:33

2018-07-12 Thread Lance Albertson
NERO Network Engineer 541.346.1713 - Work 541.346.6376 - NERO NOC ___ systems mailing list syst...@osuosl.org https://lists.osuosl.org/mailman/listinfo/systems -- Lance Albertson Director Oregon State University | Open Source Lab

[Hosting] Fwd: Unplanned Outage: Internet2 Eugene/Sunnyvale Outage, 2018-07-14 17:57-Ongoing

2018-07-14 Thread Lance Albertson
We noticed a short outage in pingdom at this time so I'm passing this along in case any of you might have noticed. -- Lance Albertson Director Oregon State University | Open Source Lab -- Forwarded message - From: Richard Hicks Date: Sat, Jul 14, 2018, 8:24 PM Subject: Unpl

Re: [osuosl-openpower] https://powerci.osuosl.org/ - flaky , not able to login

2018-07-18 Thread Lance Albertson
t - Telco/Mobile > (720) 395-4767 > > > > ___ > openpower mailing list > openpo...@osuosl.org > https://lists.osuosl.org/mailman/listinfo/openpower > > -- Lance Albertson Director Oregon State University | Open Source La

[Hosting] Fwd: [nero-membership] NERO Planned Upgrades

2018-07-20 Thread Lance Albertson
O NOC , directly. Thank you, José. ___ nero-membership mailing list nero-members...@nero.net https://www.nero.net/mailman/listinfo/nero-membership -- Lance Albertson Director Oregon State University | Open Sourc

[Hosting] Fwd: Planned maintenance: Eugene, Salem, Corvallis Optical Upgrades, multiple nights, 2018-07-31 to 2018-08-09 00:00-06:00

2018-07-26 Thread Lance Albertson
-- Richard Hicks NERO Network Engineer 541.346.1713 - Work 541.346.6376 - NERO NOC ___ systems mailing list syst...@osuosl.org https://lists.osuosl.org/mailman/listinfo/systems -- Lance Albertson Director Oregon State University | Open Source Lab

[Hosting] vip-mysql2 (mysql3) outage

2018-07-30 Thread Lance Albertson
have likely failed. All services should have been restored by around 10:24 AM PDT (1724 UTC). We've been working on getting this cluster into a master-master setup so that this would be prevented in the future. Thanks and sorry for the issues. -- Lance Albertson Director Oregon State Unive

[Hosting] web2.osuosl.org (shared web hosting server) security incident 2018-08-14

2018-08-15 Thread Lance Albertson
ur various services today and hope to have it fully enabled on more servers in the coming weeks. If you have any questions or concerns, please let us know. Thanks- [1] https://twitter.com/umplugin/status/1028911823712530437 -- Lance Albertson Director Oregon State University | Open

[Hosting] Hypervisor reboots on production Ganeti cluster

2018-08-27 Thread Lance Albertson
s. Ideally we'd put the WARN in everything modifying the expiry but most of that is inlines and we don't need the bloat. If you have any questions or concerns please let us know. Thanks! -- Lance Albertson Director Oregon State University | Open Source Lab ___

Re: [Hosting] Hypervisor reboots on production Ganeti cluster

2018-09-06 Thread Lance Albertson
7, 2018 at 1:01 PM Lance Albertson wrote: > All, > > We've been having issues with our Ganeti cluster hypervisors randomly > rebooting. So far we've had this happen four times within the last month > (including this morning) at the following times (in UTC) and duration: &

Re: [Hosting] Hypervisor reboots on production Ganeti cluster

2018-09-06 Thread Lance Albertson
I've deployed a new kernel on all of our production ganeti nodes and the systems seem to be running better now. I'll keep an eye on it for the next few days to be sure everything is still working properly. Thanks- On Thu, Sep 6, 2018 at 8:31 AM Lance Albertson wrote: > Hi all, &g

[Hosting] MAINTENANCE: pg2.osuosl.org server move and outage - Sep 24, 2018 9:00AM PDT (Sep 24 1600 UTC)

2018-09-18 Thread Lance Albertson
vulnerabilities we've encountered this year. We'll have the cables pre-plumbed and ready before the move so I don't expect the downtime to actually last any longer than around 20-30 minutes if all goes well. Please contact me ASAP if this window does not work for you and your project. Tha

Re: [Hosting] MAINTENANCE: pg2.osuosl.org server move and outage - Sep 24, 2018 9:00AM PDT (Sep 24 1600 UTC)

2018-09-24 Thread Lance Albertson
This move has been completed and the postgres service should be back online. Please let me know if you have any issues connecting. Thanks- On Tue, Sep 18, 2018 at 9:45 AM Lance Albertson wrote: > Service(s) affected: Any website or application which uses pg2.osuosl.org > for Post

[Hosting] Resolved Outage: Mailman, RT (support.osuosl.org), OSL email delivery

2018-10-05 Thread Lance Albertson
causing the machine to take a long time to boot up. I was able to bring up the VM at approximately 1pm PDT and mail delivery resumed properly. Apologies for any issues this may have caused. Thanks- -- Lance Albertson Director Oregon State University | Open Source Lab

[osuosl-openpower] POWER9 availability

2018-12-05 Thread Lance Albertson
Please let me know if you have any questions! [1] http://osuosl.org/services/powerdev/request_hosting/ -- Lance Albertson Director Oregon State University | Open Source Lab ___ openpower mailing list openpo...@osuosl.org https://lists.osuosl.org/mailman/listinfo/openpower

[osuosl-openpower] OpenStack Ocata Upgrade - Jan 2, 2019 8:30am-10:30am PST (1630-1830 UTC)

2018-12-18 Thread Lance Albertson
cerns, please let me know. -- Lance Albertson Director Oregon State University | Open Source Lab ___ openpower mailing list openpo...@osuosl.org https://lists.osuosl.org/mailman/listinfo/openpower

Re: [osuosl-openpower] OpenStack Ocata Upgrade - Jan 2, 2019 8:30am-10:30am PST (1630-1830 UTC)

2019-01-02 Thread Lance Albertson
All, Just a reminder that I will be starting this upgrade in just under 30 minutes. I will send an email once everything is back online. On Tue, Dec 18, 2018 at 11:22 AM Lance Albertson wrote: > > Users of the ppc64 OpenStack cluster, > > We're going to be performing an upgra

Re: [osuosl-openpower] OpenStack Ocata Upgrade - Jan 2, 2019 8:30am-10:30am PST (1630-1830 UTC)

2019-01-02 Thread Lance Albertson
The upgrade is finished and all services appear to be working normally now. Please send an email powerdev-requ...@osuosl.org if you have any issues related to this upgrade. Thanks- On Wed, Jan 2, 2019 at 8:03 AM Lance Albertson wrote: > > All, > > Just a reminder that I will be s

[Hosting] OSUOSL attending FOSDEM / Config Mgt Camp

2019-01-08 Thread Lance Albertson
won't have a booth but we could set up an informal meet up or individual meetups if you'd like. I arrive on Feb 1 and leave on Feb 7 and will be staying in Brussels the whole time. Please email me directly (la...@osuosl.org) or ping me on IRC if you'd like to schedule anything. Tha

Re: [osuosl-openpower] POWER9 availability

2019-01-16 Thread Lance Albertson
access. [1] http://osuosl.org/request-hosting/ -- Lance Albertson Director Oregon State University | Open Source Lab ___ openpower mailing list openpo...@osuosl.org https://lists.osuosl.org/mailman/listinfo/openpower

[Hosting] Fwd: Unplanned Outage: Internet2 Eugene/Sunnyvale Outage, 2019-01-16 13:56-13:57

2019-01-16 Thread Lance Albertson
O NOC ___ systems mailing list syst...@osuosl.org https://lists.osuosl.org/mailman/listinfo/systems -- Lance Albertson Director Oregon State University | Open Source Lab ___ Hosting mailing list host...@osuosl.org https://lists.osuosl.org/mailman/lis

[Hosting] Fwd: [Kerr_b210-announce] B210 Data Center Maintenance

2019-01-17 Thread Lance Albertson
ment on the UPS units. The UPS units will be in bypass during the maintenance. They will work on one unit at a time so both units will not be off-line at any one time. Maintenance is expected to take about 2-3 hours. Regards, Steve Fowler -- Lance Albertson Director Oregon State University |

[Hosting] Fwd: Unplanned outage: Bend Hub, 2019-02-07 07:26-08:01

2019-02-07 Thread Lance Albertson
tephen Fromm Network for Education and Research in Oregon ___ systems mailing list syst...@osuosl.org https://lists.osuosl.org/mailman/listinfo/systems -- Lance Albertson Director Oregon State University | Ope

[Hosting] Fwd: Unplanned outage: DDoS against Corvallis hub customer, 2019-02-14 06:21-06:25

2019-02-14 Thread Lance Albertson
O NOC ___ systems mailing list syst...@osuosl.org https://lists.osuosl.org/mailman/listinfo/systems -- Lance Albertson Director Oregon State University | Open Source Lab ___ Hosting mailing list host...@osuosl.org https://lists.osuos

[Hosting] Planned power grid work between March 28-30, 2019 impacting OSL data center

2019-03-26 Thread Lance Albertson
gt; on March 16, 2019 to enable the hook-up of generators, allowing for > business continuity in most buildings. If you have any additional questions or concerns, please contact me directly either via email or IRC. Thank you! [1] https://powergrid.oregonstate.edu/ -- Lance Albertson Director

[osuosl-openpower] ACTION REQUIRED: 2019 OSL OpenPOWER Cluster Survey

2019-03-29 Thread Lance Albertson
Team [1] https://forms.gle/ni22gW3y2iuT55vb9 -- Lance Albertson Director Oregon State University | Open Source Lab ___ openpower mailing list openpo...@osuosl.org https://lists.osuosl.org/mailman/listinfo/openpower

[osuosl-openpower] Planned outage for POWER9 VMs - Wednesday April 3, 2019 9:00AM PDT (1600 UTC)

2019-04-02 Thread Lance Albertson
n is to issue a "migrate" command for each POWER9 VM running at 9AM PDT (1600 UTC) which will gracefully shutdown the VM and power it up on another server. I expect the outage should last no more than a few minutes per VM. If this is a problem for your project, please let me know ASAP. Tha

Re: [osuosl-openpower] Planned outage for POWER9 VMs - Wednesday April 3, 2019 9:00AM PDT (1600 UTC)

2019-04-03 Thread Lance Albertson
working properly initially and thus resulted in a failed VM startup on the other node. Everything should be back to normal now. Thanks- On Tue, Apr 2, 2019 at 4:33 PM Lance Albertson wrote: > All, > > I need to have a short outage for all POWER9 virtual machines running on > our c

[osuosl-openpower] OpenStack Pike Upgrade - April 22, 2019 9am-11am PDT (1600-1800 UTC)

2019-04-15 Thread Lance Albertson
cerns, please let me know. -- Lance Albertson Director Oregon State University | Open Source Lab ___ openpower mailing list openpo...@osuosl.org https://lists.osuosl.org/mailman/listinfo/openpower

Re: [osuosl-openpower] OpenStack Pike Upgrade - April 22, 2019 9am-11am PDT (1600-1800 UTC)

2019-04-22 Thread Lance Albertson
! On Mon, Apr 15, 2019 at 11:35 AM Lance Albertson wrote: > Users of the ppc64le OpenStack cluster, > > We're going to be performing an upgrade of the cluster from Ocata to Pike > on April 22, 2019 at 9am-11am PST (1600-1800 UTC). Any virtual machines > that are running at the

Re: [osuosl-openpower] ACTION REQUIRED: 2019 OSL OpenPOWER Cluster Survey

2019-04-22 Thread Lance Albertson
Hi all, I wanted to resend this email again as a reminder. As of today, we only have almost 1/3 of projects having completed this survey. Thanks- On Fri, Mar 29, 2019 at 9:19 AM Lance Albertson wrote: > Hello OSU OpenPOWER development cluster users, > > In order to extend your usa

[osuosl-openpower] Projects not completing Survey

2019-05-01 Thread Lance Albertson
coder Zarafa -- Lance Albertson Director Oregon State University | Open Source Lab ___ openpower mailing list openpo...@osuosl.org https://lists.osuosl.org/mailman/listinfo/openpower

Re: [osuosl-openpower] Projects not completing Survey

2019-05-02 Thread Lance Albertson
d, May 1, 2019 at 4:12 PM Lance Albertson wrote: > Below is a list of projects that have not completed their survey according > to our records. I will start powering down virtual machines on May 6th if > we do not have any response from your project. If you think your project > being l

Re: [osuosl-openpower] Projects not completing Survey

2019-05-06 Thread Lance Albertson
May 1, 2019 at 4:12 PM Lance Albertson wrote: > Below is a list of projects that have not completed their survey according > to our records. I will start powering down virtual machines on May 6th if > we do not have any response from your project. If you think your project > being lis

[osuosl-openpower] Service disruption on Ceph storage cluster

2019-05-13 Thread Lance Albertson
ppen again. Sorry for any issues this may have caused. I'll send any further updates as they are needed. Thanks- -- Lance Albertson Director Oregon State University | Open Source Lab ___ openpower mailing list openpo...@osuosl.org https://lists.osu

[osuosl-openpower] Ongoing VM network connectivity issues since Pike upgrade

2019-05-16 Thread Lance Albertson
em. Thanks for your patience! -- Lance Albertson Director Oregon State University | Open Source Lab ___ openpower mailing list openpo...@osuosl.org https://lists.osuosl.org/mailman/listinfo/openpower

Re: [osuosl-openpower] Ongoing VM network connectivity issues since Pike upgrade

2019-05-21 Thread Lance Albertson
end another update as I have one. Thanks- On Thu, May 16, 2019 at 8:58 AM Lance Albertson wrote: > All, > > Since the upgrade to Pike we've noticed virtual machines suddenly losing > network connectivity. This issue seems to sometimes fix itself or when we > restart the ne

[Hosting] Fwd: Unplanned outage: Internet2 Sunnyvale node, 2019-05-23 10:31

2019-05-23 Thread Lance Albertson
gards, -- Stephen Fromm Network for Education and Research in Oregon ___ systems mailing list syst...@osuosl.org https://lists.osuosl.org/mailman/listinfo/systems -- Lance Albertson Director Oregon State University | Open Sourc

[Hosting] Fwd: Unplanned outage: Internet2 Sunnyvale node, 2019-05-23 10:31

2019-05-23 Thread Lance Albertson
k for Education and Research in Oregon ___ systems mailing list syst...@osuosl.org https://lists.osuosl.org/mailman/listinfo/systems -- Lance Albertson Director Oregon State University | Open Source Lab ___ Hosting mailing list host...@osuosl.org htt

Re: [osuosl-openpower] Ongoing VM network connectivity issues since Pike upgrade

2019-05-29 Thread Lance Albertson
'll continue looking into this this week. Thanks for your patience. On Tue, May 21, 2019 at 12:16 PM Lance Albertson wrote: > All, > > I wanted to send you an update on where we are at on this issue. So far > I've narrowed down the problem to happening when a VM using a pri

Re: [osuosl-openpower] Ongoing VM network connectivity issues since Pike upgrade

2019-06-07 Thread Lance Albertson
grade. Thanks- On Tue, May 21, 2019 at 12:16 PM Lance Albertson wrote: > All, > > I wanted to send you an update on where we are at on this issue. So far > I've narrowed down the problem to happening when a VM using a private > network is removed causing certain iptable rules

[osuosl-openpower] OpenStack Queens Upgrade - June 12, 2019 1-3pm PDT (2000-2200 UTC)

2019-06-07 Thread Lance Albertson
e let me know. -- Lance Albertson Director Oregon State University | Open Source Lab ___ openpower mailing list openpo...@osuosl.org https://lists.osuosl.org/mailman/listinfo/openpower

  1   2   3   >