Re: [Openstack-operators] large high-performance ephemeral storage

, and some sort of r/w mix (where write <=50% > to acknowledge this is ephemeral storage so hopefully something is using it > soon after storing). Thoughts? > > Cheers, > Blair > > On Thu., 14 Jun. 2018, 00:24 Joe Topjian, wrote: > >> Yes, you can! The kernel docume

Re: [Openstack-operators] large high-performance ephemeral storage

Yes, you can! The kernel documentation for read/write limits actually uses /dev/null in the examples :) But more seriously: while we have not architected specifically for high performance, for the past few years, we have used a zpool of cheap spindle disks and 1-2 SSD disks for caching. We have ZF

Re: [Openstack-operators] [Openstack] Certifying SDKs

Hi all, I've been meaning to reply to this thread. Volodymyr, your reply reminded me :) I agree with what you said that the SDK should support everything that the API supports. In that way, one could simply review the API reference docs and create a checklist for each possible action. I've often

Re: [Openstack-operators] mitaka/xenial libvirt issues

We think we've pinned the qemu errors down to a mismatched group ID on a handful of compute nodes. The slow systemd/libvirt is still unsolved, but at the moment that does not actually be the cause of the qemu errors. On Mon, Nov 27, 2017 at 8:04 AM, Joe Topjian wrote: > Hi all, &

Re: [Openstack-operators] mitaka/xenial libvirt issues

t; https://bugs.launchpad.net/ubuntu/+source/qemu/+bug/1647389 > > Thanks > > On Thu, Nov 23, 2017 at 6:20 PM, Joe Topjian wrote: > >> OK, thanks. We'll definitely look at downgrading in a test environment. >> >> To add some further info to this problem, here

Re: [Openstack-operators] mitaka/xenial libvirt issues

11:03 AM, Chris Sarginson wrote: > I think we may have pinned libvirt-bin as well, (1.3.1), but I can't > guarantee that, sorry - I would suggest its worth trying pinning both > initially. > > Chris > > On Thu, 23 Nov 2017 at 17:42 Joe Topjian wrote: > >> Hi Chris, >

Re: [Openstack-operators] mitaka/xenial libvirt issues

MU and related packages at the following version (you might > need to hunt down the debs manually): > > 1:2.5+dfsg-5ubuntu10.5 > > I'm certain there's a launchpad bug for Ubuntu qemu regarding this, but > don't have it to hand. > > Hope this helps, > Chris >

[Openstack-operators] mitaka/xenial libvirt issues

Hi all, We're seeing some strange libvirt issues in an Ubuntu 16.04 environment. It's running Mitaka, but I don't think this is a problem with OpenStack itself. We're in the process of upgrading this environment from Ubuntu 14.04 with the Mitaka cloud archive to 16.04. Instances are being live mi

Re: [Openstack-operators] Successful nova-network to Neutron Migration

n the migration, > like us :) > > thanks, > Belmiro > > On Sat, May 20, 2017 at 6:51 PM, Joe Topjian wrote: > >> Hi all, >> >> There probably aren't a lot of people in this situation nowadays, but for >> those that are, I wanted to report a succe

[Openstack-operators] Successful nova-network to Neutron Migration

Hi all, There probably aren't a lot of people in this situation nowadays, but for those that are, I wanted to report a successful nova-network to Neutron migration. We used NeCTAR's migration scripts which can be found here: https://github.com/NeCTAR-RC/novanet2neutron These scripts allowed us

Re: [Openstack-operators] [scientific] Resource reservation requirements (Blazar) - Forum session

On Mon, Apr 3, 2017 at 8:20 AM, Jay Pipes wrote: > On 04/01/2017 08:32 PM, Joe Topjian wrote: > >> On Sat, Apr 1, 2017 at 5:21 PM, Matt Riedemann > <mailto:mriede...@gmail.com>> wrote: >> >> On 4/1/2017 8:36 AM, Blair Bethwaite wrote: >> >

Re: [Openstack-operators] [scientific] Resource reservation requirements (Blazar) - Forum session

On Sat, Apr 1, 2017 at 5:21 PM, Matt Riedemann wrote: > On 4/1/2017 8:36 AM, Blair Bethwaite wrote: > >> Hi all, >> >> The below was suggested for a Forum session but we don't yet have a >> submission or name to chair/moderate. I, for one, would certainly be >> interested in providing input. Do w

Re: [Openstack-operators] backup to object store - tool recommendations

We use rclone quite a bit. It works great and has a wealth of features: http://rclone.org/ On Mon, Mar 27, 2017 at 7:50 AM, Nick Jones wrote: > On 27 Mar 2017, at 12:59, Marcus Furlong wrote: > > > On 27 March 2017 at 22:39, Blair Bethwaite > wrote: > > Hi all, > > Does anyone have any recom

Re: [Openstack-operators] [neutron] Modify Default Quotas

We run a similar kind of script. I think in most cases, a Floating IP means a publicly routable IP, and those are now scarce resources. Because of that, I agree with what's been mentioned about a conservative floating IP quota. Since the other resource types aren't restricted by external availabi

Re: [Openstack-operators] Flavors

e bundled. :) On Wed, Mar 15, 2017 at 10:31 PM, Joe Topjian wrote: > Another benefit of flavors is that they provide ease of use. While there > are users who are confident enough to spec out each instance they launch, I > work with a lot of users who would feel overwhelmed if they h

Re: [Openstack-operators] Flavors

Another benefit of flavors is that they provide ease of use. While there are users who are confident enough to spec out each instance they launch, I work with a lot of users who would feel overwhelmed if they had to do this. Providing a set of recommended instance specs can go a long way to lowerin

Re: [Openstack-operators] libvirt freezing when loading Nova instance nwfilters

We ran into the "virsh nwfilter-list hanging indefinitely" thing back in early January. I spent hours and I almost went insane trying to figure it out. We weren't upgrading nodes, though, it just sort of happened. I have no idea if the following was the correct way of handling this, but this ultim

Re: [Openstack-operators] Encrypted Cinder Volume Deployment

archives and has an update, do post :) Thanks, Joe On Mon, Jan 23, 2017 at 8:58 PM, Joe Topjian wrote: > Hi Kris, > > I came across that as well and I believe it has been fixed and ensures > existing volumes are accessible: > > https://github.com/openstack/nova/blob/8c3f7

Re: [Openstack-operators] Encrypted Cinder Volume Deployment

the badly hashed key/password vs’s > what was configured. > > > > > > ___ > > Kris Lindgren > > Senior Linux Systems Engineer > > GoDaddy > > > > *From: *Joe Topjian > *Date: *Monday, Ja

[Openstack-operators] Encrypted Cinder Volume Deployment

Hi all, I'm investigating the options for configuring Cinder with encrypted volumes and have a few questions. The Cinder environment is currently running Kilo which will be upgraded to something between M-O later this year. The Kilo release supports the fixed_key setting. I see fixed_key is still

Re: [Openstack-operators] VM monitoring suggestions

We have some custom scripts that run on the hypervisors which poll: virsh dominfo virsh domiflist etc The memory stats with "virsh dommemstat" are, AFAIK, not accurate since there's nothing triggering kvm / the vm to release unused memory. But all other virsh stuff works well for us. We don't re

Re: [Openstack-operators] [nova] Does anyone use the os-diagnostics API?

Hi Matt, Tim, Thanks for asking. We’ve used the API in the past as a way of getting the > usage data out of Nova. We had problems running ceilometer at scale and > this was a way of retrieving the data for our accounting reports. We > created a special policy configuration to allow authorised user

Re: [Openstack-operators] Murano in Production

Hi Serg, We were indeed hitting that bug, but the cert wasn't self-signed. It was easier for us to manually patch the Ubuntu Cloud package of Murano with the stable/mitaka fix linked in that bug report than trying to debug where OpenSSL/python/requests/etc was going awry. We might redeploy Murano

Re: [Openstack-operators] Murano in Production

Hi Serg, Thank you for sharing this information :) If I'm understanding correctly, the main reason you're using a non-clustered / corosync setup is because that's how most other components in Mirantis OpenStack are configured? Is there anything to be aware of in how Murano communicates over the a

Re: [Openstack-operators] Murano in Production

m Morrison wrote: > You could also use https://www.rabbitmq.com/maxlength.html to mitigate > overflowing on the trove vhost side. > > > Sam > > > On 19 Sep 2016, at 1:07 PM, Joe Topjian wrote: > > Thanks for everyone's input. I think I'm going to go wit

Re: [Openstack-operators] Murano in Production

burns on us, I'll be more than happy to report failure. :) On Sun, Sep 18, 2016 at 7:38 PM, Silence Dogood wrote: > I'd love to see your results on this . Very interesting stuff. > > On Sep 17, 2016 1:37 AM, "Joe Topjian" wrote: > >> Hi all, >> >

[Openstack-operators] Murano in Production

Hi all, We're planning to deploy Murano to one of our OpenStack clouds and I'm debating the RabbitMQ setup. For background: the Murano agent that runs on instances requires access to RabbitMQ. Murano is able to be configured with two RabbitMQ services: one for traditional OpenStack communication

Re: [Openstack-operators] cURL call at the end of provisioning and deprovisioning

Hi Will, What about notification events? I'm not sure what the best source of documentation is for events, but googling "OpenStack Notification Events" yields a bunch of information that should lead you in the right direction. With events, you would write a custom scripts / daemon that polls rabb

Re: [Openstack-operators] PCI Passthrough issues

Hi Blair, We only updated qemu. We're running the version of libvirt from the Kilo cloudarchive. We've been in production with our K80s for around two weeks now and have had several users report success. Thanks, Joe On Tue, Jul 19, 2016 at 5:06 PM, Blair Bethwaite wrote: > Hilariously (or not

Re: [Openstack-operators] How to create floating ip pool use nova network? thanks

In Kilo (I haven't verified Liberty or Mitaka), you can manage nova-network floating IP pools with: nova-manage floating --help nova-manage floating create --help Hope that helps! Joe On Wed, Jul 6, 2016 at 8:23 PM, 云淡风轻 <821696...@qq.com> wrote: > hi everyone, > > How to create floating ip po

Re: [Openstack-operators] PCI Passthrough issues

Hi Jon, We were also running into issues with the K80s. For our GPU nodes, we've gone with a 4.2 or 4.4 kernel. PCI Passthrough works much better in those releases. (I ran into odd issues with 4.4 and NFS, downgraded to 4.2 after a few hours of banging my head, problems went away, not a scientifi

Re: [Openstack-operators] How are folks providing GPU instance types?

Just wanted to add a few notes (I apologize for the brevity): * The wiki page is indeed the best source of information to get started. * I found that I didn't have to use EFI-based images. I wonder why that is? * PCI devices and IDs can be found by running the following on a compute node: $ lspci

Re: [Openstack-operators] RAID / stripe block storage volumes

re-raid-on-a-linux-general-purpose-cloud-server/ > 2: https://support.rackspace.com/how-to/cloud-block-storage-faq/ > 3: > https://azure.microsoft.com/en-us/documentation/articles/virtual-machines-linux-configure-raid/ > 4: https://wiki.openstack.org/wiki/LibvirtVirtioScsi > >

Re: [Openstack-operators] RAID / stripe block storage volumes

rticles/virtual-machines-linux-configure-raid/ 4: https://wiki.openstack.org/wiki/LibvirtVirtioScsi On Mon, Feb 8, 2016 at 7:18 PM, Joe Topjian wrote: > Yep. Don't get me wrong -- I agree 100% with everything you've said > throughout this thread. Applications that have native replication a

Re: [Openstack-operators] [kolla] Question about how Operators deploy

2 VIPs as well. On Fri, Feb 12, 2016 at 8:27 AM, Matt Fischer wrote: > We also use 2 VIPs. public and internal, with admin being a CNAME for > internal. > > On Fri, Feb 12, 2016 at 7:28 AM, Fox, Kevin M wrote: > >> We usually use two vips. >> >> Thanks, >> Kevin >> >> --

Re: [Openstack-operators] User Survey - Deadline Feb 24th

ot Enterprise version of OpenContrail. > Therefore, this question could split the responses because it is confusing. > > Edgar > > From: Joe Topjian > Date: Tuesday, February 9, 2016 at 8:17 AM > To: Edgar Magana > Cc: Tom Fifield , "openst...@lists.openstack.org"

Re: [Openstack-operators] User Survey - Deadline Feb 24th

Isn't it similar to asking if you use Fedora, CentOS, or RHEL? My understanding is that Juniper offers a paid/supported version of Contrail while OpenContrail is the open source version. On Tue, Feb 9, 2016 at 2:50 AM, Edgar Magana wrote: > Tom, > > For the "Which OpenStack Network (Neutron) dr

Re: [Openstack-operators] RAID / stripe block storage volumes

gt;> I have not run into anyone replicating volumes or creating redundancy at >> the VM level (beyond, as you point out, HDFS, etc.). >> >> R >> >> On Mon, Feb 8, 2016 at 6:54 PM, Joe Topjian wrote: >> >>> This is a great conversation and I really a

Re: [Openstack-operators] RAID / stripe block storage volumes

te environments. >>> >>> We've been discussing how possible it would be to translate that same >>> idea to block storage. Rather than have a large HA storage cluster (whether >>> Ceph, Gluster, NetApp, etc), is it possible to offer simple single LVM >&g

Re: [Openstack-operators] RAID / stripe block storage volumes

> I've always recommended providing multiple underlying storage services to > provide this rather than adding the overhead to the VM. So, not in any of > my systems or any I've worked with. > > R > > > > On Fri, Feb 5, 2016 at 5:56 PM, Joe Topjian wrote: > >>

[Openstack-operators] RAID / stripe block storage volumes

Hello, Does anyone have users RAID'ing or striping multiple block storage volumes from within an instance? If so, what was the experience? Good, bad, possible but with caveats? Thanks, Joe ___ OpenStack-operators mailing list OpenStack-operators@lists.

Re: [Openstack-operators] Storage backend for glance

Yup, it's definitely possible. All Glance nodes will need to share the same database as well as the same file system. Common ways of sharing the file system are to mount /var/lib/glance/images either from NFS (like you mentioned) or Gluster. I've done both in the past with no issues. The usual cav

Re: [Openstack-operators] Cinder API with multiple regions not working.

61686 | regionOne | glance | image > | > > +--+---+--+--+ > > > > But different names are working perfectly fine with glance and nova. > > Anyways thanks for all your time and efforts. If I manage to rep

Re: [Openstack-operators] Cinder API with multiple regions not working.

t; —— cinder.conf > [default] > os_region_name = regionOne > —— > > Regards.. > Salman. > > > > > > PhD, Scientific Computing > Researcher, IT Department, > Uppsala University. > Senior Cloud Architect, > SNIC. > Cloud Application Expert, > U

Re: [Openstack-operators] Galera setup testing

We do something similar: Instead of McRouter, we use the repcached patches to replicate data between two memcached nodes. We then use HAProxy as a single entry point for memcached requests. We've been doing this for 6+ months and it's been working great. It's effectively solved the issue I describ

Re: [Openstack-operators] Cinder API with multiple regions not working.

| > > > +--+--+ > —— > > Again if I remove hpc2n everything started to work fine. Can you tell me > what service I suppose to restart (if required)? > > Regards.. > Salman > > > > PhD, Sc

Re: [Openstack-operators] Cinder API with multiple regions not working.

Hi Salman, Someone mentioned this same issue yesterday in relation to Terraform (maybe a colleague of yours?), so given the two occurrences, I thought I'd look into this. I have a Liberty environment readily available, so I created a second set of volume and volumev2 endpoints for a fictional reg

Re: [Openstack-operators] Hypervisor Tuning Guide

this guide fall under a more formal schedule and would like to lead that effort, please get in contact with me. Thanks, Joe 1: https://wiki.openstack.org/wiki/Documentation/HypervisorTuningGuide 2: https://wiki.openstack.org/wiki/Documentation/HypervisorTuningGuide#How_to_Contribute On Tue, Oct 27,

[Openstack-operators] Horizon Kilo bug with nova-network and floating IPs

Hi all, I recently came across this bug and thought I'd share it for anyone else running a similar environment: https://bugs.launchpad.net/horizon/+bug/1520071 Joe ___ OpenStack-operators mailing list OpenStack-operators@lists.openstack.org http://list

Re: [Openstack-operators] instances floating IPs not reachable while restarting nova-network

Yup, this is expected. It happens for both single-host and multi-host. With the former, we have an older environment where it takes around 10 minutes for all network access to resume. That's with a few hundred tenants, a few hundred vlans, and a few hundred floating IPs all on one host, though. >F

Re: [Openstack-operators] Router associated with multiple l3 agents

Hi Matt, > It's also weird that we've only seen this when the environment has been > built using terraform. This particular customer re-creates the issue every > time they rebuild. > I work on the OpenStack support for Terraform, so I might be able to help with this. Could you provide an example T

Re: [Openstack-operators] [Nova] Question about starting nova as service versus directly

> Yes, most likely is related to permissions. Another good source of > information for troubleshooting is /var/log/upstart/nova-compute.log > Ah yes! Much easier. ___ OpenStack-operators mailing list OpenStack-operators@lists.openstack.org http://lists.o

Re: [Openstack-operators] [openstack-operators][osops] tools-contrib is open for business!

scher" wrote: > >> Is there a reason why we can't license the entire repo with Apache2 and >> if you want to contribute you agree to that? Otherwise it might become a >> bit of a nightmare. Or maybe at least do "Apache2 unless otherwise stated"? >&g

Re: [Openstack-operators] [openstack-operators][osops] tools-contrib is open for business!

Thanks, JJ! It looks like David Wahlstrom submitted a script and there's a question about license. https://review.openstack.org/#/c/247823/ Though contributions to contrib do not have to follow a certain coding style, can be very lax on error handling, etc, should they at least mention a license

Re: [Openstack-operators] [Nova] Question about starting nova as service versus directly

Hi Adam, I've seen this happen due to permission issues. Regardless of running with sudo, upstart is dropping to the "nova" user. I usually debug this by setting a shell on the nova user, sudoing/su'ing to nova, then running nova-compute from there. It should die with an error message of the caus

Re: [Openstack-operators] OPs Midcycle location discussion.

+1 Option 1 On Mon, Nov 16, 2015 at 10:01 AM, Jonathan Proulx wrote: > > Let me restate the question a bit as I think I'm hearing two different > responses that may be getting conflated. > > Option 1: There's a single Ops Midcycle that shifts around and we > look at ways to increase remote part

Re: [Openstack-operators] OpenStack Tuning Guide

Hi Kevin, Oops, noticed I didn't reply to all the first time. I think it's great to see more people who want to collect and distil knowledge like this. :) Finally, I hate diverging resources, so if something like this already > exists please speak up so we can focus our efforts on making sure th

Re: [Openstack-operators] Informal Ops Meetup?

We're currently in the Prince room by the projector. On Fri, Oct 30, 2015 at 10:07 AM, Edgar Magana wrote: > Where are you meeting? > > Edgar > > From: "Kris G. Lindgren" > Date: Thursday, October 29, 2015 at 6:37 AM > To: Sam Morrison , " > openstack-operators@lists.openstack.org" < > openstac

Re: [Openstack-operators] Informal Ops Meetup?

Hi Kris, I'll be around and am interested. On Thu, Oct 29, 2015 at 4:39 PM, Kris G. Lindgren wrote: > Hello all, > > I am not sure if you guys have looked at the schedule for Friday… but its > all working groups. I was talking with a few other operators and the idea > came up around doing an i

[Openstack-operators] Hypervisor Tuning Guide

We had a great Hypervisor Tuning Guide session yesterday! We agreed on an initial structure to the guide that will include four core sections (CPU, Memory, Network, and Disk) and common subsections to each. The etherpad[1] has this structure defined and during the session, we went through and adde

Re: [Openstack-operators] [openstack-operators][osops] Something other than NOOP in our jenkins tests

people could share what they had. And a curated one, > where everything within the curated repo follows a standard set of > conventions/guidelines. > > _______ > Kris Lindgren > Senior Linux Systems Engineer > Go

Re: [Openstack-operators] [openstack-operators][osops] Something other than NOOP in our jenkins tests

So this will require bash scripts to adhere to bashate before being accepted? Is it possible to have the check as non-voting? Does this open the door to having other file types be checked? IMHO, it's more important for the OSOps project to foster collaboration and contributions rather than worry a

Re: [Openstack-operators] Tokyo Summit Ops Design Summit Tracks - Draft Agenda

Thanks, guys! On Wed, Sep 23, 2015 at 1:33 AM, Tom Fifield wrote: > Moved it to a day earlier. > > > Regards, > > > Tom > > On 23/09/15 14:25, Tim Bell wrote: > >> I’d like Joe there too JCan we re-schedule that one ? >> >> Tim >> >>

Re: [Openstack-operators] Tokyo Summit Ops Design Summit Tracks - Draft Agenda

Hi Tom, As luck would have it, the sole session I'm doing in the Main Conference clashes with the current time slot for Hypervisor Tuning. I see that Tim is also listed as a moderator for that session, so I'm more than happy to let him run it. However, if it's not asking much, I'd really like to b

Re: [Openstack-operators] [openstack-operators] Announcement! We have everything ready to get to Stackforge!

Hi JJ, Thank you for putting all of this together! All, one decision that was made during the PAO Ops Meetup was to make the OSOps github repos "official" repos / projects. If you've contributed to the existing repos, you may have an interest in this. As well, if you p

Re: [Openstack-operators] Draft Agenda for PAO Ops Meetup (August 18, 19)

Hi Tom, On Thu, Aug 13, 2015 at 2:08 AM, Tom Fifield wrote: > Hi all, > > We're still lacking moderators for the following sessions - any takers? > > 1. Hypervisor Tuning - General Session (have a backup, but would like a > primary) > I'm happy to be the primary on this -- not a problem at all

Re: [Openstack-operators] Compressed Images

I can imagine that '.zip and .tar.gz' might have applied to raw, which can > be quite large, but ultimately, you're probably right and found text that > needs updating. > > > On Jul 29, 2015, at 12:38 PM, Joe Topjian wrote: > > > > Hello, > > > > In

[Openstack-operators] Compressed Images

Hello, In the "Create An Image" page of Horizon, it says the following: Currently only images available via an HTTP URL are supported. The image location must be accessible to the Image Service. Compressed image binaries are supported (.zip and .tar.gz.) Either I have something misconfigured, th

Re: [Openstack-operators] Palo Alto Midcycle - agenda brainstorming

Hi Tom, The list of General Session ideas is definitely shorter than past meetups, but maybe that's a good sign! It could be that past burning topics have been acknowledged and handled. If that's the case, does anyone have thoughts about extending the length of Working Group sessions so there's m

Re: [Openstack-operators] FAiled to create instance wiht openstack nova network

Hello, According to nova.conf, you're running nova-network in multi-host mode. Just to be verbose: if your OpenStack installation is an all-in-one or if you intend for all network traffic to go through the cloud controller, this setting should be changed to false. The error message is reporting:

Re: [Openstack-operators] Scaling the Ops Meetup

> -1 on paid registration, I think we need to be mindful of the smaller > openstack deployers, their voice is an important one, and their access to > the larger operations teams is invaluable to them. I like the idea of > local teams showing up because it's in the neighborhood and they don't need

Re: [Openstack-operators] Scaling the Ops Meetup

Hi Tom, I think this is a great problem to have. Difficult to solve, but it shows how popular / important these meetups are. I'm definitely in favor of a "no booths" type meetup. I feel if a company wants to sponsor, they're doing it out of good will and any recognition would come from that. I'd

Re: [Openstack-operators] Instance memory overhead

In addition to what Kris said, here are two other ways to see memory usage of qemu processes: The first is with "nova diagnostics ". By default this is an admin-only command. The second is by running "virsh dommemstat " directly on the compute node. Note that it's possible for the used memory (r

Re: [Openstack-operators] 100% CPU and hangs if syslog is restarted

> Using UDP is a great workaround but it does not feel like a fix... Additionally, it's not possible to use TLS/SSL with syslog and UDP -- TCP is required. ___ OpenStack-operators mailing list OpenStack-operators@lists.openstack.org http://lists.opensta

Re: [Openstack-operators] 100% CPU and hangs if syslog is restarted

Hello, Yeah, I ran into it last fall: http://www.gossamer-threads.com/lists/openstack/operators/41876 Good to know that this issue still exists in Juno (we're still on Icehouse). Thanks for the note. :) Joe On Thu, May 28, 2015 at 10:56 AM, George Shuklin wrote: > Hello. > > Today we've disc

[Openstack-operators] Ops Keystone / Federation Session

Hello, Better late than never, here's a summary of the Ops Keystone / Federation Session from this past Tuesday: First, I want to thank everyone from the Keystone team for attending the session -- it was very cool to have you guys on-hand to directly answer questions and give input and insight in

Re: [Openstack-operators] How do your end users use networking?

Hi Kris, Busy week! It was good seeing you in Vancouver - even if it was just in passing on the escalator ;) It is always nice to see that other people are doing the same things as > you or see the same issues as you are and that "you are not crazy". > +100 > Would it be accurate to say that

Re: [Openstack-operators] Venom vulnerability

Looks like the updated Ubuntu packages are available: http://www.ubuntu.com/usn/usn-2608-1/ On Wed, May 13, 2015 at 10:44 AM, Matt Van Winkle wrote: > Yeah, something like that would be handy. > > From: matt > Date: Wednesday, May 13, 2015 10:29 AM > To: "Daniel P. Berrange" > Cc: Matt Van

Re: [Openstack-operators] Venom vulnerability

Hello, Looking through the details of the Venom vulnerability, > https://securityblog.redhat.com/2015/05/13/venom-dont-get-bitten/, it > would appear that the QEMU processes need to be restarted. > > > > Our understanding is thus that a soft reboot of the VM is not sufficient > but a hard one wo

[Openstack-operators] Federation Ops Session at the Vancouver Summit

Hello, Following suit of the other posts, this is an announcement / reminder of the Federation Ops Session happening next Tuesday: http://sched.co/3BBs The etherpad for the session is here: https://etherpad.openstack.org/p/YVR-ops-federation I encourage everyone to add items they'd like to dis

Re: [Openstack-operators] expanding to 2nd location

Hi Jon, We're about to expand our OpenStack Cloud to a second datacenter. > Congratulations! :) > Anyone one have opinions they'd like to share as to what I would and > should be worrying about or how to structure this? What services will be shared between the two locations? Keystone with db

[Openstack-operators] Windows Instances and Volumes

Hello, I'm wondering if anyone has best practices for Windows-based instances that make heavy use of volumes? I have a user who was running SQL Server off of an iSCSI-based volume. We did a live-migration of the instance and that seemed to have caused Windows to drop the drive. Disk Manager showe

Re: [Openstack-operators] [Openstack] [nova] Cleaning up unused images in the cache

Hello, I've got a similar question about cache-manager and the presence of a > shared filesystem for instances images. > I'm currently reading the source code in order to find out how this is > managed but before I would be curious how you achieve this on production > servers. > > For example imag

Re: [Openstack-operators] nova rescue

That's all very useful advise -- thank you. :) On Sun, Mar 29, 2015 at 8:39 PM, gustavo panizzo (gfa) wrote: > > > On 03/29/2015 11:19 AM, Joe Topjian wrote: > >> Hello, >> >> Without specifying a rescue image, Nova will use the image that the >> inst

[Openstack-operators] nova rescue

Hello, Without specifying a rescue image, Nova will use the image that the instance is based on when performing a rescue. I've noticed that this is problematic for "cloud-friendly" images such as the official Ubuntu images and the newer CentOS 7 images. I'm finding that /dev/vdb still ends up mou

Re: [Openstack-operators] What are people using for configuration management? Puppet? Chef? Other?

+1 We've also been looking into stackstorm for day-to-day operational tasks. It's an awesome tool with an awesome team. On Thu, Mar 26, 2015 at 1:13 PM, John Dewey wrote: > We are also in the process of looking at stackstorm[1] as a means to > operate openstack. The ability limit playbook exe

Re: [Openstack-operators] FYI: Rabbit Heartbeat Patch Landed

We have other supporting services that require RabbitMQ and since they only accept a single host in their connection config, we need a more reliable way for them to connect. Those services work just fine with HAProxy/RabbitMQ. The OpenStack HA guide

[Openstack-operators] Live migration of instances with iscsi volumes

Hello, I just resolved an issue where migrating instances with iSCSI volumes would occasionally fail. There's a bug report here: https://bugs.launchpad.net/nova/+bug/1423772 The core cause ended up being libvirt transferring the volume paths verbatim. For example, take the situation where: comp

[Openstack-operators] Ops Meetup Monitoring/Tools Session

Hi all, I'll be moderating the Monitoring/Tools session at next week's Ops Meetup. The etherpage is here: https://etherpad.openstack.org/p/PHL-ops-tools-wg Please add items you'd like to see covered. So far, the general topics will be: * Discussion of Monasca, StackTach, and related tools. Memb

Re: [Openstack-operators] qemu 1.x to 2.0

ave even worked and we would have had to do a cloud-wide hard-reboot of instances. :/ Let me know if you have any more questions with this. :) Thanks, Joe On Tue, Feb 24, 2015 at 11:40 AM, Mathieu Gagné wrote: > Joe, > > Finally got time to check the QEMU 2.0 upgrade. =) > &g

Re: [Openstack-operators] State of Juno in Production

witched over to quantum in Grizzly. > > As I said I've not looked deeply at this so it may be a local config > issue rather than something systemic with Juno, but if it's an > important use case for you be sure to take a good look at it. > > -Jon > > On Tue, Feb 17

Re: [Openstack-operators] State of Juno in Production

Nice - thanks, Jesse. :) On Tue, Feb 17, 2015 at 10:35 AM, Jesse Keating wrote: > On 2/17/15 8:46 AM, Joe Topjian wrote: > >> >> The only issue I'm aware of is that live snapshotting is disabled. Has >> anyone re-enabled this and seen issues? What was the procedure

[Openstack-operators] State of Juno in Production

Hello, I'm beginning to plan for a Juno upgrade and wanted to get some feedback from anyone else who has gone through the upgrade and has been running Juno in production. The environment that will be upgraded is pretty basic: nova-network, no cells, Keystone v2. We run a RabbitMQ cluster, though,

Re: [Openstack-operators] [Ceilometer] Real world experience with Ceilometer deployments - Feedback requested

Hi Tim, Does anyone have any proposals regarding > > > - Possible replacements for Ceilometer that you have used instead > > It seems that many sites have written their own systems. > Sorry - I should have appended this at the end of my last post. I need to preface this with "I have never used C

Re: [Openstack-operators] [Ceilometer] Real world experience with Ceilometer deployments - Feedback requested

Hi Sandy, That said, I'd love to hear about headaches and failures of the older > StackTach release and how people are using it, or hope to use it. > We have two StackTach v2 environments, one of which has been running for almost 3 years. For that particular environment, it can be a bear to do qu

Re: [Openstack-operators] How to handle updates of public images?

I'm curious: are you using _base files? We're not and we're able to block migrate instances based on deleted images or images that were public but are now private. On Thu, Feb 5, 2015 at 2:42 PM, Belmiro Moreira < moreira.belmiro.email.li...@gmail.com> wrote: > We don't delete public images from

Re: [Openstack-operators] How to handle updates of public images?

We do exactly this. Public images are named very generically like "Ubuntu 14.04". Not even "14.04.1" or something like that. Old images are renamed and made private. Existing instances continue to run, but, as others have mentioned, if a user is using a UUID to launch instances, that will break f

Re: [Openstack-operators] RHEL 7 / CentOS 7 instances losing their network gateway

;forever" just like other distros. And so far, so good (crossing fingers). Thanks, Joe On Tue, Jan 27, 2015 at 1:53 PM, Joe Topjian wrote: > Hi George, > > All instances have only a single interface. > > Thanks, > Joe > > On Tue, Jan 27, 2015 at 1:38 PM, George Shukl

Re: [Openstack-operators] RHEL 7 / CentOS 7 instances losing their network gateway

s up with routes for the main network. > > > On 01/27/2015 06:08 PM, Joe Topjian wrote: > > Hello, > > I have run into two different OpenStack clouds where instances running > either RHEL 7 or CentOS 7 images are randomly losing their network gateway. > > There's

Re: [Openstack-operators] RHEL 7 / CentOS 7 instances losing their network gateway

o roll out Cent7 vm's. However, a huge change from Cent6 to > Cent7 was the switch from upstart/dhclient to systemd/systemd-dhcp. > > > Kris Lindgren > Senior Linux Systems Engineer > GoDaddy, LLC. > > > > From: Joe To

  1   2   >