[Openstack-operators] control guest VMs in isolated network

Hi colleagues, are there ways to control guest VMs which reside in isolated network? In general, there two methods are available: 1. use Heat's SoftwareDeployment method 2. use Qemu Guest Agent First method requires accessibility of Keystone/Heat (os-collect-agent authorizes on Keystone, rece

Re: [Openstack-operators] Mitaka and gnocchi

Sorry, I didn't notice policy.json in the first link. I'll check it. > On May 22, 2017 at 6:09 PM mate...@mailbox.org wrote: > > > Hello Gordon, > I'm trying to use gnocchi client and it always says to me that I don't have > sufficient priviligies - > 10.10.10.69 - - [22/May/2017 14:58:33] "GE

Re: [Openstack-operators] Mitaka and gnocchi

On 23/05/17 06:35 AM, mate...@mailbox.org wrote: > Sorry, I didn't notice policy.json in the first link. I'll check it. :) depending on gnocchi version, it might look internally for policy.json if it doesn't find policy.json in /etc/gnocchi/ -- gord __

Re: [Openstack-operators] Mitaka and gnocchi

As for gnocchi 2.2.1 policy.json is needed :) Now I'm fighting with ceilometer. > On May 23, 2017 at 3:16 PM gordon chung wrote: > > > > > On 23/05/17 06:35 AM, mate...@mailbox.org wrote: > > Sorry, I didn't notice policy.json in the first link. I'll check it. > > :) > > depending on gnoc

Re: [Openstack-operators] Routed provider networks...

On Mon, May 22, 2017 at 9:12 PM, Kevin Benton wrote: > The operators that were asking for the spec were using private IP space > and that is probably going to be the most common use case for routed > networks. Splitting a /21 up across the entire data center isn't really > something you would wan

Re: [Openstack-operators] [nova][ironic][scheduler][placement] IMPORTANT: Getting rid of the automated reschedule functionality

For the anti-affinity use case, it's really useful for smaller or medium size operators who want to provide some form of failure domains to users but do not have the resources to create AZ's at DC or even at rack or row scale. Don't forget that as soon as you introduce AZs, you need to grow thos

Re: [Openstack-operators] Mitaka and gnocchi

I'm trying to connect ceilometer to gnocchi. I don't use any wsgi application and ceilometer is connecting directly to gnocchi-api.The thing is that I see these kind of errors in ceilometer-collector.log  -  2017-05-23 13:29:10.961 1931583 ERROR ceilometer.dispatcher.gnocchi [-] Failed to connect

[Openstack-operators] UTC 14:00 henceforth for Ops Meet Up planning

I have picked "Tuesday, May 30, 2017 8:00 AM (Time zone: Mountain Time)" as final option(s) for the Doodle poll "Ops Meetup Preferred Time." Follow this link to open the poll: http://doodle.com/poll/bccipuc8kfm36xna ___ OpenStack-operators mailing list O

[Openstack-operators] [scientific] Reminder: Scientific WG IRC meeting - Wednesday 0900 UTC

Hello - We have a meeting on Wednesday at 0900 UTC in channel #openstack-meeting. We’d like to cover the discussion on gathering OpenStack-related research papers, and round up on activities from the summit. Today’s full agenda is here: https://wiki.openstack.org/wiki/Scientific_working_group#

[Openstack-operators] DB deadlocks due to connection string

Just wanted to put this out there to hopefully spread awareness and prevent it from happening more. We had a bug reported in Cinder of hitting a deadlock when attempting to deelte multiple volumes simultaneously: https://bugs.launchpad.net/cinder/+bug/1685818 Some were seeing it, but others were

Re: [Openstack-operators] UTC 14:00 henceforth for Ops Meet Up planning

On Tue, May 23, 2017 at 09:16:13AM -0600, David Medberry wrote: > I have picked "Tuesday, May 30, 2017 8:00 AM (Time zone: Mountain Time)" as > final option(s) for the Doodle poll "Ops Meetup Preferred Time." Hey David, Sorry, I'm sure this was stated elsewhere, but where is this meeting held? T

Re: [Openstack-operators] UTC 14:00 henceforth for Ops Meet Up planning

Ah, excellent question sean. This is an IRC meeting that occurs weekly currently at 14:00 (starting next week) on Tuesdays in Freenode at #openstack-operators (the current one is just wrapping up) On Tue, May 23, 2017 at 9:34 AM, Sean McGinnis wrote: > On Tue, May 23, 2017 at 09:16:13AM -0600,

Re: [Openstack-operators] [nova][ironic][scheduler][placement] IMPORTANT: Getting rid of the automated reschedule functionality

On 05/23/2017 09:48 AM, Marc Heckmann wrote: For the anti-affinity use case, it's really useful for smaller or medium size operators who want to provide some form of failure domains to users but do not have the resources to create AZ's at DC or even at rack or row scale. Don't forget that as so

Re: [Openstack-operators] UTC 14:00 henceforth for Ops Meet Up planning

Many thanks to David for running this poll. We desperately needed a more popular slot. Minutes for today's meeting : Meeting ended Tue May 23 15:43:14 2017 UTC. Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4) 11:43 AM O Minutes: http://eavesdrop.openstack.org/meetings/ops

Re: [Openstack-operators] DB deadlocks due to connection string

As discussed on the Cinder channel, I’ve opened https://bugs.launchpad.net/oslo.db/+bug/1692956 to see if oslo.db would be a good place to produce a warning when it detects this potential misconfiguration. Cheers, Arne On 23 May 2017, at 17:25, Sean McGinnis mailto:sean.mcgin...@gmx.com>> wro

Re: [Openstack-operators] [nova][ironic][scheduler][placement] IMPORTANT: Getting rid of the automated reschedule functionality

On 05/22/2017 03:36 PM, Sean Dague wrote: On 05/22/2017 02:45 PM, James Penick wrote: I recognize that large Ironic users expressed their concerns about IPMI/BMC communication being unreliable and not wanting to have users manually retry a baremetal instance launch. But, on this

Re: [Openstack-operators] DB deadlocks due to connection string

Excerpts from Arne Wiebalck's message of 2017-05-23 15:50:43 +: > As discussed on the Cinder channel, I’ve opened > > https://bugs.launchpad.net/oslo.db/+bug/1692956 > > to see if oslo.db would be a good place to produce a warning when it detects > this potential misconfiguration. This sound

Re: [Openstack-operators] DB deadlocks due to connection string

Would an automatic transform of mysql:// to mysql+pymysql:// be possible? Is there any reason to not have pymysql? Tim From: Arne Wiebalck Date: Tuesday, 23 May 2017 at 17:50 To: Sean McGinnis Cc: openstack-operators Subject: Re: [Openstack-operators] DB deadlocks due to connection string As

Re: [Openstack-operators] [nova][ironic][scheduler][placement] IMPORTANT: Getting rid of the automated reschedule functionality

On Tue, 2017-05-23 at 11:44 -0400, Jay Pipes wrote: > On 05/23/2017 09:48 AM, Marc Heckmann wrote: > > For the anti-affinity use case, it's really useful for smaller or > > medium  > > size operators who want to provide some form of failure domains to > > users  > > but do not have the resources to

Re: [Openstack-operators] [nova][ironic][scheduler][placement] IMPORTANT: Getting rid of the automated reschedule functionality

On 05/23/2017 12:34 PM, Marc Heckmann wrote: On Tue, 2017-05-23 at 11:44 -0400, Jay Pipes wrote: On 05/23/2017 09:48 AM, Marc Heckmann wrote: For the anti-affinity use case, it's really useful for smaller or medium size operators who want to provide some form of failure domains to users but do

Re: [Openstack-operators] DB deadlocks due to connection string

> > This sounds like something we could fix completely by dropping the > use of the offending library. I know there was a lot of work done > to get pymysql support in place. It seems like we can finish that by > removing support for the old library and redirecting mysql:// > connections to use pym

[Openstack-operators] [nova] Cells v2 FAQs

FYI, I've started a series of changes to add FAQs to the nova devref about cells v2: https://review.openstack.org/#/q/topic:cells-v2-docs The first few questions are based on things that have come up in IRC. If you have other things you'd like to see here, please reply to this thread, or bett

Re: [Openstack-operators] DB deadlocks due to connection string

On 5/23/2017 11:38 AM, Sean McGinnis wrote: This sounds like something we could fix completely by dropping the use of the offending library. I know there was a lot of work done to get pymysql support in place. It seems like we can finish that by removing support for the old library and redirecti

[Openstack-operators] Eventbrite - Ops Meet-up Mexico City

Tom, It is time to create the Eventbrite for the Ops Meet-up in Mexico City. I was told you are the man helping us on this area. I have also started the etherpad for the meet-up that we can use as a reference. https://etherpad.openstack.org/p/MEX-ops-meetup Let me know what you need to create t

[Openstack-operators] CockroachDB for Core Services

All, Now that cockroachdb has reached v1 and appears to be compatible with sqlalchemy (https://www.cockroachlabs.com/blog/building-application-cockroachdb-sqlalchemy-2/), has anyone tried using it as the backend for any of the service databases? Wondering how far away it is from production-r

Re: [Openstack-operators] DB deadlocks due to connection string

One scenario would be to change the default and allow the exceptions to opt out (e.g. mysql-pymysql ( Tim On 23.05.17, 19:08, "Matt Riedemann" wrote: On 5/23/2017 11:38 AM, Sean McGinnis wrote: >> >> This sounds like something we could fix completely by dropping the >> use of t

Re: [Openstack-operators] Mitaka and gnocchi

On 23/05/17 10:16 AM, mate...@mailbox.org wrote: > / 2017-05-23 13:29:10.961 1931583 ERROR ceilometer.dispatcher.gnocchi > [-] Failed to connect to Gnocchi./ > /2017-05-23 13:29:10.962 1931583 ERROR stevedore.extension [-] Could not > load 'gnocchi': Unexpected exception for > http//10.10.10.69:8

Re: [Openstack-operators] [nova][ironic][scheduler][placement] IMPORTANT: Getting rid of the automated reschedule functionality

On Tue, May 23, 2017 at 8:52 AM, Jay Pipes wrote: > > If Heat was more widely deployed, would you feel this way? Would you > reconsider having Heat as one of those "basic compute services" in > OpenStack, then? > > (Caveat: I haven't looked at Heat in at least a year) I haven't deployed heat in m

[Openstack-operators] [publiccloud-wg] Tomorrows meeting PublicCloudWorkingGroup

Hi everyone, First of all, really fun to see the interest for the group and the forum sessions we moderated in Boston. I hope that we can keep up that spirit and looking forward to a lot of participants in the bi-weekly meetings for this cycle. So, reminder for tomorrows meeting for the Publ

Re: [Openstack-operators] [nova][ironic][scheduler][placement] IMPORTANT: Getting rid of the automated reschedule functionality

On May 23, 2017, at 1:27 PM, James Penick wrote: > Perhaps this is a place where the TC and Foundation should step in and > foster the existence of a porcelain API. Either by constructing something > new, or by growing Nova into that thing. Oh please, not Nova. The last word that comes to mi

Re: [Openstack-operators] [nova][ironic][scheduler][placement] IMPORTANT: Getting rid of the automated reschedule functionality

On Tue, May 23, 2017 at 12:20 PM, Edward Leafe wrote: > > > Oh please, not Nova. The last word that comes to mind when thinking about > Nova code is “porcelain”. > Oh I dunno, porcelain is usually associated with so many every day objects. If we really push, we could see a movement in the right

Re: [Openstack-operators] [nova][ironic][scheduler][placement] IMPORTANT: Getting rid of the automated reschedule functionality

On Tue, 23 May 2017, James Penick wrote: I agree that a single entry point to OpenStack would be fantastic. If it existed, scheduling, quota, etc would have moved out of Nova a long time ago, and Nova at this point would be just a small VM driver. Unfortunately such a thing does not yet exist, a

Re: [Openstack-operators] DB deadlocks due to connection string

Excerpts from Sean McGinnis's message of 2017-05-23 11:38:34 -0500: > > > > This sounds like something we could fix completely by dropping the > > use of the offending library. I know there was a lot of work done > > to get pymysql support in place. It seems like we can finish that by > > removing

Re: [Openstack-operators] [nova][ironic][scheduler][placement] IMPORTANT: Getting rid of the automated reschedule functionality

On Tue, May 23, 2017 at 1:20 PM, Edward Leafe wrote: > On May 23, 2017, at 1:27 PM, James Penick wrote: > >> Perhaps this is a place where the TC and Foundation should step in and >> foster the existence of a porcelain API. Either by constructing something >> new, or by growing Nova into that

Re: [Openstack-operators] [nova][ironic][scheduler][placement] IMPORTANT: Getting rid of the automated reschedule functionality

On Tue, 23 May 2017 20:58:20 +0100 (IST), Chris Dent wrote: If we're talking big crazy changes: Why not take the "small VM driver" (presumably nova-compute) out of Nova? What stays behind is _already_ orchestration but missing some features and having a fair few bugs. I've suggested this a cou

Re: [Openstack-operators] [nova][ironic][scheduler][placement] IMPORTANT: Getting rid of the automated reschedule functionality

Thanks for the feedback, Curtis, appreciated! On 05/23/2017 04:09 PM, Curtis wrote: On Tue, May 23, 2017 at 1:20 PM, Edward Leafe wrote: On May 23, 2017, at 1:27 PM, James Penick wrote: Perhaps this is a place where the TC and Foundation should step in and foster the existence of a porce

Re: [Openstack-operators] DB deadlocks due to connection string

> On May 23, 2017, at 4:01 PM, Doug Hellmann wrote: > > Excerpts from Sean McGinnis's message of 2017-05-23 11:38:34 -0500: >>> >>> This sounds like something we could fix completely by dropping the >>> use of the offending library. I know there was a lot of work done >>> to get pymysql suppor

Re: [Openstack-operators] DB deadlocks due to connection string

On 05/23/2017 04:17 PM, Doug Hellmann wrote: On May 23, 2017, at 4:01 PM, Doug Hellmann > wrote: Excerpts from Sean McGinnis's message of 2017-05-23 11:38:34 -0500: This sounds like something we could fix completely by dropping the use of the offending librar

[Openstack-operators] [nova][ironic][scheduler][placement] IMPORTANT: NOT Getting rid of the automated reschedule functionality

Hello Dear Operators, OK, we've heard you loud and (mostly) clear. We won't remove the automated rescheduling behavior from Nova. While we will be removing the primary cause of reschedules (resource overconsumption races), we cannot yet eliminate the catchall exception handling on the compute

Re: [Openstack-operators] DB deadlocks due to connection string

per IRC deliberation, I'll do a patch in oslo.db: * detect incoming URL that is like "mysql://" with no driver (I can set this up for all DBs, though excluding sqlite:// since that one is used by tests) * log warning referring to the issue, rather than a hard-cut over ("the big issue we'll

Re: [Openstack-operators] Routed provider networks...

>Dynamic routing is absolutely necessary, though. Large blocks of 1918 addresses are scarce, even inside the DC. I just described a 65 thousand VM topology and it used a /16. Dynamic routing is not necessary or even helpful in this scenario if you plan on ever running close to your max server dens

Re: [Openstack-operators] Routed provider networks...

Kevin, should have been more clear For the specific operator that is running L3 to host, with only a few /20 blocks...dynamic routing is absolutely necessary. The /16 scenario you describe is totally fine without it. CM On Tue, May 23, 2017 at 2:40 PM, Kevin Benton wrote: > >Dynamic

[Openstack-operators] [MassivelyDistributed] IRC Meeting tomorrow15:00 UTC

Dear all, A gentle reminder for our meeting tomorrow. As usual, the agenda is available at: https://etherpad.openstack.org/p/massively_distributed_ircmeetings_2017 (line 597) Please feel free to add items. Best, ad_rien_ ___ OpenStack-operators ma

Re: [Openstack-operators] Routed provider networks...

On Mon, May 22, 2017 at 1:47 PM, Chris Marino wrote: > Hello operators, I will be talking about the new routed provider network > > features in OpenStack at a Meetup >

Re: [Openstack-operators] [nova][ironic][scheduler][placement] IMPORTANT: NOT Getting rid of the automated reschedule functionality

Thanks Jay, I wonder whether there is an easy-ish way to collect stats about the sorts of errors deployers see in that catchall, so that when this comes back around in a release or two there might be some less anecdotal data available...? Cheers, On 24 May 2017 at 06:43, Jay Pipes wrote: > Hell

Re: [Openstack-operators] [nova][ironic][scheduler][placement] IMPORTANT: NOT Getting rid of the automated reschedule functionality

On 05/23/2017 07:06 PM, Blair Bethwaite wrote: Thanks Jay, I wonder whether there is an easy-ish way to collect stats about the sorts of errors deployers see in that catchall, so that when this comes back around in a release or two there might be some less anecdotal data available...? Don't wo

Re: [Openstack-operators] [nova][ironic][scheduler][placement] IMPORTANT: NOT Getting rid of the automated reschedule functionality

On 5/23/2017 7:01 PM, Jay Pipes wrote: On 05/23/2017 07:06 PM, Blair Bethwaite wrote: Thanks Jay, I wonder whether there is an easy-ish way to collect stats about the sorts of errors deployers see in that catchall, so that when this comes back around in a release or two there might be some less

[Openstack-operators] [Nova] [Cells] Stupid question: Cells v2 & AZs

Hi Devs and Implementers, A question came up tonight in the Colorado OpenStack meetup regarding cells v2 and availability zones. Can a cell contain multiple AZs? (I assume this is yes.) Can an AZ contain mutliple cells (I assumed this is no, but now in thinking about it, that's probably not righ

Re: [Openstack-operators] [openstack-dev] [Nova] [Cells] Stupid question: Cells v2 & AZs

Hi David, AVZs are basically aggregates. In cells_v2 aggregates are defined in the cell_api, so it will be possible to have multiple AVZs per cell and AVZs that spread between different cells. Belmiro On Wed, May 24, 2017 at 5:14 AM, David Medberry wrote: > Hi Devs and Implementers, > > A quest