Re: [Openstack] Incredibly odd mysql permission error

2013-03-11 Thread Samuel Winchenbach
t; > *From:* openstack-bounces+mark.m.miller=hp@lists.launchpad.net[mailto: > openstack-bounces+mark.m.miller=hp@lists.launchpad.net] *On Behalf Of > *Samuel Winchenbach > *Sent:* Monday, March 11, 2013 11:18 AM > *To:* Sylvain Bauza > *Cc:* openstack@lists.launchpad.net &

Re: [Openstack] Incredibly odd mysql permission error

2013-03-11 Thread Miller, Mark M (EB SW Cloud - R&D - Corvallis)
@lists.launchpad.net [mailto:openstack-bounces+mark.m.miller=hp@lists.launchpad.net] On Behalf Of Samuel Winchenbach Sent: Monday, March 11, 2013 11:18 AM To: Sylvain Bauza Cc: openstack@lists.launchpad.net Subject: Re: [Openstack] Incredibly odd mysql permission error OK Someone on the IRC channel got

Re: [Openstack] Incredibly odd mysql permission error

2013-03-11 Thread Samuel Winchenbach
OK Someone on the IRC channel got me closer, but we have no idea why this would happen: this works: "root@test1:~# nova-manage --config-file=/etc/nova/nova.conf service list" Why would I have to specify the config file though? It is in the standard place. Thanks, Sam On Mon, Mar 11, 20

Re: [Openstack] Incredibly odd mysql permission error

2013-03-11 Thread Samuel Winchenbach
For completeness here the routing table, and ip listing for both test1 and test2. Doubt this will help much: http://paste2.org/p/3117125 On Mon, Mar 11, 2013 at 1:52 PM, Samuel Winchenbach wrote: > #1 - No change > #2 - All of grants are in the ip/mask form such as: 'nova'@' > 10.21.0.0/255.2

Re: [Openstack] Incredibly odd mysql permission error

2013-03-11 Thread Samuel Winchenbach
#1 - No change #2 - All of grants are in the ip/mask form such as: 'nova'@' 10.21.0.0/255.255.0.0' I have also tried adding 'nova'@'test1' and 'nova'@'10.21.0.1'. No change. #3 - I changed the SQL connection string over to IP instead of hostname. No change. I didn't restart nova-api because it

Re: [Openstack] Incredibly odd mysql permission error

2013-03-11 Thread Samuel Winchenbach
I enabled general_log in /etc/mysql/my.cnf Here are the results of connecting from "test1", "test2" and using the client: http://paste2.org/p/3115525 I purposefully used the real password in case there is a problem with it. I changed before submitting post. here is a raw packet TCP dump (tcpd

Re: [Openstack] Incredibly odd mysql permission error

2013-03-11 Thread Sylvain Bauza
So as to reproduce the nova-manage SQL command, I would recommand to tcpdump -A port 3306 on the host and get the SQL trace on what's failing. Could you please explain further what is your HA config ? Are you using pacemaker/heartbeat or any VIP ? -Sylvain Le 11/03/2013 14:23, Samuel Winchen

Re: [Openstack] Incredibly odd mysql permission error

2013-03-11 Thread Samuel Winchenbach
Does anyone think this could be an openstack bug? I just want to check before submitting a bug report. Sam On Fri, Mar 8, 2013 at 4:02 PM, Jay Pipes wrote: > Sorry, I really can't think of anything :( > > On 03/08/2013 03:52 PM, Samuel Winchenbach wrote: > > I dropped those users and no chang

Re: [Openstack] Incredibly odd mysql permission error

2013-03-08 Thread Jay Pipes
Sorry, I really can't think of anything :( On 03/08/2013 03:52 PM, Samuel Winchenbach wrote: > I dropped those users and no change. > > I also set up general logging in mysql but it really doesn't provide any > additional information. Any idea for a next step I could take? > > I am almost at th

Re: [Openstack] Incredibly odd mysql permission error

2013-03-08 Thread Samuel Winchenbach
I dropped those users and no change. I also set up general logging in mysql but it really doesn't provide any additional information. Any idea for a next step I could take? I am almost at the point of taking a tcpdump and trying to recreate the salted password. :/ Thanks for the help Sam

Re: [Openstack] Incredibly odd mysql permission error

2013-03-08 Thread Jay Pipes
I'm stumped :( Looks like everything is set up correctly to me. What is interested is that your nova user access works from test2, but there is no nova@test2 user in the mysql.user table. What about doing a DROP USER nova@test1; FLUSH PRIVILEGES; and then see if that fixes things... since the nova@

Re: [Openstack] Incredibly odd mysql permission error

2013-03-08 Thread Samuel Winchenbach
http://paste2.org/p/3085807 On Fri, Mar 8, 2013 at 2:46 PM, Jay Pipes wrote: > Please paste the results of SELECT User, Host, Password FROM mysql.user > when running as root... > > Thanks! > -jay > > On 03/08/2013 02:25 PM, Samuel Winchenbach wrote: > > Here are my grants. I don't know if this

Re: [Openstack] Incredibly odd mysql permission error

2013-03-08 Thread Samuel Winchenbach
oops. guess I need to change the root password now :P On Fri, Mar 8, 2013 at 3:09 PM, Samuel Winchenbach wrote: > > http://paste2.org/p/3085807 > > > On Fri, Mar 8, 2013 at 2:46 PM, Jay Pipes wrote: > >> Please paste the results of SELECT User, Host, Password FROM mysql.user >> when running as

Re: [Openstack] Incredibly odd mysql permission error

2013-03-08 Thread Jay Pipes
Please paste the results of SELECT User, Host, Password FROM mysql.user when running as root... Thanks! -jay On 03/08/2013 02:25 PM, Samuel Winchenbach wrote: > Here are my grants. I don't know if this helps, but I did verify that > the password was identical for each grant: http://paste2.org/p

Re: [Openstack] Incredibly odd mysql permission error

2013-03-08 Thread Samuel Winchenbach
Here are my grants. I don't know if this helps, but I did verify that the password was identical for each grant: http://paste2.org/p/3085361 On Fri, Mar 8, 2013 at 2:17 PM, Samuel Winchenbach wrote: > root@test1:/var/log# mysql -hmysql-ha -unova -p > -e"SELECT

Re: [Openstack] Incredibly odd mysql permission error

2013-03-08 Thread Samuel Winchenbach
root@test1:/var/log# mysql -hmysql-ha -unova -p -e"SELECT User, Host, Password FROM mysql.user;" ERROR 1142 (42000) at line 1: SELECT command denied to user 'nova'@'test1' for table 'user' On Fri, Mar 8, 2013 at 2:06 PM, Jay Pipes wrote: > What does this show? >

Re: [Openstack] Incredibly odd mysql permission error

2013-03-08 Thread Jay Pipes
What does this show? mysql -hmysql-ha -unova -p -e"SELECT User, Host, Password FROM mysql.user" -jay On 03/08/2013 01:46 PM, Samuel Winchenbach wrote: > Sorry, that must have been a copy and paste error. Here is what I > actually ran: > > http://paste2.org/p/3084996 > > > On Fri, Mar 8, 2013

Re: [Openstack] Incredibly odd mysql permission error

2013-03-08 Thread Samuel Winchenbach
Sorry, that must have been a copy and paste error. Here is what I actually ran: http://paste2.org/p/3084996 On Fri, Mar 8, 2013 at 12:40 PM, Jay Pipes wrote: > On 03/08/2013 12:19 PM, Samuel Winchenbach wrote: > > Hi All, > > > > I have two nodes (test1 and test2) that I am trying to set up i

Re: [Openstack] Incredibly odd mysql permission error

2013-03-08 Thread Jay Pipes
On 03/08/2013 12:19 PM, Samuel Winchenbach wrote: > Hi All, > > I have two nodes (test1 and test2) that I am trying to set up in a > highly available configuration. > > During the setup process I tried running "nova-manage service list" on > both nodes. It worked fine on test2, but fails on tes

[Openstack] Incredibly odd mysql permission error

2013-03-08 Thread Samuel Winchenbach
Hi All, I have two nodes (test1 and test2) that I am trying to set up in a highly available configuration. During the setup process I tried running "nova-manage service list" on both nodes. It worked fine on test2, but fails on test1 even though I can connect to the database with the mysql clie