I tried 1 minute ago to create another instance. Nova api reports the following:
RROR oslo_db.sqlalchemy.engines [req-cac96dee-d91b-48cb-831b-31f95cffa2f4 89f76bc5de5545f381da2c10c7df7f15 59f1f232ce28409593d66d8f6495e434 - default default] Database connection was found disconnected; reconnecting: DBConnectionError: (pymysql.err.OperationalError) (2013, 'Lost connection to MySQL server during query') [SQL: u'SELECT 1'] (Background on this error at: http://sqlalche.me/e/e3q8) 2018-11-12 23:07:28.813 4224 ERROR oslo_db.sqlalchemy.engines Traceback (most recent call last): 2018-11-12 23:07:28.813 4224 ERROR oslo_db.sqlalchemy.engines File "/usr/lib/python2.7/site-packages/oslo_db/sqlalchemy/engines.py", line 73, in _connect_ping_listener 2018-11-12 23:07:28.813 4224 ERROR oslo_db.sqlalchemy.engines connection.scalar(select([1])) 2018-11-12 23:07:28.813 4224 ERROR oslo_db.sqlalchemy.engines File "/usr/lib64/python2.7/site-packages/sqlalchemy/engine/base.py", line 880, in scalar 2018-11-12 23:07:28.813 4224 ERROR oslo_db.sqlalchemy.engines return self.execute(object, *multiparams, **params).scalar() 2018-11-12 23:07:28.813 4224 ERROR oslo_db.sqlalchemy.engines File "/usr/lib64/python2.7/site-packages/sqlalchemy/engine/base.py", line 948, in execute 2018-11-12 23:07:28.813 4224 ERROR oslo_db.sqlalchemy.engines return meth(self, multiparams, params) 2018-11-12 23:07:28.813 4224 ERROR oslo_db.sqlalchemy.engines File "/usr/lib64/python2.7/site-packages/sqlalchemy/sql/elements.py", line 269, in _execute_on_connection 2018-11-12 23:07:28.813 4224 ERROR oslo_db.sqlalchemy.engines return connection._execute_clauseelement(self, multiparams, params) 2018-11-12 23:07:28.813 4224 ERROR oslo_db.sqlalchemy.engines File "/usr/lib64/python2.7/site-packages/sqlalchemy/engine/base.py", line 1060, in _execute_clauseelement 2018-11-12 23:07:28.813 4224 ERROR oslo_db.sqlalchemy.engines compiled_sql, distilled_params 2018-11-12 23:07:28.813 4224 ERROR oslo_db.sqlalchemy.engines File "/usr/lib64/python2.7/site-packages/sqlalchemy/engine/base.py", line 1200, in _execute_context 2018-11-12 23:07:28.813 4224 ERROR oslo_db.sqlalchemy.engines context) 2018-11-12 23:07:28.813 4224 ERROR oslo_db.sqlalchemy.engines File "/usr/lib64/python2.7/site-packages/sqlalchemy/engine/base.py", line 1409, in _handle_dbapi_exception 2018-11-12 23:07:28.813 4224 ERROR oslo_db.sqlalchemy.engines util.raise_from_cause(newraise, exc_info) 2018-11-12 23:07:28.813 4224 ERROR oslo_db.sqlalchemy.engines File "/usr/lib64/python2.7/site-packages/sqlalchemy/util/compat.py", line 203, in raise_from_cause 2018-11-12 23:07:28.813 4224 ERROR oslo_db.sqlalchemy.engines reraise(type(exception), exception, tb=exc_tb, cause=cause) 2018-11-12 23:07:28.813 4224 ERROR oslo_db.sqlalchemy.engines File "/usr/lib64/python2.7/site-packages/sqlalchemy/engine/base.py", line 1193, in _execute_context 2018-11-12 23:07:28.813 4224 ERROR oslo_db.sqlalchemy.engines context) 2018-11-12 23:07:28.813 4224 ERROR oslo_db.sqlalchemy.engines File "/usr/lib64/python2.7/site-packages/sqlalchemy/engine/default.py", line 507, in do_execute 2018-11-12 23:07:28.813 4224 ERROR oslo_db.sqlalchemy.engines cursor.execute(statement, parameters) 2018-11-12 23:07:28.813 4224 ERROR oslo_db.sqlalchemy.engines File "/usr/lib/python2.7/site-packages/pymysql/cursors.py", line 166, in execute 2018-11-12 23:07:28.813 4224 ERROR oslo_db.sqlalchemy.engines result = self._query(query) 2018-11-12 23:07:28.813 4224 ERROR oslo_db.sqlalchemy.engines File "/usr/lib/python2.7/site-packages/pymysql/cursors.py", line 322, in _query 2018-11-12 23:07:28.813 4224 ERROR oslo_db.sqlalchemy.engines conn.query(q) 2018-11-12 23:07:28.813 4224 ERROR oslo_db.sqlalchemy.engines File "/usr/lib/python2.7/site-packages/pymysql/connections.py", line 856, in query 2018-11-12 23:07:28.813 4224 ERROR oslo_db.sqlalchemy.engines self._affected_rows = self._read_query_result(unbuffered=unbuffered) 2018-11-12 23:07:28.813 4224 ERROR oslo_db.sqlalchemy.engines File "/usr/lib/python2.7/site-packages/pymysql/connections.py", line 1057, in _read_query_result 2018-11-12 23:07:28.813 4224 ERROR oslo_db.sqlalchemy.engines result.read() 2018-11-12 23:07:28.813 4224 ERROR oslo_db.sqlalchemy.engines File "/usr/lib/python2.7/site-packages/pymysql/connections.py", line 1340, in read 2018-11-12 23:07:28.813 4224 ERROR oslo_db.sqlalchemy.engines first_packet = self.connection._read_packet() 2018-11-12 23:07:28.813 4224 ERROR oslo_db.sqlalchemy.engines File "/usr/lib/python2.7/site-packages/pymysql/connections.py", line 987, in _read_packet 2018-11-12 23:07:28.813 4224 ERROR oslo_db.sqlalchemy.engines packet_header = self._read_bytes(4) 2018-11-12 23:07:28.813 4224 ERROR oslo_db.sqlalchemy.engines File "/usr/lib/python2.7/site-packages/pymysql/connections.py", line 1033, in _read_bytes 2018-11-12 23:07:28.813 4224 ERROR oslo_db.sqlalchemy.engines CR.CR_SERVER_LOST, "Lost connection to MySQL server during query") 2018-11-12 23:07:28.813 4224 ERROR oslo_db.sqlalchemy.engines DBConnectionError: (pymysql.err.OperationalError) (2013, 'Lost connection to MySQL server during query') [SQL: u'SELECT 1'] (Background on this error at: http://sqlalche.me/e/e3q8) 2018-11-12 23:07:28.813 4224 ERROR oslo_db.sqlalchemy.engines 2018-11-12 23:07:28.831 4224 ERROR oslo_db.sqlalchemy.engines [req-cac96dee-d91b-48cb-831b-31f95cffa2f4 89f76bc5de5545f381da2c10c7df7f15 59f1f232ce28409593d66d8f6495e434 - default default] Database connection was found disconnected; reconnecting: DBConnectionError: (pymysql.err.OperationalError) (2013, 'Lost connection to MySQL server during query') [SQL: u'SELECT 1'] (Background on this error at: http://sqlalche.me/e/e3q8) 2018-11-12 23:07:28.831 4224 ERROR oslo_db.sqlalchemy.engines Traceback (most recent call last): 2018-11-12 23:07:28.831 4224 ERROR oslo_db.sqlalchemy.engines File "/usr/lib/python2.7/site-packages/oslo_db/sqlalchemy/engines.py", line 73, in _connect_ping_listener 2018-11-12 23:07:28.831 4224 ERROR oslo_db.sqlalchemy.engines connection.scalar(select([1])) 2018-11-12 23:07:28.831 4224 ERROR oslo_db.sqlalchemy.engines File "/usr/lib64/python2.7/site-packages/sqlalchemy/engine/base.py", line 880, in scalar 2018-11-12 23:07:28.831 4224 ERROR oslo_db.sqlalchemy.engines return self.execute(object, *multiparams, **params).scalar() 2018-11-12 23:07:28.831 4224 ERROR oslo_db.sqlalchemy.engines File "/usr/lib64/python2.7/site-packages/sqlalchemy/engine/base.py", line 948, in execute 2018-11-12 23:07:28.831 4224 ERROR oslo_db.sqlalchemy.engines return meth(self, multiparams, params) 2018-11-12 23:07:28.831 4224 ERROR oslo_db.sqlalchemy.engines File "/usr/lib64/python2.7/site-packages/sqlalchemy/sql/elements.py", line 269, in _execute_on_connection 2018-11-12 23:07:28.831 4224 ERROR oslo_db.sqlalchemy.engines return connection._execute_clauseelement(self, multiparams, params) 2018-11-12 23:07:28.831 4224 ERROR oslo_db.sqlalchemy.engines File "/usr/lib64/python2.7/site-packages/sqlalchemy/engine/base.py", line 1060, in _execute_clauseelement 2018-11-12 23:07:28.831 4224 ERROR oslo_db.sqlalchemy.engines compiled_sql, distilled_params 2018-11-12 23:07:28.831 4224 ERROR oslo_db.sqlalchemy.engines File "/usr/lib64/python2.7/site-packages/sqlalchemy/engine/base.py", line 1200, in _execute_context 2018-11-12 23:07:28.831 4224 ERROR oslo_db.sqlalchemy.engines context) 2018-11-12 23:07:28.831 4224 ERROR oslo_db.sqlalchemy.engines File "/usr/lib64/python2.7/site-packages/sqlalchemy/engine/base.py", line 1409, in _handle_dbapi_exception 2018-11-12 23:07:28.831 4224 ERROR oslo_db.sqlalchemy.engines util.raise_from_cause(newraise, exc_info) 2018-11-12 23:07:28.831 4224 ERROR oslo_db.sqlalchemy.engines File "/usr/lib64/python2.7/site-packages/sqlalchemy/util/compat.py", line 203, in raise_from_cause 2018-11-12 23:07:28.831 4224 ERROR oslo_db.sqlalchemy.engines reraise(type(exception), exception, tb=exc_tb, cause=cause) 2018-11-12 23:07:28.831 4224 ERROR oslo_db.sqlalchemy.engines File "/usr/lib64/python2.7/site-packages/sqlalchemy/engine/base.py", line 1193, in _execute_context 2018-11-12 23:07:28.831 4224 ERROR oslo_db.sqlalchemy.engines context) 2018-11-12 23:07:28.831 4224 ERROR oslo_db.sqlalchemy.engines File "/usr/lib64/python2.7/site-packages/sqlalchemy/engine/default.py", line 507, in do_execute 2018-11-12 23:07:28.831 4224 ERROR oslo_db.sqlalchemy.engines cursor.execute(statement, parameters) 2018-11-12 23:07:28.831 4224 ERROR oslo_db.sqlalchemy.engines File "/usr/lib/python2.7/site-packages/pymysql/cursors.py", line 166, in execute 2018-11-12 23:07:28.831 4224 ERROR oslo_db.sqlalchemy.engines result = self._query(query) 2018-11-12 23:07:28.831 4224 ERROR oslo_db.sqlalchemy.engines File "/usr/lib/python2.7/site-packages/pymysql/cursors.py", line 322, in _query 2018-11-12 23:07:28.831 4224 ERROR oslo_db.sqlalchemy.engines conn.query(q) 2018-11-12 23:07:28.831 4224 ERROR oslo_db.sqlalchemy.engines File "/usr/lib/python2.7/site-packages/pymysql/connections.py", line 856, in query 2018-11-12 23:07:28.831 4224 ERROR oslo_db.sqlalchemy.engines self._affected_rows = self._read_query_result(unbuffered=unbuffered) 2018-11-12 23:07:28.831 4224 ERROR oslo_db.sqlalchemy.engines File "/usr/lib/python2.7/site-packages/pymysql/connections.py", line 1057, in _read_query_result 2018-11-12 23:07:28.831 4224 ERROR oslo_db.sqlalchemy.engines result.read() 2018-11-12 23:07:28.831 4224 ERROR oslo_db.sqlalchemy.engines File "/usr/lib/python2.7/site-packages/pymysql/connections.py", line 1340, in read 2018-11-12 23:07:28.831 4224 ERROR oslo_db.sqlalchemy.engines first_packet = self.connection._read_packet() 2018-11-12 23:07:28.831 4224 ERROR oslo_db.sqlalchemy.engines File "/usr/lib/python2.7/site-packages/pymysql/connections.py", line 987, in _read_packet 2018-11-12 23:07:28.831 4224 ERROR oslo_db.sqlalchemy.engines packet_header = self._read_bytes(4) 2018-11-12 23:07:28.831 4224 ERROR oslo_db.sqlalchemy.engines File "/usr/lib/python2.7/site-packages/pymysql/connections.py", line 1033, in _read_bytes 2018-11-12 23:07:28.831 4224 ERROR oslo_db.sqlalchemy.engines CR.CR_SERVER_LOST, "Lost connection to MySQL server during query") 2018-11-12 23:07:28.831 4224 ERROR oslo_db.sqlalchemy.engines DBConnectionError: (pymysql.err.OperationalError) (2013, 'Lost connection to MySQL server during query') [SQL: u'SELECT 1'] (Background on this error at: http://sqlalche.me/e/e3q8) Anything went wrong while upgrade to queens in nova database ? # nova-manage db online_data_migrations /usr/lib/python2.7/site-packages/oslo_db/sqlalchemy/enginefacade.py:332: NotSupportedWarning: Configuration option(s) ['use_tpool'] not supported exception.NotSupportedWarning Running batches of 50 until complete 1 rows matched query migrate_instances_add_request_spec, 0 migrated +---------------------------------------------+--------------+-----------+ | Migration | Total Needed | Completed | +---------------------------------------------+--------------+-----------+ | delete_build_requests_with_no_instance_uuid | 0 | 0 | | migrate_aggregate_reset_autoincrement | 0 | 0 | | migrate_aggregates | 0 | 0 | | migrate_instance_groups_to_api_db | 0 | 0 | | migrate_instances_add_request_spec | 1 | 0 | | migrate_keypairs_to_api_db | 0 | 0 | | migrate_quota_classes_to_api_db | 0 | 0 | | migrate_quota_limits_to_api_db | 0 | 0 | | migration_migrate_to_uuid | 0 | 0 | | populate_missing_availability_zones | 0 | 0 | | populate_uuids | 0 | 0 | | service_uuids_online_data_migration | 0 | 0 | +---------------------------------------------+--------------+-----------+ Il giorno lun 12 nov 2018 alle ore 22:40 Slawomir Kaplonski < skapl...@redhat.com> ha scritto: > Hi, > > From logs which You attached it looks that Your neutron-metadata-agent > can’t connect to nova-api service. Please check if nova-metadata-api is > reachable from node where Your neutron-metadata-agent is running. > > > Wiadomość napisana przez Ignazio Cassano <ignaziocass...@gmail.com> w > dniu 12.11.2018, o godz. 22:34: > > > > Hello again, > > I have another installation of ocata . > > On ocata the metadata for a network id is displayed by ps -afe like this: > > /usr/bin/python2 /bin/neutron-ns-metadata-proxy > --pid_file=/var/lib/neutron/external/pids/c4731392-9b91-4663-adb3-b10b5ebcc4f1.pid > --metadata_proxy_socket=/var/lib/neutron/metadata_proxy > --network_id=c4731392-9b91-4663-adb3-b10b5ebcc4f1 > --state_path=/var/lib/neutron --metadata_port=80 --metadata_proxy_user=996 > --metadata_proxy_group=993 > --log-file=neutron-ns-metadata-proxy-c4731392-9b91-4663-adb3-b10b5ebcc4f1.log > --log-dir=/var/log/neutron > > > > On queens like this: > > haproxy -f > /var/lib/neutron/ns-metadata-proxy/e8ba8c09-a7dc-4a22-876e-b8d4187a23fe.conf > > > > Is it the correct behaviour ? > > Yes, that is correct. It was changed some time ago, see > https://bugs.launchpad.net/neutron/+bug/1524916 > > > > > Regards > > Ignazio > > > > > > > > Il giorno lun 12 nov 2018 alle ore 21:37 Slawomir Kaplonski < > skapl...@redhat.com> ha scritto: > > Hi, > > > > Can You share logs from Your haproxy-metadata-proxy service which is > running in qdhcp namespace? There should be some info about reason of those > errors 500. > > > > > Wiadomość napisana przez Ignazio Cassano <ignaziocass...@gmail.com> w > dniu 12.11.2018, o godz. 19:49: > > > > > > Hi All, > > > I upgraded manually my centos 7 openstack ocata to pike. > > > All worked fine. > > > Then I upgraded from pike to Queens and instances stopped to reach > metadata on 169.254.169.254 with error 500. > > > I am using isolated metadata true in my dhcp conf and in dhcp > namespace the port 80 is in listen. > > > Please, anyone can help me? > > > Regards > > > Ignazio > > > > > > _______________________________________________ > > > OpenStack-operators mailing list > > > OpenStack-operators@lists.openstack.org > > > > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators > > > > — > > Slawek Kaplonski > > Senior software engineer > > Red Hat > > > > — > Slawek Kaplonski > Senior software engineer > Red Hat > >
_______________________________________________ OpenStack-operators mailing list OpenStack-operators@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators
