To me, that looks like your network configuration has a problem and
fuel/nailgun is bubbling up the error when trying to provision neutron on the
node.
Since it is a fresh install, I would consider starting over, and validating
network config in fuel before restarting your deployment.
I don't have a ton of experience with fuel, but that's my best guess based on
my limited exposure to it and what I see in your logs/traceback. That
highlighted line is not a neutron call, that's within fuel, presumably when it
is trying to provision neutron.
-Chris
> On Feb 23, 2017, at 6:33 PM, Mario Romascanu <ro...@mindtrek.dnsalias.net>
> wrote:
>
> Hi
>
> I was able to get to a certain point in deploying an environment on a fresh
> installation of Mirantis OpenStack 9.0 but I am stuck with this bizarre error
> in the controller deployment
> It does not seem to be a communication issue - but rather a logical one - an
> error 500 on a particular REST call URI to neutron and 200 OK otherwise on
> different neutron URI.
> This occurs after deploying Ubuntu and Openstack on the Controller node
> I am at a loss of what the issue is here - any suggestion welcome
>
> "Deployment has failed. All nodes are finished. Failed tasks:
> Task[connectivity_tests/4] Stopping the deployment process!"
> "Failed to deploy node 'MOS_Controller (1e:c1)': Task connectivity_tests
> failed on node 4"
> [root@fuel ~]# fuel node
> id | status | name | cluster | ip | mac
> | roles | pending_roles | online | group_id
> ---+---------+------------------------+---------+-----------+-------------------+------------+---------------+--------+---------
> 3 | stopped | MOS_Compute (46:4a) | 1 | 10.20.0.4 |
> 00:50:56:98:46:4a | compute | | 1 | 1
> 4 | error | MOS_Controller (1e:c1) | 1 | 10.20.0.3 |
> 00:50:56:98:1e:c1 | controller | | 1 | 1
>
> An in the logs I see the traceback partially included - the root cause is
> highlighted
>
> Date Level Message
> 2017-02-23 20:56:14 ERROR [7fad42058880] (base) Unexpected exception
> occured
> Traceback (most recent call last):
> File "/usr/lib/python2.7/site-packages/nailgun/api/v1/handlers/base.py",
> line 297, in handle_errors
> return func(cls, *args, **kwargs)
> File "<string>", line 2, in GET
> File "/usr/lib/python2.7/site-packages/nailgun/api/v1/handlers/base.py",
> line 355, in validate
> return func(cls, *args, **kwargs)
> File "<string>", line 2, in GET
> File "/usr/lib/python2.7/site-packages/nailgun/api/v1/handlers/base.py",
> line 381, in serialize
> resp = func(cls, *args, **kwargs)
> File
> "/usr/lib/python2.7/site-packages/nailgun/api/v1/handlers/network_configuration.py",
> line 92, in GET
> return self.serializer.serialize_for_cluster(cluster)
> File
> "/usr/lib/python2.7/site-packages/nailgun/objects/serializers/network_configuration.py",
> line 112, in serialize_for_cluster
> result = cls.serialize_net_groups_and_vips(cluster, allocate_vips)
> File
> "/usr/lib/python2.7/site-packages/nailgun/objects/serializers/network_configuration.py",
> line 42, in serialize_net_groups_and_vips
> nets = cluster.network_groups + [
> File
> "/usr/lib/python2.7/site-packages/nailgun/db/sqlalchemy/models/cluster.py",
> line 159, in network_groups
> net_list.extend(ng.networks)
> File "/usr/lib64/python2.7/site-packages/sqlalchemy/orm/attributes.py",
> line 237, in __get__
> return self.impl.get(instance_state(instance), dict_)
> File "/usr/lib64/python2.7/site-packages/sqlalchemy/orm/attributes.py",
> line 583, in get
> value = self.callable_(state, passive)
> File "/usr/lib64/python2.7/site-packages/sqlalchemy/orm/strategies.py",
> line 532, in _load_for_state
> return self._emit_lazyload(session, state, ident_key, passive)
> File "<string>", line 1, in <lambda>
> File "/usr/lib64/python2.7/site-packages/sqlalchemy/orm/strategies.py",
> line 602, in _emit_lazyload
> result = q.all()
> File "/usr/lib64/python2.7/site-packages/sqlalchemy/orm/query.py", line
> 2588, in all
> return list(self)
> File "/usr/lib64/python2.7/site-packages/sqlalchemy/orm/loading.py", line
> 86, in instances
> util.raise_from_cause(err)
> File "/usr/lib64/python2.7/site-packages/sqlalchemy/util/compat.py", line
> 200, in raise_from_cause
> reraise(type(exception), exception, tb=exc_tb, cause=cause)
> File "/usr/lib64/python2.7/site-packages/sqlalchemy/orm/loading.py", line
> 71, in instances
> rows = [proc(row) for row in fetch]
> File "/usr/lib64/python2.7/site-packages/sqlalchemy/orm/loading.py", line
> 428, in _instance
> loaded_instance, populate_existing, populators)
> File "/usr/lib64/python2.7/site-packages/sqlalchemy/orm/loading.py", line
> 486, in _populate_full
> dict_[key] = getter(row)
> File "/usr/lib64/python2.7/site-packages/sqlalchemy/sql/type_api.py", line
> 1030, in process
> return process_value(value, dialect)
> File
> "/usr/lib/python2.7/site-packages/nailgun/db/sqlalchemy/models/fields.py",
> line 32, in process_result_value
> value = jsonutils.loads(value)
> File "/usr/lib/python2.7/site-packages/oslo_serialization/jsonutils.py",
> line 229, in loads
> return json.loads(encodeutils.safe_decode(s, encoding), **kwargs)
> File "/usr/lib64/python2.7/json/__init__.py", line 338, in loads
> return _default_decoder.decode(s)
> File "/usr/lib64/python2.7/json/decoder.py", line 365, in decode
> obj, end = self.raw_decode(s, idx=_w(s, 0).end())
> File "/usr/lib64/python2.7/json/decoder.py", line 381, in raw_decode
> obj, end = self.scan_once(s, idx)
> ValueError: Expecting ',' delimiter: line 1 column 320 (char 319)
> [pid: 3540|app: 0|req: 656/1262] 192.168.198.222 () {50 vars in 905 bytes}
> [Thu Feb 23 20:56:14 2017] GET
> /api/clusters/1/network_configuration/neutron?_=1487883365500 => generated 39
> bytes in 11 msecs (HTTP/1.1 500) 5 headers in 223 bytes (2 switches on core 0)
> [pid: 3539|app: 0|req: 607/1263] 192.168.198.222 () {50 vars in 841 bytes}
> [Thu Feb 23 20:56:15 2017] GET /api/clusters?_=1487883365502 => generated 627
> bytes in 4 msecs (HTTP/1.1 200) 4 headers in 185 bytes (2 switches on core 0)
> [pid: 3539|app: 0|req: 608/1264] 192.168.198.222 () {50 vars in 835 bytes}
> [Thu Feb 23 20:56:15 2017] GET /api/tasks?_=1487883365504 => generated 286
> bytes in 4 msecs (HTTP/1.1 200) 4 headers in 185 bytes (2 switches on core 0)
> [pid: 3540|app: 0|req: 657/1265] 192.168.198.222 () {50 vars in 835 bytes}
> [Thu Feb 23 20:56:15 2017] GET /api/nodes?_=1487883365503 => generated 2
> bytes in 13 msecs (HTTP/1.1 200) 4 headers in 185 bytes (2 switches on core 0)
> [pid: 3540|app: 0|req: 658/1266] 192.168.198.222 () {50 vars in 851 bytes}
> [Thu Feb 23 20:56:29 2017] GET /api/notifications?_=1487883365506 =>
> generated 769 bytes in 4 msecs (HTTP/1.1 200) 4 headers in 185 bytes (2
> switches on core 0)
> [pid: 3539|app: 0|req: 609/1267] 192.168.198.222 () {50 vars in 869 bytes}
> [Thu Feb 23 20:56:29 2017] GET /api/nodes/allocation/stats?_=1487883365505 =>
> generated 30 bytes in 17 msecs (HTTP/1.1 200) 4 headers in 185 bytes (2
> switches on core 0)
> [pid: 3539|app: 0|req: 610/1268] 192.168.198.222 () {50 vars in 867 bytes}
> [Thu Feb 23 20:56:38 2017] GET /api/clusters/1/attributes?_=1487883365508 =>
> generated 35673 bytes in 20 msecs (HTTP/1.1 200) 4 headers in 185 bytes (2
> switches on core 0)
> [pid: 3539|app: 0|req: 611/1269] 192.168.198.222 () {50 vars in 871 bytes}
> [Thu Feb 23 20:56:38 2017] GET /api/clusters/1/plugin_links?_=1487883365510
> => generated 2 bytes in 4 msecs (HTTP/1.1 200) 4 headers in 185 bytes (2
> switches on core 0)
> [pid: 3539|app: 0|req: 612/1270] 192.168.198.222 () {50 vars in 861 bytes}
> [Thu Feb 23 20:56:38 2017] GET /api/nodes?cluster_id=1&_=1487883365511 =>
> generated 2 bytes in 12 msecs (HTTP/1.1 200) 4 headers in 185 bytes (2
> switches on core 0)
> [pid: 3539|app: 0|req: 613/1271] 192.168.198.222 () {50 vars in 845 bytes}
> [Thu Feb 23 20:56:38 2017] GET /api/clusters/1?_=1487883365507 => generated
> 625 bytes in 4 msecs (HTTP/1.1 200) 4 headers in 185 bytes (2 switches on
> core 0)
> [pid: 3539|app: 0|req: 614/1272] 192.168.198.222 () {50 vars in 861 bytes}
> [Thu Feb 23 20:56:38 2017] GET /api/tasks?cluster_id=1&_=1487883365512 =>
> generated 2 bytes in 3 msecs (HTTP/1.1 200) 4 headers in 185 bytes (2
> switches on core 0)
> [pid: 3539|app: 0|req: 615/1273] 192.168.198.222 () {50 vars in 871 bytes}
> [Thu Feb 23 20:56:38 2017] GET /api/nodegroups?cluster_id=1&_=1487883365513
> => generated 67 bytes in 3 msecs (HTTP/1.1 200) 4 headers in 185 bytes (2
> switches on core 0)
> [pid: 3540|app: 0|req: 659/1274] 192.168.198.222 () {50 vars in 857 bytes}
> [Thu Feb 23 20:56:38 2017] GET /api/clusters/1/roles?_=1487883365509 =>
> generated 7255 bytes in 67 msecs (HTTP/1.1 200) 4 headers in 185 bytes (2
> switches on core 0)
> [pid: 3540|app: 0|req: 660/1275] 192.168.198.222 () {50 vars in 845 bytes}
> [Thu Feb 23 20:56:38 2017] GET /api/releases/2?_=1487883365515 => generated
> 45381 bytes in 7 msecs (HTTP/1.1 200) 4 headers in 185 bytes (2 switches on
> core 0)
> [pid: 3539|app: 0|req: 616/1276] 192.168.198.222 () {50 vars in 905 bytes}
> [Thu Feb 23 20:56:38 2017] GET
> /api/clusters/1/network_configuration/neutron?_=1487883365514 => generated
> 2441 bytes in 29 msecs (HTTP/1.1 200) 4 headers in 185 bytes (2 switches on
> core 0)
> [pid: 3540|app: 0|req: 661/1277] 192.168.198.222 () {50 vars in 887 bytes}
> [Thu Feb 23 20:56:39 2017] GET
> /api/v1/clusters/1/vmware_attributes?_=1487883365516 => generated 4068 bytes
> in 21 msecs (HTTP/1.1 200) 4 headers in 185 bytes (2 switches on core 0)
> [pid: 3539|app: 0|req: 617/1278] 192.168.198.222 () {50 vars in 851 bytes}
> [Thu Feb 23 20:56:49 2017] GET /api/notifications?_=1487883365518 =>
> generated 769 bytes in 3 msecs (HTTP/1.1 200) 4 headers in 185 bytes (2
> switches on core 0)
> [pid: 3540|app: 0|req: 662/1279] 192.168.198.222 () {50 vars in 869 bytes}
> [Thu Feb 23 20:56:49 2017] GET /api/nodes/allocation/stats?_=1487883365517 =>
> generated 30 bytes in 16 msecs (HTTP/1.1 200) 4 headers in 185 bytes (2
> switches on core 0)
> [pid: 3539|app: 0|req: 618/1280] 192.168.198.222 () {50 vars in 851 bytes}
> [Thu Feb 23 20:57:09 2017] GET /api/notifications?_=1487883365520 =>
> generated 769 bytes in 6 msecs (HTTP/1.1 200) 4 headers in 185 bytes (2
> switches on core 0)
> [pid: 3540|app: 0|req: 663/1281] 192.168.198.222 () {50 vars in 869 bytes}
> [Thu Feb 23 20:57:09 2017] GET /api/nodes/allocation/stats?_=1487883365519 =>
> generated 30 bytes in 17 msecs (HTTP/1.1 200) 4 headers in 185 bytes (2
> switches on core 0)
> [pid: 3539|app: 0|req: 619/1282] 192.168.198.222 () {50 vars in 851 bytes}
> [Thu Feb 23 20:57:29 2017] GET /api/notifications?_=1487883365522 =>
> generated 769 bytes in 4 msecs (HTTP/1.1 200) 4 headers in 185 bytes (2
> switches on core 0)
> [pid: 3540|app: 0|req: 664/1283] 192.168.198.222 () {50 vars in 869 bytes}
> [Thu Feb 23 20:57:29 2017] GET /api/nodes/allocation/stats?_=1487883365521 =>
> generated 30 bytes in 19 msecs (HTTP/1.1 200) 4 headers in 185 bytes (2
> switches on core 0)
> [pid: 3539|app: 0|req: 620/1284] 192.168.198.222 () {50 vars in 811 bytes}
> [Thu Feb 23 20:57:44 2017] GET /api/notifications?_=1487883365524 =>
> generated 769 bytes in 27 msecs (HTTP/1.1 200) 4 headers in 185 bytes (2
> switches on core 0)
> [pid: 3539|app: 0|req: 621/1285] 192.168.198.222 () {50 vars in 839 bytes}
> [Thu Feb 23 20:57:44 2017] GET /api/version?_=1487883365525 => generated 110
> bytes in 2 msecs (HTTP/1.1 200) 4 headers in 185 bytes (2 switches on core 0)
> [pid: 3539|app: 0|req: 622/1286] 192.168.198.222 () {50 vars in 841 bytes}
> [Thu Feb 23 20:57:44 2017] GET /api/settings?_=1487883365526 => generated 563
> bytes in 3 msecs (HTTP/1.1 200) 4 headers in 185 bytes (2 switches on core 0)
> [pid: 3539|app: 0|req: 623/1287] 192.168.198.222 () {50 vars in 841 bytes}
> [Thu Feb 23 20:57:44 2017] GET /api/clusters?_=1487883365527 => generated 627
> bytes in 4 msecs (HTTP/1.1 200) 4 headers in 185 bytes (2 switches on core 0)
> [pid: 3539|app: 0|req: 624/1288] 192.168.198.222 () {50 vars in 835 bytes}
> [Thu Feb 23 20:57:44 2017] GET /api/tasks?_=1487883365529 => generated 286
> bytes in 4 msecs (HTTP/1.1 200) 4 headers in 185 bytes (2 switches on core 0)
> [pid: 3539|app: 0|req: 625/1289] 192.168.198.222 () {50 vars in 835 bytes}
> [Thu Feb 23 20:57:44 2017] GET /api/nodes?_=1487883365528 => generated 2
> bytes in 13 msecs (HTTP/1.1 200) 4 headers in 185 bytes (2 switches on core 0)
> [pid: 3540|app: 0|req: 665/1290] 192.168.198.222 () {50 vars in 829 bytes}
> [Thu Feb 23 20:57:44 2017] GET /api/nodes/allocation/stats?_=1487883365523 =>
> generated 30 bytes in 81 msecs (HTTP/1.1 200) 4 headers in 185 bytes (2
> switches on core 0)
> [pid: 3539|app: 0|req: 626/1291] 192.168.198.222 () {50 vars in 851 bytes}
> [Thu Feb 23 20:57:44 2017] GET /api/notifications?_=1487883365531 =>
> generated 769 bytes in 4 msecs (HTTP/1.1 200) 4 headers in 185 bytes (2
> switches on core 0)
> [pid: 3540|app: 0|req: 666/1292] 192.168.198.222 () {50 vars in 869 bytes}
> [Thu Feb 23 20:57:44 2017] GET /api/nodes/allocation/stats?_=1487883365530 =>
> generated 30 bytes in 16 msecs (HTTP/1.1 200) 4 headers in 185 bytes (2
> switches on core 0)
> [pid: 3540|app: 0|req: 667/1293] 192.168.198.222 () {50 vars in 867 bytes}
> [Thu Feb 23 20:57:46 2017] GET /api/clusters/1/attributes?_=1487883365533 =>
> generated 35673 bytes in 21 msecs (HTTP/1.1 200) 4 headers in 185 bytes (2
> switches on core 0)
> [pid: 3540|app: 0|req: 668/1294] 192.168.198.222 () {50 vars in 871 bytes}
> [Thu Feb 23 20:57:46 2017] GET /api/clusters/1/plugin_links?_=1487883365535
> => generated 2 bytes in 4 msecs (HTTP/1.1 200) 4 headers in 185 bytes (2
> switches on core 0)
> [pid: 3540|app: 0|req: 669/1295] 192.168.198.222 () {50 vars in 845 bytes}
> [Thu Feb 23 20:57:46 2017] GET /api/clusters/1?_=1487883365532 => generated
> 625 bytes in 3 msecs (HTTP/1.1 200) 4 headers in 185 bytes (2 switches on
> core 0)
> [pid: 3540|app: 0|req: 670/1296] 192.168.198.222 () {50 vars in 861 bytes}
> [Thu Feb 23 20:57:46 2017] GET /api/nodes?cluster_id=1&_=1487883365536 =>
> generated 2 bytes in 12 msecs (HTTP/1.1 200) 4 headers in 185 bytes (2
> switches on core 0)
> [pid: 3540|app: 0|req: 671/1297] 192.168.198.222 () {50 vars in 861 bytes}
> [Thu Feb 23 20:57:46 2017] GET /api/tasks?cluster_id=1&_=1487883365537 =>
> generated 2 bytes in 3 msecs (HTTP/1.1 200) 4 headers in 185 bytes (2
> switches on core 0)
> [pid: 3540|app: 0|req: 672/1298] 192.168.198.222 () {50 vars in 871 bytes}
> [Thu Feb 23 20:57:46 2017] GET /api/nodegroups?cluster_id=1&_=1487883365538
> => generated 67 bytes in 2 msecs (HTTP/1.1 200) 4 headers in 185 bytes (2
> switches on core 0)
> [pid: 3539|app: 0|req: 627/1299] 192.168.198.222 () {50 vars in 857 bytes}
> [Thu Feb 23 20:57:46 2017] GET /api/clusters/1/roles?_=1487883365534 =>
> generated 7255 bytes in 70 msecs (HTTP/1.1 200) 4 headers in 185 bytes (2
> switches on core 0)
> [pid: 3539|app: 0|req: 628/1300] 192.168.198.222 () {50 vars in 845 bytes}
> [Thu Feb 23 20:57:46 2017] GET /api/releases/2?_=1487883365540 => generated
> 45381 bytes in 14 msecs (HTTP/1.1 200) 4 headers in 185 bytes (2 switches on
> core 0)
> [pid: 3540|app: 0|req: 673/1301] 192.168.198.222 () {50 vars in 905 bytes}
> [Thu Feb 23 20:57:46 2017] GET
> /api/clusters/1/network_configuration/neutron?_=1487883365539 => generated
> 2441 bytes in 40 msecs (HTTP/1.1 200) 4 headers in 185 bytes (2 switches on
> core 0)
> [pid: 3540|app: 0|req: 674/1302] 192.168.198.222 () {50 vars in 887 bytes}
> [Thu Feb 23 20:57:46 2017] GET
> /api/v1/clusters/1/vmware_attributes?_=1487883365541 => generated 4068 bytes
> in 19 msecs (HTTP/1.1 200) 4 headers in 185 bytes (2 switches on core 0)
> [pid: 3539|app: 0|req: 629/1303] 192.168.198.222 () {50 vars in 851 bytes}
> [Thu Feb 23 20:58:04 2017] GET /api/notifications?_=1487883365543 =>
> generated 769 bytes in 3 msecs (HTTP/1.1 200) 4 headers in 185 bytes (2
> switches on core 0)
> [pid: 3540|app: 0|req: 675/1304] 192.168.198.222 () {50 vars in 869 bytes}
> [Thu Feb 23 20:58:04 2017] GET /api/nodes/allocation/stats?_=1487883365542 =>
> generated 30 bytes in 16 msecs (HTTP/1.1 200) 4 headers in 185 bytes (2
> switches on core 0)
> [pid: 3539|app: 0|req: 630/1305] 192.168.198.222 () {50 vars in 851 bytes}
> [Thu Feb 23 20:58:24 2017] GET /api/notifications?_=1487883365545 =>
> generated 769 bytes in 5 msecs (HTTP/1.1 200) 4 headers in 185 bytes (2
> switches on core 0)
> [pid: 3540|app: 0|req: 676/1306] 192.168.198.222 () {50 vars in 869 bytes}
> [Thu Feb 23 20:58:24 2017] GET /api/nodes/allocation/stats?_=1487883365544 =>
> generated 30 bytes in 19 msecs (HTTP/1.1 200) 4 headers in 185 bytes (2
> switches on core 0)
> [pid: 3539|app: 0|req: 631/1307] 192.168.198.222 () {50 vars in 851 bytes}
> [Thu Feb 23 20:58:44 2017] GET /api/notifications?_=1487883365547 =>
> generated 769 bytes in 3 msecs (HTTP/1.1 200) 4 headers in 185 bytes (2
> switches on core 0)
> [pid: 3540|app: 0|req: 677/1308] 192.168.198.222 () {50 vars in 869 bytes}
> [Thu Feb 23 20:58:44 2017] GET /api/nodes/allocation/stats?_=1487883365546 =>
> generated 30 bytes in 16 msecs (HTTP/1.1 200) 4 headers in 185 bytes (2
> switches on core 0)
> [pid: 3540|app: 0|req: 678/1309] 192.168.198.222 () {50 vars in 851 bytes}
> [Thu Feb 23 20:59:04 2017] GET /api/notifications?_=1487883365549 =>
> generated 769 bytes in 5 msecs (HTTP/1.1 200) 4 headers in 185 bytes (2
> switches on core 0)
> [pid: 3539|app: 0|req: 632/1310] 192.168.198.222 () {50 vars in 869 bytes}
> [Thu Feb 23 20:59:04 2017] GET /api/nodes/allocation/stats?_=1487883365548 =>
> generated 30 bytes in 22 msecs (HTTP/1.1 200) 4 headers in 185 bytes (2
> switches on core 0)
> [pid: 3539|app: 0|req: 633/1311] 192.168.198.222 () {50 vars in 851 bytes}
> [Thu Feb 23 20:59:24 2017] GET /api/notifications?_=1487883365551 =>
> generated 769 bytes in 5 msecs (HTTP/1.1 200) 4 headers in 185 bytes (2
> switches on core 0)
>
>
>
>
> This email has been checked for viruses by Avast antivirus software.
> www.avast.com
>
>
> _______________________________________________
> Mailing list: http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
> Post to : openstack@lists.openstack.org
> Unsubscribe : http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
_______________________________________________
Mailing list: http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
Post to : openstack@lists.openstack.org
Unsubscribe : http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
