Public bug reported:
With the current implement, under certain curcumstance, router
information(including internal-stored data, iptables settings) kept and
set up by metering-agent could be inconsistent with what the neutron-
server suppose it to be. And that cause the metering-agent to run into a
problematic status.
Version:
Neutron master commit 7327e8c21(Merge "Fix update target tenant RBAC external
path")
Steps to reproduce:
1. Create internal net, subnet, router. Set external gateway for router, add
interface to the router for the created subnet.
2. Create neutron metering label and rule.
3. Make sure the created router is added to an active l3-agent.
4. Create another router and set its external gateway.
5. Use 'neutron l3-agent-router-remove' to remove the first router from the l3
agent.
6. Use 'neutron l3-agent-router-add' to add the first router back to the l3
agent by which it was hosted.
(In the following logs, the first created router's id is
0dd35572-735a-4ecd-8429-e047ed385100, the second one is
392820a7-3908-49eb-98a3-967f1638432f)
Logs in metering-agent.log after step 4:
2016-05-11 18:32:49.915 26409 DEBUG neutron.agent.linux.utils
[req-0515c6f8-cbd1-47d4-a45e-f89e6b6e97b4 - - - - -] Running command: ['sudo',
'ip', 'netns', 'exec', 'qrouter-392820a7-3908-49eb-98a3-967f1638432f',
'iptables-save'] create_process
/usr/local/lib/python2.7/dist-packages/neutron/agent/linux/utils.py:82
2016-05-11 18:32:49.926 26409 ERROR neutron.agent.linux.utils
[req-0515c6f8-cbd1-47d4-a45e-f89e6b6e97b4 - - - - -] Exit code: 1; Stdin: ;
Stdout: ; Stderr: Cannot open network namespace
"qrouter-392820a7-3908-49eb-98a3-967f1638432f": No such file or directory
2016-05-11 18:32:49.926 26409 DEBUG oslo_concurrency.lockutils
[req-0515c6f8-cbd1-47d4-a45e-f89e6b6e97b4 - - - - -] Releasing semaphore
"iptables-qrouter-392820a7-3908-49eb-98a3-967f1638432f" lock
/usr/local/lib/python2.7/dist-packages/oslo_concurrency/lockutils.py:225
2016-05-11 18:32:49.927 26409 ERROR
neutron.services.metering.agents.metering_agent
[req-0515c6f8-cbd1-47d4-a45e-f89e6b6e97b4 - - - - -] Driver
neutron.services.metering.drivers.iptables.iptables_driver.IptablesMeteringDriver:update_routers
runtime error
2016-05-11 18:32:49.927 26409 ERROR
neutron.services.metering.agents.metering_agent Traceback (most recent call
last):
2016-05-11 18:32:49.927 26409 ERROR
neutron.services.metering.agents.metering_agent File
"/usr/local/lib/python2.7/dist-packages/neutron/services/metering/agents/metering_agent.py",
line 177, in _invoke_driver
2016-05-11 18:32:49.927 26409 ERROR
neutron.services.metering.agents.metering_agent return
getattr(self.metering_driver, func_name)(context, meterings)
2016-05-11 18:32:49.927 26409 ERROR
neutron.services.metering.agents.metering_agent File
"/usr/local/lib/python2.7/dist-packages/oslo_log/helpers.py", line 48, in
wrapper
2016-05-11 18:32:49.927 26409 ERROR
neutron.services.metering.agents.metering_agent return method(*args,
**kwargs)
2016-05-11 18:32:49.927 26409 ERROR
neutron.services.metering.agents.metering_agent File
"/usr/local/lib/python2.7/dist-packages/neutron/services/metering/drivers/iptables/iptables_driver.py",
line 125, in update_routers
2016-05-11 18:32:49.927 26409 ERROR
neutron.services.metering.agents.metering_agent
self._process_associate_metering_label(router)
2016-05-11 18:32:49.927 26409 ERROR
neutron.services.metering.agents.metering_agent File
"/usr/local/lib/python2.7/dist-packages/neutron/services/metering/drivers/iptables/iptables_driver.py",
line 228, in _process_associate_metering_label
2016-05-11 18:32:49.927 26409 ERROR
neutron.services.metering.agents.metering_agent
rm.metering_labels[label_id] = label
2016-05-11 18:32:49.927 26409 ERROR
neutron.services.metering.agents.metering_agent File
"/usr/local/lib/python2.7/dist-packages/neutron/services/metering/drivers/iptables/iptables_driver.py",
line 58, in __exit__
2016-05-11 18:32:49.927 26409 ERROR
neutron.services.metering.agents.metering_agent self.im.apply()
2016-05-11 18:32:49.927 26409 ERROR
neutron.services.metering.agents.metering_agent File
"/usr/local/lib/python2.7/dist-packages/neutron/agent/linux/iptables_manager.py",
line 431, in apply
2016-05-11 18:32:49.927 26409 ERROR
neutron.services.metering.agents.metering_agent return self._apply()
2016-05-11 18:32:49.927 26409 ERROR
neutron.services.metering.agents.metering_agent File
"/usr/local/lib/python2.7/dist-packages/neutron/agent/linux/iptables_manager.py",
line 439, in _apply
2016-05-11 18:32:49.927 26409 ERROR
neutron.services.metering.agents.metering_agent return
self._apply_synchronized()
2016-05-11 18:32:49.927 26409 ERROR
neutron.services.metering.agents.metering_agent File
"/usr/local/lib/python2.7/dist-packages/neutron/agent/linux/iptables_manager.py",
line 465, in _apply_synchronized
2016-05-11 18:32:49.927 26409 ERROR
neutron.services.metering.agents.metering_agent save_output =
self.execute(args, run_as_root=True)
2016-05-11 18:32:49.927 26409 ERROR
neutron.services.metering.agents.metering_agent File
"/usr/local/lib/python2.7/dist-packages/neutron/agent/linux/utils.py", line
137, in execute
2016-05-11 18:32:49.927 26409 ERROR
neutron.services.metering.agents.metering_agent raise RuntimeError(msg)
2016-05-11 18:32:49.927 26409 ERROR
neutron.services.metering.agents.metering_agent RuntimeError: Exit code: 1;
Stdin: ; Stdout: ; Stderr: Cannot open network namespace
"qrouter-392820a7-3908-49eb-98a3-967f1638432f": No such file or directory
2016-05-11 18:32:49.927 26409 ERROR
neutron.services.metering.agents.metering_agent
2016-05-11 18:32:49.927 26409 ERROR
neutron.services.metering.agents.metering_agent
...Skipping...
2016-05-11 18:33:06.776 26409 DEBUG neutron.agent.linux.utils [-] Running
command: ['sudo', 'ip', 'netns', 'exec',
'qrouter-392820a7-3908-49eb-98a3-967f1638432f', 'iptables', '-t', 'filter',
'-L', 'neutron-meter-l-1811c9b9-f9b', '-n', '-v', '-x', '-Z'] create_process
/usr/local/lib/python2.7/dist-packages/neutron/agent/linux/utils.py:82
2016-05-11 18:33:06.787 26409 ERROR neutron.agent.linux.utils [-] Exit code: 1;
Stdin: ; Stdout: ; Stderr: Cannot open network namespace
"qrouter-392820a7-3908-49eb-98a3-967f1638432f": No such file or directory
2016-05-11 18:33:06.788 26409 ERROR
neutron.services.metering.drivers.iptables.iptables_driver [-] Failed to get
traffic counters, router: {u'status': u'ACTIVE', u'name': u'router-two',
u'gw_port_id': u'729fba34-4aec-415d-bef8-b93b34e69536', u'admin_state_up':
False, u'tenant_id': u'e60fad96858e431cb3da4f0c1ab2d733', u'_metering_labels':
[{u'rules': [{u'remote_ip_prefix': u'192.168.88.0/24', u'direction': u'egress',
u'metering_label_id': u'1811c9b9-f9b0-40a6-839a-250b20674860', u'id':
u'007e3f9e-4e7c-439d-a606-db6ad0ae11ca', u'excluded': False}], u'id':
u'1811c9b9-f9b0-40a6-839a-250b20674860'}, {u'rules': [{u'remote_ip_prefix':
u'192.168.88.0/24', u'direction': u'ingress', u'metering_label_id':
u'd05ce8fb-c013-4ae4-bac5-de1be3f65331', u'id':
u'6c580a70-5dec-4225-ade8-44e03fdc9f1b', u'excluded': False}], u'id':
u'd05ce8fb-c013-4ae4-bac5-de1be3f65331'}], u'id':
u'392820a7-3908-49eb-98a3-967f1638432f'}
2016-05-11 18:33:06.788 26409 ERROR
neutron.services.metering.drivers.iptables.iptables_driver Traceback (most
recent call last):
2016-05-11 18:33:06.788 26409 ERROR
neutron.services.metering.drivers.iptables.iptables_driver File
"/usr/local/lib/python2.7/dist-packages/neutron/services/metering/drivers/iptables/iptables_driver.py",
line 359, in get_traffic_counters
2016-05-11 18:33:06.788 26409 ERROR
neutron.services.metering.drivers.iptables.iptables_driver chain,
wrap=False, zero=True)
2016-05-11 18:33:06.788 26409 ERROR
neutron.services.metering.drivers.iptables.iptables_driver File
"/usr/local/lib/python2.7/dist-packages/neutron/agent/linux/iptables_manager.py",
line 669, in get_traffic_counters
2016-05-11 18:33:06.788 26409 ERROR
neutron.services.metering.drivers.iptables.iptables_driver current_table =
self.execute(args, run_as_root=True)
2016-05-11 18:33:06.788 26409 ERROR
neutron.services.metering.drivers.iptables.iptables_driver File
"/usr/local/lib/python2.7/dist-packages/neutron/agent/linux/utils.py", line
137, in execute
2016-05-11 18:33:06.788 26409 ERROR
neutron.services.metering.drivers.iptables.iptables_driver raise
RuntimeError(msg)
2016-05-11 18:33:06.788 26409 ERROR
neutron.services.metering.drivers.iptables.iptables_driver RuntimeError: Exit
code: 1; Stdin: ; Stdout: ; Stderr: Cannot open network namespace
"qrouter-392820a7-3908-49eb-98a3-967f1638432f": No such file or directory
2016-05-11 18:33:06.788 26409 ERROR
neutron.services.metering.drivers.iptables.iptables_driver
2016-05-11 18:33:06.788 26409 ERROR
neutron.services.metering.drivers.iptables.iptables_driver
2016-05-11 18:33:06.790 26409 DEBUG neutron.agent.linux.utils [-] Running
command: ['sudo', 'ip', 'netns', 'exec',
'qrouter-392820a7-3908-49eb-98a3-967f1638432f', 'iptables', '-t', 'filter',
'-L', 'neutron-meter-l-d05ce8fb-c01', '-n', '-v', '-x', '-Z'] create_process
/usr/local/lib/python2.7/dist-packages/neutron/agent/linux/utils.py:82
Logs in metering-agent.log after step 5:
2016-05-11 18:33:36.805 26409 ERROR
neutron.services.metering.drivers.iptables.iptables_driver [-] Failed to get
traffic counters, router: {u'status': u'ACTIVE', u'name': u'router-two',
u'gw_port_id': u'729fba34-4aec-415d-bef8-b93b34e69536', u'admin_state_up':
False, u'tenant_id': u'e60fad96858e431cb3da4f0c1ab2d733', u'_metering_labels':
[{u'rules': [{u'remote_ip_prefix': u'192.168.88.0/24', u'direction': u'egress',
u'metering_label_id': u'1811c9b9-f9b0-40a6-839a-250b20674860', u'id':
u'007e3f9e-4e7c-439d-a606-db6ad0ae11ca', u'excluded': False}], u'id':
u'1811c9b9-f9b0-40a6-839a-250b20674860'}, {u'rules': [{u'remote_ip_prefix':
u'192.168.88.0/24', u'direction': u'ingress', u'metering_label_id':
u'd05ce8fb-c013-4ae4-bac5-de1be3f65331', u'id':
u'6c580a70-5dec-4225-ade8-44e03fdc9f1b', u'excluded': False}], u'id':
u'd05ce8fb-c013-4ae4-bac5-de1be3f65331'}], u'id':
u'392820a7-3908-49eb-98a3-967f1638432f'}
2016-05-11 18:33:36.805 26409 ERROR
neutron.services.metering.drivers.iptables.iptables_driver Traceback (most
recent call last):
2016-05-11 18:33:36.805 26409 ERROR
neutron.services.metering.drivers.iptables.iptables_driver File
"/usr/local/lib/python2.7/dist-packages/neutron/services/metering/drivers/iptables/iptables_driver.py",
line 359, in get_traffic_counters
2016-05-11 18:33:36.805 26409 ERROR
neutron.services.metering.drivers.iptables.iptables_driver chain,
wrap=False, zero=True)
2016-05-11 18:33:36.805 26409 ERROR
neutron.services.metering.drivers.iptables.iptables_driver File
"/usr/local/lib/python2.7/dist-packages/neutron/agent/linux/iptables_manager.py",
line 669, in get_traffic_counters
2016-05-11 18:33:36.805 26409 ERROR
neutron.services.metering.drivers.iptables.iptables_driver current_table =
self.execute(args, run_as_root=True)
2016-05-11 18:33:36.805 26409 ERROR
neutron.services.metering.drivers.iptables.iptables_driver File
"/usr/local/lib/python2.7/dist-packages/neutron/agent/linux/utils.py", line
137, in execute
2016-05-11 18:33:36.805 26409 ERROR
neutron.services.metering.drivers.iptables.iptables_driver raise
RuntimeError(msg)
2016-05-11 18:33:36.805 26409 ERROR
neutron.services.metering.drivers.iptables.iptables_driver RuntimeError: Exit
code: 1; Stdin: ; Stdout: ; Stderr: Cannot open network namespace
"qrouter-392820a7-3908-49eb-98a3-967f1638432f": No such file or directory
2016-05-11 18:33:36.805 26409 ERROR
neutron.services.metering.drivers.iptables.iptables_driver
2016-05-11 18:33:36.805 26409 ERROR
neutron.services.metering.drivers.iptables.iptables_driver
2016-05-11 18:33:36.807 26409 DEBUG neutron.agent.linux.utils [-] Running
command: ['sudo', 'ip', 'netns', 'exec',
'qrouter-0dd35572-735a-4ecd-8429-e047ed385100', 'iptables', '-t', 'filter',
'-L', 'neutron-meter-l-1811c9b9-f9b', '-n', '-v', '-x', '-Z'] create_process
/usr/local/lib/python2.7/dist-packages/neutron/agent/linux/utils.py:82
2016-05-11 18:33:36.818 26409 ERROR neutron.agent.linux.utils [-] Exit code: 1;
Stdin: ; Stdout: ; Stderr: Cannot open network namespace
"qrouter-0dd35572-735a-4ecd-8429-e047ed385100": No such file or directory
Logs in metering-agent.log after step 6:
2016-05-11 18:34:06.856 26409 ERROR
neutron.services.metering.drivers.iptables.iptables_driver [-] Failed to get
traffic counters, router: {u'status': u'ACTIVE', u'name': u'router-one',
u'gw_port_id': u'a4137128-27d6-43d6-b04d-a5df8ea62b02', u'admin_state_up':
True, u'tenant_id': u'e60fad96858e431cb3da4f0c1ab2d733', u'_metering_labels':
[{u'rules': [{u'remote_ip_prefix': u'192.168.88.0/24', u'direction': u'egress',
u'metering_label_id': u'1811c9b9-f9b0-40a6-839a-250b20674860', u'id':
u'007e3f9e-4e7c-439d-a606-db6ad0ae11ca', u'excluded': False}], u'id':
u'1811c9b9-f9b0-40a6-839a-250b20674860'}, {u'rules': [{u'remote_ip_prefix':
u'192.168.88.0/24', u'direction': u'ingress', u'metering_label_id':
u'd05ce8fb-c013-4ae4-bac5-de1be3f65331', u'id':
u'6c580a70-5dec-4225-ade8-44e03fdc9f1b', u'excluded': False}], u'id':
u'd05ce8fb-c013-4ae4-bac5-de1be3f65331'}], u'id':
u'0dd35572-735a-4ecd-8429-e047ed385100'}
2016-05-11 18:34:06.856 26409 ERROR
neutron.services.metering.drivers.iptables.iptables_driver Traceback (most
recent call last):
2016-05-11 18:34:06.856 26409 ERROR
neutron.services.metering.drivers.iptables.iptables_driver File
"/usr/local/lib/python2.7/dist-packages/neutron/services/metering/drivers/iptables/iptables_driver.py",
line 359, in get_traffic_counters
2016-05-11 18:34:06.856 26409 ERROR
neutron.services.metering.drivers.iptables.iptables_driver chain,
wrap=False, zero=True)
2016-05-11 18:34:06.856 26409 ERROR
neutron.services.metering.drivers.iptables.iptables_driver File
"/usr/local/lib/python2.7/dist-packages/neutron/agent/linux/iptables_manager.py",
line 669, in get_traffic_counters
2016-05-11 18:34:06.856 26409 ERROR
neutron.services.metering.drivers.iptables.iptables_driver current_table =
self.execute(args, run_as_root=True)
2016-05-11 18:34:06.856 26409 ERROR
neutron.services.metering.drivers.iptables.iptables_driver File
"/usr/local/lib/python2.7/dist-packages/neutron/agent/linux/utils.py", line
137, in execute
2016-05-11 18:34:06.856 26409 ERROR
neutron.services.metering.drivers.iptables.iptables_driver raise
RuntimeError(msg)
2016-05-11 18:34:06.856 26409 ERROR
neutron.services.metering.drivers.iptables.iptables_driver RuntimeError: Exit
code: 1; Stdin: ; Stdout: ; Stderr: iptables: No chain/target/match by that
name.
2016-05-11 18:34:06.856 26409 ERROR
neutron.services.metering.drivers.iptables.iptables_driver
2016-05-11 18:34:06.856 26409 ERROR
neutron.services.metering.drivers.iptables.iptables_driver
2016-05-11 18:34:06.858 26409 DEBUG neutron.agent.linux.utils [-] Running
command: ['sudo', 'ip', 'netns', 'exec',
'qrouter-0dd35572-735a-4ecd-8429-e047ed385100', 'iptables', '-t', 'filter',
'-L', 'neutron-meter-l-d05ce8fb-c01', '-n', '-v', '-x', '-Z'] create_process
/usr/local/lib/python2.7/dist-packages/neutron/agent/linux/utils.py:82
2016-05-11 18:34:06.911 26409 ERROR neutron.agent.linux.utils [-] Exit code: 1;
Stdin: ; Stdout: ; Stderr: iptables: No chain/target/match by that name.
** Affects: neutron
Importance: Undecided
Assignee: HuntXu (huntxu)
Status: In Progress
** Changed in: neutron
Status: New => In Progress
** Changed in: neutron
Assignee: (unassigned) => HuntXu (huntxu)
--
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to neutron.
https://bugs.launchpad.net/bugs/1580548
Title:
Metering-agent's routers information is inconsistent with neutron-
server
Status in neutron:
In Progress
Bug description:
With the current implement, under certain curcumstance, router
information(including internal-stored data, iptables settings) kept
and set up by metering-agent could be inconsistent with what the
neutron-server suppose it to be. And that cause the metering-agent to
run into a problematic status.
Version:
Neutron master commit 7327e8c21(Merge "Fix update target tenant RBAC external
path")
Steps to reproduce:
1. Create internal net, subnet, router. Set external gateway for router, add
interface to the router for the created subnet.
2. Create neutron metering label and rule.
3. Make sure the created router is added to an active l3-agent.
4. Create another router and set its external gateway.
5. Use 'neutron l3-agent-router-remove' to remove the first router from the
l3 agent.
6. Use 'neutron l3-agent-router-add' to add the first router back to the l3
agent by which it was hosted.
(In the following logs, the first created router's id is
0dd35572-735a-4ecd-8429-e047ed385100, the second one is
392820a7-3908-49eb-98a3-967f1638432f)
Logs in metering-agent.log after step 4:
2016-05-11 18:32:49.915 26409 DEBUG neutron.agent.linux.utils
[req-0515c6f8-cbd1-47d4-a45e-f89e6b6e97b4 - - - - -] Running command: ['sudo',
'ip', 'netns', 'exec', 'qrouter-392820a7-3908-49eb-98a3-967f1638432f',
'iptables-save'] create_process
/usr/local/lib/python2.7/dist-packages/neutron/agent/linux/utils.py:82
2016-05-11 18:32:49.926 26409 ERROR neutron.agent.linux.utils
[req-0515c6f8-cbd1-47d4-a45e-f89e6b6e97b4 - - - - -] Exit code: 1; Stdin: ;
Stdout: ; Stderr: Cannot open network namespace
"qrouter-392820a7-3908-49eb-98a3-967f1638432f": No such file or directory
2016-05-11 18:32:49.926 26409 DEBUG oslo_concurrency.lockutils
[req-0515c6f8-cbd1-47d4-a45e-f89e6b6e97b4 - - - - -] Releasing semaphore
"iptables-qrouter-392820a7-3908-49eb-98a3-967f1638432f" lock
/usr/local/lib/python2.7/dist-packages/oslo_concurrency/lockutils.py:225
2016-05-11 18:32:49.927 26409 ERROR
neutron.services.metering.agents.metering_agent
[req-0515c6f8-cbd1-47d4-a45e-f89e6b6e97b4 - - - - -] Driver
neutron.services.metering.drivers.iptables.iptables_driver.IptablesMeteringDriver:update_routers
runtime error
2016-05-11 18:32:49.927 26409 ERROR
neutron.services.metering.agents.metering_agent Traceback (most recent call
last):
2016-05-11 18:32:49.927 26409 ERROR
neutron.services.metering.agents.metering_agent File
"/usr/local/lib/python2.7/dist-packages/neutron/services/metering/agents/metering_agent.py",
line 177, in _invoke_driver
2016-05-11 18:32:49.927 26409 ERROR
neutron.services.metering.agents.metering_agent return
getattr(self.metering_driver, func_name)(context, meterings)
2016-05-11 18:32:49.927 26409 ERROR
neutron.services.metering.agents.metering_agent File
"/usr/local/lib/python2.7/dist-packages/oslo_log/helpers.py", line 48, in
wrapper
2016-05-11 18:32:49.927 26409 ERROR
neutron.services.metering.agents.metering_agent return method(*args,
**kwargs)
2016-05-11 18:32:49.927 26409 ERROR
neutron.services.metering.agents.metering_agent File
"/usr/local/lib/python2.7/dist-packages/neutron/services/metering/drivers/iptables/iptables_driver.py",
line 125, in update_routers
2016-05-11 18:32:49.927 26409 ERROR
neutron.services.metering.agents.metering_agent
self._process_associate_metering_label(router)
2016-05-11 18:32:49.927 26409 ERROR
neutron.services.metering.agents.metering_agent File
"/usr/local/lib/python2.7/dist-packages/neutron/services/metering/drivers/iptables/iptables_driver.py",
line 228, in _process_associate_metering_label
2016-05-11 18:32:49.927 26409 ERROR
neutron.services.metering.agents.metering_agent
rm.metering_labels[label_id] = label
2016-05-11 18:32:49.927 26409 ERROR
neutron.services.metering.agents.metering_agent File
"/usr/local/lib/python2.7/dist-packages/neutron/services/metering/drivers/iptables/iptables_driver.py",
line 58, in __exit__
2016-05-11 18:32:49.927 26409 ERROR
neutron.services.metering.agents.metering_agent self.im.apply()
2016-05-11 18:32:49.927 26409 ERROR
neutron.services.metering.agents.metering_agent File
"/usr/local/lib/python2.7/dist-packages/neutron/agent/linux/iptables_manager.py",
line 431, in apply
2016-05-11 18:32:49.927 26409 ERROR
neutron.services.metering.agents.metering_agent return self._apply()
2016-05-11 18:32:49.927 26409 ERROR
neutron.services.metering.agents.metering_agent File
"/usr/local/lib/python2.7/dist-packages/neutron/agent/linux/iptables_manager.py",
line 439, in _apply
2016-05-11 18:32:49.927 26409 ERROR
neutron.services.metering.agents.metering_agent return
self._apply_synchronized()
2016-05-11 18:32:49.927 26409 ERROR
neutron.services.metering.agents.metering_agent File
"/usr/local/lib/python2.7/dist-packages/neutron/agent/linux/iptables_manager.py",
line 465, in _apply_synchronized
2016-05-11 18:32:49.927 26409 ERROR
neutron.services.metering.agents.metering_agent save_output =
self.execute(args, run_as_root=True)
2016-05-11 18:32:49.927 26409 ERROR
neutron.services.metering.agents.metering_agent File
"/usr/local/lib/python2.7/dist-packages/neutron/agent/linux/utils.py", line
137, in execute
2016-05-11 18:32:49.927 26409 ERROR
neutron.services.metering.agents.metering_agent raise RuntimeError(msg)
2016-05-11 18:32:49.927 26409 ERROR
neutron.services.metering.agents.metering_agent RuntimeError: Exit code: 1;
Stdin: ; Stdout: ; Stderr: Cannot open network namespace
"qrouter-392820a7-3908-49eb-98a3-967f1638432f": No such file or directory
2016-05-11 18:32:49.927 26409 ERROR
neutron.services.metering.agents.metering_agent
2016-05-11 18:32:49.927 26409 ERROR
neutron.services.metering.agents.metering_agent
...Skipping...
2016-05-11 18:33:06.776 26409 DEBUG neutron.agent.linux.utils [-] Running
command: ['sudo', 'ip', 'netns', 'exec',
'qrouter-392820a7-3908-49eb-98a3-967f1638432f', 'iptables', '-t', 'filter',
'-L', 'neutron-meter-l-1811c9b9-f9b', '-n', '-v', '-x', '-Z'] create_process
/usr/local/lib/python2.7/dist-packages/neutron/agent/linux/utils.py:82
2016-05-11 18:33:06.787 26409 ERROR neutron.agent.linux.utils [-] Exit code:
1; Stdin: ; Stdout: ; Stderr: Cannot open network namespace
"qrouter-392820a7-3908-49eb-98a3-967f1638432f": No such file or directory
2016-05-11 18:33:06.788 26409 ERROR
neutron.services.metering.drivers.iptables.iptables_driver [-] Failed to get
traffic counters, router: {u'status': u'ACTIVE', u'name': u'router-two',
u'gw_port_id': u'729fba34-4aec-415d-bef8-b93b34e69536', u'admin_state_up':
False, u'tenant_id': u'e60fad96858e431cb3da4f0c1ab2d733', u'_metering_labels':
[{u'rules': [{u'remote_ip_prefix': u'192.168.88.0/24', u'direction': u'egress',
u'metering_label_id': u'1811c9b9-f9b0-40a6-839a-250b20674860', u'id':
u'007e3f9e-4e7c-439d-a606-db6ad0ae11ca', u'excluded': False}], u'id':
u'1811c9b9-f9b0-40a6-839a-250b20674860'}, {u'rules': [{u'remote_ip_prefix':
u'192.168.88.0/24', u'direction': u'ingress', u'metering_label_id':
u'd05ce8fb-c013-4ae4-bac5-de1be3f65331', u'id':
u'6c580a70-5dec-4225-ade8-44e03fdc9f1b', u'excluded': False}], u'id':
u'd05ce8fb-c013-4ae4-bac5-de1be3f65331'}], u'id':
u'392820a7-3908-49eb-98a3-967f1638432f'}
2016-05-11 18:33:06.788 26409 ERROR
neutron.services.metering.drivers.iptables.iptables_driver Traceback (most
recent call last):
2016-05-11 18:33:06.788 26409 ERROR
neutron.services.metering.drivers.iptables.iptables_driver File
"/usr/local/lib/python2.7/dist-packages/neutron/services/metering/drivers/iptables/iptables_driver.py",
line 359, in get_traffic_counters
2016-05-11 18:33:06.788 26409 ERROR
neutron.services.metering.drivers.iptables.iptables_driver chain,
wrap=False, zero=True)
2016-05-11 18:33:06.788 26409 ERROR
neutron.services.metering.drivers.iptables.iptables_driver File
"/usr/local/lib/python2.7/dist-packages/neutron/agent/linux/iptables_manager.py",
line 669, in get_traffic_counters
2016-05-11 18:33:06.788 26409 ERROR
neutron.services.metering.drivers.iptables.iptables_driver current_table =
self.execute(args, run_as_root=True)
2016-05-11 18:33:06.788 26409 ERROR
neutron.services.metering.drivers.iptables.iptables_driver File
"/usr/local/lib/python2.7/dist-packages/neutron/agent/linux/utils.py", line
137, in execute
2016-05-11 18:33:06.788 26409 ERROR
neutron.services.metering.drivers.iptables.iptables_driver raise
RuntimeError(msg)
2016-05-11 18:33:06.788 26409 ERROR
neutron.services.metering.drivers.iptables.iptables_driver RuntimeError: Exit
code: 1; Stdin: ; Stdout: ; Stderr: Cannot open network namespace
"qrouter-392820a7-3908-49eb-98a3-967f1638432f": No such file or directory
2016-05-11 18:33:06.788 26409 ERROR
neutron.services.metering.drivers.iptables.iptables_driver
2016-05-11 18:33:06.788 26409 ERROR
neutron.services.metering.drivers.iptables.iptables_driver
2016-05-11 18:33:06.790 26409 DEBUG neutron.agent.linux.utils [-] Running
command: ['sudo', 'ip', 'netns', 'exec',
'qrouter-392820a7-3908-49eb-98a3-967f1638432f', 'iptables', '-t', 'filter',
'-L', 'neutron-meter-l-d05ce8fb-c01', '-n', '-v', '-x', '-Z'] create_process
/usr/local/lib/python2.7/dist-packages/neutron/agent/linux/utils.py:82
Logs in metering-agent.log after step 5:
2016-05-11 18:33:36.805 26409 ERROR
neutron.services.metering.drivers.iptables.iptables_driver [-] Failed to get
traffic counters, router: {u'status': u'ACTIVE', u'name': u'router-two',
u'gw_port_id': u'729fba34-4aec-415d-bef8-b93b34e69536', u'admin_state_up':
False, u'tenant_id': u'e60fad96858e431cb3da4f0c1ab2d733', u'_metering_labels':
[{u'rules': [{u'remote_ip_prefix': u'192.168.88.0/24', u'direction': u'egress',
u'metering_label_id': u'1811c9b9-f9b0-40a6-839a-250b20674860', u'id':
u'007e3f9e-4e7c-439d-a606-db6ad0ae11ca', u'excluded': False}], u'id':
u'1811c9b9-f9b0-40a6-839a-250b20674860'}, {u'rules': [{u'remote_ip_prefix':
u'192.168.88.0/24', u'direction': u'ingress', u'metering_label_id':
u'd05ce8fb-c013-4ae4-bac5-de1be3f65331', u'id':
u'6c580a70-5dec-4225-ade8-44e03fdc9f1b', u'excluded': False}], u'id':
u'd05ce8fb-c013-4ae4-bac5-de1be3f65331'}], u'id':
u'392820a7-3908-49eb-98a3-967f1638432f'}
2016-05-11 18:33:36.805 26409 ERROR
neutron.services.metering.drivers.iptables.iptables_driver Traceback (most
recent call last):
2016-05-11 18:33:36.805 26409 ERROR
neutron.services.metering.drivers.iptables.iptables_driver File
"/usr/local/lib/python2.7/dist-packages/neutron/services/metering/drivers/iptables/iptables_driver.py",
line 359, in get_traffic_counters
2016-05-11 18:33:36.805 26409 ERROR
neutron.services.metering.drivers.iptables.iptables_driver chain,
wrap=False, zero=True)
2016-05-11 18:33:36.805 26409 ERROR
neutron.services.metering.drivers.iptables.iptables_driver File
"/usr/local/lib/python2.7/dist-packages/neutron/agent/linux/iptables_manager.py",
line 669, in get_traffic_counters
2016-05-11 18:33:36.805 26409 ERROR
neutron.services.metering.drivers.iptables.iptables_driver current_table =
self.execute(args, run_as_root=True)
2016-05-11 18:33:36.805 26409 ERROR
neutron.services.metering.drivers.iptables.iptables_driver File
"/usr/local/lib/python2.7/dist-packages/neutron/agent/linux/utils.py", line
137, in execute
2016-05-11 18:33:36.805 26409 ERROR
neutron.services.metering.drivers.iptables.iptables_driver raise
RuntimeError(msg)
2016-05-11 18:33:36.805 26409 ERROR
neutron.services.metering.drivers.iptables.iptables_driver RuntimeError: Exit
code: 1; Stdin: ; Stdout: ; Stderr: Cannot open network namespace
"qrouter-392820a7-3908-49eb-98a3-967f1638432f": No such file or directory
2016-05-11 18:33:36.805 26409 ERROR
neutron.services.metering.drivers.iptables.iptables_driver
2016-05-11 18:33:36.805 26409 ERROR
neutron.services.metering.drivers.iptables.iptables_driver
2016-05-11 18:33:36.807 26409 DEBUG neutron.agent.linux.utils [-] Running
command: ['sudo', 'ip', 'netns', 'exec',
'qrouter-0dd35572-735a-4ecd-8429-e047ed385100', 'iptables', '-t', 'filter',
'-L', 'neutron-meter-l-1811c9b9-f9b', '-n', '-v', '-x', '-Z'] create_process
/usr/local/lib/python2.7/dist-packages/neutron/agent/linux/utils.py:82
2016-05-11 18:33:36.818 26409 ERROR neutron.agent.linux.utils [-] Exit code:
1; Stdin: ; Stdout: ; Stderr: Cannot open network namespace
"qrouter-0dd35572-735a-4ecd-8429-e047ed385100": No such file or directory
Logs in metering-agent.log after step 6:
2016-05-11 18:34:06.856 26409 ERROR
neutron.services.metering.drivers.iptables.iptables_driver [-] Failed to get
traffic counters, router: {u'status': u'ACTIVE', u'name': u'router-one',
u'gw_port_id': u'a4137128-27d6-43d6-b04d-a5df8ea62b02', u'admin_state_up':
True, u'tenant_id': u'e60fad96858e431cb3da4f0c1ab2d733', u'_metering_labels':
[{u'rules': [{u'remote_ip_prefix': u'192.168.88.0/24', u'direction': u'egress',
u'metering_label_id': u'1811c9b9-f9b0-40a6-839a-250b20674860', u'id':
u'007e3f9e-4e7c-439d-a606-db6ad0ae11ca', u'excluded': False}], u'id':
u'1811c9b9-f9b0-40a6-839a-250b20674860'}, {u'rules': [{u'remote_ip_prefix':
u'192.168.88.0/24', u'direction': u'ingress', u'metering_label_id':
u'd05ce8fb-c013-4ae4-bac5-de1be3f65331', u'id':
u'6c580a70-5dec-4225-ade8-44e03fdc9f1b', u'excluded': False}], u'id':
u'd05ce8fb-c013-4ae4-bac5-de1be3f65331'}], u'id':
u'0dd35572-735a-4ecd-8429-e047ed385100'}
2016-05-11 18:34:06.856 26409 ERROR
neutron.services.metering.drivers.iptables.iptables_driver Traceback (most
recent call last):
2016-05-11 18:34:06.856 26409 ERROR
neutron.services.metering.drivers.iptables.iptables_driver File
"/usr/local/lib/python2.7/dist-packages/neutron/services/metering/drivers/iptables/iptables_driver.py",
line 359, in get_traffic_counters
2016-05-11 18:34:06.856 26409 ERROR
neutron.services.metering.drivers.iptables.iptables_driver chain,
wrap=False, zero=True)
2016-05-11 18:34:06.856 26409 ERROR
neutron.services.metering.drivers.iptables.iptables_driver File
"/usr/local/lib/python2.7/dist-packages/neutron/agent/linux/iptables_manager.py",
line 669, in get_traffic_counters
2016-05-11 18:34:06.856 26409 ERROR
neutron.services.metering.drivers.iptables.iptables_driver current_table =
self.execute(args, run_as_root=True)
2016-05-11 18:34:06.856 26409 ERROR
neutron.services.metering.drivers.iptables.iptables_driver File
"/usr/local/lib/python2.7/dist-packages/neutron/agent/linux/utils.py", line
137, in execute
2016-05-11 18:34:06.856 26409 ERROR
neutron.services.metering.drivers.iptables.iptables_driver raise
RuntimeError(msg)
2016-05-11 18:34:06.856 26409 ERROR
neutron.services.metering.drivers.iptables.iptables_driver RuntimeError: Exit
code: 1; Stdin: ; Stdout: ; Stderr: iptables: No chain/target/match by that
name.
2016-05-11 18:34:06.856 26409 ERROR
neutron.services.metering.drivers.iptables.iptables_driver
2016-05-11 18:34:06.856 26409 ERROR
neutron.services.metering.drivers.iptables.iptables_driver
2016-05-11 18:34:06.858 26409 DEBUG neutron.agent.linux.utils [-] Running
command: ['sudo', 'ip', 'netns', 'exec',
'qrouter-0dd35572-735a-4ecd-8429-e047ed385100', 'iptables', '-t', 'filter',
'-L', 'neutron-meter-l-d05ce8fb-c01', '-n', '-v', '-x', '-Z'] create_process
/usr/local/lib/python2.7/dist-packages/neutron/agent/linux/utils.py:82
2016-05-11 18:34:06.911 26409 ERROR neutron.agent.linux.utils [-] Exit code:
1; Stdin: ; Stdout: ; Stderr: iptables: No chain/target/match by that name.
To manage notifications about this bug go to:
https://bugs.launchpad.net/neutron/+bug/1580548/+subscriptions
--
Mailing list: https://launchpad.net/~yahoo-eng-team
Post to : yahoo-eng-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~yahoo-eng-team
More help : https://help.launchpad.net/ListHelp
