Hello Brian,
Thanks for the info.
I looked into the code for the metering agent and discovered the following:
1. The metering agent on compute nodes doesn't get notified about DVR
routers running on the node.
2. For DVR routers it tries to meter the rfp- interface(for floating
ips) on the snat- namespace. This is wrong. The rfp- interface lies on
the qrouter- namespace on compute nodes. Also, there is a qg- interface
on the snat- namespace on network nodes (for performing NAT) which
should be metered too.
3. There is a race condition whereby the metering agent is notified
about a router before its namespaces are created. The agent ends up not
adding the metering rules for those namespaces and this leads to
unrecorded traffic.
I addressed those issues in a change:
https://review.openstack.org/#/c/621165/. Any feedback is appreciated.
Best regards,
Alex
On 26/10/2018 21:49, Brian Haley wrote:
On 10/25/2018 08:06 AM, Alexandru Sorodoc wrote:
Hello,
I'm trying to set up metering for neutron in Pike. I tested it with a
centralized router and it works, but when I try with a distributed
router it
doesn't record any usage samples. I have one compute node and one
network node
and I've created an instance with a floating ip.
The metering agent isn't very well maintained, and I don't see any
open bugs similar to this issue. The only thing I can remember is
this abandoned change regarding traffic counters for DVR routers -
https://review.openstack.org/#/c/486493/ but there was no follow-on
from the author.
The best thing to do would be to try and reproduce it on the master
branch (or Rocky) and file a bug.
> I think this is because the router is running on network1. Why is it
> running on
> network1 and why does it seem that the l3 agent on compute1 does the
actual
> routing?
The compute node will do all the routing when a floating IP is
associated, the router on network1 is for default snat traffic when
there is no floating IP and the instance tries to communicate out the
external network.
-Brian
openstack router show public-router2
+-------------------------+----------------------------------------------------+
| Field |
Value |
+-------------------------+----------------------------------------------------+
| admin_state_up |
UP |
| availability_zone_hints
| |
| availability_zones |
nova |
| created_at |
2018-10-05T12:07:32Z |
| description | |
| distributed |
True |
| external_gateway_info | {"network_id":
"b96473ce- |
| | 94f6-464f-a703-5285fb8ff3d3",
"enable_snat": true, |
| | "external_fixed_ips":
[{"subnet_id": |
| |
"6c08c3d9-7df1-4bec-b847-19f80b9d1764", |
| | "ip_address":
"192.168.252.102"}]} |
| flavor_id |
None |
| ha |
False |
| id |
37c1794b-58d1-4d0d-b34b-944ca411b86b |
| name |
public-router2 |
| project_id |
fe203109e67f4e39b066c9529f9fc35d |
| revision_number |
5 |
| routes | |
| status |
ACTIVE |
| tags | |
| updated_at |
2018-10-05T12:09:36Z |
+-------------------------+----------------------------------------------------+
openstack network agent list
+-----------+------------+-----------+-------------------+-------+-------+--------------+
| ID | Agent Type | Host | Availability Zone | Alive |
State | Binary |
+-----------+------------+-----------+-------------------+-------+-------+--------------+
| 14b9ea75- | L3 agent | compute1. | nova | :-) | UP |
neutron-l3-a |
| 1dc1-4e37 | | localdoma | | | | gent |
| -a2b0-508 | | in | | | | |
| 3d336916d | | | | | | |
| 26139ec1- | Metering | compute1. | None | :-) | UP |
neutron- |
| f4f9-4bb3 | agent | localdoma | | | | metering- |
| -aebb-c35 | | in | | | | agent |
| 3a36ed79c | | | | | | |
| 2a54971f- | DHCP agent | network1. | nova | :-) | UP |
neutron- |
| 9849-4ed2 | | localdoma | | | | dhcp-agent |
| -b009-00e | | in | | | | |
| 45eb4d255 | | | | | | |
| 443c0b49- | Open | compute1. | None | :-) | UP |
neutron- |
| 4484-44d2 | vSwitch | localdoma | | | | openvswitch- |
| -a704-32a | agent | in | | | | agent |
| 92ffe6982 | | | | | | |
| 5d00a219 | L3 agent | network1. | nova | :-) | UP |
neutron-vpn- |
| -abce- | | localdoma | | | | agent |
| 48ca- | | in | | | | |
| ba1e-d962 | | | | | | |
| 01bd7de3 | | | | | | |
| bc3458b4 | Open | network1. | None | :-) | UP |
neutron- |
| -250e- | vSwitch | localdoma | | | | openvswitch- |
| 4adf-90e0 | agent | in | | | | agent |
| -110a1a7f | | | | | | |
| 6ccb | | | | | | |
| c29f9da8- | Metering | network1. | None | :-) | UP |
neutron- |
| ca58-4a11 | agent | localdoma | | | | metering- |
| -b500-a25 | | in | | | | agent |
| 3f820808e | | | | | | |
| cdce667d- | Metadata | network1. | None | :-) | UP |
neutron- |
| faa4 | agent | localdoma | | | | metadata- |
| -49ed- | | in | | | | agent |
| 83ee-e0e5 | | | | | | |
| a352d482 | | | | | | |
| cf5ae104- | Metadata | compute1. | None | :-) | UP |
neutron- |
| 49d7-4c85 | agent | localdoma | | | | metadata- |
| -a252-cc5 | | in | | | | agent |
| 9a9a12789 | | | | | | |
+-----------+------------+-----------+-------------------+-------+-------+--------------+
If I check the node on which my distributed router is running it
tells me that
it's running on the network node:
neutron l3-agent-list-hosting-router
37c1794b-58d1-4d0d-b34b-944ca411b86b
+--------------------------------------+----------------------+----------------+-------+----------+
| id | host |
admin_state_up | alive | ha_state |
+--------------------------------------+----------------------+----------------+-------+----------+
| 5d00a219-abce-48ca-ba1e-d96201bd7de3 | network1.localdomain |
True | :-) | |
+--------------------------------------+----------------------+----------------+-------+----------+
If I check the iptable rules for the router on the compute and
network nodes by running:
ip netns exec qrouter-37c1794b-58d1-4d0d-b34b-944ca411b86b iptables
-nv -L
I see that compute1 records the traffic while network1 doesn't. Also,
I did some
debugging and found out that the metering agent on compute1 receives
an empty
list of routers when querying the routers that it should monitor.
Source:
https://github.com/openstack/neutron/blob/stable/pike/neutron/services/metering/agents/metering_agent.py#L177-L189
https://github.com/openstack/neutron/blob/stable/pike/neutron/db/metering/metering_rpc.py#L33-L57
I think this is because the router is running on network1. Why is it
running on
network1 and why does it seem that the l3 agent on compute1 does the
actual
routing?
Thanks,
Alex
_______________________________________________
Mailing list:
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
Post to : openstack@lists.openstack.org
Unsubscribe :
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
_______________________________________________
Mailing list: http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
Post to : openstack@lists.openstack.org
Unsubscribe : http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
