[
https://issues.apache.org/jira/browse/CLOUDSTACK-721?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ilia Shakitko updated CLOUDSTACK-721:
-------------------------------------
Description:
The usage server can give the system usage of each virtual machine (such as
running time, ServiceOffering, IPAddress, Volume, Template, ISO, Port
Forwarding Rule, Network offering), except the network bytes sent/received.
This problem only exists in CloudStack 4.0. In CloudStack 3.0.2, the usage
server works well.
Analysis steps:
1. Check the CloudStack logs
We can get the logs in /var/log/cloud/management/management-server.log
2012-12-18 15:05:51,130 DEBUG [agent.transport.Request]
(AgentManager-Handler-8:null) Seq 1-1158742136: Processing: { Ans: , MgmtId:
345051509349, via: 1, Ver: v1, Flags: 10,
[{"NetworkUsageAnswer":{"routerName":"r-4-VM","bytesSent":5928,"bytesReceived":6188,"result":true,"details":"","wait":0}}]
}
2012-12-18 15:05:51,130 DEBUG [agent.transport.Request] (RouterMonitor-1:null)
Seq 1-1158742136: Received: { Ans: , MgmtId: 345051509349, via: 1, Ver: v1,
Flags: 10, { NetworkUsageAnswer } }
2012-12-18 15:05:51,130 DEBUG [agent.manager.AgentManagerImpl]
(RouterMonitor-1:null) Details from executing class
com.cloud.agent.api.NetworkUsageCommand:
2012-12-18 15:05:51,131 WARN
[network.router.VirtualNetworkApplianceManagerImpl] (RouterMonitor-1:null)
unable to find stats for account: 2
We can see that AgentManager works well. It can get the network usage. However,
there is a problem in subsequent data processing.
2. Check the source code and the value in database.
We can see the data processing procedure in
/cloud-server/src/com/cloud/network/router/VirtualNetworkApplianceManagerImpl.java
(I have deleted some unrelated source codes)
public void run() {
for (DomainRouterVO router : routers) {
String privateIP = router.getPrivateIpAddress();
// from domain_router table (id=4,
name=r-4-VM, vm_instance.private_ip_address=SOME_IP_ADDRESS1)
if (privateIP != null) {
List<? extends Nic> routerNics = _nicDao.listByVmId(router.getId());
// from nics table (instance_id=4). Found 3 records.
Return (networkid= 204/202/200)
for (Nic routerNic : routerNics) {
Network network = _networkMgr.getNetwork(routerNic.getNetworkId());
// from nics table.
if (network.getTrafficType() == TrafficType.Public) {
// get traffic_type in networks table. Return
Guest/Control/Public.
UserStatisticsVO previousStats =
_statsDao.findBy(router.getAccountId(),
router.getDataCenterIdToDeployIn(), network.getId(), null,
router.getId(), router.getType().toString());
if (answer != null) {
Transaction txn = Transaction.open(Transaction.CLOUD_DB);
try {
txn.start();
UserStatisticsVO stats = _statsDao.lock(router.getAccountId(),
router.getDataCenterIdToDeployIn(), network.getId(),
routerNic.getIp4Address(), router.getId(), router.getType().toString());
// the parameter is “2, 1, 200, SOME_IP_ADDRESS2, 4”. It’s inconsistent to
database.
if (stats == null) {
s_logger.warn("unable to find stats for account: " +
router.getAccountId());
continue;
}
………………..
}
We can see the data initialization in
/cloud-server/src/com/cloud/vm/dao/DomainRouterDaoImpl.java
public void addRouterToGuestNetwork(VirtualRouter router, Network guestNetwork)
{
if (stats == null) {
stats = new UserStatisticsVO(router.getAccountId(),
router.getDataCenterIdToDeployIn(), null, router.getId(),
// the parameter is “2, 1, null, 204, 4”. It’s consistent to
database.
router.getType().toString(), guestNetwork.getId());
_userStatsDao.persist(stats);
}
}
3. Check the database. We can see the data is same to data in data
initialization in Step (2), but different to data in data processing.
mysql> select
id,data_center_id,account_id,public_ip_address,device_id,device_type,network_id
from user_statistics;
+----+----------------+------------+-------------------+-----------+--------------+------------+
| id | data_center_id | account_id | public_ip_address | device_id |
device_type | network_id |
+----+----------------+------------+-------------------+-----------+--------------+------------+
| 1 | 1 | 2 | NULL |
4 | DomainRouter | 204 |
| 2 | 1 | 2 | NULL |
6 | DomainRouter | 205 |
4. Testing. If we insert a record into the databases. The record is same to
data processing. It will work!
| 3 | 1 | 2 | SOME_IP_ADDRESS2 | 4 |
DomainRouter | 200 |
... after 5 minutes, we can get the VR data traffic.It looks like
| id | data_center_id | account_id | public_ip_address | device_id |
device_type | network_id | net_bytes_received | net_bytes_sent |
current_bytes_received | current_bytes_sent | agg_bytes_received |
agg_bytes_sent |
| 3 | 1 | 2 | SOME_IP_ADDRESS2 | 4 |
DomainRouter | 200 | 0 |
0 | 2284239008 | 33613977 |
2284239008 | 33613977 |
>From Step 1 to Step 4, we can conclude that the inconsistent of data
>initialization and data processing procedure is the root cause of this problem.
was:
The usage server can give the system usage of each virtual machine (such as
running time, ServiceOffering, IPAddress, Volume, Template, ISO, Port
Forwarding Rule, Network offering), except the network bytes sent/received.
This problem only exists in CloudStack 4.0. In CloudStack 3.0.2, the usage
server works well.
Analysis steps:
1. Check the CloudStack logs
We can get the logs in /var/log/cloud/management/management-server.log
2012-12-18 15:05:51,130 DEBUG [agent.transport.Request]
(AgentManager-Handler-8:null) Seq 1-1158742136: Processing: { Ans: , MgmtId:
345051509349, via: 1, Ver: v1, Flags: 10,
[{"NetworkUsageAnswer":{"routerName":"r-4-VM","bytesSent":5928,"bytesReceived":6188,"result":true,"details":"","wait":0}}]
}
2012-12-18 15:05:51,130 DEBUG [agent.transport.Request] (RouterMonitor-1:null)
Seq 1-1158742136: Received: { Ans: , MgmtId: 345051509349, via: 1, Ver: v1,
Flags: 10, { NetworkUsageAnswer } }
2012-12-18 15:05:51,130 DEBUG [agent.manager.AgentManagerImpl]
(RouterMonitor-1:null) Details from executing class
com.cloud.agent.api.NetworkUsageCommand:
2012-12-18 15:05:51,131 WARN
[network.router.VirtualNetworkApplianceManagerImpl] (RouterMonitor-1:null)
unable to find stats for account: 2
We can see that AgentManager works well. It can get the network usage. However,
there is a problem in subsequent data processing.
2. Check the source code and the value in database.
We can see the data processing procedure in
/cloud-server/src/com/cloud/network/router/VirtualNetworkApplianceManagerImpl.java
(I have deleted some unrelated source codes)
{code:java}
public void run() {
for (DomainRouterVO router : routers) {
String privateIP = router.getPrivateIpAddress();
// from domain_router table (id=4,
name=r-4-VM, vm_instance.private_ip_address=169.254.1.17)
if (privateIP != null) {
List<? extends Nic> routerNics = _nicDao.listByVmId(router.getId());
// from nics table (instance_id=4). Found 3
records. Return (networkid= 204/202/200)
for (Nic routerNic : routerNics) {
Network network = _networkMgr.getNetwork(routerNic.getNetworkId());
// from nics table.
if (network.getTrafficType() == TrafficType.Public) {
// get traffic_type in networks table. Return
Guest/Control/Public.
UserStatisticsVO previousStats =
_statsDao.findBy(router.getAccountId(),
router.getDataCenterIdToDeployIn(), network.getId(), null,
router.getId(), router.getType().toString());
if (answer != null) {
Transaction txn = Transaction.open(Transaction.CLOUD_DB);
try {
txn.start();
UserStatisticsVO stats = _statsDao.lock(router.getAccountId(),
router.getDataCenterIdToDeployIn(), network.getId(),
routerNic.getIp4Address(), router.getId(), router.getType().toString());
// the parameter is “2, 1, 200, 10.11.102.168, 4”. It’s inconsistent to
database.
if (stats == null) {
s_logger.warn("unable to find stats for account: " +
router.getAccountId());
continue;
}
………………..
}
{code}
We can see the data initialization in
/cloud-server/src/com/cloud/vm/dao/DomainRouterDaoImpl.java
{code:java}
public void addRouterToGuestNetwork(VirtualRouter router,
Network guestNetwork) {
if (stats == null) {
stats = new UserStatisticsVO(router.getAccountId(),
router.getDataCenterIdToDeployIn(), null, router.getId(),
// the parameter
is “2, 1, null, 204, 4”. It’s consistent to database.
router.getType().toString(), guestNetwork.getId());
_userStatsDao.persist(stats);
}
}
{code:java}
3. Check the database. We can see the data is same to data in data
initialization in Step (2), but different to data in data processing.
mysql> select
id,data_center_id,account_id,public_ip_address,device_id,device_type,network_id
from user_statistics;
+----+----------------+------------+-------------------+-----------+--------------+------------+
| id | data_center_id | account_id | public_ip_address | device_id |
device_type | network_id |
+----+----------------+------------+-------------------+-----------+--------------+------------+
| 1 | 1 | 2 | NULL | 4 | DomainRouter |
204 |
| 2 | 1 | 2 | NULL | 6 | DomainRouter |
205 |
(4) Testing. If we insert a record into the databases. The record is same
to data processing. It will work!
| 3 | 1 | 2 | 10.11.102.168 | 4 | DomainRouter |
200 |
After 5 minutes, we can get the VR data traffic.It looks like
| id | data_center_id | account_id | public_ip_address | device_id |
device_type | network_id | net_bytes_received | net_bytes_sent |
current_bytes_received | current_bytes_sent | agg_bytes_received |
agg_bytes_sent |
| 3 | 1 | 2 | 10.11.102.168 | 4 | DomainRouter |
200 | 0 | 0 | 2284239008 | 33613977 |
2284239008 | 33613977 |
>From Step 1 to Step 4, we can conclude that the inconsistent of data
>initialization and data processing procedure is the root cause of this problem.
> Bytes sent/received in user statistics is empty (CloudStack 4.0)
> ----------------------------------------------------------------
>
> Key: CLOUDSTACK-721
> URL: https://issues.apache.org/jira/browse/CLOUDSTACK-721
> Project: CloudStack
> Issue Type: Bug
> Security Level: Public(Anyone can view this level - this is the
> default.)
> Components: Usage
> Affects Versions: 4.0.0
> Reporter: Ilia Shakitko
> Labels: network, usage, user_statistics
> Original Estimate: 168h
> Remaining Estimate: 168h
>
> The usage server can give the system usage of each virtual machine (such as
> running time, ServiceOffering, IPAddress, Volume, Template, ISO, Port
> Forwarding Rule, Network offering), except the network bytes sent/received.
> This problem only exists in CloudStack 4.0. In CloudStack 3.0.2, the usage
> server works well.
> Analysis steps:
> 1. Check the CloudStack logs
> We can get the logs in /var/log/cloud/management/management-server.log
> 2012-12-18 15:05:51,130 DEBUG [agent.transport.Request]
> (AgentManager-Handler-8:null) Seq 1-1158742136: Processing: { Ans: , MgmtId:
> 345051509349, via: 1, Ver: v1, Flags: 10,
> [{"NetworkUsageAnswer":{"routerName":"r-4-VM","bytesSent":5928,"bytesReceived":6188,"result":true,"details":"","wait":0}}]
> }
> 2012-12-18 15:05:51,130 DEBUG [agent.transport.Request]
> (RouterMonitor-1:null) Seq 1-1158742136: Received: { Ans: , MgmtId:
> 345051509349, via: 1, Ver: v1, Flags: 10, { NetworkUsageAnswer } }
> 2012-12-18 15:05:51,130 DEBUG [agent.manager.AgentManagerImpl]
> (RouterMonitor-1:null) Details from executing class
> com.cloud.agent.api.NetworkUsageCommand:
> 2012-12-18 15:05:51,131 WARN
> [network.router.VirtualNetworkApplianceManagerImpl] (RouterMonitor-1:null)
> unable to find stats for account: 2
> We can see that AgentManager works well. It can get the network usage.
> However, there is a problem in subsequent data processing.
>
> 2. Check the source code and the value in database.
> We can see the data processing procedure in
> /cloud-server/src/com/cloud/network/router/VirtualNetworkApplianceManagerImpl.java
> (I have deleted some unrelated source codes)
> public void run() {
> for (DomainRouterVO router : routers) {
> String privateIP = router.getPrivateIpAddress();
> // from domain_router table (id=4,
> name=r-4-VM, vm_instance.private_ip_address=SOME_IP_ADDRESS1)
>
> if (privateIP != null) {
> List<? extends Nic> routerNics = _nicDao.listByVmId(router.getId());
> // from nics table (instance_id=4). Found 3 records.
> Return (networkid= 204/202/200)
> for (Nic routerNic : routerNics) {
> Network network = _networkMgr.getNetwork(routerNic.getNetworkId());
> // from nics table.
> if (network.getTrafficType() == TrafficType.Public) {
> // get traffic_type in networks table.
> Return Guest/Control/Public.
> UserStatisticsVO previousStats =
> _statsDao.findBy(router.getAccountId(),
> router.getDataCenterIdToDeployIn(), network.getId(), null,
> router.getId(), router.getType().toString());
>
> if (answer != null) {
> Transaction txn = Transaction.open(Transaction.CLOUD_DB);
> try {
> txn.start();
> UserStatisticsVO stats =
> _statsDao.lock(router.getAccountId(),
> router.getDataCenterIdToDeployIn(), network.getId(),
> routerNic.getIp4Address(), router.getId(), router.getType().toString());
> // the parameter is “2, 1, 200, SOME_IP_ADDRESS2, 4”. It’s inconsistent to
> database.
> if (stats == null) {
> s_logger.warn("unable to find stats for account: " +
> router.getAccountId());
> continue;
> }
> ………………..
> }
> We can see the data initialization in
> /cloud-server/src/com/cloud/vm/dao/DomainRouterDaoImpl.java
> public void addRouterToGuestNetwork(VirtualRouter router, Network
> guestNetwork) {
> if (stats == null) {
> stats = new UserStatisticsVO(router.getAccountId(),
> router.getDataCenterIdToDeployIn(), null, router.getId(),
> // the parameter is “2, 1, null, 204, 4”. It’s consistent
> to database.
> router.getType().toString(), guestNetwork.getId());
> _userStatsDao.persist(stats);
> }
> }
> 3. Check the database. We can see the data is same to data in data
> initialization in Step (2), but different to data in data processing.
> mysql> select
> id,data_center_id,account_id,public_ip_address,device_id,device_type,network_id
> from user_statistics;
> +----+----------------+------------+-------------------+-----------+--------------+------------+
> | id | data_center_id | account_id | public_ip_address | device_id |
> device_type | network_id |
> +----+----------------+------------+-------------------+-----------+--------------+------------+
> | 1 | 1 | 2 | NULL |
> 4 | DomainRouter | 204 |
> | 2 | 1 | 2 | NULL |
> 6 | DomainRouter | 205 |
>
> 4. Testing. If we insert a record into the databases. The record is same to
> data processing. It will work!
> | 3 | 1 | 2 | SOME_IP_ADDRESS2 | 4 |
> DomainRouter | 200 |
> ... after 5 minutes, we can get the VR data traffic.It looks like
> | id | data_center_id | account_id | public_ip_address | device_id |
> device_type | network_id | net_bytes_received | net_bytes_sent |
> current_bytes_received | current_bytes_sent | agg_bytes_received |
> agg_bytes_sent |
> | 3 | 1 | 2 | SOME_IP_ADDRESS2 | 4 |
> DomainRouter | 200 | 0 |
> 0 | 2284239008 | 33613977 |
> 2284239008 | 33613977 |
>
> From Step 1 to Step 4, we can conclude that the inconsistent of data
> initialization and data processing procedure is the root cause of this
> problem.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
