Re: [ovs-dev] OFtest failures

[Lori Jakab]

Hi Ben,

On 10/16/13 1:16 PM, Simon Horman wrote:
Hi Ben,
On Tue, Oct 15, 2013 at 09:55:12AM -0700, Ben Pfaff wrote:
I see that "make check-oftest" reports 8 failures and 1 error.  (I
haven't updated OFtest in ages, maybe newer versions would give
different results.)  I don't have time to chase these down this week,
but it would be a much appreciated public service if anyone else wants
to take a look.

README-OFTest has instructions for running OFtest.  I got these results
with "make check-oftest OFTFLAGS='--verbose -T all'".
I have taken a look at the errors you list below in the following
environment.

OFtest: 926c30c ("netdev-dummy: Count rx packets regardless of source")
OVS:    48a5f33 ("OPENFLOW-1.1+: note on table-miss")

The revisions above were the head of the master branches
of the respective trees for each project earlier this afternoon.

I also applied the following patch to ovs as OFtest seems
to want to use a different (the new?) port.

--- begin ---
--- a/tests/run-oftest
+++ b/tests/run-oftest
@@ -74,7 +74,7 @@ for port in p1 p2 p3 p4; do
                                 options:pstream=punix:$OVS_RUNDIR/$port
  done
  run ovs-vsctl \
-    -- set-controller br0 tcp:127.0.0.1 \
+    -- set-controller br0 tcp:127.0.0.1:6653 \
      -- set controller br0 connection-mode=out-of-band max-backoff=1000
  
  # Run OFTest.
--- end ---

I have not yet investigated to see if there are any errors
that you didn't hit using -T all.

I started looking at this too before Simon sent this email, with the 
same OFtest and OVS revisions.  I had to apply the same patch to 
tests/run-oftest to make it work, so I suggest applying it to the tree.

I don't receive any ERROR (just like Simon), but I get 11 FAILed tests.  
On my setup pktact.WildcardPriorityWithDelete and 
pktact.WildcardPriority are passed (unlike in Simon's case).

Below are the 5 FAILed tests not on your list.

======================================================================
FAIL: load.FlowRemovedLoad
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/moriarty/src/oftest/tests/load.py", line 290, in runTest
    delete_all_flows(self.controller)
  File "/home/moriarty/src/oftest/src/python/oftest/testutils.py", line 
46, in delete_all_flows
    do_barrier(ctrl)
  File "/home/moriarty/src/oftest/src/python/oftest/testutils.py", line 
498, in do_barrier
    raise AssertionError("barrier failed")
AssertionError: barrier failed

======================================================================
FAIL: message_types.QueueConfigReply
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/moriarty/src/oftest/tests/message_types.py", line 683, in 
runTest
    self.assertTrue(response.type == ofp.OFPT_QUEUE_GET_CONFIG_REPLY, 
"Reply is not Queue Config Reply")
  File "/home/moriarty/src/oftest/src/python/oftest/base_tests.py", 
line 97, in assertTrue
    unittest.TestCase.assertTrue(self, cond, msg)
AssertionError: Reply is not Queue Config Reply

======================================================================
FAIL: message_types.ErrorMsg
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/moriarty/src/oftest/tests/message_types.py", line 105, in 
runTest
    'Switch did not reply with error message')
  File "/home/moriarty/src/oftest/src/python/oftest/base_tests.py", 
line 97, in assertTrue
    unittest.TestCase.assertTrue(self, cond, msg)
AssertionError: Switch did not reply with error message

======================================================================
FAIL: counters.RxBytPerPort
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/moriarty/src/oftest/tests/counters.py", line 244, in runTest
    rx_bytes=byte_count)
  File "/home/moriarty/src/oftest/src/python/oftest/testutils.py", line 
1470, in verify_port_stats
    "Port RX byte counter is not updated correctly (expected increase 
of %d, got increase of %d)" % (rx_bytes, rx_bytes_diff))
  File "/home/moriarty/src/oftest/src/python/oftest/base_tests.py", 
line 97, in assertTrue
    unittest.TestCase.assertTrue(self, cond, msg)
AssertionError: Port RX byte counter is not updated correctly (expected 
increase of 500, got increase of 0)

======================================================================
FAIL: counters.RxPktPerPort
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/moriarty/src/oftest/tests/counters.py", line 173, in runTest
    verify_port_stats(self, of_ports[0], initial=initial_stats, 
rx_pkts=num_pkts)
  File "/home/moriarty/src/oftest/src/python/oftest/testutils.py", line 
1464, in verify_port_stats
    test.assertEqual(rx_pkts,rx_pkts_diff,"Port RX packet counter is 
not updated correctly (expected increase of %d, got increase of %d)" % 
(rx_pkts, rx_pkts_diff))
AssertionError: Port RX packet counter is not updated correctly 
(expected increase of 5, got increase of 0)



======================================================================
ERROR: runTest (pktact.DirectPacketControllerQueue)
----------------------------------------------------------------------
Traceback (most recent call last):
   File "/home/blp/nicira/oftest/tests/pktact.py", line 349, in runTest
     self.handleFlow()
   File "/home/blp/nicira/oftest/tests/pktact.py", line 389, in handleFlow
     + str(self.portQueuesGet(queue_stats, egress_port)))
   File "/home/blp/nicira/oftest/tests/pktact.py", line 353, in portQueuesGet
     for qs in queue_stats.stats:
AttributeError: queue_op_failed_error_msg instance has no attribute 'stats'
This tests passes in my environment.

======================================================================
FAIL: runTest (port_stats.MultiFlowStats)
----------------------------------------------------------------------
Traceback (most recent call last):
   File "/home/blp/nicira/oftest/tests/port_stats.py", line 258, in runTest
     initTxInPort, initRxInPort + num_pkt1s + num_pkt2s)
   File "/home/blp/nicira/oftest/tests/port_stats.py", line 115, in verifyStats
     "Packet received does not match number sent")
   File "/home/blp/nicira/oftest/src/python/oftest/base_tests.py", line 88, in 
assertTrue
     unittest.TestCase.assertTrue(self, cond, msg)
AssertionError: Packet received does not match number sent
I will send a patch for this shortly.

======================================================================
FAIL: runTest (port_stats.SingleFlowStats)
----------------------------------------------------------------------
Traceback (most recent call last):
   File "/home/blp/nicira/oftest/tests/port_stats.py", line 179, in runTest
     verifyStats(self, ingress_port, test_timeout, initTxInPort, initRxInPort + 
num_sends)
   File "/home/blp/nicira/oftest/tests/port_stats.py", line 115, in verifyStats
     "Packet received does not match number sent")
   File "/home/blp/nicira/oftest/src/python/oftest/base_tests.py", line 88, in 
assertTrue
     unittest.TestCase.assertTrue(self, cond, msg)
AssertionError: Packet received does not match number sent
Ditto (its the same patch)

======================================================================
FAIL: runTest (flow_expire.FlowExpire)
----------------------------------------------------------------------
Traceback (most recent call last):
   File "/home/blp/nicira/oftest/tests/flow_expire.py", line 88, in runTest
     'Flow table entry does not match')
AssertionError: Flow table entry does not match
I see this but have not addressed it.

======================================================================
FAIL: runTest (detailed_contr_sw_messages.DeleteEmerFlow)
----------------------------------------------------------------------
Traceback (most recent call last):
   File "/home/blp/nicira/oftest/tests/detailed_contr_sw_messages.py", line 
468, in runTest
     'Test Failed ')
   File "/home/blp/nicira/oftest/src/python/oftest/base_tests.py", line 88, in 
assertTrue
     unittest.TestCase.assertTrue(self, cond, msg)
AssertionError: Test Failed
Ditto

======================================================================
FAIL: runTest (flow_stats.SingleFlowStats)
----------------------------------------------------------------------
Traceback (most recent call last):
   File "/home/blp/nicira/oftest/tests/flow_stats.py", line 156, in runTest
     self.verifyStats(flow_mod_msg, match, ofp.OFPP_NONE, test_timeout, 0)
   File "/home/blp/nicira/oftest/tests/flow_stats.py", line 97, in verifyStats
     "Matches do not match")
AssertionError: Matches do not match
Ditto

======================================================================
FAIL: runTest (pktact.WildcardPriorityWithDelete)
----------------------------------------------------------------------
Traceback (most recent call last):
   File "/home/blp/nicira/oftest/tests/pktact.py", line 1150, in runTest
     self.verifyFlow(of_ports[0], of_ports[2])
   File "/home/blp/nicira/oftest/tests/pktact.py", line 1062, in verifyFlow
     receive_pkt_verify(self, egp, pkt, inp)
   File "/home/blp/nicira/oftest/src/python/oftest/testutils.py", line 309, in 
receive_pkt_verify
     "Did not receive packet port " + str(check_port))
   File "/home/blp/nicira/oftest/src/python/oftest/base_tests.py", line 88, in 
assertTrue
     unittest.TestCase.assertTrue(self, cond, msg)
AssertionError: Did not receive packet port 3
I will send a patch (its the same patch again).

======================================================================
FAIL: runTest (pktact.WildcardPriority)
----------------------------------------------------------------------
Traceback (most recent call last):
   File "/home/blp/nicira/oftest/tests/pktact.py", line 1119, in runTest
     self.verifyFlow(of_ports[0], of_ports[3])
   File "/home/blp/nicira/oftest/tests/pktact.py", line 1062, in verifyFlow
     receive_pkt_verify(self, egp, pkt, inp)
   File "/home/blp/nicira/oftest/src/python/oftest/testutils.py", line 309, in 
receive_pkt_verify
     "Did not receive packet port " + str(check_port))
   File "/home/blp/nicira/oftest/src/python/oftest/base_tests.py", line 88, in 
assertTrue
     unittest.TestCase.assertTrue(self, cond, msg)
AssertionError: Did not receive packet port 4
Ditto

======================================================================
FAIL: runTest (pktact.DirectLlcPackets)
----------------------------------------------------------------------
Traceback (most recent call last):
   File "/home/blp/nicira/oftest/tests/pktact.py", line 2530, in runTest
     IS_NOT_SNAP,
   File "/home/blp/nicira/oftest/tests/pktact.py", line 2510, in testPacket
     testPacketEthTypeNotEth(title, llc, is_snap)
   File "/home/blp/nicira/oftest/tests/pktact.py", line 2506, in 
testPacketEthTypeNotEth
     self.testPktsAgainstFlow(pkts, act, match)
   File "/home/blp/nicira/oftest/tests/pktact.py", line 1999, in 
testPktsAgainstFlow
     self.testPktAgainstFlow(title, pkt, acts, match, expected_result)
   File "/home/blp/nicira/oftest/tests/pktact.py", line 2071, in 
testPktAgainstFlow
     "Did not receive packet, expected a match")
   File "/home/blp/nicira/oftest/src/python/oftest/base_tests.py", line 88, in 
assertTrue
     unittest.TestCase.assertTrue(self, cond, msg)
AssertionError: Did not receive packet, expected a match
I see a different error.

======================================================================
FAIL: pktact.DirectLlcPackets
----------------------------------------------------------------------
Traceback (most recent call last):
   File "/home/horms/projects/openvswitch/oftest/tests/pktact.py", line 2550, 
in runTest
     IS_SNAP_NOT_IP,
   File "/home/horms/projects/openvswitch/oftest/tests/pktact.py", line 2515, 
in testPacket
     testPacketEthTypeNotEth(title, llc, is_snap)
   File "/home/horms/projects/openvswitch/oftest/tests/pktact.py", line 2511, 
in testPacketEthTypeNotEth
     self.testPktsAgainstFlow(pkts, act, match)
   File "/home/horms/projects/openvswitch/oftest/tests/pktact.py", line 2001, 
in testPktsAgainstFlow
     self.testPktAgainstFlow(title, pkt, acts, match, expected_result)
   File "/home/horms/projects/openvswitch/oftest/tests/pktact.py", line 2090, 
in testPktAgainstFlow
     self.assertTrue(rcv_pkt is None, "Received packet, expected drop")
   File 
"/home/horms/projects/openvswitch/oftest/src/python/oftest/base_tests.py", line 
97, in assertTrue
     unittest.TestCase.assertTrue(self, cond, msg)
AssertionError: Received packet, expected drop

_______________________________________________
dev mailing list
dev@openvswitch.org
http://openvswitch.org/mailman/listinfo/dev

_______________________________________________
dev mailing list
dev@openvswitch.org
http://openvswitch.org/mailman/listinfo/dev